Industrial product multi-channel detection method and system based on reinforcement learning

Abstract

This invention provides a multi-channel inspection method and system for industrial products based on reinforcement learning, comprising: step S1: acquiring a dataset of inspection results from adjacent processes of the same workpiece; step S2: filtering the inspection information in the dataset of inspection results; step S3: setting control conditions; and step S4: solving for the optimal solution of the control conditions. The multi-channel inspection strategy based on reinforcement learning proposed in this invention continuously learns from environmental experience and, with appropriate rewards, ultimately finds the optimal solution to the value function. This significantly improves the overall inspection efficiency and effectively solves the problem of waste caused by single-process inspection leading to multiple-channel inspections.

Description

Technical Field

[0001] This invention relates to the field of industrial production, and more specifically, to a multi-channel detection method and system for industrial products based on reinforcement learning. Background Technology

[0002] The production process for industrial products is often lengthy, and inspection equipment is typically installed after critical production steps. Defects that don't affect the quality of the current product in a previous step may become quality-critical in a later step. Traditional multi-stage inspection processes only check the pre-set inspection results for that specific step. Some defects that are detectable in a previous step may not have a substantial impact on the current step, but after further processing or other treatments, they might only be detected as non-compliant (NG) and scrapped at a later stage. This leads to low efficiency and wasted inspection time.

[0003] Patent document CN111368970A discloses a sensor optimization placement method based on deep reinforcement learning, relating to the fields of structural health monitoring and vibration testing. First, it derives the sensor optimization placement criteria for engineering structures based on mutual information correlation theory. Then, it uses a deep reinforcement learning-based optimization algorithm to quickly and effectively find the optimal solution to the objective function, i.e., the optimal sensor placement positions for the structure. Its drawbacks are: 1. It requires a specially designed model for the problem, and the optimal solution is obtained using reinforcement learning, resulting in poor versatility; 2. It uses a dedicated neural network for calculating the value function, leading to poor reusability. The corresponding value function code can be directly written according to the actual scenario, and the intermediate process of the neural network is unpredictable, resulting in poor interpretability. Summary of the Invention

[0004] In view of the shortcomings of the prior art, the purpose of this invention is to provide a multi-channel inspection method and system for industrial products based on reinforcement learning.

[0005] A reinforcement learning-based multichannel inspection method for industrial products, according to the present invention, includes:

[0006] Step S1: Obtain the inspection result dataset of adjacent processes for the same workpiece;

[0007] Step S2: Filter the detection information in the detection result dataset;

[0008] Step S3: Set the control conditions;

[0009] Step S4: Solve for the optimal solution of the control conditions.

[0010] Preferably, in step S1:

[0011] The detection results of all workpieces appearing in adjacent processes are collected and stored in a detection result information file. The detection result information file format includes CSV and BMP, and contains information such as the identifier of the workpiece, the position of each detection result, the type of detection result, the size of the detection result, and the shape and number of detection results. The detection result information file is read by reading tools, including Pandas and OpenCV. Workpiece matching is performed in adjacent processes using workpiece identifiers, including OCRSTRING, to obtain the detection result dataset of adjacent processes for the same workpiece.

[0012] Preferably, in step S2:

[0013] The method for generating multichannel detection strategies selects detection results as a prerequisite for setting control conditions based on the actual situation, and filters suitable detection results based on experience or observed features, including:

[0014] The test results have prominent shapes, including irregular test results with many uneven edges;

[0015] The size of the test result is larger than the preset standard;

[0016] The categories of test results that fall within the preset standards include test results for burrs, foreign objects, and those where the shape difference between preceding and following processes is greater than the preset standards.

[0017] The number of similar test results is greater than the preset value.

[0018] Preferably, in step S3:

[0019] The setting of control conditions is a multi-dimensional, multi-objective nonlinear programming problem. Multiple conditions are set based on various features including size, shape, and category. During application, based on the correlation between the detection results of adjacent processes, the size, shape, category, location, and grayscale distribution of the detection results are used as basic conditions. The control of the number of basic conditions is defined by the following variables:

[0020] Under the new control conditions, when the optical inspection result of the preceding process is NG, the number of workpieces with the optical inspection result of the following process being NG is represented as A, and the number of workpieces with the optical inspection result of the following process being OK is represented as B.

[0021] Under the new control conditions, the number of workpieces whose optical inspection results in the preceding process are OK and whose optical inspection results in the following process are NG is represented as C, and the number of workpieces whose optical inspection results in the following process are OK is represented as D.

[0022] The false positive rate x for the control conditions is:

[0023] x = B / A

[0024] The NG ratio y for optical inspection in the subsequent process is:

[0025] y = (A+B) / (A+B+C+D)

[0026] The accuracy z of the card control method is:

[0027] z = (A+D) / (A+B+C+D);

[0028] The planned restrictions and adjustments are set as follows: x does not exceed the first threshold; y does not exceed the second threshold; z is greater than the third threshold; the control conditions are set according to the basic conditions and restrictions.

[0029] Preferably, in step S4:

[0030] Policy-based reinforcement learning is used to find the optimal solution for the checkpoint condition. The set checkpoint condition is used as the policy in reinforcement learning, and the state space, action space, reward function, and Q-value function are defined as follows:

[0031] State={OK_OK, NG_OK, OK_NG, NG_NG}

[0032] Where OK indicates that the optical inspection result is OK, NG indicates that the optical inspection result is NG, OK_OK indicates that the optical inspection result of the previous process is OK and the optical inspection result of the next process is OK, NG_0K indicates that the optical inspection result of the previous process is NG and the optical inspection result of the next process is OK; OK_NG indicates that the optical inspection result of the previous process is OK and the optical inspection result of the next process is NG; NG_NG indicates that the optical inspection result of the previous process is NG and the optical inspection result of the next process is NG.

[0033] Aciton = {Controlled, Uncontrolled}

[0034] Reward = -|xT x |-|yT y |-|zT z |

[0035] Among them, T x T represents the first threshold. y T represents the second threshold. z This represents the third threshold. If the state vector matches the target threshold, the reward function is 0; otherwise, the reward function will be a negative value related to the degree of deviation from the target threshold.

[0036] Q(S i A i )←Q(S i A i)+α(R i +γ·max(Q(S i+1 ,a))-Q(S i A i ))

[0037] Among them, S i Indicates the current state, A i R represents the action to be performed in the current state. i S represents the reward obtained after performing an action in the current state. i+1 Q(S) represents the new state after taking an action in the current state, α represents the learning rate, γ represents the discount factor, and Q(S) represents the new state after taking an action in the current state. i A i ) indicates the state S i Begin execution of Action A i The expected value of the future returns that can be obtained, max(Q(S) i+1 , a)) is in state S i+1 The maximum Q value of all possible actions;

[0038] In each experimental step, based on the current state S i Choose Action A i A new state S is observed. i+1 and the corresponding reward R i Then update the Q-value function until the function converges.

[0039] According to the present invention, a multi-channel inspection system for industrial products based on reinforcement learning includes:

[0040] Module M1: Obtains the dataset of inspection results from adjacent processes for the same workpiece;

[0041] Module M2: Filters detection information in the detection result dataset;

[0042] Module M3: Sets the card control conditions;

[0043] Module M4: Solve for the optimal solution of the card control conditions.

[0044] Preferably, in module M1:

[0045] The detection results of all workpieces appearing in adjacent processes are collected and stored in a detection result information file. The detection result information file format includes CSV and BMP, and contains information such as the identifier of the workpiece, the position of each detection result, the type of detection result, the size of the detection result, and the shape and number of detection results. The detection result information file is read by reading tools, including Pandas and OpenCV. Workpiece matching is performed in adjacent processes using workpiece identifiers, including OCRSTRING, to obtain the detection result dataset of adjacent processes for the same workpiece.

[0046] Preferably, in module M2:

[0047] The method for generating multichannel detection strategies selects detection results as a prerequisite for setting control conditions based on the actual situation, and filters suitable detection results based on experience or observed features, including:

[0048] The test results have prominent shapes, including irregular test results with many uneven edges;

[0049] The size of the test result is larger than the preset standard;

[0050] The categories of test results that fall within the preset standards include test results for burrs, foreign objects, and those where the shape difference between preceding and following processes is greater than the preset standards.

[0051] The number of similar test results is greater than the preset value.

[0052] Preferably, in module M3:

[0053] The setting of control conditions is a multi-dimensional, multi-objective nonlinear programming problem. Multiple conditions are set based on various features including size, shape, and category. During application, based on the correlation between the detection results of adjacent processes, the size, shape, category, location, and grayscale distribution of the detection results are used as basic conditions. The control of the number of basic conditions is defined by the following variables:

[0054] Under the new control conditions, when the optical inspection result of the preceding process is NG, the number of workpieces with the optical inspection result of the following process being NG is represented as A, and the number of workpieces with the optical inspection result of the following process being OK is represented as B.

[0055] Under the new control conditions, the number of workpieces whose optical inspection results in the preceding process are OK and whose optical inspection results in the following process are NG is represented as C, and the number of workpieces whose optical inspection results in the following process are OK is represented as D.

[0056] The false positive rate x for the control conditions is:

[0057] x = B / A

[0058] The NG ratio y for optical inspection in the subsequent process is:

[0059] y = (A+B) / (A+B+C+D)

[0060] The accuracy z of the card control method is:

[0061] z = (A+D) / (A+B+C+D);

[0062] The planned restrictions and adjustments are set as follows: x does not exceed the first threshold; y does not exceed the second threshold; z is greater than the third threshold; the control conditions are set according to the basic conditions and restrictions.

[0063] Preferably, in module M4:

[0064] Policy-based reinforcement learning is used to find the optimal solution for the checkpoint condition. The set checkpoint condition is used as the policy in reinforcement learning, and the state space, action space, reward function, and Q-value function are defined as follows:

[0065] State={0K-OK, NG_OK, OK_NG, NG_NG}

[0066] Where OK indicates that the optical inspection result is OK, NG indicates that the optical inspection result is NG, OK_0K indicates that the optical inspection result of the previous process is OK and the optical inspection result of the next process is OK, NG_OK indicates that the optical inspection result of the previous process is NG and the optical inspection result of the next process is OK; OK_NG indicates that the optical inspection result of the previous process is OK and the optical inspection result of the next process is NG; NG_NG indicates that the optical inspection result of the previous process is NG and the optical inspection result of the next process is NG.

[0067] Aciton = {Controlled, Uncontrolled}

[0068] Reward = -|xT x |-|yT y |-|zT z |

[0069] Among them, T x T represents the first threshold. y T represents the second threshold. z This represents the third threshold. If the state vector matches the target threshold, the reward function is 0; otherwise, the reward function will be a negative value related to the degree of deviation from the target threshold.

[0070] Q(S i A i )←Q(S i A i)+α(R i +γ·max(Q(S i+1 ,a))-Q(S i A i ))

[0071] Among them, S i Indicates the current state, A i R represents the action to be performed in the current state. i S represents the reward obtained after performing an action in the current state. i+1 Q(S) represents the new state after taking an action in the current state, α represents the learning rate, γ represents the discount factor, and Q(S) represents the new state after taking an action in the current state. i A i ) indicates the state S i Begin execution of Action A i The expected value of the future returns that can be obtained, max(Q(S) i+1 , a)) is in state S i+1 The maximum Q value of all possible actions;

[0072] In each experimental step, based on the current state S i Choose Action A i A new state S is observed. i+1 and the corresponding reward R i Then update the Q-value function until the function converges.

[0073] Compared with the prior art, the present invention has the following beneficial effects:

[0074] 1. The multi-channel detection strategy based on reinforcement learning proposed in this invention learns continuously from the experience of the environment and is given certain rewards, so that it eventually finds the optimal solution of the value function. This can greatly improve the overall detection efficiency and effectively solve the problem of waste caused by single-channel detection.

[0075] 2. This invention does not require spending a lot of time building a corresponding model for the environment. Based on the characteristics of Monte Carlo optimal solutions, it can be self-trained and continuously optimized in the environment.

[0076] 3. The implementation process of this invention can directly use highly interpretable logic functions, avoiding the problem of poor interpretability of neural networks, and it also has strong reusability. Attached Figure Description

[0077] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0078] Figure 1 This is a flowchart illustrating the overall technical implementation process. Detailed Implementation

[0079] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0080] Example 1:

[0081] According to the present invention, a multi-channel detection method for industrial products based on reinforcement learning is provided, such as... Figure 1 As shown, it includes:

[0082] Step S1: Obtain the inspection result dataset of adjacent processes for the same workpiece;

[0083] Specifically, in step S1:

[0084] The detection results of all workpieces appearing in adjacent processes are collected and stored in a detection result information file. The detection result information file format includes CSV and BMP, and contains information such as the identifier of the workpiece, the position of each detection result, the type of detection result, the size of the detection result, and the shape and number of detection results. The detection result information file is read by reading tools, including Pandas and OpenCV. Workpiece matching is performed in adjacent processes using workpiece identifiers, including OCRSTRING, to obtain the detection result dataset of adjacent processes for the same workpiece.

[0085] Step S2: Filter the detection information in the detection result dataset;

[0086] Specifically, in step S2:

[0087] The method for generating multichannel detection strategies selects detection results as a prerequisite for setting control conditions based on the actual situation, and filters suitable detection results based on experience or observed features, including:

[0088] The test results have prominent shapes, including irregular test results with many uneven edges;

[0089] The size of the test result is larger than the preset standard;

[0090] The categories of test results that fall within the preset standards include test results for burrs, foreign objects, and those where the shape difference between preceding and following processes is greater than the preset standards.

[0091] The number of similar test results is greater than the preset value.

[0092] Step S3: Set the control conditions;

[0093] Specifically, in step S3:

[0094] The setting of control conditions is a multi-dimensional, multi-objective nonlinear programming problem. Multiple conditions are set based on various features including size, shape, and category. During application, based on the correlation between the detection results of adjacent processes, the size, shape, category, location, and grayscale distribution of the detection results are used as basic conditions. The control of the number of basic conditions is defined by the following variables:

[0095] Under the new control conditions, when the optical inspection result of the preceding process is NG, the number of workpieces with the optical inspection result of the following process being NG is represented as A, and the number of workpieces with the optical inspection result of the following process being OK is represented as B.

[0096] Under the new control conditions, the number of workpieces whose optical inspection results in the preceding process are OK and whose optical inspection results in the following process are NG is represented as C, and the number of workpieces whose optical inspection results in the following process are OK is represented as D.

[0097] The false positive rate x for the control conditions is:

[0098] x = B / A

[0099] The NG ratio y for optical inspection in the subsequent process is:

[0100] y = (A+B) / (A+B+C+D)

[0101] The accuracy z of the card control method is:

[0102] z = (A+D) / (A+B+C+D);

[0103] The planned restrictions and adjustments are set as follows: x does not exceed the first threshold; y does not exceed the second threshold; z is greater than the third threshold; the control conditions are set according to the basic conditions and restrictions.

[0104] Step S4: Solve for the optimal solution of the control conditions.

[0105] Specifically, in step S4:

[0106] Policy-based reinforcement learning is used to find the optimal solution for the checkpoint condition. The set checkpoint condition is used as the policy in reinforcement learning, and the state space, action space, reward function, and Q-value function are defined as follows:

[0107] State={OK_OK,NG_OK,OK_NG,NG_NG}

[0108] Where OK indicates that the optical inspection result is OK, NG indicates that the optical inspection result is NG, OK_OK indicates that the optical inspection result of the previous process is OK and the optical inspection result of the next process is OK, NG_OK indicates that the optical inspection result of the previous process is NG and the optical inspection result of the next process is OK; OK_NG indicates that the optical inspection result of the previous process is OK and the optical inspection result of the next process is NG; NG_NG indicates that the optical inspection result of the previous process is NG and the optical inspection result of the next process is NG.

[0109] Aciton = {Controlled, Uncontrolled}

[0110] Reward = -|xT x |-|yT y |-|zT z |

[0111] Among them, T x T represents the first threshold. y T represents the second threshold. z This represents the third threshold. If the state vector matches the target threshold, the reward function is 0; otherwise, the reward function will be a negative value related to the degree of deviation from the target threshold.

[0112] Q(S i A i )←Q(S i A i )+α(R i +γ·max(Q(S i+1 ,a))-Q(S i A i ))

[0113] Among them, S i Indicates the current state, A i R represents the action to be performed in the current state. i S represents the reward obtained after performing an action in the current state. i+1 Q(S) represents the new state after taking an action in the current state, α represents the learning rate, γ represents the discount factor, and Q(S) represents the new state after taking an action in the current state. i A i ) indicates the state S i Begin execution of Action A i The expected value of the future returns that can be obtained, max(Q(S) i+1 , a)) is in state S i+1 The maximum Q value of all possible actions;

[0114] In each experimental step, based on the current state S i Choose Action A i A new state S is observed. i+1and the corresponding reward R i Then update the Q-value function until the function converges.

[0115] Example 2:

[0116] Example 2 is a preferred embodiment of Example 1, and is used to illustrate the present invention in more detail.

[0117] The present invention also provides a reinforcement learning-based multichannel inspection system for industrial products. The reinforcement learning-based multichannel inspection system for industrial products can be implemented by executing the process steps of the reinforcement learning-based multichannel inspection method for industrial products. That is, those skilled in the art can understand the reinforcement learning-based multichannel inspection method for industrial products as a preferred embodiment of the reinforcement learning-based multichannel inspection system for industrial products.

[0118] According to the present invention, a multi-channel inspection system for industrial products based on reinforcement learning includes:

[0119] Module M1: Obtains the dataset of inspection results from adjacent processes for the same workpiece;

[0120] Specifically, in module M1:

[0121] The detection results of all workpieces appearing in adjacent processes are collected and stored in a detection result information file. The detection result information file format includes CSV and BMP, and contains information such as the identifier of the workpiece, the position of each detection result, the type of detection result, the size of the detection result, and the shape and number of detection results. The detection result information file is read by reading tools, including Pandas and OpenCV. Workpiece matching is performed in adjacent processes using workpiece identifiers, including OCRSTRING, to obtain the detection result dataset of adjacent processes for the same workpiece.

[0122] Module M2: Filters detection information in the detection result dataset;

[0123] Specifically, in module M2:

[0124] The method for generating multichannel detection strategies selects detection results as a prerequisite for setting control conditions based on the actual situation, and filters suitable detection results based on experience or observed features, including:

[0125] The test results have prominent shapes, including irregular test results with many uneven edges;

[0126] The size of the test result is larger than the preset standard;

[0127] The categories of test results that fall within the preset standards include test results for burrs, foreign objects, and those where the shape difference between preceding and following processes is greater than the preset standards.

[0128] The number of similar test results is greater than the preset value.

[0129] Module M3: Sets the card control conditions;

[0130] Specifically, in module M3:

[0131] The setting of control conditions is a multi-dimensional, multi-objective nonlinear programming problem. Multiple conditions are set based on various features including size, shape, and category. During application, based on the correlation between the detection results of adjacent processes, the size, shape, category, location, and grayscale distribution of the detection results are used as basic conditions. The control of the number of basic conditions is defined by the following variables:

[0132] Under the new control conditions, when the optical inspection result of the preceding process is NG, the number of workpieces with the optical inspection result of the following process being NG is represented as A, and the number of workpieces with the optical inspection result of the following process being OK is represented as B.

[0133] Under the new control conditions, the number of workpieces whose optical inspection results in the preceding process are OK and whose optical inspection results in the following process are NG is represented as C, and the number of workpieces whose optical inspection results in the following process are OK is represented as D.

[0134] The false positive rate x for the control conditions is:

[0135] x = B / A

[0136] The NG ratio y for optical inspection in the subsequent process is:

[0137] y = (A+B) / (A+B+C+D)

[0138] The accuracy z of the card control method is:

[0139] z = (A+D) / (A+B+C+D);

[0140] The planned restrictions and adjustments are set as follows: x does not exceed the first threshold; y does not exceed the second threshold; z is greater than the third threshold; the control conditions are set according to the basic conditions and restrictions.

[0141] Module M4: Solve for the optimal solution of the card control conditions.

[0142] Specifically, in module M4:

[0143] Policy-based reinforcement learning is used to find the optimal solution for the checkpoint condition. The set checkpoint condition is used as the policy in reinforcement learning, and the state space, action space, reward function, and Q-value function are defined as follows:

[0144] State={OK_OK,NG_OK,OK_NG,NG_NG}

[0145] Where OK indicates that the optical inspection result is OK, NG indicates that the optical inspection result is NG, OK_OK indicates that the optical inspection result of the previous process is OK and the optical inspection result of the next process is OK, NG_OK indicates that the optical inspection result of the previous process is NG and the optical inspection result of the next process is OK; OK_NG indicates that the optical inspection result of the previous process is OK and the optical inspection result of the next process is NG; NG_NG indicates that the optical inspection result of the previous process is NG and the optical inspection result of the next process is NG.

[0146] Aciton = {Controlled, Uncontrolled}

[0147] Reward = -|xT x |-|yT y |-|zT z |

[0148] Among them, T x T represents the first threshold. y T represents the second threshold. z This represents the third threshold. If the state vector matches the target threshold, the reward function is 0; otherwise, the reward function will be a negative value related to the degree of deviation from the target threshold.

[0149] Q(S i A i )←Q(S i A i )+α(R i +γ·max(Q(S i+1 ,a))-Q(S i A i ))

[0150] Among them, S i Indicates the current state, A i R represents the action to be performed in the current state. i S represents the reward obtained after performing an action in the current state. i+1 Q(S) represents the new state after taking an action in the current state, α represents the learning rate, γ represents the discount factor, and Q(S) represents the new state after taking an action in the current state. i A i ) indicates the state S i Begin execution of Action A i The expected value of the future returns that can be obtained, max(Q(S) i+1 , a)) is in state S i+1 The maximum Q value of all possible actions;

[0151] In each experimental step, based on the current state S i Choose Action A i A new state S is observed. i+1 Then, update the Q-value function with the corresponding reward Ri until the function converges.

[0152] Example 3:

[0153] Example 3 is a preferred example of Example 1, and is used to illustrate the present invention in more detail.

[0154] This patent proposes a method for generating multi-channel inspection strategies for industrial products based on reinforcement learning. This method can pre-screen products based on the characteristics of inspection results from preceding and following processes, and set appropriate control conditions according to actual conditions. During the preceding production process, potentially defective products from subsequent processes are intercepted and repaired, reducing waste of production resources. The control conditions are set using a Monte Carlo method based on reinforcement learning, employing a reward mechanism to optimize the control conditions.

[0155] Step 1: Obtain the inspection result dataset for adjacent processes of the same workpiece. First, collect the inspection results of all workpieces appearing in adjacent processes and store them in an inspection result information file. The file format includes CSV, BMP, etc., and contains information such as the workpiece identifier, the position of each inspection result, the type of inspection result, the size of the inspection result, and the shape and quantity of the inspection results. Read the inspection result information file using tools such as Pandas and OpenCV. Then, use the workpiece identifiers (such as OCRSTRING) to perform workpiece matching in adjacent processes, thereby obtaining the inspection result dataset for adjacent processes of the same workpiece.

[0156] Step two involves filtering the detection information in the detection result dataset. The generation method for multichannel detection strategies requires selecting detection results with appropriate features as a prerequisite for setting control conditions, based on the actual situation. Suitable detection results can be selected based on experience or observed features, including but not limited to:

[0157] The test results have a prominent shape, such as irregular test results with many uneven edges;

[0158] The test results are relatively large in size;

[0159] The categories of test results are very special, such as burrs, foreign objects, and test results with significant differences in shape between the preceding and following processes;

[0160] There are a large number of similar test results.

[0161] All of these can serve as criteria for screening detection results. After gaining a general understanding of the detection result dataset, we can prepare for setting subsequent control conditions.

[0162] Step 3, Setting Control Conditions. Setting control conditions is a multivariate, multi-objective nonlinear programming problem. Multiple conditions can be set based on features such as size, shape, and category. In application, based on the correlation between the detection results of adjacent processes, the size, shape, category, location, and grayscale distribution of the detection results are used as basic conditions. A number of control measures are then set for these basic conditions, and the following variables are defined:

[0163] 1) Under the new control conditions, the number of workpieces whose optical inspection results in the preceding process are NG and whose optical inspection results in the following process are NG is represented as A, and the number of workpieces whose optical inspection results in the following process are OK is represented as B.

[0164] 2) Under the new control conditions, the number of workpieces whose optical inspection results in the preceding process are OK and whose optical inspection results in the following process are NG is represented as C, and the number of workpieces whose optical inspection results in the following process are OK is represented as D.

[0165] 3) The error rate of the control condition is expressed as x = B / A, the NG ratio of the optical inspection in the later process is expressed as y = (A+B) / (A+B+C+D), and the accuracy of the control method is expressed as z = (A+D) / (A+B+C+D).

[0166] Based on the above conditions, the planning restrictions are set in the following three ways: 1) x cannot exceed the first threshold; 2) y cannot exceed the second threshold; 3) z must be greater than the third threshold.

[0167] Complete the setting of card control conditions based on basic and restrictive conditions.

[0168] Step 4: Find the optimal solution for the checkpoint conditions. A policy-based reinforcement learning method is used to find the optimal solution for the checkpoint conditions. The checkpoint conditions set in Step 3 are used as the policy in reinforcement learning. The state space (State), action space (Action), reward function (Reward), and Q-value function are defined as follows:

[0169] 1) State = {OK_OK, NG_OK, OK_NG, NG_NG}. Where OK represents that the optical inspection result is OK, NG represents that the optical inspection result is NG, and OK_OK means that the optical inspection result of the previous process is 0K and the optical inspection result of the subsequent process is OK;

[0170] 2) Aciton = {Controlled, Not Controlled};

[0171] 3) Reward = -|xT x |-|yT y |-|zT z |. Among them, T x T represents the first threshold. y T represents the second threshold. z This represents the third threshold. In this case, if the state vector matches the target threshold, the reward function is 0; otherwise, the reward function will be a negative value related to the degree of deviation from the target threshold.

[0172] 4) Q(S) i A i )←Q(S i A i )+α(R i +γ·max(Q(S i+1 ,a))-Q(S i A i )). Among them, S i Indicates the current state, A i R represents the action to be performed in the current state. i S represents the reward obtained after performing an action in the current state. i+1 Q(S) represents the new state after taking an action in the current state, α represents the learning rate, γ represents the discount factor, and Q(S) represents the new state after taking an action in the current state. i A i ) indicates the state S i Begin execution of Action A i The expected value of the future returns that can be obtained, max(Q(S) i+1 , a)) is in state S i+1 The maximum Q value of all possible actions.

[0173] In each experimental step, based on the current state S i Choose one action A i Then observe the new state S i+1 and the corresponding reward R i Then update the Q-value function until the function converges.

[0174] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0175] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A multi-channel detection method for industrial products based on reinforcement learning, characterized in that, include: Step S1: Obtain the inspection result dataset of adjacent processes for the same workpiece; Step S2: Filter the detection information in the detection result dataset; Step S3: Set the control conditions; Step S4: Solve for the optimal solution of the control conditions; In step S3: The setting of control conditions is a multi-dimensional, multi-objective nonlinear programming problem. Multiple conditions are set based on various features including size, shape, and category. During application, based on the correlation between the detection results of adjacent processes, the size, shape, category, location, and grayscale distribution of the detection results are used as basic conditions. The control of the number of basic conditions is defined by the following variables: Under the new control conditions, when the optical inspection result of the preceding process is NG, the number of workpieces with the optical inspection result of the following process being NG is represented as A, and the number of workpieces with the optical inspection result of the following process being OK is represented as B. Under the new control conditions, the number of workpieces whose optical inspection results in the preceding process are OK and whose optical inspection results in the following process are NG is represented as C, and the number of workpieces whose optical inspection results in the following process are OK is represented as D. The false positive rate x for the control conditions is: x = B / A The NG ratio y for optical inspection in the subsequent process is: y = (A+B) / (A+B+C+D) The accuracy z of the card control method is: z = (A+D) / (A+B+C+D); The planned restrictions and adjustments are set as follows: x does not exceed the first threshold; y does not exceed the second threshold; z is greater than the third threshold; the control conditions are set according to the basic conditions and restrictions.

2. The reinforcement learning-based multichannel detection method for industrial products according to claim 1, characterized in that, In step S1: Collect the inspection results of all workpieces appearing in adjacent processes and store them in the inspection result information file. The format of the inspection result information file includes CSV and BMP, and contains information including the identifier of the workpiece, the location of each inspection result, the type of inspection result, the size of the inspection result, and the shape and quantity of the inspection results. The detection result information file is read using reading tools, including Pandas and OpenCV. Workpiece identification is used to match workpieces in adjacent processes. Workpiece identification includes OCRSTRING. This yields a dataset of detection results for adjacent processes of the same workpiece.

3. The reinforcement learning-based multichannel detection method for industrial products according to claim 1, characterized in that, In step S2: The method for generating multichannel detection strategies selects detection results as a prerequisite for setting control conditions based on the actual situation, and filters suitable detection results based on experience or observed features, including: The test results have prominent shapes, including irregular test results with many uneven edges; The size of the test result is larger than the preset standard; The categories of test results that fall within the preset standards include test results for burrs, foreign objects, and those where the shape difference between preceding and following processes is greater than the preset standards. The number of similar test results is greater than the preset value.

4. The reinforcement learning-based multichannel detection method for industrial products according to claim 1, characterized in that, In step S4: Policy-based reinforcement learning is used to find the optimal solution for the checkpoint condition. The set checkpoint condition is used as the policy in reinforcement learning, and the state space, action space, reward function, and Q-value function are defined as follows: State={OK_OK,NG_OK,OK_NG,NG_NG} Where OK indicates that the optical inspection result is OK, NG indicates that the optical inspection result is NG, OK_OK indicates that the optical inspection result of the previous process is OK and the optical inspection result of the next process is OK, and NG_OK indicates that the optical inspection result of the previous process is NG and the optical inspection result of the next process is OK. OK_NG indicates that the optical inspection result of the previous process is OK and the optical inspection result of the subsequent process is NG; NG_NG indicates that the optical inspection result of the previous process is NG and the optical inspection result of the subsequent process is NG; Aciton = {Controlled, Uncontrolled} Reward=-|x-T x |-|y-T y |-|z-T z | Among them, T x T represents the first threshold. y T represents the second threshold. z This represents the third threshold. If the state vector matches the target threshold, the reward function is 0; otherwise, the reward function will be a negative value related to the degree of deviation from the target threshold. Q(S i ,A i )←Q(S i ,A i )+α(R i +γ·max(Q(S i+1 ,a))-Q(S i ,A i )) Among them, S i Indicates the current state, A i R represents the action to be performed in the current state. i S represents the reward obtained after performing an action in the current state. i+1 Q(S) represents the new state after taking an action in the current state, α represents the learning rate, γ represents the discount factor, and Q(S) represents the new state after taking an action in the current state. i A i ) indicates the state S i Begin execution of Action A i The expected value of the future returns that can be obtained, max(Q(S) i+1 ,a)) is in state S i+1 The maximum Q value of all possible actions; In each experimental step, based on the current state S i Choose Action A i A new state S is observed. i+1 and the corresponding reward R i Then update the Q-value function until the function converges.

5. A multi-channel inspection system for industrial products based on reinforcement learning, characterized in that, include: Module M1: Obtains the dataset of inspection results from adjacent processes for the same workpiece; Module M2: Filters detection information in the detection result dataset; Module M3: Sets the card control conditions; Module M4: Solve for the optimal solution of the card control conditions; In module M3: The setting of control conditions is a multi-dimensional, multi-objective nonlinear programming problem. Multiple conditions are set based on various features including size, shape, and category. During application, based on the correlation between the detection results of adjacent processes, the size, shape, category, location, and grayscale distribution of the detection results are used as basic conditions. The control of the number of basic conditions is defined by the following variables: Under the new control conditions, when the optical inspection result of the preceding process is NG, the number of workpieces with the optical inspection result of the following process being NG is represented as A, and the number of workpieces with the optical inspection result of the following process being OK is represented as B. Under the new control conditions, the number of workpieces whose optical inspection results in the preceding process are OK and whose optical inspection results in the following process are NG is represented as C, and the number of workpieces whose optical inspection results in the following process are OK is represented as D. The false positive rate x for the control conditions is: x = B / A The NG ratio y for optical inspection in the subsequent process is: y = (A+B) / (A+B+C+D) The accuracy z of the card control method is: z = (A+D) / (A+B+C+D); The planned restrictions and adjustments are set as follows: x does not exceed the first threshold; y does not exceed the second threshold; z is greater than the third threshold; the control conditions are set according to the basic conditions and restrictions.

6. The reinforcement learning-based multi-channel inspection system for industrial products according to claim 5, characterized in that, In module M1: Collect the inspection results of all workpieces appearing in adjacent processes and store them in the inspection result information file. The format of the inspection result information file includes CSV and BMP, and contains information including the identifier of the workpiece, the location of each inspection result, the type of inspection result, the size of the inspection result, and the shape and quantity of the inspection results. The detection result information file is read using reading tools, including Pandas and OpenCV. Workpiece identification is used to match workpieces in adjacent processes. Workpiece identification includes OCRSTRING. This yields a dataset of detection results for adjacent processes of the same workpiece.

7. The reinforcement learning-based multi-channel inspection system for industrial products according to claim 5, characterized in that, In module M2: The method for generating multichannel detection strategies selects detection results as a prerequisite for setting control conditions based on the actual situation, and filters suitable detection results based on experience or observed features, including: The test results have prominent shapes, including irregular test results with many uneven edges; The size of the test result is larger than the preset standard; The categories of test results that fall within the preset standards include test results for burrs, foreign objects, and those where the shape difference between preceding and following processes is greater than the preset standards. The number of similar test results is greater than the preset value.

8. The reinforcement learning-based multi-channel inspection system for industrial products according to claim 5, characterized in that, In module M4: Policy-based reinforcement learning is used to find the optimal solution for the checkpoint condition. The set checkpoint condition is used as the policy in reinforcement learning, and the state space, action space, reward function, and Q-value function are defined as follows: State={OK_OK,NG_OK,OK_NG,NG_NG} Where OK indicates that the optical inspection result is OK, NG indicates that the optical inspection result is NG, OK_OK indicates that the optical inspection result of the previous process is OK and the optical inspection result of the next process is OK, and NG_OK indicates that the optical inspection result of the previous process is NG and the optical inspection result of the next process is OK. OK_NG indicates that the optical inspection result of the previous process is OK and the optical inspection result of the subsequent process is NG; NG_NG indicates that the optical inspection result of the previous process is NG and the optical inspection result of the subsequent process is NG; Aciton = {Controlled, Uncontrolled} Reward=-|x-T x |-|y-T y |-|z-T z | Among them, T x T represents the first threshold. y T represents the second threshold. z This represents the third threshold. If the state vector matches the target threshold, the reward function is 0; otherwise, the reward function will be a negative value related to the degree of deviation from the target threshold. Q(S i ,A i )←Q(S i ,A i )+α(R i +γ·max(Q(S i+1 ,a))-Q(S i ,A i )) Among them, S i Indicates the current state, A i R represents the action to be performed in the current state. i S represents the reward obtained after performing an action in the current state. i+1 Q(S) represents the new state after taking an action in the current state, α represents the learning rate, γ represents the discount factor, and Q(S) represents the new state after taking an action in the current state. i A i ) indicates the state S i Begin execution of Action A i The expected value of the future returns that can be obtained, max(Q(S) i+1 ,a)) is in state S i+1 The maximum Q value of all possible actions; In each experimental step, based on the current state S i Choose Action A i A new state S is observed. i+1 and the corresponding reward R i Then update the Q-value function until the function converges.

Images