An industrial vision inspection data processing method

Abstract

The present invention discloses an industrial vision inspection data processing method, which extracts product appearance data from product inspection images, constructs a preliminary screening model for product appearance, comprehensively evaluates product quality from two perspectives of color and geometry, and conducts a preliminary screening of products to ensure the comprehensiveness of data processing. Then, the preliminarily screened products are re-inspected. The image acquisition accuracy is analyzed and evaluated through product inspection images, and the image acquisition performance is comprehensively evaluated through periodic data. Then, it is combined with the product re-inspection results to deeply evaluate the vision inspection efficiency and generate a vision control signal to dynamically adjust the vision acquisition mode, realizing the vision inspection requirements under abnormal environmental change states, thereby ensuring the continuous stability of vision inspection efficiency. Furthermore, by evaluating the preliminary screening inspection efficiency of the product appearance preliminary screening model, a model optimization signal is generated to optimize and upgrade the product appearance preliminary screening model, realizing model self-growth and improving the preliminary screening inspection accuracy of the model.

Description

Technical Field

[0001] The present invention relates to the technical field of visual detection, and in particular, to an industrial visual detection data processing method. Background Art

[0002] Industrial visual detection technology, by simulating the visual function of humans and applying machine learning and image processing technologies, realizes precise control of product quality, providing a brand-new intelligent solution for industrial manufacturing. Industrial visual detection mainly relies on computer vision technology to obtain image information of the target object through a camera or other sensors, and uses image processing and recognition algorithms to analyze the image, thereby realizing automatic detection of product quality;

[0003] However, the existing industrial visual acquisition mode is easily affected by changes in the detection environment. During the industrial visual detection process, there are defects in poor environmental adaptability. For example, infrared imaging detection is suitable for high-temperature environments but is easily affected by temperature changes; binocular camera detection is fast, efficient, and convenient, but is easily restricted by light; line scan camera detection is suitable for assembly lines, but has strict requirements for the object motion control of products;

[0004] Therefore, in the state of abnormal changes in the visual detection environment, the traditional mode cannot effectively adapt to the changing scenarios, resulting in insufficient image acquisition accuracy and poor stability, leading to incomplete data processing and inaccurate product quality evaluation, thus affecting the visual detection efficiency and the industrial production detection effect;

[0005] In view of the above technical defects, a solution is now proposed. Summary of the Invention

[0006] The purpose of the present invention is to solve the technical defects that the existing industrial visual acquisition mode is easily affected by changes in the detection environment, cannot effectively adapt to the changing scenarios, resulting in insufficient image acquisition accuracy and poor stability, leading to incomplete data processing and inaccurate product quality evaluation, thus affecting the visual detection efficiency and the industrial production detection effect. The technical solution of the present invention constructs a preliminary screening model for product appearance by using product appearance data, comprehensively evaluates product quality and conducts preliminary screening on products to ensure the comprehensiveness of data processing. By evaluating the image acquisition accuracy and image acquisition performance, and then combining them with the product re-inspection results, the visual detection efficiency is deeply evaluated and the visual acquisition mode is dynamically adjusted to meet the visual detection requirements in the state of abnormal environmental changes, thereby ensuring the continuous stability of the visual detection efficiency. Furthermore, by evaluating the preliminary screening detection efficiency of the product appearance preliminary screening model, the model self-growth is realized, and the preliminary screening detection accuracy of the model is improved.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] An industrial vision inspection data processing method includes the following steps:

[0009] Step 1, the vision acquisition unit monitors and obtains product appearance data: The vision acquisition device is used to obtain the product inspection images on the production line, and the image pixels and image frame rate of the product inspection images are collected. Then, the image acquisition period T is set, and the actual product image A is obtained at regular intervals through the product inspection images. And the required product image B is obtained according to the user's requirements. The product appearance data of the actual product image A and the required product image B are collected through an image analysis tool;

[0010] Step 2, the data processing unit preliminarily analyzes the product appearance data: A product appearance preliminary screening model is constructed, and the product appearance data of the actual product image A and the required product image B are compared to comprehensively evaluate the product quality. And the products with unqualified quality are preliminarily screened. Then, the products with unqualified quality after the preliminary screening are re-inspected to obtain the qualified rate of the product re-inspection;

[0011] Step 3, the core analysis unit deeply analyzes and evaluates the vision inspection efficiency: By combining the image pixels and image frame rate of the product inspection images, the image acquisition accuracy is evaluated. Then, a change curve of the image acquisition accuracy is constructed and curve analysis is carried out to comprehensively evaluate the image acquisition performance. And the image acquisition performance is combined with the qualified rate of the product re-inspection to evaluate the vision inspection efficiency and generate corresponding vision control signals;

[0012] Step 4, the vision control unit receives the vision control signal and feeds back to the vision acquisition unit to adjust the vision acquisition mode;

[0013] Step 5, the model optimization unit evaluates the preliminary screening detection efficiency of the product appearance preliminary screening model, generates a model optimization signal and sends it to the data processing unit to optimize and upgrade the product appearance preliminary screening model.

[0014] Further, the specific process of constructing the product appearance preliminary screening model is as follows:

[0015] The product appearance data includes color parameters and geometric parameters; the color parameters include RGB color vectors and HSV color vectors; the geometric parameters include the displacement vectors of pixel point pairs;

[0016] A1, compare the color parameters of the actual product image A and the required product image B;

[0017] A1-1, extract n0 pixel points of the actual product image A and the required product image B respectively, and construct a parameter analysis sub-model to analyze the color parameters;

[0018] A1-2. Construct the RGB parameter matrix Crgb and substitute it into the parameter analysis sub-model: By comparing the RGB color vectors of pixel points, construct the RGB comparison matrix Cpc to obtain the RGB comprehensive comparison coefficient Xrgb;

[0019] A1-3. Construct the HSV parameter matrix Dhsv and substitute it into the parameter analysis sub-model: By comparing the HSV color vectors of pixel points, construct the HSV comparison matrix Dpc to obtain the HSV comprehensive comparison coefficient Xhsv;

[0020] A2. Compare the geometric parameters of the actual product image A and the required product image B;

[0021] A2-1. Extract n1 pairs of pixel points from the actual product image A and the required product image B respectively, so as to construct the geometric parameter matrix of the actual image A and the required product image B;

[0022] A2-2. Obtain the geometric comparison matrix Pdb through the geometric parameter matrices of the actual image A and the required product image B;

[0023] A2-3. Obtain the geometric comprehensive comparison coefficient Xjh by analyzing the geometric comparison matrix Pdb;

[0024] A3. Combine the RGB comprehensive comparison coefficient Xrgb, the HSV comprehensive comparison coefficient Xhsv and the geometric comprehensive comparison coefficient Xjh to obtain the product quality evaluation index ZA, determine the product quality of the actual product image A, and thus conduct a preliminary screening of the product.

[0025] Furthermore, the specific process of constructing the parameter analysis sub-model is as follows:

[0026] Input the parameter matrix ψ1 and the parameter matrix ψ2 into the parameter analysis sub-model;

[0027] Obtain the parameter comparison matrix ψpc by subtracting the parameter matrix ψ1 and the parameter matrix ψ2;

[0028] Then, obtain the deviation coefficient of the column vector by calculating the mean value of the column vectors of the parameter comparison matrix ψpc;

[0029] Furthermore, comprehensively obtain the comprehensive comparison coefficient of the parameter comparison matrix ψpc to evaluate the degree of parameter comparison deviation;

[0030] Output the comprehensive comparison coefficient of the parameter comparison matrix ψpc.

[0031] Furthermore, the specific process of obtaining the RGB comprehensive comparison coefficient Xrgb is as follows:

[0032] The RGB color vector is denoted as C: C = <R, G, B>, where R represents the intensity of the red component, G represents the intensity of the green component, and B represents the intensity of the blue component;

[0033] Mark any two pixel points formed in the actual product image A as i, and mark the RGB color vector of pixel point i as Ci: Ci = <Ri, Gi, Bi>;

[0034] Mark any two pixel points formed in the product requirement image B as j, and mark the RGB color vector of pixel point j as Cj: Cj = <Rj, Gj, Bj>;

[0035] Construct an RGB parameter matrix Crgb through the RGB color vectors of n0 pixel points;

[0036] Construct the RGB parameter matrix Crgb of the actual product image A in sequence through the RGB color vectors of n0 pixel points of the actual product image A and the product requirement image B i and the RGB parameter matrix Crgb of the product requirement image B j , and substitute them into the parameter analysis sub-model to construct an RGB comparison matrix Cpc;

[0037] Obtain the deviation coefficients of the red component intensity, green component intensity, and blue component intensity of the pixel points of the actual product image A from the RGB comparison matrix Cpc, and mark them as φr, φg, and φb in sequence, and then comprehensively obtain the RGB comprehensive comparison coefficient Xrgb of the actual product image A.

[0038] Furthermore, the specific process of obtaining the HSV comprehensive comparison coefficient Xhsv is as follows:

[0039] The HSV color vector is denoted as D: D = <H, S, V>, where H represents hue, S represents saturation, and V represents value;

[0040] Mark the HSV color vector of pixel point i as Di: Di = <Hi, Si, Vi>;

[0041] Mark the HSV color vector of pixel point j as Dj: Dj = <Hj, Sj, Vj>;

[0042] Construct an HSV parameter matrix Dhsv through the HSV color vectors of n0 pixel points;

[0043] Construct the HSV parameter matrix Dhsv of the actual product image A in sequence through the HSV color vectors of n0 pixel points of the actual product image A and the product requirement image B i and the HSV parameter matrix Dhsv of the product requirement image B j, substitute into the parameter analysis sub-model, and construct the HSV comparison matrix Dpc;

[0044] Obtain the deviation coefficients of the hue, saturation, and lightness of the pixel points of the actual product image A from the HSV comparison matrix Dpc, and mark them as φh, φs, and φv in sequence. Then, comprehensively obtain the HSV comprehensive comparison coefficient Xhsv of the actual product image A.

[0045] Furthermore, the specific process of obtaining the geometric comprehensive comparison coefficient Xjh is as follows:

[0046] Mark any pixel point pair of the product demand image B as J. The pixel point pair J includes pixel point De and pixel point Df, and mark the displacement vector of the pixel point pair J as Displacement vector Includes the length value Jl and the direction angle Jr from pixel point De to pixel point Df. Construct the geometric parameter matrix P1 through the displacement vectors of n1 pixel point pairs of the product demand image B.

[0047] Mark any pixel point pair of the actual product image A as I. The pixel point pair I includes pixel point Da and pixel point Db, and mark the displacement vector of the pixel point pair I as Displacement vector Includes the length value Il and the direction angle Ir from pixel point Da to pixel point Db. Construct the geometric parameter matrix P2 through the displacement vectors of n1 pixel point pairs of the actual product image A.

[0048] Subtract the geometric parameter matrix P2 from the geometric parameter matrix P1 to obtain the geometric comparison matrix Pdb, obtain the deviation coefficients of the displacement length and displacement angle of the actual product image A, and mark them as ψl and ψr in sequence. Then, comprehensively obtain the geometric comprehensive comparison coefficient Xjh of the actual product image A.

[0049] Furthermore, the specific process of deeply analyzing and evaluating the visual detection efficiency is as follows:

[0050] Mark the image pixels and image frame rate of the product detection image as XSp and ZLp respectively. Set the data measurement period Tp, and combine the image pixels XSp and the image frame rate ZLp to obtain the image acquisition accuracy coefficient Xtj;

[0051] Construct the change curve S0 between the image acquisition accuracy coefficient Xtj and the data measurement period Tp, and perform curve analysis: By extracting n2 points of the change curve S0, obtain the overall coefficient Zs and the fluctuation coefficient σs of the change curve S0, and then obtain the image acquisition performance evaluation coefficient XN;

[0052] Reconstruct the change curve between the qualified rate HG of product reinspection and the image acquisition performance evaluation coefficient XN, and fit to generate the change function F0. Combine the change function F0 with the image acquisition performance evaluation coefficient XN to obtain the visual detection efficiency coefficient Xsj.

[0053] Set the evaluation interval of the visual detection efficiency coefficient Xsj, evaluate the visual detection efficiency level through interval comparison, and generate corresponding visual control signals.

[0054] In summary, due to the adoption of the above technical solutions, the beneficial effects of the present invention are as follows:

[0055] The present invention extracts product appearance data from product detection images, constructs a product appearance preliminary screening model, starts from two aspects of color and geometry, comprehensively evaluates product quality and conducts preliminary screening on products to ensure the comprehensiveness of data processing. Then, reinspect the preliminarily screened products, analyze and evaluate the image acquisition accuracy through product detection images, comprehensively evaluate the image acquisition performance through periodic data, and then combine it with the product reinspection results to deeply evaluate the visual detection efficiency and generate visual control signals to dynamically adjust the visual acquisition mode to meet the visual detection requirements under abnormal environmental change states, thereby ensuring the continuous stability of visual detection efficiency. Furthermore, by evaluating the preliminary screening detection efficiency of the product appearance preliminary screening model, generate model optimization signals to optimize and upgrade the product appearance preliminary screening model, realize model self-growth, and improve the preliminary screening detection accuracy of the model.

[0056] That is, the technical solution of the present invention is to solve the defect of incomplete data processing and various problems caused thereby. Through various data processing, the data processing is made more comprehensive and more comprehensive; through the improved data processing method, the effect of more accurate product appearance detection is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 Shows a flowchart of the present invention;

[0058] Figure 2 Shows a block diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0060] Embodiment 1:

[0061] As Figure 1 - Figure 2As shown in the figure, an industrial vision inspection data processing method includes the following steps:

[0062] S1. The vision acquisition unit monitors and obtains product appearance data: The vision acquisition device is used to obtain the product inspection images of the production line, and the image pixels and image frame rate of the product inspection images are collected. Then, the image acquisition period Ta is set to regularly obtain the actual product image A from the product inspection images, and the product requirement image B is obtained according to the preset user requirements.

[0063] The user requirements refer to the appearance requirements of the product by the user. For example, if the product is a box and is a cuboid, the user requirements are the appearance parameters of the length, width, and height of the product. The 3D model can be built based on the parameters of the user requirements, and then the product requirement image B is generated.

[0064] The product appearance data includes color parameters and geometric parameters. The product appearance data of the actual product image A and the product requirement image B are obtained through an image analysis tool. Among them, the color parameters include RGB color vectors and HSV color vectors; the geometric parameters include the displacement vectors of pixel point pairs.

[0065] S2. The data processing unit preliminarily analyzes the product appearance data: A product appearance preliminary screening model is constructed. From the two perspectives of color and geometry, the product appearance data of the actual product image A and the product requirement image B are compared, so as to comprehensively evaluate the product quality. The products with unqualified quality are preliminarily screened, and the unqualified products after preliminary screening are re-inspected in the inspection room to obtain the qualified rate HG of the product re-inspection.

[0066] Among them, the environmental conditions in the inspection room are kept constant, so as to ensure that the image acquisition accuracy coefficient in the inspection room is in a high-precision and stable performance state, and the product re-inspection is carried out to obtain the qualified rate HG of the product re-inspection. The qualified rate HG of the product re-inspection refers to the proportion of the number of products qualified in the re-inspection among the products unqualified in the preliminary screening.

[0067] When the qualified rate HG of the product re-inspection result is 0, that is, all the preliminarily screened products are unqualified in quality, it means that the accuracy rate of the preliminary screening is 100%; the higher the qualified rate HG of the product re-inspection, the lower the accuracy rate and the worse the effect of the preliminary screening.

[0068] The specific process of constructing the product appearance preliminary screening model is as follows:

[0069] A1. Compare the color parameters of the actual product image A and the product requirement image B.

[0070] A1-1. Extract n0 pixel points from the actual product image A and the product requirement image B respectively, and construct a parameter analysis sub-model to analyze the color parameters.

[0071] The specific process of constructing the parameter analysis sub-model is as follows:

[0072] Input the parameter matrix ψ1 and the parameter matrix ψ2 into the parameter analysis sub-model;

[0073] Obtain the parameter comparison matrix ψpc by subtracting the parameter matrix ψ1 from the parameter matrix ψ2;

[0074] Then, obtain the deviation coefficient of the column vectors by calculating the mean of the column vectors of the parameter comparison matrix ψpc;

[0075] Furthermore, comprehensively obtain the comprehensive comparison coefficient of the parameter comparison matrix ψpc to evaluate the degree of parameter comparison deviation;

[0076] Output the comprehensive comparison coefficient of the parameter comparison matrix ψpc;

[0077] A1-2, construct the RGB parameter matrix Crgb and substitute it into the parameter analysis sub-model: Compare through the RGB color vectors of pixel points to construct the RGB comparison matrix Cpc to obtain the RGB comprehensive comparison coefficient Xrgb. The specific process is as follows:

[0078] Mark the RGB color vector as C: C = <R, G, B>, where R refers to the intensity of the red component, G refers to the intensity of the green component, B refers to the intensity of the blue component, and R, G, B are all in the interval [0, 255];

[0079] Mark any two pixel points formed in the actual product image A as i, and mark the RGB color vector of pixel point i as Ci: Ci = <Ri, Gi, Bi>;

[0080] Mark any two pixel points formed in the product demand image B as j, and mark the RGB color vector of pixel point j as Cj: Cj = <Rj, Gj, Bj>;

[0081] Among them, Ci = <Ri, Gi, Bi>, where Ri refers to the intensity of the red component of pixel point i, Gi refers to the intensity of the green component of pixel point i, Bi refers to the intensity of the blue component of pixel point i, and Ri, Gi, Bi are all in the interval [0, 255];

[0082] The parameter meaning of Cj = <Rj, Gj, Bj> is the same and will not be elaborated;

[0083] Construct the RGB parameter matrix Crgb through the RGB color vectors of n0 pixel points:

[0084]

[0085] The RGB parameter matrix Crgb of the actual product image A i and the RGB parameter matrix Crgb of the product demand image B jSubstitute the parameters into the analysis sub-model and output the RGB comprehensive comparison coefficient Xrgb of the actual product image A. The specific process is as follows:

[0086] Construct the RGB parameter matrix Crgb of the actual product image A in sequence through the RGB color vectors of n0 pixel points of the actual product image A and the required product image B i and the RGB parameter matrix Crgb of the required product image B j , and then construct the RGB comparison matrix Cpc: Cpc = Crgb i - Crgb j ;

[0087] Thus, obtain:

[0088]

[0089] That is,

[0090] By calculating the mean value of the column vectors of the RGB comparison matrix Cpc, obtain the deviation coefficients of the red component intensity, green component intensity, and blue component intensity of the pixel points of the actual product image A, and mark them as φr, φg, and φb in sequence;

[0091] Among them, the deviation coefficient φr of the red component intensity:

[0092] The deviation coefficient φg of the green component intensity:

[0093] The deviation coefficient φb of the blue component intensity:

[0094] Through the deviation coefficients of the red component intensity, green component intensity, and blue component intensity of the pixel points of the actual product image A, comprehensively obtain the RGB comprehensive comparison coefficient Xrgb of the actual product image A:

[0095] Xrgb = α1 * φr + α2 * φg + α3 * φb

[0096] Among them, α1, α2, and α3 are the weight factor coefficients of the deviation coefficient φr of the red component intensity, the deviation coefficient φg of the green component intensity, and the deviation coefficient φb of the blue component intensity respectively, and α1, α2, and α3 are all greater than 0; the weight factor coefficients are preset after being measured through a large amount of data. When the deviation coefficient φr of the red component intensity, the deviation coefficient φg of the green component intensity, and the deviation coefficient φb of the blue component intensity are higher, the RGB comprehensive comparison coefficient Xrgb of the actual product image A is higher, indicating that the RGB comprehensive comparison deviation degree of the actual product image A is greater;

[0097] A1-3, construct the HSV parameter matrix Dhsv and substitute it into the parameter analysis sub-model: By comparing the HSV color vectors of pixel points, construct the HSV comparison matrix Dpc to obtain the HSV comprehensive comparison coefficient Xhsv. The specific process is as follows:

[0098] Mark the HSV color vector as D: D = <H, S, V>, where H refers to hue, S refers to saturation, and V refers to value;

[0099] Mark the HSV color vector of pixel point i as Di: Di = <Hi, Si, Vi>, and mark the HSV color vector of pixel point j as Dj: Dj = <Hj, Sj, Vj>;

[0100] Among them, Di = <Hi, Si, Vi>, where Hi refers to the hue of pixel point i, Si refers to the saturation of pixel point i, and Vi refers to the value of pixel point i;

[0101] The parameter meanings of Dj = <Hj, Sj, Vj> are the same as above and will not be elaborated;

[0102] Construct the HSV parameter matrix Dhsv through the HSV color vectors of n0 pixel points:

[0103]

[0104] Substitute the HSV parameter matrix Dhsv of the actual product image A i and the HSV parameter matrix Dhsv of the product requirement image B j into the parameter analysis sub-model, and output the HSV comprehensive comparison coefficient Xhsv of the actual product image A. The specific process is as follows:

[0105] Through the HSV color vectors of n0 pixel points of the actual product image A and the product requirement image B, successively construct the HSV parameter matrix Dhsv of the actual product image A i and the HSV parameter matrix Dhsv of the product requirement image B j , and then construct the HSV comparison matrix Dpc: Dpc = Dhsv i -Dhsv j ;

[0106] Thus, obtain:

[0107]

[0108] That is,

[0109] By calculating the mean of the column vectors of the HSV comparison matrix Dpc, obtain the deviation coefficients of the hue, saturation, and value of the pixel points of the actual product image A, and mark them as φh, φs, and φv in sequence;

[0110] Among them, the hue deviation coefficient φh:

[0111] The saturation deviation coefficient φs:

[0112] The lightness deviation coefficient φv:

[0113] Through the hue deviation coefficient φh, saturation deviation coefficient φs, and lightness deviation coefficient φv of the pixel points of the actual product image A, the HSV comprehensive comparison coefficient Xhsv of the actual product image A is comprehensively obtained:

[0114] Xhsv = β1 * φh + β2 * φs + β3 * φv

[0115] Among them, β1, β2, and β3 are the weight factor coefficients of the hue deviation coefficient φh, saturation deviation coefficient φs, and lightness deviation coefficient φv respectively, and β1, β2, and β3 are all greater than 0; when the hue deviation coefficient φh, saturation deviation coefficient φs, and lightness deviation coefficient φv are higher, the HSV comprehensive comparison coefficient Xhsv of the actual product image A is higher, indicating that the HSV comprehensive comparison deviation degree of the actual product image A is greater;

[0116] A2. Compare the geometric parameters of the actual product image A and the product demand image B;

[0117] A2-1. Respectively extract n1 pixel point pairs of the actual product image A and the product demand image B, so as to construct the geometric parameter matrix of the actual image A and the product demand image B;

[0118] A2-2. Through the geometric parameter matrix of the actual image A and the product demand image B, obtain the geometric comparison matrix Pdb;

[0119] A2-3. Through analyzing the geometric comparison matrix Pdb, obtain the geometric comprehensive comparison coefficient Xjh;

[0120] The specific process is as follows:

[0121] Extract any two pixel points to form a pixel point pair. Mark any pixel point pair of the product demand image B as J; mark the pixel point pair formed by any two pixel points of the actual product image A as I;

[0122] The pixel point pair J includes pixel points De and Df, and mark the displacement vector of the pixel point pair J as The displacement vector Includes the length value Jl and direction angle Jr from pixel point De to pixel point Df, that is

[0123] Among them, a reference line L is set in the product requirement image B 1 , and let the direction angle of the reference line L 1 be 0°, then the direction angle Jr of the displacement vector is the angular value of the included angle between the line connecting the pixel point De and the pixel point Df and the reference line L 1 ;

[0124] The pixel point pair I includes the pixel point Da and the pixel point Db, and the displacement vector of the pixel point pair I is marked as The displacement vector includes the length value Il and the direction angle Ir from the pixel point Da to the pixel point Db, that is

[0125] Among them, according to the coordinates of the reference line L 1 in the product requirement image B, a reference line L 2 is set in the actual product image A, and let the direction angle of the reference line L 2 be 0°, then the direction angle Ir of the displacement vector is the angular value of the included angle between the line connecting the pixel point Da and the pixel point Db and the reference line L 2 ;

[0126] It should be noted that the pixel point pair J is set corresponding to the pixel point pair I. For example, in the product requirement image B, a certain edge line of the required product is marked. The edge line goes from any edge angle to its adjacent edge angle. By connecting the pixel points of the two edge angles, the displacement vector is formed. Then, in the actual product image A, the edge line of the actual product corresponding to the displacement vector of the required product is marked. By connecting the two edge angles set correspondingly in the actual product image A, the displacement vector

[0127] A geometric parameter matrix P1 is constructed through the displacement vectors of n1 pixel point pairs in the product requirement image B;

[0128] A geometric parameter matrix P2 is constructed through the displacement vectors of n1 pixel point pairs in the actual product image A;

[0129] Among them,

[0130] By subtracting the geometric parameter matrix P1 and the geometric parameter matrix P2, a geometric comparison matrix Pdb is obtained;

[0131]

[0132] Furthermore, by calculating the mean value of the column vectors of the geometric comparison matrix Pdb, the deviation coefficients of the displacement length and displacement angle of the actual product image A are obtained, and are marked as ψl and ψr in sequence;

[0133] Among them, the displacement length deviation coefficient ψl:

[0134] The displacement angle deviation coefficient ψr:

[0135] Through the displacement length deviation coefficient ψl and displacement angle deviation coefficient ψr of the actual product image A, the geometric comprehensive comparison coefficient Xjh of the actual product image A is comprehensively obtained:

[0136] Xjh = γ1 * ψl + γ2 * ψr;

[0137] Among them, γ1 and γ2 are the weight factor coefficients of the displacement length deviation coefficient ψl and displacement angle deviation coefficient ψr respectively, and both γ1 and γ2 are greater than 0; when the displacement length deviation coefficient ψl and displacement angle deviation coefficient ψr are higher, the geometric comprehensive comparison coefficient Xjh of the actual product image A is higher, indicating that the geometric comprehensive comparison deviation degree of the actual product image A is greater;

[0138] A3. By combining the RGB comprehensive comparison coefficient Xrgb, the HSV comprehensive comparison coefficient Xhsv and the geometric comprehensive comparison coefficient Xjh, the product quality evaluation index ZA is obtained to determine the product quality of the actual product image A, thereby performing a preliminary screening of the product;

[0139] A3-1. By combining the RGB comprehensive comparison coefficient Xrgb and the HSV comprehensive comparison coefficient Xhsv of the actual product image A, the color comprehensive comparison coefficient Xsc of the actual product image A is obtained, and combined with the geometric comprehensive comparison coefficient Xjh of the actual product image A, the product quality evaluation index ZA is obtained:

[0140]

[0141] Among them, μ1 and μ2 are the weight indices of the color comprehensive comparison coefficient Xsc and the geometric comprehensive comparison coefficient Xjh respectively, and both μ1 and μ2 are greater than 0; when the color comprehensive comparison coefficient Xsc and the geometric comprehensive comparison coefficient Xjh are higher, the product quality evaluation index ZA is lower, indicating that when the color difference and geometric size difference of the detected product are greater, the product quality is worse;

[0142] The above formula can comprehensively evaluate the product quality from two aspects: color Xsc and geometry Xjh. Among them, the color is comprehensively evaluated from two aspects: RGB and HSV. The "+1" is to avoid the situation where all X coefficients are 0, resulting in a denominator of 0 in the formula. The X coefficients are all comparison / deviation coefficients of the corresponding parameters. When the X parameter is higher, it indicates that the deviation / difference between the actual product and the requirement is greater, indicating that the product quality detected in terms of appearance is worse.

[0143] A3 - 2, set the threshold Gz of the product quality evaluation index ZA. When the product quality evaluation index ZA is lower than the threshold Gz, it is determined that the product quality of the actual image A of the product is unqualified, thus realizing the preliminary screening of the products with unqualified quality. This threshold Gz is an empirical value and is set according to application requirements and accuracy requirements.

[0144] S3, the core analysis unit deeply analyzes and evaluates the visual detection efficiency: through the image pixels and image frame rate of the product detection image, set the data measurement period Tp to regularly evaluate the image acquisition accuracy, and then construct the change curve between the image acquisition accuracy coefficient Xtj and the data measurement period Tp, comprehensively evaluate the image acquisition performance, and combine it with the qualified rate HG of the product re - inspection to evaluate the visual detection efficiency and generate the corresponding visual control signal.

[0145] S3 - 1, mark the image pixels and image frame rate of the product detection image as XSp and ZLp respectively.

[0146] Set the data measurement period Tp. By combining the image pixels XSp and the image frame rate ZLp, and regularly obtain the image acquisition accuracy coefficient Xtj according to the data measurement period Tp: Xtj = ω1 * XSp + ω2 * ZLp.

[0147] Among them, ω1 and ω2 are the weight factor coefficients of the image pixels XSp and the image frame rate ZLp respectively, and both ω1 and ω2 are greater than 0. When the image pixels XSp and the image frame rate ZLp are higher, the image acquisition accuracy coefficient Xtj is higher, indicating that the image acquisition accuracy of the product detection image is high and the quality is good.

[0148] S3 - 2, with the image acquisition accuracy coefficient Xtj as the ordinate and the data measurement period Tp as the abscissa, and the abscissa represents the time sequence of the data measurement period, that is, the first data measurement period, the second data measurement period,..., and so on, construct the change curve S0 between the image acquisition accuracy coefficient Xtj and the data measurement period Tp, and conduct curve analysis.

[0149] S3-201, Extract n2 points of the change curve S0. Mark any point on the change curve S0 and its coordinates as q(Xq, Yq). Calculate the average value through the ordinates of the n2 points of the change curve S0 to obtain the overall coefficient Zs of the change curve S0: When the overall coefficient Zs of the change curve S0 is higher, it indicates that the overall level of the image acquisition accuracy coefficient Xtj is higher and the image acquisition quality is better;

[0150] S3-202, Furthermore, calculate the standard deviation through the ordinates of the n2 points of the change curve S0 to obtain the fluctuation coefficient σs of the change curve S0: When the fluctuation coefficient σs of the change curve S0 is lower, it indicates that the fluctuation change level of the image acquisition accuracy coefficient Xtj is lower and the image acquisition level is more stable;

[0151] S3-203, Combine the overall coefficient Zs and the fluctuation coefficient σs of the change curve S0 to obtain the image acquisition performance evaluation coefficient XN: XN = λ1*Zs - λ2*σs;

[0152] Among them, λ1 and λ2 are the weight factor coefficients of the overall coefficient Zs and the fluctuation coefficient σs respectively, and both λ1 and λ2 are greater than 0; when the overall coefficient Zs of the change curve S0 is higher and the fluctuation coefficient σs is lower, the image acquisition performance evaluation coefficient XN is higher, indicating better image acquisition performance;

[0153] S3-3, Construct the change curve between the qualified rate HG of product re-inspection and the image acquisition performance evaluation coefficient XN, and fit to generate the change function F0. Among them, F0(XN) = HG. Substitute the image acquisition performance evaluation coefficient XN into the change function F0, and then obtain the corresponding qualified rate HG of product re-inspection;

[0154] S3-4, Combine the change function F0 and the image acquisition performance evaluation coefficient XN to obtain the visual detection efficiency coefficient Xsj:

[0155] When the image acquisition performance evaluation coefficient XN is higher and the qualified rate HG of product re-inspection is lower, the visual detection efficiency coefficient Xsj is higher, indicating better evaluation of visual detection efficiency;

[0156] S3-5, Set the evaluation interval of the visual detection efficiency coefficient Xsj as [f1, f2], and evaluate the visual detection efficiency level through interval comparison to generate the corresponding visual control signal;

[0157] S3-501, When the visual detection efficiency coefficient Xsj is lower than f1, generate a level I visual control signal, and evaluate the visual detection efficiency level as level I, indicating poor visual detection efficiency;

[0158] S3-502. When the visual detection efficiency coefficient Xsj is within the evaluation interval [f1, f2], a level II visual regulation signal is generated, and the visual detection efficiency level is evaluated as level II, indicating that the visual detection efficiency is average.

[0159] S3-503. When the visual detection efficiency coefficient Xsj is higher than f2, a level III visual regulation signal is generated, and the visual detection efficiency level is evaluated as level III, indicating that the visual detection efficiency is good.

[0160] S4. The visual regulation unit receives the visual regulation signal and feeds it back to the visual acquisition unit to adjust the visual acquisition mode.

[0161] S4-1. When receiving the level I visual regulation signal, by setting multiple visual acquisition modes, select the visual acquisition mode with the optimal image acquisition performance according to the product detection environmental conditions to meet the visual detection requirements under abnormal changes in light conditions and temperature conditions, so as to ensure the continuous stability of the visual detection efficiency.

[0162] S4-2. When receiving the level II visual regulation signal, start the test plan for multiple visual acquisition modes. For example, lidar acquisition mode, infrared camera acquisition mode, line scan image acquisition mode, etc. Evaluate the corresponding image acquisition performance under multiple visual acquisition modes, and select the visual acquisition mode with the optimal image acquisition performance for preparation, so as to directly apply it when receiving the level I visual regulation signal, ensuring the continuity and working efficiency of product detection.

[0163] S4-3. When receiving the level III visual regulation signal, no processing is performed.

[0164] S5. The model optimization unit evaluates the initial screening detection efficiency of the product appearance initial screening model, generates a model optimization signal and sends it to the data processing unit to optimize and upgrade the product appearance initial screening model.

[0165] S5-1. Combine the pass rate HG of product re-inspection with the image acquisition performance evaluation coefficient XN to obtain the initial screening detection efficiency coefficient Xcs:

[0166] When the pass rate HG of product re-inspection and the image acquisition performance evaluation coefficient XN are higher, the initial screening detection efficiency coefficient Xcs is lower, indicating that in the state where the image acquisition performance evaluation coefficient XN is high, the pass rate HG of product re-inspection is high and the accuracy of initial screening is low, indicating that the data processing effect of the product appearance initial screening model is worse and the initial screening detection efficiency is poor.

[0167] S5-2. Set the threshold f0 of the initial screening detection efficiency coefficient Xcs. When the initial screening detection efficiency coefficient Xcs is lower than the threshold f0, a model optimization signal is generated to adjust the preset parameters of the product appearance initial screening model. The preset parameters include the weight factor coefficient and the weight index. By feeding back the re-inspection results of the product quality to the product appearance initial screening model, the initial screening error results are corrected, so as to continuously train the model to achieve optimization and upgrade.

[0168] Among them, the preset parameters include the weight factor coefficients α1, α2, α3, β1, β2, β3, γ1, and γ2, and the weight indices μ1 and μ2. Mark the initial screening product quality evaluation index as ZA1, then re-inspect the product to obtain the re-inspection results of the product quality, mark the re-inspection product quality evaluation index as ZA2, and compare the initial screening product quality evaluation index ZA1 and the re-inspection product quality evaluation index ZA2 to correct the initial screening results. For example, when ZA2 > ZA1, it means that the initial screening result is incorrect and the value is on the small side. Therefore, a large amount of re-inspection data is required to adaptively increase or decrease the preset parameters, so that the initial screening product quality evaluation index ZA1 increases and continuously approaches the re-inspection product quality evaluation index ZA2, improving the fitting accuracy of the mathematical model and realizing the optimization and upgrade of the model.

[0169] It should be noted that this method is applied to an industrial vision detection data processing system, which includes a vision acquisition unit, a data processing unit, a core analysis unit, a vision regulation unit, and a model optimization unit. Among them, the vision acquisition unit, the data processing unit, the core analysis unit, the vision regulation unit, and the model optimization unit are communicatively connected.

[0170] In summary, the present invention extracts the product appearance data from the product detection image, constructs a product appearance initial screening model, starts from two aspects of color and geometry, comprehensively evaluates the product quality and conducts an initial screening of the product to ensure the comprehensiveness of data processing. Then, the initially screened product is re-inspected, the image acquisition accuracy is analyzed and evaluated through the product detection image, and the image acquisition performance is comprehensively evaluated through periodic data. Then, it is combined with the product re-inspection results to deeply evaluate the vision detection efficiency and generate a vision regulation signal to dynamically adjust the vision acquisition mode, realizing the vision detection requirements under abnormal environmental change conditions, thus ensuring the continuous stability of the vision detection efficiency. Furthermore, by evaluating the initial screening detection efficiency of the product appearance initial screening model, a model optimization signal is generated to optimize and upgrade the product appearance initial screening model, realizing model self-growth and improving the initial screening detection accuracy of the model.

[0171] The setting of the interval and the size of the threshold is for the convenience of comparison. Regarding the size of the threshold, it depends on the amount of sample data and the base quantity set by those skilled in the art for each group of sample data; as long as it does not affect the proportional relationship between the parameter and the quantified value.

[0172] The above formulas are all dimensionless and take their numerical values for calculation. The formulas are obtained by collecting a large amount of data for software simulation to get a formula closest to the actual situation. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0173] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent substitutions or changes should be covered within the protection scope of the present invention.

Claims

1. A method for processing industrial visual inspection data, characterized in that: The following steps are involved: Step 1: The visual acquisition unit monitors and acquires product appearance data: the product inspection image of the production line is acquired through the visual acquisition device, and the image pixels and image frame rate of the product inspection image are acquired, and then the image acquisition cycle T is set, the actual product image A is acquired through the product inspection image at a fixed time, and the product requirement image B is acquired through the user requirement preset, and the product appearance data of the actual product image A and the product requirement image B are acquired through the image analysis tool; Step 2: The data processing unit preliminarily analyzes the product appearance data: constructs a product appearance preliminary screening model, compares the product appearance data of the actual product image A and the product requirement image B, thereby comprehensively evaluating the product quality, and preliminarily screening the unqualified products, and then re-inspecting the unqualified products after the preliminary screening to obtain the qualified rate of the product re-inspection; Step three, the core analysis unit deeply analyzes and evaluates the visual inspection performance: by combining the image pixels and image frame rate of the product inspection image, the image acquisition accuracy is evaluated, and then a change curve of the image acquisition accuracy is constructed and a curve analysis is performed, so as to comprehensively evaluate the image acquisition performance, and the image acquisition performance is combined with the qualified rate of product re-inspection, the visual inspection performance is evaluated and the corresponding visual control signal is generated; wherein, the image pixels and image frame rate of the product inspection image are marked as XSp and ZLp respectively, and the data calculation period Tp is set. By combining the image pixels XSp and the image frame rate ZLp, the image acquisition accuracy coefficient Xtj is obtained regularly, and a change curve S0 between the image acquisition accuracy coefficient Xtj and the data calculation period Tp is constructed, and a curve analysis is performed; Step 4: the visual control unit receives the visual control signal, and then feeds it back to the visual acquisition unit to adjust the visual acquisition mode; Step five: The model optimization unit evaluates the initial screening performance of the product appearance initial screening model, generates a model optimization signal and sends it to the data processing unit to optimize and upgrade the product appearance initial screening model.

2. The industrial visual inspection data processing method according to claim 1, characterized in that: The specific process of building a product appearance preliminary screening model is as follows: Product appearance data includes color parameters and geometric parameters; color parameters include RGB color vectors and HSV color vectors; geometric parameters include displacement vectors of pixel pairs; A1, compare the color parameters of the actual product image A and the required product image B; A1-1, extract n0 pixels of the actual product image A and the required product image B respectively, and build a parameter analysis sub-model to analyze the color parameters; A1-2, construct the RGB parameter matrix Crogb and substitute it into the parameter analysis sub-model: compare the RGB color vectors of the pixels, construct the RGB contrast matrix Cpc to obtain the RGB comprehensive contrast coefficient Xrgb; A1-3, construct the HSV parameter matrix Dhsv and substitute it into the parameter analysis sub-model: compare the HSV color vectors of the pixels, construct the HSV comparison matrix Dpc to obtain the HSV comprehensive comparison coefficient Xhsv; A2, compare the geometric parameters of the actual product image A and the required product image B; A2-1, extract n1 pixel pairs of the product actual image A and the product requirement image B respectively, so as to construct the geometric parameter matrix of the actual image A and the product requirement image B; A2-2, obtain the geometric contrast matrix Pdb through the geometric parameter matrices of the actual image A and the product requirement image B; A2-3, by analyzing the geometric contrast matrix Pdb, obtain the geometric comprehensive contrast coefficient Xjh; A3, by combining the RGB comprehensive contrast coefficient Xrgb, the HSV comprehensive contrast coefficient Xhsv and the geometric comprehensive contrast coefficient Xjh, the product quality assessment index ZA is obtained to determine the product quality of the actual product image A, thereby performing preliminary product screening.

3. The industrial visual inspection data processing method according to claim 2, characterized in that: The specific process of constructing the parameter analysis sub-model is as follows: Input parameter matrix ψ1 and parameter matrix ψ2 to the parameter analysis sub-model; Obtain the parameter comparison matrix ψpc by subtracting the parameter matrix ψ1 from the parameter matrix ψ2; Then, the column vector of the parameter comparison matrix ψpc is averaged to obtain the deviation coefficient of the column vector; Then, the comprehensive comparison coefficient of the parameter comparison matrix ψpc is obtained to evaluate the degree of parameter comparison deviation; The comprehensive contrast coefficient of the output parameter contrast matrix ψpc.

4. The industrial visual inspection data processing method according to claim 3 is characterized in that: The specific process of obtaining the RGB comprehensive contrast coefficient Xrgb is: The RGB color vector is labeled C: C =<R,G,B> , where R refers to the intensity of the red component, G refers to the intensity of the green component, and B refers to the intensity of the blue component; Mark any pixel of the actual product image A as i, and mark the RGB color vector of pixel i as Ci: Ci =<Ri,Gi,Bi> ; Mark any pixel of the product demand image B as j, and mark the RGB color vector of pixel j as Cj: Cj =<Rj,Gj,Bj> ; Construct the RGB parameter matrix Crogb through the RGB color vectors of n0 pixels; The RGB parameter matrix C rgb of the actual product image A is constructed in turn through the RGB color vectors of n0 pixels of the actual product image A and the required product image B. i And the RGB parameter matrix Crogb of the product demand image B j , and substitute it into the parameter analysis sub-model to construct the RGB contrast matrix Cpc; The deviation coefficients of the red component intensity, green component intensity, and blue component intensity of the pixel points of the actual product image A are obtained by the RGB contrast matrix Cpc, and are marked as φr, φg, and φb respectively, and then the RGB comprehensive contrast coefficient Xrgb of the actual product image A is comprehensively obtained.

5. The industrial visual inspection data processing method according to claim 4, characterized in that: The specific process of obtaining the HSV comprehensive contrast coefficient Xhsv is: The HSV color vector is marked as D: D =<H,S,V> , where H refers to hue, S refers to saturation, and V refers to brightness; Mark the HSV color vector of pixel i as Di: Di ​​=<Hi,Si,Vi> ; The HSV color vector of pixel j is marked as Dj: Dj =<Hj,Sj,Vj> ; Construct the HSV parameter matrix Dhsv through the HSV color vectors of n0 pixels; The HSV parameter matrix Dhsv of the actual product image A is constructed in turn through the HSV color vectors of n0 pixels of the actual product image A and the required product image B. i And the HSV parameter matrix Dhsv of product demand image B j , and substitute it into the parameter analysis sub-model to construct the HSV comparison matrix Dpc; The deviation coefficients of the hue, saturation, and brightness of the pixel points of the actual product image A are obtained by the HSV contrast matrix Dpc, and are marked as φh, φs, and φv respectively, so as to comprehensively obtain the HSV comprehensive contrast coefficient Xhsv of the actual product image A.

6. The industrial visual inspection data processing method according to claim 5, characterized in that: The specific process of obtaining the geometric comprehensive contrast coefficient Xjh is: The pixel pair formed by any two pixels of the product demand image B is marked as J. The pixel pair J includes pixel De and pixel Df. The displacement vector of the pixel pair J is Displacement Vector Including the length value Jl and direction angle Jr from pixel point De to pixel point Df, Construct the geometric parameter matrix P1 through the displacement vectors of n1 pixel pairs of the product demand image B; The pixel pair formed by any two pixels of the actual product image A is marked as I. The pixel pair I includes pixel Da and pixel Db. The displacement vector of the pixel pair I is Displacement Vector Including the length value Il and direction angle Ir from pixel Da to pixel Db, Construct the geometric parameter matrix P2 through the displacement vectors of n1 pixel pairs of the actual product image A; By subtracting the geometric parameter matrix P1 from the geometric parameter matrix P2, the geometric contrast matrix Pdb is obtained, and the deviation coefficients of the displacement length and displacement angle of the actual image A of the product are obtained, and marked as ψl and ψr respectively, and then the geometric comprehensive contrast coefficient Xjh of the actual image A of the product is comprehensively obtained.

7. The industrial visual inspection data processing method according to claim 6, characterized in that: The specific process of in-depth analysis and evaluation of visual inspection performance is as follows: By extracting n2 points of the change curve S0, the overall coefficient Zs and the fluctuation coefficient σs of the change curve S0 are obtained, and then the image acquisition performance evaluation coefficient XN is obtained; Reconstruct the change curve between the qualified rate HG of product re-inspection and the image acquisition performance evaluation coefficient XN, and fit the change function F0 to generate the change function F0. By combining the change function F0 with the image acquisition performance evaluation coefficient XN, the visual inspection efficiency coefficient Xsj is obtained; Set the evaluation interval of the visual detection efficiency coefficient Xsj, evaluate the visual detection efficiency level through interval comparison, and generate the corresponding visual control signal.

Images