Industrial visual inspection data processing method

Abstract

The invention discloses an industrial visual inspection data processing method, which comprises the following steps of: extracting product appearance data through a product inspection image, constructing a product appearance preliminary screening model, comprehensively evaluating product quality from two angles of color and geometry, preliminarily screening products, ensuring the comprehensiveness of data processing, re-inspecting the preliminarily screened products, and determining the quality of the products. Image acquisition precision is analyzed and evaluated through a product detection image, image acquisition performance is comprehensively evaluated through periodic data, then the image acquisition performance is combined with a product reinspection result, visual detection efficiency is deeply evaluated, a visual regulation and control signal is generated, a visual acquisition mode is dynamically adjusted, and visual detection requirements in an environment abnormal change state are met. Therefore, the continuous stability of the visual detection efficiency is ensured, the product appearance preliminary screening model is optimized and upgraded by evaluating the preliminary screening detection efficiency of the product appearance preliminary screening model and generating a model optimization signal, the self-growth of the model is realized, and the preliminary screening detection accuracy of the model is improved.

Description

Technical Field

[0001] The present invention relates to the field of visual inspection technology, and in particular to an industrial visual inspection data processing method. Background Art

[0002] Industrial visual inspection technology simulates human visual functions and uses machine learning and image processing technologies to achieve precise control of product quality, providing a new intelligent solution for industrial manufacturing. Industrial visual inspection mainly relies on computer vision technology, obtains image information of target objects through cameras or other sensors, and uses image processing and recognition algorithms to analyze images, thereby achieving automated inspection of product quality.

[0003] However, the existing industrial vision acquisition mode is easily affected by changes in the detection environment. In the process of industrial vision detection, there is a defect of poor environmental adaptability. For example, infrared imaging detection is suitable for high temperature environments, but it is easily affected by temperature changes; binocular camera detection is fast, efficient and convenient, but it is easily restricted by light; line scanning camera detection is suitable for assembly lines, but it has strict requirements on the object motion control of the product;

[0004] Therefore, when the visual inspection environment changes abnormally, the traditional model cannot effectively adapt to the changing scenes, which will cause insufficient image acquisition accuracy and poor stability, resulting in incomplete data processing and inaccurate product quality assessment, thus affecting the visual inspection efficiency and industrial production inspection results;

[0005] In view of the above technical defects, a solution is now proposed. Summary of the invention

[0006] The purpose of the present invention is to solve the technical defects that the existing industrial visual acquisition mode is easily affected by changes in the detection environment and cannot effectively adapt to changing scenes, resulting in insufficient image acquisition accuracy and poor stability, leading to incomplete data processing and inaccurate product quality assessment, thereby affecting the visual inspection efficiency and industrial production inspection effect. The technical solution of the present invention uses product appearance data and constructs a product appearance preliminary screening model to comprehensively evaluate product quality and perform preliminary screening on the product to ensure the comprehensiveness of data processing, and by evaluating image acquisition accuracy and image acquisition performance, and then combining it with the product re-inspection results, deeply evaluates visual inspection efficiency and dynamically adjusts the visual acquisition mode to achieve visual inspection requirements under abnormal environmental changes, thereby ensuring the continuous stability of visual inspection efficiency, and then by evaluating the preliminary screening detection efficiency of the product appearance preliminary screening model, the self-growth of the model is achieved, and the accuracy of the preliminary screening detection of the model is improved.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] An industrial visual inspection data processing method comprises the following steps:

[0009] Step 1: The visual acquisition unit monitors and acquires product appearance data: the product inspection image of the production line is acquired through the visual acquisition device, and the image pixels and image frame rate of the product inspection image are acquired, and then the image acquisition cycle T is set, the actual product image A is acquired through the product inspection image at a fixed time, and the product requirement image B is acquired through the user requirement preset, and the product appearance data of the actual product image A and the product requirement image B are acquired through the image analysis tool;

[0010] Step 2: The data processing unit preliminarily analyzes the product appearance data: constructs a product appearance preliminary screening model, compares the product appearance data of the actual product image A and the product requirement image B, thereby comprehensively evaluating the product quality, and preliminarily screening the unqualified products, and then re-inspecting the unqualified products after the preliminary screening to obtain the qualified rate of the product re-inspection;

[0011] Step 3: The core analysis unit deeply analyzes and evaluates the visual inspection performance: by combining the image pixels and image frame rate of the product inspection image, the image acquisition accuracy is evaluated, and then a curve of image acquisition accuracy change is constructed and analyzed, so as to comprehensively evaluate the image acquisition performance, and combine the image acquisition performance with the qualified rate of product re-inspection, evaluate the visual inspection performance and generate corresponding visual control signals;

[0012] Step 4: the visual control unit receives the visual control signal, and then feeds it back to the visual acquisition unit to adjust the visual acquisition mode;

[0013] Step five: The model optimization unit evaluates the initial screening performance of the product appearance initial screening model, generates a model optimization signal and sends it to the data processing unit to optimize and upgrade the product appearance initial screening model.

[0014] Furthermore, the specific process of constructing the product appearance preliminary screening model is as follows:

[0015] Product appearance data includes color parameters and geometric parameters; color parameters include RGB color vectors and HSV color vectors; geometric parameters include displacement vectors of pixel pairs;

[0016] A1, compare the color parameters of the actual product image A and the required product image B;

[0017] A1-1, extract n0 pixels of the actual product image A and the required product image B respectively, and build a parameter analysis sub-model to analyze the color parameters;

[0018] A1-2, construct the RGB parameter matrix Crogb and substitute it into the parameter analysis sub-model: compare the RGB color vectors of the pixels, construct the RGB contrast matrix Cpc to obtain the RGB comprehensive contrast coefficient Xrgb;

[0019] A1-3, construct the HSV parameter matrix Dhsv and substitute it into the parameter analysis sub-model: compare the HSV color vectors of the pixels, construct the HSV comparison matrix Dpc to obtain the HSV comprehensive comparison coefficient Xhsv;

[0020] A2, compare the geometric parameters of the actual product image A and the required product image B;

[0021] A2-1, extract n1 pixel pairs of the product actual image A and the product requirement image B respectively, so as to construct the geometric parameter matrix of the actual image A and the product requirement image B;

[0022] A2-2, obtain the geometric contrast matrix Pdb through the geometric parameter matrices of the actual image A and the product requirement image B;

[0023] A2-3, by analyzing the geometric contrast matrix Pdb, obtain the geometric comprehensive contrast coefficient Xjh;

[0024] A3, by combining the RGB comprehensive contrast coefficient Xrgb, the HSV comprehensive contrast coefficient Xhsv and the geometric comprehensive contrast coefficient Xjh, the product quality assessment index ZA is obtained to determine the product quality of the actual product image A, thereby performing preliminary product screening.

[0025] Furthermore, the specific process of constructing the parameter analysis sub-model is as follows:

[0026] Input parameter matrix ψ1 and parameter matrix ψ2 to the parameter analysis sub-model;

[0027] Obtain the parameter comparison matrix ψpc by subtracting the parameter matrix ψ1 from the parameter matrix ψ2;

[0028] Then, the column vector of the parameter comparison matrix ψpc is averaged to obtain the deviation coefficient of the column vector;

[0029] Then, the comprehensive comparison coefficient of the parameter comparison matrix ψpc is obtained to evaluate the degree of parameter comparison deviation;

[0030] The comprehensive contrast coefficient of the output parameter contrast matrix ψpc.

[0031] Furthermore, the specific process of obtaining the RGB comprehensive contrast coefficient Xrgb is as follows:

[0032] The RGB color vector is labeled C: C =<R,G,B> , where R refers to the intensity of the red component, G refers to the intensity of the green component, and B refers to the intensity of the blue component;

[0033] The pixel point formed by any two pixels of the actual product image A is marked as i, and the RGB color vector of pixel point i is marked as Ci: Ci =<Ri,Gi,Bi> ;

[0034] The pixel point formed by any two pixels of the product demand image B is marked as j, and the RGB color vector of pixel point j is marked as Cj: Cj =<Rj,Gj,Bj> ;

[0035] Construct the RGB parameter matrix Crogb through the RGB color vectors of n0 pixels;

[0036] The RGB parameter matrix C rgb of the actual product image A is constructed in turn through the RGB color vectors of n0 pixels of the actual product image A and the required product image B. i And the RGB parameter matrix Crogb of the product demand image B j , and substitute it into the parameter analysis sub-model to construct the RGB contrast matrix Cpc;

[0037] The deviation coefficients of the red component intensity, green component intensity, and blue component intensity of the pixel points of the actual product image A are obtained by the RGB contrast matrix Cpc, and are marked as φr, φg, and φb respectively, and then the RGB comprehensive contrast coefficient Xrgb of the actual product image A is comprehensively obtained.

[0038] Furthermore, the specific process of obtaining the HSV comprehensive contrast coefficient Xhsv is:

[0039] The HSV color vector is marked as D: D =<H,S,V> , where H refers to hue, S refers to saturation, and V refers to brightness;

[0040] Mark the HSV color vector of pixel i as Di: Di ​​=<Hi,Si,Vi> ;

[0041] The HSV color vector of pixel j is marked as Dj: Dj =<Hj,Sj,Vj> ;

[0042] Construct the HSV parameter matrix Dhsv through the HSV color vectors of n0 pixels;

[0043] The HSV parameter matrix Dhsv of the actual product image A is constructed in turn through the HSV color vectors of n0 pixels of the actual product image A and the required product image B. i And the HSV parameter matrix Dhsv of product demand image B j, and substitute it into the parameter analysis sub-model to construct the HSV comparison matrix Dpc;

[0044] The deviation coefficients of the hue, saturation, and brightness of the pixel points of the actual product image A are obtained by the HSV contrast matrix Dpc, and are marked as φh, φs, and φv respectively, so as to comprehensively obtain the HSV comprehensive contrast coefficient Xhsv of the actual product image A.

[0045] Furthermore, the specific process of obtaining the geometric comprehensive contrast coefficient Xjh is as follows:

[0046] Mark any pixel pair of the product demand image B as J. Pixel pair J includes pixel De and pixel Df. The displacement vector of pixel pair J is Displacement Vector Including the length value Jl and direction angle Jr from pixel point De to pixel point Df, Construct the geometric parameter matrix P1 through the displacement vectors of n1 pixel pairs of the product demand image B;

[0047] Mark any pixel pair of the actual product image A as I. Pixel pair I includes pixel Da and pixel Db. The displacement vector of pixel pair I is Displacement Vector Including the length value Il and direction angle Ir from pixel Da to pixel Db, Construct the geometric parameter matrix P2 through the displacement vectors of n1 pixel pairs of the actual product image A;

[0048] By subtracting the geometric parameter matrix P1 from the geometric parameter matrix P2, the geometric contrast matrix Pdb is obtained, and the deviation coefficients of the displacement length and displacement angle of the actual image A of the product are obtained, and are marked as ψl and ψr respectively, and then the geometric comprehensive contrast coefficient Xjh of the actual image A of the product is comprehensively obtained.

[0049] Furthermore, the specific process of in-depth analysis and evaluation of visual inspection performance is as follows:

[0050] The image pixels and image frame rate of the product inspection image are marked as XSp and ZLp respectively, and the data measurement period Tp is set. The image acquisition accuracy coefficient Xtj is obtained by combining the image pixels XSp and the image frame rate ZLp;

[0051] Construct the change curve S0 between the image acquisition accuracy coefficient Xtj and the data measurement period Tp, and perform curve analysis: by extracting n2 points of the change curve S0, obtain the overall coefficient Zs and the fluctuation coefficient σs of the change curve S0, and then obtain the image acquisition performance evaluation coefficient XN;

[0052] Reconstruct the change curve between the qualified rate HG of product re-inspection and the image acquisition performance evaluation coefficient XN, and fit the change function F0 to generate the change function F0. By combining the change function F0 with the image acquisition performance evaluation coefficient XN, the visual inspection efficiency coefficient Xsj is obtained;

[0053] Set the evaluation interval of the visual detection efficiency coefficient Xsj, evaluate the visual detection efficiency level through interval comparison, and generate the corresponding visual control signal.

[0054] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0055] The present invention extracts product appearance data through product inspection images and constructs a product appearance preliminary screening model. Starting from the two angles of color and geometry, the product quality is comprehensively evaluated and the products are preliminarily screened to ensure the comprehensiveness of data processing. The preliminarily screened products are then re-inspected. The image acquisition accuracy is evaluated through product inspection image analysis. The image acquisition performance is comprehensively evaluated through periodic data, which is then combined with the product re-inspection results. The visual inspection efficiency is deeply evaluated and a visual control signal is generated. The visual acquisition mode is dynamically adjusted to meet the visual inspection requirements under abnormal environmental changes, thereby ensuring the continuous stability of the visual inspection efficiency. Furthermore, by evaluating the preliminary screening detection efficiency of the product appearance preliminary screening model, a model optimization signal is generated to optimize and upgrade the product appearance preliminary screening model, thereby achieving model self-growth and improving the model's preliminary screening detection accuracy.

[0056] That is, in order to solve the defects of incomplete data processing and various problems caused by it, the technical solution of the present invention makes data processing more comprehensive and integrated by performing various data processing; through the improved data processing method, a more accurate product appearance inspection effect is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of the process of the present invention is shown;

[0058] Figure 2 A module schematic diagram of the present invention is shown. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] Embodiment 1:

[0061] like Figure 1-Figure 2As shown, a method for processing industrial visual inspection data includes the following steps:

[0062] S1, the visual acquisition unit monitors and acquires product appearance data: the product inspection image of the production line is acquired through the visual acquisition device, and the image pixels and image frame rate of the product inspection image are acquired, and then the image acquisition cycle Ta is set to obtain the actual product image A through the product inspection image at a fixed time, and the product requirement image B is acquired according to the user requirement preset;

[0063] User demand refers to the user's demand for the appearance of the product. For example, if the product is a box in the shape of a cuboid, the user demand is the appearance parameters of the length, width, and height of the product. 3D modeling can be performed based on the parameters of the user demand, and then the product demand image B is generated;

[0064] Product appearance data includes color parameters and geometric parameters. The product appearance data of the actual product image A and the required product image B are obtained through image analysis tools. The color parameters include RGB color vectors and HSV color vectors; the geometric parameters include the displacement vectors of pixel pairs;

[0065] S2, the data processing unit preliminarily analyzes the product appearance data: constructs a product appearance preliminary screening model, compares the product appearance data of the actual product image A and the product demand image B from the perspectives of color and geometry, so as to comprehensively evaluate the product quality, preliminarily screen out the unqualified products, and re-inspect the unqualified products after the preliminary screening through the inspection room to obtain the qualified rate HG of the product re-inspection;

[0066] Among them, the environmental conditions of the inspection room are kept constant, so as to ensure that the image acquisition accuracy coefficient of the inspection room is in a high-precision and stable performance state, and the product re-inspection is carried out to obtain the qualified rate HG of the product re-inspection. The qualified rate HG of the product re-inspection refers to the proportion of the number of products that pass the re-inspection to the products that fail the initial screening;

[0067] When the product re-inspection result is a qualified rate HG = 0, that is, all the products in the initial screening are of unqualified quality, which means that the accuracy of the initial screening is 100%; when the qualified rate HG of the product re-inspection is higher, it means that the accuracy of the initial screening is lower and the effect is worse;

[0068] The specific process of building a product appearance preliminary screening model is as follows:

[0069] A1, compare the color parameters of the actual product image A and the required product image B;

[0070] A1-1, extract n0 pixels of the actual product image A and the required product image B respectively, and build a parameter analysis sub-model to analyze the color parameters;

[0071] The specific process of constructing the parameter analysis sub-model is as follows:

[0072] Input parameter matrix ψ1 and parameter matrix ψ2 to the parameter analysis sub-model;

[0073] Obtain the parameter comparison matrix ψpc by subtracting the parameter matrix ψ1 from the parameter matrix ψ2;

[0074] Then, the column vector of the parameter comparison matrix ψpc is averaged to obtain the deviation coefficient of the column vector;

[0075] Then, the comprehensive comparison coefficient of the parameter comparison matrix ψpc is obtained to evaluate the degree of parameter comparison deviation;

[0076] The comprehensive contrast coefficient of the output parameter contrast matrix ψpc;

[0077] A1-2, construct the RGB parameter matrix Crogb and substitute it into the parameter analysis sub-model: compare the RGB color vectors of the pixels, construct the RGB contrast matrix Cpc to obtain the RGB comprehensive contrast coefficient Xrgb. The specific process is as follows:

[0078] The RGB color vector is labeled C: C =<R,G,B> , where R refers to the intensity of the red component, G refers to the intensity of the green component, and B refers to the intensity of the blue component. R, G, and B are all in the range [0,255];

[0079] The pixel point formed by any two pixels of the actual product image A is marked as i, and the RGB color vector of pixel point i is marked as Ci: Ci =<Ri,Gi,Bi> ;

[0080] The pixel point formed by any two pixels of the product demand image B is marked as j, and the RGB color vector of pixel point j is marked as Cj: Cj =<Rj,Gj,Bj> ;

[0081] Where Ci=<Ri,Gi,Bi> , where Ri refers to the intensity of the red component of pixel i, Gi refers to the intensity of the green component of pixel i, and Bi refers to the intensity of the blue component of pixel i. Ri, Gi, and Bi are all in the interval [0,255];

[0082] Cj=<Rj,Gj,Bj> The meaning of the parameters is the same, so I won’t go into details.

[0083] Construct the RGB parameter matrix Crogb through the RGB color vector of n0 pixels:

[0084]

[0085] The RGB parameter matrix C rgb of the actual product image A i And the RGB parameter matrix Crogb of the product demand image B jSubstitute into the parameter analysis sub-model and output the RGB comprehensive contrast coefficient Xrgb of the actual product image A. The specific process is as follows:

[0086] The RGB parameter matrix C rgb of the actual product image A is constructed in turn through the RGB color vectors of n0 pixels of the actual product image A and the required product image B. i And the RGB parameter matrix Crogb of the product demand image B j , and then construct the RGB contrast matrix Cpc: Cpc = Crb i -Crgb j ;

[0087] Obtained from this:

[0088]

[0089] Right now,

[0090] By calculating the mean of the column vectors of the RGB contrast matrix Cpc, the deviation coefficients of the red component intensity, green component intensity, and blue component intensity of the pixel points of the actual product image A are obtained and marked as φr, φg, and φb respectively;

[0091] Among them, the deviation coefficient φr of the red component intensity is:

[0092] Deviation coefficient φg of green component intensity:

[0093] Deviation coefficient φb of the blue component intensity:

[0094] The RGB comprehensive contrast coefficient Xrgb of the actual product image A is obtained by comprehensively calculating the deviation coefficients of the red component intensity, green component intensity, and blue component intensity of the pixel points of the actual product image A:

[0095] Xrgb=α1*φr+α2*φg+α3*φb

[0096] Among them, α1, α2 and α3 are weight factor coefficients of the deviation coefficient φr of the red component intensity, the deviation coefficient φg of the green component intensity and the deviation coefficient φb of the blue component intensity, respectively, and α1, α2 and α3 are all greater than 0; the weight factor coefficients are preset and obtained after a large amount of data measurement, and when the deviation coefficient φr of the red component intensity, the deviation coefficient φg of the green component intensity and the deviation coefficient φb of the blue component intensity are higher, the RGB comprehensive contrast coefficient Xrgb of the actual product image A is higher, indicating that the degree of deviation of the RGB comprehensive contrast of the actual product image A is greater;

[0097] A1-3, construct the HSV parameter matrix Dhsv and substitute it into the parameter analysis sub-model: compare the HSV color vectors of the pixels, construct the HSV comparison matrix Dpc to obtain the HSV comprehensive comparison coefficient Xhsv, the specific process is:

[0098] The HSV color vector is marked as D: D =<H,S,V> , where H refers to hue, S refers to saturation, and V refers to brightness;

[0099] Mark the HSV color vector of pixel i as Di: Di ​​=<Hi,Si,Vi> , mark the HSV color vector of pixel j as Dj: Dj =<Hj,Sj,Vj> ;

[0100] Where Di=<Hi,Si,Vi> , where Hi refers to the hue of pixel i, Si refers to the saturation of pixel i, and Vi refers to the brightness of pixel i;

[0101] Dj=<Hj,Sj,Vj> The meaning of the parameters is the same as above and will not be repeated here;

[0102] Construct the HSV parameter matrix Dhsv through the HSV color vector of n0 pixels:

[0103]

[0104] The HSV parameter matrix Dhsv of the actual product image A i And the HSV parameter matrix Dhsv of product demand image B j Substitute into the parameter analysis sub-model and output the HSV comprehensive contrast coefficient Xhsv of the actual product image A. The specific process is as follows:

[0105] The HSV parameter matrix Dhsv of the actual product image A is constructed in turn through the HSV color vectors of n0 pixels of the actual product image A and the required product image B. i And the HSV parameter matrix Dhsv of product demand image B j , and then construct the HSV comparison matrix Dpc: Dpc = Dhsv i -Dhsv j ;

[0106] Obtained from this:

[0107]

[0108] Right now,

[0109] By calculating the mean of the column vectors of the HSV contrast matrix Dpc, the deviation coefficients of the hue, saturation, and brightness of the pixel points of the actual product image A are obtained and marked as φh, φs, and φv respectively;

[0110] Among them, the hue deviation coefficient φh:

[0111] Saturation deviation coefficient φs:

[0112] Brightness deviation coefficient φv:

[0113] The HSV comprehensive contrast coefficient Xhsv of the actual product image A is obtained by comprehensively obtaining the hue deviation coefficient φh, saturation deviation coefficient φs and brightness deviation coefficient φv of the pixel points of the actual product image A:

[0114] Xhsv=β1*φh+β2*φs+β3*φv

[0115] Among them, β1, β2 and β3 are weight factor coefficients of the hue deviation coefficient φh, the saturation deviation coefficient φs and the lightness deviation coefficient φv, respectively, and β1, β2 and β3 are all greater than 0; when the hue deviation coefficient φh, the saturation deviation coefficient φs and the lightness deviation coefficient φv are higher, the HSV comprehensive contrast coefficient Xhsv of the actual product image A is higher, indicating that the HSV comprehensive contrast deviation of the actual product image A is greater;

[0116] A2, compare the geometric parameters of the actual product image A and the required product image B;

[0117] A2-1, extract n1 pixel pairs of the product actual image A and the product requirement image B respectively, so as to construct the geometric parameter matrix of the actual image A and the product requirement image B;

[0118] A2-2, obtain the geometric contrast matrix Pdb through the geometric parameter matrices of the actual image A and the product requirement image B;

[0119] A2-3, by analyzing the geometric contrast matrix Pdb, obtain the geometric comprehensive contrast coefficient Xjh;

[0120] The specific process is:

[0121] Extract any two pixels to form a pixel pair, mark any pixel pair of the product demand image B as J; mark any pixel pair formed by any two pixels of the product actual image A as I;

[0122] The pixel pair J includes pixel De and pixel Df. The displacement vector of the pixel pair J is Displacement Vector Including the length value Jl and direction angle Jr from pixel point De to pixel point Df, that is,

[0123] In which, the baseline L is set in the product demand image B 1 , let the baseline L 1 The direction angle is 0°, then the displacement vector The direction angle Jr is the angle between the line connecting the pixel point De to the pixel point Df and the reference line L 1 The angle value between them;

[0124] The pixel pair I includes pixel Da and pixel Db. The displacement vector of the pixel pair I is Displacement Vector Including the length value Il and direction angle Ir from pixel point Da to pixel point Db, that is,

[0125] Among them, according to the baseline L of the product demand image B 1 coordinates, set the reference line L in the actual product image A 2 , let the baseline L 2 The direction angle is 0°, then the displacement vector The direction angle Ir is the angle between the line connecting the pixel point Da to the pixel point Db and the reference line L 2 The angle value between them;

[0126] It should be noted that the pixel pair J is arranged corresponding to the pixel pair I. For example, in the product demand image B, a certain edge line of the required product is marked. The edge line runs from any edge corner to its adjacent edge corner. By connecting the pixel points of the two edge corners, a displacement vector is formed. Then mark the displacement vector between the actual product and the required product from the actual product image A The corresponding edge lines are connected by connecting the two corresponding edge angles set in the actual image A of the product to form a displacement vector

[0127] Construct the geometric parameter matrix P1 through the displacement vectors of n1 pixel pairs of the product demand image B;

[0128] Construct the geometric parameter matrix P2 through the displacement vectors of n1 pixel pairs of the actual product image A;

[0129] in,

[0130] The geometric parameter matrix P1 is subtracted from the geometric parameter matrix P2 to obtain the geometric contrast matrix Pdb;

[0131]

[0132] Then, by finding the mean of the column vectors of the geometric contrast matrix Pdb, the deviation coefficients of the displacement length and displacement angle of the actual product image A are obtained and marked as ψl and ψr respectively;

[0133] Among them, the displacement length deviation coefficient ψl is:

[0134] Displacement angle deviation coefficient ψr:

[0135] The geometric comprehensive contrast coefficient Xjh of the actual product image A is obtained by using the displacement length deviation coefficient ψl and the displacement angle deviation coefficient ψr of the actual product image A:

[0136] Xjh=γ1*ψl+γ2*ψr;

[0137] Among them, γ1 and γ2 are weight factor coefficients of the displacement length deviation coefficient ψl and the displacement angle deviation coefficient ψr, respectively, and both γ1 and γ2 are greater than 0; when the displacement length deviation coefficient ψl and the displacement angle deviation coefficient ψr are higher, the geometric comprehensive contrast coefficient Xjh of the actual image A of the product is higher, indicating that the degree of geometric comprehensive contrast deviation of the actual image A of the product is greater;

[0138] A3, by combining the RGB comprehensive contrast coefficient Xrgb, the HSV comprehensive contrast coefficient Xhsv and the geometric comprehensive contrast coefficient Xjh, obtain the product quality assessment index ZA, determine the product quality of the actual product image A, and thus perform preliminary product screening;

[0139] A3-1, by combining the RGB comprehensive contrast coefficient Xrgb and the HSV comprehensive contrast coefficient Xhsv of the actual product image A, the color comprehensive contrast coefficient Xsc of the actual product image A is obtained, and combined with the geometric comprehensive contrast coefficient Xjh of the actual product image A, the product quality evaluation index ZA is obtained:

[0140]

[0141] Among them, μ1 and μ2 are weight indexes of the color comprehensive contrast coefficient Xsc and the geometric comprehensive contrast coefficient Xjh, respectively, and both μ1 and μ2 are greater than 0; when the color comprehensive contrast coefficient Xsc and the geometric comprehensive contrast coefficient Xjh are higher, the product quality assessment index ZA is lower, which means that when the color difference and geometric size difference of the tested product are greater, the product quality is worse;

[0142] The above formula can comprehensively evaluate product quality from two perspectives: color Xsc and geometry Xjh. The color is comprehensively evaluated from two perspectives: RGB and HSV. The "+1" is to avoid the situation where the X coefficients are all 0, resulting in the denominator of the formula being 0. The X coefficients are the comparison / deviation coefficients of the corresponding parameters. The higher the X parameter, the greater the deviation / difference between the actual product and the demand, indicating that the product quality detected in appearance is worse.

[0143] A3-2, set a threshold Gz of the product quality evaluation index ZA. When the product quality evaluation index ZA is lower than the threshold Gz, the product quality of the actual product image A is determined to be unqualified, thereby achieving a preliminary screening of unqualified products. The threshold Gz is an empirical value and is set according to application requirements and accuracy requirements;

[0144] S3, the core analysis unit deeply analyzes and evaluates the visual inspection performance: through the image pixels and image frame rate of the product inspection image, set the data calculation period Tp to regularly evaluate the image acquisition accuracy, and then construct the change curve between the image acquisition accuracy coefficient Xtj and the data calculation period Tp, comprehensively evaluate the image acquisition performance, and combine it with the qualified rate HG of the product re-inspection, evaluate the visual inspection performance and generate corresponding visual control signals;

[0145] S3-1, mark the image pixels and image frame rate of the product inspection image as XSp and ZLp respectively;

[0146] Set the data calculation period Tp, combine the image pixel XSp and the image frame rate ZLp, and obtain the image acquisition accuracy coefficient Xtj regularly according to the data calculation period Tp: Xtj=ω1*XSp+ω2*ZLp;

[0147] Among them, ω1 and ω2 are weight factor coefficients of image pixel XSp and image frame rate ZLp respectively, and ω1 and ω2 are both greater than 0; when the image pixel XSp and image frame rate ZLp are higher, the image acquisition accuracy coefficient Xtj is higher, and the image acquisition accuracy and quality of the product inspection image are high;

[0148] S3-2, with the image acquisition accuracy coefficient Xtj as the ordinate and the data calculation period Tp as the abscissa, and the abscissa represents the time sequence of the data calculation period, that is, the first data calculation period, the second data calculation period, ..., and so on, construct a change curve S0 between the image acquisition accuracy coefficient Xtj and the data calculation period Tp, and perform curve analysis;

[0149] S3-201, extract n2 points of the change curve S0, mark any point of the change curve S0 and its coordinates as q (Xq, Yq), calculate the average value through the ordinates of the n2 points of the change curve S0, and obtain the overall coefficient Zs of the change curve S0: When the overall coefficient Zs of the variation curve S0 is higher, it means that the overall level of the image acquisition accuracy coefficient Xtj is higher and the image acquisition quality is better;

[0150] S3-202, and then calculate the standard deviation through the ordinates of the n2 points of the change curve S0 to obtain the fluctuation coefficient σs of the change curve S0: When the fluctuation coefficient σs of the variation curve S0 is lower, it means that the fluctuation level of the image acquisition accuracy coefficient Xtj is lower and the image acquisition level is more stable;

[0151] S3-203, by combining the overall coefficient Zs and the fluctuation coefficient σs of the variation curve S0, an image acquisition performance evaluation coefficient XN is obtained: XN = λ1*Zs-λ2*σs;

[0152] Among them, λ1 and λ2 are weight factor coefficients of the overall coefficient Zs and the fluctuation coefficient σs, respectively, and both λ1 and λ2 are greater than 0; when the overall coefficient Zs of the variation curve S0 is higher and the fluctuation coefficient σs is lower, the image acquisition performance evaluation coefficient XN is higher, indicating that the image acquisition performance is better;

[0153] S3-3, construct a change curve between the qualified rate HG of product re-inspection and the image acquisition performance evaluation coefficient XN, and fit to generate a change function F0, where F0(XN)=HG, substitute the image acquisition performance evaluation coefficient XN into the change function F0, and then obtain the corresponding qualified rate HG of product re-inspection;

[0154] S3-4, by combining the variation function F0 with the image acquisition performance evaluation coefficient XN, obtain the visual detection efficiency coefficient Xsj:

[0155] When the image acquisition performance evaluation coefficient XN is higher and the product re-inspection pass rate HG is lower, the visual inspection efficiency coefficient Xsj is higher, and the visual inspection efficiency is better;

[0156] S3-5, setting the evaluation interval of the visual detection efficiency coefficient Xsj to [f1, f2], evaluating the visual detection efficiency level through interval comparison, and generating a corresponding visual control signal;

[0157] S3-501, when the visual inspection efficiency coefficient Xsj is lower than f1, a level I visual control signal is generated, and the visual inspection efficiency level is evaluated as level I, indicating that the visual inspection efficiency is poor;

[0158] S3-502, when the visual inspection efficiency coefficient Xsj is in the evaluation interval [f1, f2], a level II visual control signal is generated, and the visual inspection efficiency level is evaluated as level II, indicating that the visual inspection efficiency is average;

[0159] S3-503, when the visual inspection efficiency coefficient Xsj is higher than f2, a level III visual control signal is generated, and the visual inspection efficiency level is evaluated as level III, indicating that the visual inspection efficiency is good;

[0160] S4, the visual control unit receives the visual control signal, and then feeds it back to the visual acquisition unit to adjust the visual acquisition mode;

[0161] S4-1, when receiving the Level I visual control signal, by setting a variety of visual acquisition modes, select the visual acquisition mode with the best image acquisition performance according to the product inspection environment conditions, so as to meet the visual inspection needs under abnormal changes in light and temperature conditions, thereby ensuring the continuous stability of visual inspection performance;

[0162] S4-2, when receiving the Level II visual control signal, start multiple visual acquisition mode test plans, such as laser radar acquisition mode, infrared camera acquisition mode, line scan image acquisition mode, etc., evaluate the corresponding image acquisition performance under multiple visual acquisition modes, and select the visual acquisition mode with the best image acquisition performance for preparation, so that it can be directly applied when receiving the Level I visual control signal, to ensure the continuity and work efficiency of product inspection;

[0163] S4-3, when receiving the level III visual control signal, no processing is performed;

[0164] S5, the model optimization unit evaluates the initial screening detection efficiency of the product appearance initial screening model, generates a model optimization signal and sends it to the data processing unit to optimize and upgrade the product appearance initial screening model;

[0165] S5-1, by combining the qualified rate HG of product re-inspection with the image acquisition performance evaluation coefficient XN, the initial screening test efficiency coefficient Xcs is obtained:

[0166] When the qualified rate HG of product re-inspection and the image acquisition performance evaluation coefficient XN are higher, the primary screening test efficiency coefficient Xcs is lower, which means that when the image acquisition performance evaluation coefficient XN is high, the qualified rate HG of product re-inspection is high and the accuracy of primary screening is low, which means that the data processing effect of the product appearance primary screening model is worse and the primary screening test efficiency is poor;

[0167] S5-2, setting a threshold value f0 of the initial screening test efficiency coefficient Xcs. When the initial screening test efficiency coefficient Xcs is lower than the threshold value f0, a model optimization signal is generated to adjust the preset parameters of the product appearance initial screening model. The preset parameters include a weight factor coefficient and a weight index. By feeding back the re-inspection results of product quality to the product appearance initial screening model, the initial screening error results are corrected, thereby continuously training the model to achieve optimization and upgrading;

[0168] Among them, the preset parameters include weight factor coefficients α1, α2 and α3, β1, β2 and β3, γ1 and γ2, and weight indexes μ1 and μ2; the product quality assessment index of the initial screening is marked as ZA1, and then the product is re-inspected to obtain the re-inspection result of the product quality, and the re-inspected product quality assessment index is marked as ZA2, and the initial screening product quality assessment index ZA1 and the re-inspected product quality assessment index ZA2 are compared, so as to correct the error of the initial screening result. For example, when ZA2>ZA1, it means that the initial screening result is wrong and the value is too small. Therefore, a large amount of re-inspection data is required to adaptively increase or decrease the preset parameters, so that the initial screening product quality assessment index ZA1 increases and continuously approaches the re-inspected product quality assessment index ZA2, thereby improving the fitting accuracy of the mathematical model and realizing the optimization and upgrading of the model;

[0169] It should be noted that the present method is applied to an industrial visual inspection data processing system, which includes a visual acquisition unit, a data processing unit, a core analysis unit, a visual control unit and a model optimization unit, wherein the visual acquisition unit, the data processing unit, the core analysis unit, the visual control unit and the model optimization unit are communicatively connected.

[0170] To summarize, the present invention extracts product appearance data through product inspection images, and constructs a preliminary screening model for product appearance. Starting from the two perspectives of color and geometry, the product quality is comprehensively evaluated and the products are preliminarily screened to ensure the comprehensiveness of data processing, and then the preliminarily screened products are re-inspected. The image acquisition accuracy is evaluated through product inspection image analysis, and the image acquisition performance is comprehensively evaluated through periodic data, which is then combined with the product re-inspection results, and the visual inspection efficiency is deeply evaluated and a visual control signal is generated. The visual acquisition mode is dynamically adjusted to realize the visual inspection requirements under abnormal environmental changes, thereby ensuring the continuous stability of the visual inspection efficiency, and then by evaluating the preliminary screening detection efficiency of the product appearance preliminary screening model, a model optimization signal is generated to optimize and upgrade the product appearance preliminary screening model, realize model self-growth, and improve the model's preliminary screening detection accuracy.

[0171] The size of the interval and threshold is set to facilitate comparison. The size of the threshold depends on the amount of sample data and the number of bases set by technical personnel in this field for each group of sample data; as long as it does not affect the proportional relationship between the parameter and the quantized value.

[0172] The above formulas are all dimensionless and numerical calculations. The formula is a formula obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0173] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for processing industrial visual inspection data, characterized in that: The following steps are involved: Step 1: The visual acquisition unit monitors and acquires product appearance data: the product inspection image of the production line is acquired through the visual acquisition device, and the image pixels and image frame rate of the product inspection image are acquired, and then the image acquisition cycle T is set, the actual product image A is acquired through the product inspection image at a fixed time, and the product requirement image B is acquired through the user requirement preset, and the product appearance data of the actual product image A and the product requirement image B are acquired through the image analysis tool; Step 2: The data processing unit preliminarily analyzes the product appearance data: constructs a product appearance preliminary screening model, compares the product appearance data of the actual product image A and the product requirement image B, thereby comprehensively evaluating the product quality, and preliminarily screening the unqualified products, and then re-inspecting the unqualified products after the preliminary screening to obtain the qualified rate of the product re-inspection; Step 3: The core analysis unit deeply analyzes and evaluates the visual inspection performance: by combining the image pixels and image frame rate of the product inspection image, the image acquisition accuracy is evaluated, and then a curve of image acquisition accuracy change is constructed and analyzed, so as to comprehensively evaluate the image acquisition performance, and combine the image acquisition performance with the qualified rate of product re-inspection, evaluate the visual inspection performance and generate corresponding visual control signals; Step 4: the visual control unit receives the visual control signal, and then feeds it back to the visual acquisition unit to adjust the visual acquisition mode; Step five: The model optimization unit evaluates the initial screening performance of the product appearance initial screening model, generates a model optimization signal and sends it to the data processing unit to optimize and upgrade the product appearance initial screening model.

2. The industrial visual inspection data processing method according to claim 1, characterized in that: The specific process of building a product appearance preliminary screening model is as follows: Product appearance data includes color parameters and geometric parameters; color parameters include RGB color vectors and HSV color vectors; geometric parameters include displacement vectors of pixel pairs; A1, compare the color parameters of the actual product image A and the required product image B; A1-1, extract n0 pixels of the actual product image A and the required product image B respectively, and build a parameter analysis sub-model to analyze the color parameters; A1-2, construct the RGB parameter matrix Crogb and substitute it into the parameter analysis sub-model: compare the RGB color vectors of the pixels, construct the RGB contrast matrix Cpc to obtain the RGB comprehensive contrast coefficient Xrgb; A1-3, construct the HSV parameter matrix Dhsv and substitute it into the parameter analysis sub-model: compare the HSV color vectors of the pixels, construct the HSV comparison matrix Dpc to obtain the HSV comprehensive comparison coefficient Xhsv; A2, compare the geometric parameters of the actual product image A and the required product image B; A2-1, extract n1 pixel pairs of the product actual image A and the product requirement image B respectively, so as to construct the geometric parameter matrix of the actual image A and the product requirement image B; A2-2, obtain the geometric contrast matrix Pdb through the geometric parameter matrices of the actual image A and the product requirement image B; A2-3, by analyzing the geometric contrast matrix Pdb, obtain the geometric comprehensive contrast coefficient Xjh; A3, by combining the RGB comprehensive contrast coefficient Xrgb, the HSV comprehensive contrast coefficient Xhsv and the geometric comprehensive contrast coefficient Xjh, the product quality assessment index ZA is obtained to determine the product quality of the actual product image A, thereby performing preliminary product screening.

3. The industrial visual inspection data processing method according to claim 2, characterized in that: The specific process of constructing the parameter analysis sub-model is as follows: Input parameter matrix ψ1 and parameter matrix ψ2 to the parameter analysis sub-model; Obtain the parameter comparison matrix ψpc by subtracting the parameter matrix ψ1 from the parameter matrix ψ2; Then, the column vector of the parameter comparison matrix ψpc is averaged to obtain the deviation coefficient of the column vector; Then, the comprehensive comparison coefficient of the parameter comparison matrix ψpc is obtained to evaluate the degree of parameter comparison deviation; The comprehensive contrast coefficient of the output parameter contrast matrix ψpc.

4. The industrial visual inspection data processing method according to claim 3 is characterized in that: The specific process of obtaining the RGB comprehensive contrast coefficient Xrgb is: The RGB color vector is labeled C: C =<R,G,B> , where R refers to the intensity of the red component, G refers to the intensity of the green component, and B refers to the intensity of the blue component; Mark any pixel of the actual product image A as i, and mark the RGB color vector of pixel i as Ci: Ci =<Ri,Gi,Bi> ; Mark any pixel of the product demand image B as j, and mark the RGB color vector of pixel j as Cj: Cj =<Rj,Gj,Bj> ; Construct the RGB parameter matrix Crogb through the RGB color vectors of n0 pixels; The RGB parameter matrix C rgb of the actual product image A is constructed in turn through the RGB color vectors of n0 pixels of the actual product image A and the required product image B. i And the RGB parameter matrix Crogb of the product demand image B j , and substitute it into the parameter analysis sub-model to construct the RGB contrast matrix Cpc; The deviation coefficients of the red component intensity, green component intensity, and blue component intensity of the pixel points of the actual product image A are obtained by the RGB contrast matrix Cpc, and are marked as φr, φg, and φb respectively, and then the RGB comprehensive contrast coefficient Xrgb of the actual product image A is comprehensively obtained.

5. The industrial visual inspection data processing method according to claim 4, characterized in that: The specific process of obtaining the HSV comprehensive contrast coefficient Xhsv is: The HSV color vector is marked as D: D =<H,S,V> , where H refers to hue, S refers to saturation, and V refers to brightness; Mark the HSV color vector of pixel i as Di: Di ​​=<Hi,Si,Vi> ; The HSV color vector of pixel j is marked as Dj: Dj =<Hj,Sj,Vj> ; Construct the HSV parameter matrix Dhsv through the HSV color vectors of n0 pixels; The HSV parameter matrix Dhsv of the actual product image A is constructed in turn through the HSV color vectors of n0 pixels of the actual product image A and the required product image B. i And the HSV parameter matrix Dhsv of product demand image B j , and substitute it into the parameter analysis sub-model to construct the HSV comparison matrix Dpc; The deviation coefficients of the hue, saturation, and brightness of the pixel points of the actual product image A are obtained by the HSV contrast matrix Dpc, and are marked as φh, φs, and φv respectively, so as to comprehensively obtain the HSV comprehensive contrast coefficient Xhsv of the actual product image A.

6. The industrial visual inspection data processing method according to claim 5, characterized in that: The specific process of obtaining the geometric comprehensive contrast coefficient Xjh is: The pixel pair formed by any two pixels of the product demand image B is marked as J. The pixel pair J includes pixel De and pixel Df. The displacement vector of the pixel pair J is Displacement Vector Including the length value Jl and direction angle Jr from pixel point De to pixel point Df, Construct the geometric parameter matrix P1 through the displacement vectors of n1 pixel pairs of the product demand image B; The pixel pair formed by any two pixels of the actual product image A is marked as I. The pixel pair I includes pixel Da and pixel Db. The displacement vector of the pixel pair I is Displacement Vector Including the length value Il and direction angle Ir from pixel Da to pixel Db, Construct the geometric parameter matrix P2 through the displacement vectors of n1 pixel pairs of the actual product image A; By subtracting the geometric parameter matrix P1 from the geometric parameter matrix P2, the geometric contrast matrix Pdb is obtained, and the deviation coefficients of the displacement length and displacement angle of the actual image A of the product are obtained, and marked as ψl and ψr respectively, and then the geometric comprehensive contrast coefficient Xjh of the actual image A of the product is comprehensively obtained.

7. The industrial visual inspection data processing method according to claim 6, characterized in that: The specific process of in-depth analysis and evaluation of visual inspection performance is as follows: The image pixels and image frame rate of the product inspection image are marked as XSp and ZLp respectively, and the data measurement period Tp is set. By combining the image pixels XSp and the image frame rate ZLp, the image acquisition accuracy coefficient Xtj is obtained regularly; Construct the change curve S0 between the image acquisition accuracy coefficient Xtj and the data measurement period Tp, and perform curve analysis: by extracting n2 points of the change curve S0, obtain the overall coefficient Zs and the fluctuation coefficient σs of the change curve S0, and then obtain the image acquisition performance evaluation coefficient XN; Reconstruct the change curve between the qualified rate HG of product re-inspection and the image acquisition performance evaluation coefficient XN, and fit the change function F0 to generate the change function F0. By combining the change function F0 with the image acquisition performance evaluation coefficient XN, the visual inspection efficiency coefficient Xsj is obtained; Set the evaluation interval of the visual detection efficiency coefficient Xsj, evaluate the visual detection efficiency level through interval comparison, and generate the corresponding visual control signal.

Images