Image processing method and device suitable for mold monitor

Abstract

The invention discloses an image processing method and device suitable for a mold monitor, the method comprising: step S1, collecting a template image of an empty mold cavity and a contrast image of a full mold cavity; step S2, image preprocessing; step S3, extracting M ROI areas of the template image and simultaneously extracting M ROI areas corresponding to the contrast image, adopting an adaptive strategy to analyze each ROI area of ​​the template image, obtaining the number of optimal pixel intervals of each ROI area and the weighting coefficient corresponding to each pixel interval, and obtaining a difference standard threshold of the corresponding ROI area; step S4, acquiring a source image and performing preprocessing; step S5, analyzing the pixel distribution interval of each ROI of the source image, and counting the pixel distribution of each ROI; step S6, for each source image ROI corresponding to the template image ROI, judging whether the current ROI area is a defective area according to the difference value of a single ROI of the source image and the difference standard threshold.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image processing method and device suitable for a mold monitor. Background Art

[0002] In the plastic manufacturing industry and metal die-casting industry, whether the injection molding machine can operate stably and efficiently directly determines the production output value and product quality. Due to the complex workshop conditions, although the injection molding machine adopts mold protection measures such as low-voltage protection and ejector alarm, it is still impossible to avoid the phenomenon of mold compression, which directly leads to the need for mold repair, which not only brings huge additional expenses such as repair costs, but also brings losses in working hours and output, which may lead to untimely delivery and loss of customers. Therefore, in order to avoid the phenomenon of mold compression during the production process of injection molding machines, many manufacturers have purchased mold monitor systems, which collect pictures through the camera installed on the injection molding machine, and use the advanced image processing algorithms in the monitor to enhance, sharpen, and contrast the pictures, detect whether the product is normally formed or taken out normally, and give comparison results to replace the human eye to monitor the mold and products in real time, thereby effectively improving the company's intelligent production and automated production level, reducing labor costs, and improving the company's competitiveness.

[0003] Existing mold monitor algorithms can be simply divided into two categories: traditional algorithms and deep learning algorithms. Among them, although the deep learning algorithm has extremely high accuracy, its early algorithm training stage requires a large amount of valid data. The algorithm model is pre-trained in a high-performance computer or server. Its algorithm deployment has the disadvantages of high price, large data demand, weak portability and weak interpretability. In actual production deployment, it exists more as a trial and auxiliary. There are two main types of traditional algorithms in the field of mold monitoring:

[0004] 1) Image similarity matching based on image structural features: Collect multiple template images, and compare the structural features of the images with the current image in turn. If the structural features of one of the template images are basically consistent with the structural features of the current image (consistency is determined by setting the structural similarity threshold), the current image is judged to be OK, otherwise it is judged to be NG. The disadvantages of this method are: it uses more parameters, which is not conducive to on-site debugging and maintenance by non-R&D personnel; it uses more image preprocessing methods (overexposure, underexposure, contour, histogram stretching, corrosion, expansion, etc.), and the single running time of the image algorithm is too long, making it difficult to achieve real-time performance.

[0005] 2) Image similarity matching based on a single pixel: Collect multiple template images and compare all the pixels of the image with the current image one by one. If the pixel value of any pixel in one of the template images is basically consistent with the pixel value of the corresponding pixel in the current image (consistency is determined by setting upper and lower floating thresholds), the current image is judged as OK, otherwise it is judged as NG. The disadvantage of this method is that it is extremely sensitive to ambient light and is prone to false alarms. Summary of the invention

[0006] In order to overcome the deficiencies of the above-mentioned prior art, the purpose of the present invention is to provide an image processing method and device suitable for a mold monitor, so as to solve the influence of ambient lighting on the collected pictures, and realize an adaptive, high-performance, real-time, low false alarm and missed detection rate mold monitor algorithm.

[0007] To achieve the above object, the present invention provides an image processing method suitable for a mold monitor, comprising the following steps:

[0008] Step S1, using an infrared camera to collect a template image of an empty mold cavity and a comparison image of a full mold cavity;

[0009] Step S2, preprocessing the acquired template image and comparison image;

[0010] Step S3, extracting M ROI regions of the template image, and extracting M ROI regions corresponding to the comparison image at the same time, using an adaptive strategy to analyze each ROI region of the template image, performing automatic optimization calculation of relevant parameters of the pixel set, determining the number of optimal pixel intervals of each ROI region and the weighting coefficient corresponding to each pixel interval, and obtaining the difference value of the ROI region corresponding to the template image and the comparison image as the difference standard threshold of the corresponding ROI region;

[0011] Step S4, using an infrared camera to obtain a source image, and preprocessing the obtained source image;

[0012] Step S5, analyzing the pixel distribution interval of each ROI of the source image, and counting the pixel distribution of each ROI;

[0013] Step S6, for each ROI of the source image corresponding to the ROI of the template image, calculate the interval distance of the same pixel interval, and obtain the difference value of the single ROI of the source image, and determine whether the current ROI area is a defective area based on the difference value of the single ROI of the source image and the corresponding difference standard threshold obtained in step S3, and then determine whether the source image has defects.

[0014] Preferably, when acquiring the template image, multiple template images are stored as needed, and fitting calculations are performed on all template images based on the least squares method, and then the fitting results are compared with all template images one by one, and the template image with the smallest difference is selected as the final template image.

[0015] Preferably, step S3 further comprises:

[0016] Step S300, extracting M ROI regions of the template image;

[0017] Step S301, merging continuous pixel intervals in each ROI region image of the template image, preliminarily determining the distribution of multiple pixel distribution intervals in each ROI region, and determining the number of pixels in each pixel distribution interval (Image_template[m][i][n]);

[0018] Step S302, for each pixel interval distribution of each ROI region of the template image, calculate the weighted value of each pixel distribution interval in the ROI region of the template image according to the pixel distribution interval of the corresponding ROI region in the comparison image;

[0019] Step S303, using the obtained weighted values ​​of the pixel intervals of the m-th ROI region, obtain the weighted square difference (Se[m][i]) of the m-th ROI region when the number of intervals of the m-th ROI region is N[i];

[0020] Step S304, by comparing the maximum values ​​among the i Se[m][i], the optimal number of pixel intervals (N[m]) and the corresponding weighting coefficient group (Coeff[m][n]) of the ROI area are obtained;

[0021] Step S305, based on the obtained optimal number of pixel intervals (N[m]) and the corresponding weighting coefficient group (Coeff[m][n]) of each ROI area, for the comparison image ROI corresponding to each ROI of the template image, the interval distance of the same pixel interval is calculated, and the weighted pixel interval distances of the same ROI area are summed according to the weighting coefficient to obtain the difference value of a single ROI of the comparison image as the difference standard threshold of the ROI area.

[0022] Preferably, in step S301, if the ROI area is a rectangular frame, when counting the ROI pixel distribution, all pixels in the rectangular frame are traversed, and then the pixel distribution of each pixel distribution interval is counted according to the number of pixel distribution intervals distributed in the current pixel interval; if the ROI area is a non-rectangular frame, an external rectangular frame of the shape is first drawn, and all pixels in the external rectangular frame are traversed, and only the pixel distribution of the ROI area is counted, and then the pixel distribution of each pixel distribution interval is counted according to the number of pixel distribution intervals distributed in the current pixel interval.

[0023] Preferably, in step S305, the calculation formula of the interval distance of the same pixel interval is as follows:

[0024]

[0025] Wherein, Distance[m,n] represents the distance between the mth ROI of the contrast image and the mth ROI of the template image in the nth pixel interval, Image_template[m][n] represents the number of pixels in the nth pixel interval of the mth ROI region in the template image, and Image_contrast[m][n] represents the number of pixels in the nth pixel interval of the mth ROI region in the contrast image.

[0026] Preferably, the calculation formula of the difference standard threshold of a single ROI of the contrast image is:

[0027]

[0028] Among them, Value_Threshould [m] Represents the difference value of the mth ROI area in the comparison image and the template image, which is used as the difference standard threshold. Coeff[m][n] represents the weighting coefficient of the nth pixel interval in the mth ROI area of ​​the template image. N[m] represents the number of pixel intervals in the mth ROI of the template image.

[0029] Preferably, step S6 further comprises:

[0030] Step S600, for each source image ROI corresponding to the template image ROI, calculating the interval distance of the same pixel intervals;

[0031] Step S601, according to the weighting coefficient of each pixel interval in the ROI area, summing the weighted pixel interval distances of the ROI area to obtain the difference value of a single ROI of the source image;

[0032] In step S602, the difference values ​​of the obtained ROI regions of the source image are compared with the corresponding difference standard thresholds obtained in step S3. If they are greater than, the current ROI region is determined to be a defective region and the source image has defects. If the difference values ​​of all ROIs of the source image are less than the corresponding difference standard thresholds, it is determined that the source image does not have defects.

[0033] Preferably, in step S602, if the number of ROIs is 1, the ROI of the template image and the ROI of the source image are directly compared; if the number of ROIs is greater than or equal to 1, the ROI regions with the same serial number on the template image and the source image need to be compared.

[0034] Preferably, in step S6, the maximum ROI difference value in the current source image is compared with the difference standard threshold. If it is greater than 1 / 3 of the difference standard threshold, the current source image is automatically saved as a new template image while determining that the image has no defects.

[0035] To achieve the above object, the present invention further provides an image processing device suitable for a mold monitor, comprising:

[0036] An image acquisition unit, used to acquire a template image of an empty mold cavity, a comparison image of a full mold cavity, and a source image to be detected using an infrared camera;

[0037] An image preprocessing unit, used for preprocessing the acquired template image, comparison image, and source image;

[0038] An adaptive analysis unit is used to extract M ROI regions of the template image and M ROI regions corresponding to the comparison image, and adopt an adaptive strategy to analyze each ROI region of the template image, perform automatic optimization calculation of relevant parameters of the pixel set, determine the number of optimal pixel intervals of each ROI region and the weighting coefficient corresponding to each pixel interval, and obtain the difference value of the ROI region corresponding to the template image and the comparison image as the difference standard threshold of the corresponding ROI region;

[0039] The defect detection unit is used to analyze the pixel distribution interval of each ROI of the source image, count the pixel distribution of each ROI, calculate the interval distance of the same pixel interval of the ROI of the source image corresponding to each template image ROI, and obtain the difference value of a single ROI of the source image. According to the difference value of a single ROI of the source image and the corresponding difference standard threshold obtained by the adaptive analysis unit, it is determined whether the current ROI area is a defective area, and then whether the source image has defects.

[0040] Compared with the prior art, the present invention has the following advantages:

[0041] 1. Without setting any parameters, an adaptive, high-performance, real-time, low false alarm and missed detection image processing method suitable for mold monitors can be realized

[0042] 2. Solved the impact of ambient lighting on collected images. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flowchart of the steps of an image processing method applicable to a mold monitor of the present invention;

[0044] Figure 2 A system architecture diagram of an image processing device suitable for a mold monitor according to the present invention;

[0045] Figure 3 is an overall flow chart of an image processing method applicable to a mold monitor in an embodiment of the present invention;

[0046] Figure 4 A flowchart of an adaptive analysis strategy in a specific embodiment of the present invention;

[0047] Figure 5 Detailed flowchart of the image processing method of the mold monitor in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following describes the implementation of the present invention through specific examples and in conjunction with the accompanying drawings. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific examples, and the details in this specification can also be modified and changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0049] Figure 1 FIG. 1 is a flow chart of the steps of an image processing method suitable for a mold monitor according to the present invention. Figure 1 As shown, the present invention provides an image processing method suitable for a mold monitor, comprising the following steps:

[0050] Step S1, using an infrared camera to capture a template image of an empty mold cavity and a comparison image of a full mold cavity.

[0051] Specifically, step S1 further includes:

[0052] Step S100, adjusting the position of the infrared camera, the focal length of the lens, the camera exposure value and the gain value, and adjusting the position of the infrared light.

[0053] In a specific embodiment of the present invention, an infrared camera can be obtained by installing an infrared filter on the camera lens, and then adjusting the position of the camera and the lens focal length, and adjusting the position of the infrared lamp to obtain a clear image, and at the same time, adjusting the exposure value and gain value of the camera to avoid overexposure and underexposure of the collected image.

[0054] Step S101, using the infrared camera to collect a template image of the empty mold cavity, recorded as Image_template.

[0055] Preferably, when acquiring a template image, multiple template images can be stored as needed, and all template images can be fitted based on the least squares method, and then the fitting results are compared with all template images one by one, and the template image with the smallest difference is selected as the final template image.

[0056] Step S102, obtaining a contrast image of the full mold cavity through a camera, denoted as image_contrast.

[0057] Step S2, preprocessing the acquired template image and comparison image.

[0058] In a specific embodiment of the present invention, the collected template image and contrast image are preprocessed using the histogram equalization in the image enhancement technology. Specifically, the image is grayscale mapped so that the converted image has approximately the same number of pixels at each grayscale level, and the image with a small grayscale dynamic range is transformed into an image with a higher contrast, a large image grayscale dynamic range, and clearer information. In this way, the preprocessing of the collected image in the form of histogram equalization basically eliminates the interference of ambient light (illumination light, natural light scattering), thereby ensuring the stability of the image imaging quality.

[0059] Step S3, extracting M ROI (Region of Interest) areas of the template image and M ROI areas corresponding to the comparison image, using an adaptive strategy to analyze each ROI area of ​​the template image, automatically optimizing the relevant parameters of the pixel set, determining the number of optimal pixel intervals of each ROI area and the weighting coefficient corresponding to each pixel interval, and obtaining the difference value of the ROI area corresponding to the template image and the comparison image. In the present invention, the size, shape and number of the ROI areas of the template image and the comparison image are completely consistent, so the ROI of the comparison image can be determined by extracting the ROI of the template image.

[0060] Specifically, step S3 further includes:

[0061] Step S300: extracting M ROI regions of the template image.

[0062] In a specific embodiment of the present invention, ROIs of any shape and any number may be drawn as required, but the present invention is not limited thereto.

[0063] Step S301 , merge the continuous pixel intervals in each ROI region image of the template image, preliminarily determine the distribution of multiple pixel intervals in each ROI region, and determine the number of pixels in each pixel distribution interval Image_template[m][i][n].

[0064] For a binary image with grayscale distribution between 0 and 255, the continuous pixel intervals in the image in the mth (m≤M) ROI are merged, and the number of merged pixel intervals is recorded as N. In a specific embodiment of the present invention, an average division strategy is adopted (the average division strategy refers to the average region division of the 256 values), N[m][i]∈{8,16,32,64,128}, and it is preliminarily determined that there are 5 pixel interval distributions in the mth ROI area. For example, N[3][1] indicates that for the first pixel interval distribution of the third ROI, the number of its pixel distribution intervals is 8, and these 8 pixel distribution intervals are:

[0065] [0,31],[32,63],[64,95],[96,127],[128,159],[160,191],[192,223],[224,255].

[0066] Each ROI region has an optimal number of pixel distribution intervals, which can be regarded as the attribute of the ROI. The present invention needs to select the optimal number of pixel distribution intervals from the five pixel interval distributions. Of course, the pixel interval distribution is not limited to five types, and the present invention is only used as an example.

[0067] For each pixel interval distribution, count the number of pixels in each pixel distribution interval of each pixel interval distribution Image_template[m][i][n]. If the ROI area is a rectangular frame, then when counting the ROI pixel distribution, traverse all the pixels in the rectangular frame, and then count the pixel distribution of each pixel distribution interval according to the number of pixel distribution intervals of the current pixel interval distribution, that is, the number of pixels Image_template[m][i][n]. If the ROI area is a non-rectangular frame, it is necessary to first draw an outer rectangular frame of the shape, and traverse all the pixels in the outer rectangular frame, only count the pixel distribution of the ROI area, and then count the pixel distribution of each pixel distribution interval according to the number of pixel distribution intervals of the current pixel interval distribution. That is to say, when counting the pixel distribution of ROI, the points in the rectangular frame of ROI area can be directly counted. However, if the ROI area is not a rectangular frame, it is necessary to find the circumscribed rectangle first, and then judge whether each point in the rectangle is in the rectangle. If it is in, it will be counted, otherwise it will not be counted. After the statistics, the pixel distribution interval is divided, and the number of pixels in each pixel distribution interval is calculated Image_template[m][i][n].

[0068] Step S302 : for each pixel interval distribution of each ROI region of the template image, a weighted value of each pixel distribution interval in the ROI region of the template image is calculated according to the pixel distribution interval of the corresponding ROI region in the comparison image.

[0069] It should be noted here that the determination of the pixel distribution interval of the corresponding ROI area in the comparison image is similar to that of the template image, and will not be described in detail here.

[0070] Specifically, for N[m][i], select the i-th value and calculate the weighted value Coeff[m,i,n] of its n-th (n≤N) pixel interval. The calculation formula is:

[0071]

[0072] Wherein, Coeff[m,i,n] represents the weighted value of the nth pixel interval in the mth ROI region when N[i] is selected as the number of pixel intervals. For example, in this embodiment, Coeff[2][3][4] represents the weighted coefficient of the fourth pixel interval (i.e., [24,31]) for the second ROI when the number of pixel intervals is 32; Image_contrast[m][i][n] represents the number of pixels in the nth pixel interval in the mth ROI region of the contrast image when N[i] is selected as the number of pixel intervals; Image_template[m][i][n] represents the number of pixels in the nth pixel interval in the mth ROI region of the template image when N[i] is selected as the number of pixel intervals.

[0073] Step S303, using the obtained weighted values ​​of each pixel interval of the mth ROI region, obtain the weighted square difference Se[m][i] of the mth ROI when the number of intervals of the mth ROI region is N[i]. In a specific embodiment of the present invention, the calculation formula of the weighted square difference is as follows:

[0074]

[0075] Step S304, by comparing the maximum values ​​among the i Se[m][i], the optimal number of pixel intervals N[m] and the corresponding weighting coefficient group Coeff[m][n] of the ROI area are obtained.

[0076] Step S305, based on the obtained optimal pixel interval number N[m] of each ROI area and the corresponding weighting coefficient group Coeff[m][n], for the comparison image ROI corresponding to each ROI of the template image, the interval distance of the same pixel interval is calculated, and the weighted pixel interval distance of the same ROI area is summed according to the weighting coefficient to obtain the difference value of a single ROI of the comparison image as the difference standard threshold of the ROI area.

[0077] In a specific embodiment of the present invention, for each template image ROI corresponding to the comparison image ROI, the interval distance of the same pixel interval is calculated, and the calculation formula is:

[0078]

[0079] Wherein, Distance[m,n] represents the distance between the mth ROI of the contrast image and the mth ROI of the template image in the nth pixel interval, Image_template[m][n] represents the number of pixels in the nth pixel interval of the mth ROI region in the template image, and Image_contrast[m][n] represents the number of pixels in the nth pixel interval of the mth ROI region in the contrast image.

[0080] According to the weighting coefficient, the weighted pixel interval distances in the same ROI area are summed to obtain the difference value of a single ROI in the contrast image. The calculation formula is:

[0081]

[0082] Among them, Value_Threshould [m] It represents the difference between the mth ROI region in the comparison image and the template image, which is used as the difference standard threshold. Coeff[m][n] represents the weighting coefficient of the nth pixel interval in the mth ROI region of the template image, and N[m] represents the number of pixel intervals in the mth ROI.

[0083] Step S4, using an infrared camera to acquire a source image, and preprocessing the acquired source image.

[0084] In a specific embodiment of the present invention, the acquisition and preprocessing of the source image are the same as the processing of the template image and the comparison image described above, and will not be described in detail herein.

[0085] Step S5, analyzing the pixel distribution interval of each ROI of the source image, and counting the pixel distribution of each ROI.

[0086] In a specific embodiment of the present invention, for the source image, the pixel distribution interval of each ROI of the source image is determined according to the optimal number of pixel intervals N[m] of each ROI region determined by the template image in step S3, and then the pixel distribution of each pixel distribution interval is counted.

[0087] Step S6, for each ROI of the source image corresponding to the ROI of the template image, calculate the interval distance of the same pixel interval, and obtain the difference value of the single ROI of the source image, and determine whether the current ROI area is a defective area based on the difference value of the single ROI of the source image and the corresponding difference standard threshold obtained in step S3, and then determine whether the source image has defects.

[0088] Specifically, step S6 further includes:

[0089] Step S600, for each source image ROI corresponding to the template image ROI, calculate the interval distance of the same pixel interval. The calculation formula is as follows:

[0090]

[0091] Step S601, according to the weighting coefficient of each pixel interval in the ROI area, the weighted pixel interval distance of the ROI area is summed to obtain the difference value of a single ROI of the source image, and the calculation formula is as follows:

[0092]

[0093] Where Distance[m,n] represents the distance between the mth ROI of the source image and the mth ROI of the kth template image in the nth pixel interval, Value[m] represents the difference between the source image and the mth ROI in the template image, Coeff[m][n] represents the weighting coefficient of the nth pixel interval in the mth ROI, Image_src[m][n] represents the number of pixels in the nth pixel interval of the mth ROI in the source image, Image_template[m][n] represents the number of pixels in the nth pixel interval of the mth ROI in the template image, and N[m] represents the number of pixel intervals in the mth ROI.

[0094] Step S602: Compare the difference values ​​of each ROI region of the source image obtained with the corresponding difference standard threshold Value_Threshould obtained in step S3. [m] Compare them, if it is greater than, then determine that the current ROI area is a defective area, and the source image has defects.

[0095] If the difference values ​​of all ROIs of the source image are less than the corresponding difference standard threshold, it is determined that the source image has no defects.

[0096] Specifically, if the number of ROIs is 1, the ROI of the template image and the ROI of the source image can be directly compared as above; if the number of ROIs is greater than or equal to 1, the ROI regions with the same serial number on the template image and the source image need to be compared as above.

[0097] Preferably, in order to further reduce the impact of strong fluctuations in ambient light or infrared light on image acquisition, the maximum ROI difference value in the current source image can be compared with the difference standard threshold. If it is greater than 1 / 3 of the difference standard threshold Value_Threshould, the current source image is automatically saved as a new template image while judging the image to be OK.

[0098] Figure 2FIG. 1 is a system architecture diagram of an image processing device suitable for a mold monitor according to the present invention. Figure 2 As shown, the present invention is an image processing device suitable for a mold monitor, comprising:

[0099] The image acquisition unit 201 is used to acquire the template image of the empty mold cavity, the comparison image of the full mold cavity and the source image to be detected by using an infrared camera.

[0100] Before image acquisition, it is necessary to adjust the infrared camera position and lens focal length, camera exposure value and gain value, and adjust the position of the infrared light.

[0101] In a specific embodiment of the present invention, an infrared camera can be obtained by installing an infrared filter on the camera lens, and then adjusting the position of the camera and the focal length of the lens, and adjusting the position of the infrared lamp to obtain a clear image. At the same time, the exposure value and gain value of the camera are adjusted to avoid overexposure and underexposure in the collected pictures.

[0102] Specifically, the image acquisition unit 201 further includes:

[0103] The template image acquisition module uses the infrared camera to acquire the template image of the empty mold cavity, which is recorded as Image_template.

[0104] Preferably, when acquiring a template image, multiple template images can be stored as needed, and all template images can be fitted based on the least squares method, and then the fitting results are compared with all template images one by one, and the template image with the smallest difference is selected as the final template image.

[0105] The contrast image acquisition module is used to obtain the contrast image of the full mold cavity through the infrared camera, which is recorded as image_contrast.

[0106] The source image acquisition module is used to obtain the source image to be detected through the infrared camera, which is recorded as Image_src.

[0107] The image preprocessing unit 202 is used to preprocess the acquired template image, comparison image, and source image.

[0108] In a specific embodiment of the present invention, the collected template image and contrast image are preprocessed using the histogram equalization in the image enhancement technology. Specifically, the image is grayscale mapped so that the converted image has approximately the same number of pixels at each grayscale level, and the image with a small grayscale dynamic range is transformed into an image with a higher contrast, a large image grayscale dynamic range, and clearer information. In this way, the preprocessing of the collected image in the form of histogram equalization basically eliminates the interference of ambient light (illumination light, natural light scattering), thereby ensuring the stability of the image imaging quality.

[0109] The adaptive analysis unit 203 is used to extract M ROI (Region of Interest) areas of the template image and simultaneously extract M ROI areas corresponding to the comparison image, adopt an adaptive strategy to analyze each ROI area of ​​the template image, perform automatic optimization calculation of relevant parameters of the pixel set, determine the number of optimal pixel intervals of each ROI area and the weighting coefficient corresponding to each pixel interval, and obtain the difference value of the ROI area corresponding to the template image and the comparison image.

[0110] Specifically, the adaptive analysis unit 203 further includes:

[0111] The ROI region extraction module is used to extract M ROI regions of the template image.

[0112] In a specific embodiment of the present invention, ROIs of any shape and any number may be drawn as required, but the present invention is not limited thereto.

[0113] The pixel distribution interval preliminary determination module is used to merge the continuous pixel intervals in each ROI area of ​​the template image, preliminarily determine the distribution of multiple pixel distribution intervals in each ROI area, and determine the number of pixels in each pixel distribution interval Image_template[m][i][n].

[0114] For a binary image with grayscale distribution between 0 and 255, the continuous pixel intervals in the image in the mth (m≤M) ROI are merged, and the number of merged pixel intervals is recorded as N. In a specific embodiment of the present invention, an average division strategy is adopted (the average division strategy refers to the average region division of the 256 values), N[m][i]∈{8,16,32,64,128}, and it is preliminarily determined that there are 5 pixel interval distributions in the mth ROI area. For example, N[3][1] indicates that for the first pixel interval distribution of the third ROI, the number of its pixel distribution intervals is 8, and these 8 pixel distribution intervals are:

[0115] [0,31],[32,63],[64,95],[96,127],[128,159],[160,191],[192,223],[224,255].

[0116] Each ROI region has an optimal number of pixel distribution intervals, which can be regarded as the attribute of the ROI. The present invention needs to select the optimal number of pixel distribution intervals from the five pixel interval distributions. Of course, the pixel interval distribution is not limited to five types, and the present invention is only used as an example.

[0117] For each pixel interval distribution, count the number of pixels in each pixel distribution interval of each pixel interval distribution Image_template[m][i][n]. If the ROI area is a rectangular frame, then when counting the ROI pixel distribution, traverse all the pixels in the rectangular frame, and then count the pixel distribution of each pixel distribution interval according to the number of pixel distribution intervals of the current pixel interval distribution, that is, the number of pixels Image_template[m][i][n]. If the ROI area is a non-rectangular frame, it is necessary to first draw an outer rectangular frame of the shape, and traverse all the pixels in the outer rectangular frame, only count the pixel distribution of the ROI area, and then count the pixel distribution of each pixel distribution interval according to the number of pixel distribution intervals of the current pixel interval distribution. That is to say, when counting the pixel distribution of ROI, the points in the rectangular frame of ROI area can be directly counted. However, if the ROI area is not a rectangular frame, it is necessary to find the circumscribed rectangle first, and then judge whether each point in the rectangle is in the rectangle. If it is in, it will be counted, otherwise it will not be counted. After the statistics, the pixel distribution interval is divided, and the number of pixels in each pixel distribution interval is calculated Image_template[m][i][n].

[0118] The weight coefficient calculation module is used to calculate the weighted value of each pixel distribution interval in each ROI area of ​​the template image according to the pixel distribution interval of the corresponding ROI area in the comparison image for each pixel interval distribution in the ROI area of ​​the template image.

[0119] It should be noted here that the determination of the pixel distribution interval of the corresponding ROI area in the comparison image is similar to that of the template image, and will not be described in detail here.

[0120] Specifically, for N[m][i], select the i-th value and calculate the weighted value Coeff[m,i,n] of its n-th (n≤N) pixel interval. The calculation formula is:

[0121]

[0122] Wherein, Coeff[m,i,n] represents the weighted value of the nth pixel interval in the mth ROI region when N[i] is selected as the number of pixel intervals. For example, in this embodiment, Coeff[2][3][4] represents the weighted coefficient of the fourth pixel interval (i.e., [24,31]) for the second ROI when the number of pixel intervals is 32; Image_contrast[m][i][n] represents the number of pixels in the nth pixel interval in the mth ROI region of the contrast image when N[i] is selected as the number of pixel intervals; Image_template[m][i][n] represents the number of pixels in the nth pixel interval in the mth ROI region of the template image when N[i] is selected as the number of pixel intervals.

[0123] The weighted square difference calculation module is used to obtain the weighted square difference Se[m][i] of the mth ROI when the number of intervals in the mth ROI is N[i] by using the weighted values ​​of each pixel interval in the mth ROI region. In a specific embodiment of the present invention, the calculation formula of the weighted square difference is as follows:

[0124]

[0125] The optimization calculation module is used to obtain the optimal number of pixel intervals N[m] and the corresponding weighting coefficient group Coeff[m][n] of the ROI area by comparing the maximum values ​​among i Se[m][i].

[0126] The difference standard threshold calculation module is used to calculate the interval distance of the same pixel interval for the comparison image ROI corresponding to each ROI of the template image according to the obtained optimal pixel interval number N[m] and the corresponding weighting coefficient group Coeff[m][n] for each ROI area, and sum the weighted pixel interval distances of the same ROI area according to the weighting coefficient to obtain the difference value of a single ROI of the comparison image as the difference standard threshold of the ROI area.

[0127] In a specific embodiment of the present invention, for each template image ROI corresponding to the comparison image ROI, the interval distance of the same pixel interval is calculated, and the calculation formula is:

[0128]

[0129] Wherein, Distance[m,n] represents the distance between the mth ROI of the contrast image and the mth ROI of the template image in the nth pixel interval, Image_template[m][n] represents the number of pixels in the nth pixel interval of the mth ROI region in the template image, and Image_contrast[m][n] represents the number of pixels in the nth pixel interval of the mth ROI region in the contrast image.

[0130] According to the weighting coefficient, the weighted pixel interval distances in the same ROI area are summed to obtain the difference value of a single ROI in the contrast image. The calculation formula is:

[0131]

[0132] Among them, Value_Threshould [m] It represents the difference between the mth ROI region in the comparison image and the template image, which is used as the difference standard threshold. Coeff[m][n] represents the weighting coefficient of the nth pixel interval in the mth ROI region of the template image, and N[m] represents the number of pixel intervals in the mth ROI.

[0133] The defect detection unit 204 is used to analyze the pixel distribution interval of each ROI of the source image, count the pixel distribution of each ROI, calculate the interval distance of the same pixel interval of the ROI of the source image corresponding to each template image ROI, and obtain the difference value of a single ROI of the source image. According to the difference value of a single ROI of the source image and the corresponding difference standard threshold obtained by the adaptive analysis unit 203, it is determined whether the current ROI area is a defective area, and then it is determined whether the source image has defects.

[0134] Specifically, the defect detection unit 204 further includes:

[0135] The source image pixel distribution determination module is used to analyze the pixel distribution interval of each ROI of the source image and count the pixel distribution of each ROI.

[0136] In a specific embodiment of the present invention, for the source image, the pixel distribution interval of each ROI of the source image is determined according to the optimal pixel interval number N[m] of each ROI area determined by the adaptive analysis unit 203, and then the pixel distribution of each pixel distribution interval is counted.

[0137] The interval distance calculation module is used to calculate the interval distance of the same pixel interval for each source image ROI corresponding to the template image ROI. The calculation formula is as follows:

[0138]

[0139] The single ROI difference value calculation module is used to sum the weighted pixel interval distances of the ROI area according to the weighting coefficient of each pixel interval in the ROI area to obtain the difference value of a single ROI of the source image. The calculation formula is as follows:

[0140]

[0141] Where Distance[m,n] represents the distance between the mth ROI of the source image and the mth ROI of the kth template image in the nth pixel interval, Value[m] represents the difference between the source image and the mth ROI in the template image, Coeff[m][n] represents the weighting coefficient of the nth pixel interval in the mth ROI, Image_src[m][n] represents the number of pixels in the nth pixel interval of the mth ROI in the source image, Image_template[m][n] represents the number of pixels in the nth pixel interval of the mth ROI in the template image, and N[m] represents the number of pixel intervals in the mth ROI.

[0142] The defect judgment module is used to compare the difference value of each ROI area of ​​the source image obtained with the corresponding difference standard threshold Value_Threshould obtained by the adaptive analysis unit [m] Compare them, if it is greater than, then determine that the current ROI area is a defective area, and the source image has defects.

[0143] If the difference values ​​of all ROIs of the source image are less than the corresponding difference standard threshold, it is determined that the source image has no defects.

[0144] Example

[0145] Figure 3 FIG. 1 is a flow chart of an image processing method according to an embodiment of the present invention. Figure 3 As shown, the image processing process of the present invention applicable to the mold monitor is as follows:

[0146] Step 1. Install an infrared filter on the camera lens, adjust the camera position and lens focal length, adjust the position of the infrared light to obtain a clear image; adjust the camera exposure value and gain value to avoid overexposure or underexposure.

[0147] Step 2, obtain the template image of the empty mold cavity through the camera, recorded as Image_template.

[0148] Step 3: Obtain a contrast image of the full mold cavity through a camera, denoted as image_contrast.

[0149] Step 4: preprocess the template image and the comparison image.

[0150] In this embodiment, the preprocessing mainly utilizes the form of histogram equalization in image enhancement technology to perform grayscale mapping on the image, so that the converted image has approximately the same number of pixels at each grayscale level, and the image with a small grayscale dynamic range is transformed into an image with higher contrast, larger grayscale dynamic range, and clearer information.

[0151] Step 5: Draw the ROI area of ​​the template image, and the number is denoted as M.

[0152] Step 6: Adopt an adaptive strategy to analyze the pixel distribution interval of each ROI of the template image, calculate the weighted value of each pixel distribution interval, and determine the pixel interval distribution of the ROI.

[0153] like Figure 4 As shown in the figure, for a binary image with grayscale distribution between 0 and 255, the continuous pixel intervals in the image in the mth (m≤M) ROI can be merged, and the number of merged pixel intervals is recorded as N (here, the equal division strategy is adopted, N[m][i]∈{8,16,32,64,128}, a total of 5). Select the i-th value and calculate the weighted value Coeff[m,i,n] of its nth (n≤N) pixel interval. The calculation formula is:

[0154]

[0155] Among them, Coeff[m,i,n] represents the weighted value of the nth pixel interval in the mth ROI area when N[i] is selected as the number of pixel intervals, Image_contrast[m][i][n] represents the number of pixels in the nth pixel interval in the mth ROI area of ​​the contrast image when N[i] is selected as the number of pixel intervals; Image_template[m][i][n] represents the number of pixels in the nth pixel interval in the mth ROI area of ​​the template image when N[i] is selected as the number of pixel intervals.

[0156] Calculate the weighted square difference Se[m][i] of the mth ROI when the number of intervals is N[i]. By comparing and selecting the maximum value among the i Se[m][i], the optimal number of pixel intervals N[m] and the corresponding weighted coefficient group Coeff[m][n] of the ROI are obtained. The calculation formula of the weighted square difference is:

[0157]

[0158] The pseudo code for finding the optimal number of pixel intervals N[m] and the selection strategy of the weighted array Coeff[m][n] in the mth ROI is:

[0159]

[0160] For each template image ROI corresponding to the contrast image ROI, the interval distance of the same pixel interval is calculated, and the calculation formula is:

[0161]

[0162] Wherein, Distance[m,n] represents the distance between the mth ROI of the contrast image and the mth ROI of the template image in the nth pixel interval, Image_template[m][n] represents the number of pixels in the nth pixel interval of the mth ROI region in the template image, and Image_contrast[m][n] represents the number of pixels in the nth pixel interval of the mth ROI region in the contrast image.

[0163] According to the weighting coefficient, the weighted pixel interval distances in the same ROI area are summed to obtain the difference value of a single ROI in the contrast image. The calculation formula is:

[0164]

[0165] Among them, Value_Threshould [m] It represents the difference between the mth ROI region in the comparison image and the template image, which is used as the difference standard threshold. Coeff[m][n] represents the weighting coefficient of the nth pixel interval in the mth ROI region of the template image, and N[m] represents the number of pixel intervals in the mth ROI.

[0166] Step 7, such as Figure 5 As shown, the source image is acquired by an infrared camera.

[0167] Step 8: pre-process the source image, analyze the pixel distribution interval of each ROI of the source image, and count the pixel distribution of each ROI.

[0168] Step 9: For each source image ROI corresponding to the template image ROI, calculate the interval distance of the same pixel interval. The calculation formula is:

[0169]

[0170] And by setting up the weighting coefficient, the weighted pixel interval distances in the same ROI area are summed to obtain the difference value of a single ROI of the source image. The calculation formula is:

[0171]

[0172] in,

[0173] Distance[m,n]: The distance between the mth ROI of the source image and the mth ROI of the kth template image in the nth pixel interval

[0174] Value[m]: The difference between the source image and the mth ROI region in the template image

[0175] Coeff[m][n]: weighting coefficient of the nth pixel interval in the mth ROI area

[0176] Image_src[m][n]: The number of pixels in the nth pixel interval of the mth ROI region in the source image

[0177] Image_template[m][n]: The number of pixels in the nth pixel interval of the mth ROI region in the template image

[0178] N[m]: The number of pixel intervals in the mth ROI

[0179] Step 10: Compare the Value[m] obtained in step 9 with the Value_Threshould obtained in step 6. [m] Compare them one by one. If they are greater than, the ROI area is determined to be a defect area and the source image has defects. If the difference values ​​of all ROIs are less than the threshold, it is determined that the source image does not have defects. The pseudo code structure is:

[0180] Do Result←true

[0181] For m←1toM(total number of ROI regions)

[0182] Do If Value[m]>Value_Threshould[m]

[0183] then Result←false

[0184] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Any person skilled in the art may modify and alter the above embodiments without violating the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be as set forth in the claims.

Claims

1. An image processing method suitable for a mold monitor comprises the following steps: Step S1, using an infrared camera to collect a template image of an empty mold cavity and a comparison image of a full mold cavity; Step S2, preprocessing the acquired template image and comparison image; Step S3, extracting M ROI regions of the template image, and extracting M ROI regions corresponding to the comparison image at the same time, using an adaptive strategy to analyze each ROI region of the template image, performing automatic optimization calculation of relevant parameters of the pixel set, determining the number of optimal pixel intervals of each ROI region and the weighting coefficient corresponding to each pixel interval, and obtaining the difference value of the ROI region corresponding to the template image and the comparison image as the difference standard threshold of the corresponding ROI region; Step S3 further comprises: Step S300, extracting M ROI regions of the template image; Step S301, merge the continuous pixel intervals in each ROI area image of the template image, preliminarily determine the distribution of multiple pixel distribution intervals of each ROI area, and determine the number of pixels in each pixel distribution interval Image_template[m][i][n]; if the ROI area is a rectangular frame, when counting the ROI pixel distribution, traverse all the pixels in the rectangular frame, and then count the pixel distribution of each pixel distribution interval according to the number of pixel distribution intervals of the current pixel interval distribution; if the ROI area is a non-rectangular frame, first draw the circumscribed rectangular frame of the ROI area, and traverse all the pixels in the external rectangular frame, only count the pixel distribution of the ROI area, and then count the pixel distribution of each pixel distribution interval according to the number of pixel distribution intervals of the current pixel interval distribution; Step S302, for each pixel interval distribution of each ROI region of the template image, calculate the weighted value of each pixel distribution interval in the ROI region of the template image according to the pixel distribution interval of the corresponding ROI region in the comparison image; Step S303, using the obtained weighted values ​​of the pixel intervals of the m-th ROI region, obtain the weighted square difference Se[m][i] of the m-th ROI when the number of intervals of the m-th ROI region is N[i]; Step S304, by comparing the maximum values ​​among i Se[m][i], the optimal number of pixel intervals N[m] and the corresponding weighting coefficient group Coeff[m][n] of the ROI area are obtained; Step S305, according to the obtained optimal pixel interval number N[m] of each ROI area and the corresponding weighting coefficient group Coeff[m][n], for the comparison image ROI corresponding to each ROI of the template image, the interval distance of the same pixel interval is calculated, and the weighted pixel interval distance of the same ROI area is summed according to the weighting coefficient to obtain the difference value of a single ROI of the comparison image as the difference standard threshold of the ROI area; the calculation formula of the interval distance of the same pixel interval is as follows Wherein, Distance[m,n] represents the distance between the mth ROI of the contrast image and the mth ROI of the template image in the nth pixel interval, Image_template[m][n] represents the number of pixels in the nth pixel interval of the mth ROI region in the template image, and Image_contrast[m][n] represents the number of pixels in the nth pixel interval of the mth ROI region in the contrast image. The calculation formula of the difference standard threshold of a single ROI of the contrast image is: Among them, Value_Threshould [m] Represents the difference value of the mth ROI region in the comparison image and the template image, which is used as the difference standard threshold. Coeff[m][n] represents the weighting coefficient of the nth pixel interval in the mth ROI region of the template image. N[m] represents the number of pixel intervals in the mth ROI of the template image. Step S4, using an infrared camera to obtain a source image, and preprocessing the obtained source image; Step S5, analyzing the pixel distribution interval of each ROI of the source image, and counting the pixel distribution of each ROI; Step S6, for each ROI of the source image corresponding to the ROI of the template image, calculate the interval distance of the same pixel interval, and obtain the difference value of the single ROI of the source image, and determine whether the current ROI area is a defective area based on the difference value of the single ROI of the source image and the corresponding difference standard threshold obtained in step S3, and then determine whether the source image has defects.

2. An image processing method suitable for a mold monitor as claimed in claim 1, characterized in that: When acquiring a template image, multiple template images are stored as needed, and all template images are fitted based on the least squares method. The fitting results are then compared with all template images one by one, and the template image with the smallest difference is selected as the final template image.

3. The image processing method for a mold monitor according to claim 1, characterized in that: Step S6 further comprises: Step S600, for each source image ROI corresponding to the template image ROI, calculating the interval distance of the same pixel intervals; Step S601, according to the weighting coefficient of each pixel interval in the ROI area, summing the weighted pixel interval distances of the ROI area to obtain the difference value of a single ROI of the source image; In step S602, the difference values ​​of the obtained ROI regions of the source image are compared with the corresponding difference standard thresholds obtained in step S3. If they are greater than, the current ROI region is determined to be a defective region and the source image has defects. If the difference values ​​of all ROIs of the source image are less than the corresponding difference standard thresholds, it is determined that the source image does not have defects.

4. The image processing method for a mold monitor as claimed in claim 3, characterized in that: In step S602 , if the number of ROIs is 1, the ROI of the template image and the ROI of the source image are directly compared; if the number of ROIs is greater than or equal to 1, the ROI regions with the same serial number on the template image and the source image need to be compared.

5. The image processing method for a mold monitor according to claim 1, characterized in that: In step S6, the maximum ROI difference value in the current source image is compared with the difference standard threshold. If it is greater than 1 / 3 of the difference standard threshold, the current source image is automatically saved as a new template image while determining that the image has no defects.

6. An image processing device suitable for a mold monitor, comprising: An image acquisition unit, used to acquire a template image of an empty mold cavity, a comparison image of a full mold cavity, and a source image to be detected using an infrared camera; An image preprocessing unit, used for preprocessing the acquired template image, comparison image, and source image; An adaptive analysis unit is used to extract M ROI regions of the template image and M ROI regions corresponding to the comparison image, and adopt an adaptive strategy to analyze each ROI region of the template image, perform automatic optimization calculation of relevant parameters of the pixel set, determine the number of optimal pixel intervals of each ROI region and the weighting coefficient corresponding to each pixel interval, and obtain the difference value of the ROI region corresponding to the template image and the comparison image as the difference standard threshold of the corresponding ROI region; The adaptive analysis unit is more specifically used for: Extract M ROI regions of the template image; Merge the continuous pixel intervals in each ROI region image of the template image, preliminarily determine the distribution of multiple pixel distribution intervals of each ROI region, and determine the number of pixels in each pixel distribution interval Image_template[m][i][n]; if the ROI region is a rectangular frame, when counting the ROI pixel distribution, traverse all the pixels in the rectangular frame, and then count the pixel distribution of each pixel distribution interval according to the number of pixel distribution intervals of the current pixel interval distribution; if the ROI region is a non-rectangular frame, first draw the outer rectangular frame of the ROI region, and traverse all the pixels in the outer rectangular frame, only count the pixel distribution of the ROI region, and then count the pixel distribution of each pixel distribution interval according to the number of pixel distribution intervals of the current pixel interval distribution; For each pixel interval distribution of each ROI region of the template image, a weighted value of each pixel distribution interval in the ROI region of the template image is calculated according to the pixel distribution interval of the corresponding ROI region in the comparison image; Using the obtained weighted values ​​of each pixel interval of the m-th ROI region, when the number of intervals of the m-th ROI region is N[i], the weighted square difference Se[m][i] of the m-th ROI is obtained; By comparing the maximum values ​​among i Se[m][i], the optimal number of pixel intervals N[m] and the corresponding weighting coefficient group Coeff[m][n] of the ROI area are obtained; According to the obtained optimal pixel interval number N[m] of each ROI area and the corresponding weighting coefficient group Coeff[m][n], for each ROI of the template image corresponding to the comparison image ROI, the interval distance of the same pixel interval is calculated, and the weighted pixel interval distance of the same ROI area is summed according to the weighting coefficient to obtain the difference value of a single ROI of the comparison image as the difference standard threshold of the ROI area; the calculation formula of the interval distance of the same pixel interval is as follows Wherein, Distance[m,n] represents the distance between the mth ROI of the contrast image and the mth ROI of the template image in the nth pixel interval, Image_template[m][n] represents the number of pixels in the nth pixel interval of the mth ROI region in the template image, and Image_contrast[m][n] represents the number of pixels in the nth pixel interval of the mth ROI region in the contrast image. The calculation formula of the difference standard threshold of a single ROI of the contrast image is: Among them, Value_Threshould [m] Represents the difference value of the mth ROI region in the comparison image and the template image, which is used as the difference standard threshold. Coeff[m][n] represents the weighting coefficient of the nth pixel interval in the mth ROI region of the template image. N[m] represents the number of pixel intervals in the mth ROI of the template image. The defect detection unit is used to analyze the pixel distribution interval of each ROI of the source image, count the pixel distribution of each ROI, calculate the interval distance of the same pixel interval of the ROI of the source image corresponding to each template image ROI, and obtain the difference value of a single ROI of the source image. According to the difference value of a single ROI of the source image and the corresponding difference standard threshold obtained by the adaptive analysis unit, it is determined whether the current ROI area is a defective area, and then whether the source image has defects.

Images