Instant noodle surface foreign matter detection method and system based on deep learning

Abstract

The application discloses a kind of instant noodle surface foreign matter detection method and system based on deep learning, to the instant noodle image collected Gamma correction;Adopt eight field noise reduction processing, according to the number of non-white points in eight fields to judge noise point, and remove it, then the local image is enhanced, and the flaw point is black, and the non-flaw is white, to facilitate subsequent more accurate identification of foreign matter;Again, the background of the cake image is removed, and the impurity interference outside the background is eliminated;And all the blind hole black holes in the cake are averaged, finally, the contrast between foreign matter and cake is improved using Fourier transform, and then the processed sample data is simulated and trained using a deep learning training model, and finally an instant noodle foreign matter recognition model is obtained, and then the model is used to identify and detect foreign matter in instant noodles.The application greatly improves the precision and efficiency of instant noodle foreign matter recognition, and meets the needs of instant noodle foreign matter automatic visual recognition.

Description

Technical Field

[0001] This invention belongs to the field of foreign object detection technology on food surfaces, and in particular relates to a method and system for detecting foreign objects on the surface of instant noodles based on deep learning. Background Technology

[0002] Instant noodles, a quick-cooking food, are widely loved. Their main ingredient is a noodle cake, made by stacking layers of thin noodles to form a round or square cake-like structure. However, during the noodle cake manufacturing process, foreign objects such as yellow spots, black spots, or other impurities may remain on the surface. To ensure noodle cake quality and a positive consumer experience, the surface of the noodles needs to be inspected for foreign objects before packaging.

[0003] With the mature development of visual inspection technology, visual inspection systems have gradually been applied in food inspection. For example, the existing patent publication number CN113487584A discloses a method for detecting yellow spots on instant noodles based on deep learning. This patent discloses a series of processing steps for instant noodle images, but does not explain the purpose and significance of these processing steps, nor does it explain the technical effects achieved by these processing steps or the technical problems to be solved. Therefore, this patent does not actually solve the problem of detecting yellow spots on instant noodles.

[0004] For instant noodle cakes, due to the characteristics of their surface structure, the color of the cakes is yellow, making it quite difficult to identify yellow foreign objects from them. Furthermore, the cakes are made up of several thin noodles stacked in layers, which are stacked irregularly and without obvious layers. The top layer is not completely covered by noodles and has some through grooves, which makes it difficult to inspect for foreign objects. In particular, the presence of blind holes and grooves can easily lead to false judgments or misjudgments during visual identification.

[0005] Therefore, it is necessary to develop a method and system for detecting foreign objects on the surface of instant noodles based on deep learning to solve the above problems. Summary of the Invention

[0006] One of the main objectives of this invention is to provide a method for detecting foreign objects on the surface of instant noodles based on deep learning, which can effectively identify yellow and black spots on instant noodles and has high reliability in the identification results.

[0007] This invention achieves the above objective through the following technical solution: a method for detecting foreign objects on the surface of instant noodles based on deep learning, comprising the following steps: S1) Use a camera to capture images of instant noodle samples to obtain sample data; S2) Preprocess the sample data: S21) Perform Gamma correction on the instant noodle sample image to expand the shadow areas in the sample image; S22) Perform neighborhood noise reduction and neighborhood image enhancement processing on the image after Gamma correction: This includes calculating the number of non-white points in the eight neighborhoods of each pixel region. If the number of non-white points is less than a set value, then all pixels in the pixel region are determined to be noise points. Calculate the average gray value of all pixels remaining after removing noise points from the eight neighborhoods of each pixel (x,y). If the average gray value is greater than a preset maximum gray value threshold, then the gray value of the pixel (x,y) is the average gray value plus a preset gain value. When the average gray value is less than a preset minimum gray value threshold, then the gray value of the pixel (x,y) is the average gray value minus the gain value. S23) Perform image separation processing on the image processed in step S22) to remove the background; S24) Perform black hole interference removal processing on the image processed in step S23), which includes using a color image to output three-channel separation, extracting the RGB three-color channel image, using one of the channel images after channel separation to perform image type conversion, using the CIELab color space conversion algorithm to make the black holes in the pancake uniform, so that all the black holes in the pancake are displayed with the same gray level. S25) Use Fourier transform to improve the contrast between foreign objects and the dough; S3) Classify and label the preprocessed sample data as good or bad products, and mark the location of foreign objects on the sample images; S4) Construct a differential convolutional neural network foreign object recognition model, and train it using the sample data processed in step S3) to obtain the network connection weight matrix; S5) Randomly select 70% of the data obtained in step S3) as the training set, 15% as the validation set, and the remaining 15% as the test set. S6) Expand the 70% training set randomly selected in step S5) using samples; S7) Use the extended training set obtained in step S6) to train the differential convolutional neural network foreign object recognition model constructed in step S4) to obtain the trained instant noodle foreign object recognition model. S8) Set up cameras directly above and below the instant noodle cake to be detected, acquire images of the instant noodle cake, and then preprocess it according to the preprocessing method described in step 2). Then, use the instant noodle foreign object recognition model described in step S7) to detect foreign objects in the preprocessed noodle cake image and determine whether it is a good product.

[0008] Furthermore, the Gamma correction process in S21) includes: S211) Normalize the sample data image according to formula 1), where formula 1) is: S = p / 225, where p is the actual gray value of the pixel in the sample data image and S is the result after normalization. S212) Pre-compensation: Pre-compensation is performed on the normalized result according to formula 2), where formula 2) is: Where B is the pre-compensation result and G is the Gamma value, which is the input setting value; S213) Inverse Normalization: The result after pre-compensation is inversely normalized according to formula 3), where formula 3) is: , where R is the result of inverse normalization; S214) Integrate Equations 1)-3) to obtain Equation 4): ; Save the generated dataset dictionary, and perform a dictionary lookup on all pixels to complete the Gamma correction process.

[0009] Further, the image separation processing in step S23) includes calculating the minimum grayscale value of the pixels in the entire instant noodle image; setting a grayscale threshold T based on the minimum grayscale value and a set minimum grayscale threshold, wherein the grayscale threshold T is close to the minimum grayscale threshold and greater than the minimum grayscale value; and calculating the binarized image according to formula 5), ​​wherein formula 5) is: ; Where f(x,y) represents the binarized result, and p(x,y) represents the gray value of the current point.

[0010] Furthermore, the sample expansion method in step S6) includes one or more of the following combinations: rotation, mirroring, increasing brightness within a range, cropping, and absolute brightness peak values ​​of random positioning points.

[0011] Another object of the present invention is to provide a foreign object detection system for instant noodles based on deep learning, comprising: The image data acquisition module uses multiple cameras to collect image data of instant noodles and form a dataset. The image preprocessing module performs Gamma calibration, neighborhood noise reduction, and neighborhood image enhancement on the dataset in sequence, removes image background and black hole interference, and finally performs Fourier transform on it to obtain the processed data. The image data classification module classifies the processed data into defective products and good products, and assigns different labels. If it is a defective product, it is assigned a label of 0; otherwise, it is assigned a label of 1. The image data expansion module expands the labeled processed data by means of rotation, mirroring, increasing brightness within a range, cropping, and absolute brightness peak values ​​of random positioning points to obtain expanded data; The differential convolutional neural network module uses the expanded data from the image data expansion module for training to obtain the network connection weight matrix; The image data training module uses the extended data to train the differential convolutional neural network module to obtain a trained instant noodle foreign object recognition model. The image recognition and comparison module uses the image to be detected, which has been preprocessed by the image preprocessing module, to perform recognition and detection, and to determine whether it is a good product.

[0012] Compared with existing technologies, the beneficial effects of this invention's deep learning-based method and system for detecting foreign objects on the surface of instant noodles are as follows: By designing a systematic image processing step, gamma correction is performed on the acquired instant noodle images to expand the shadow areas in the noodle cake image, revealing clearer actual details, making the blind holes and black holes in the noodle cake and the outlines of the surrounding noodles clearer, and diluting the grayscale of the black holes to avoid misjudging foreign objects at the blind holes and black holes; eight-domain noise reduction processing is used to determine noise points based on the number of non-white points in the eight domains and remove them, and then local image enhancement processing is performed to turn blemishes black and non-white points black. The defective areas are whitened to facilitate more accurate identification of foreign objects. The background of the noodle image is then removed to eliminate interference from impurities outside the background. All blind holes and black holes in the noodle image are averaged out to remove their interference. Finally, Fourier transform is used to improve the contrast between the foreign object area and the noodle image, obtaining sample data. This data is then used to train a deep learning model, which is simulated and trained to obtain a foreign object recognition model for instant noodles. This model is then used to identify foreign objects in instant noodles and determine whether the noodles are qualified. This significantly improves the accuracy and efficiency of foreign object recognition in instant noodles, meeting the needs for automated visual recognition of foreign objects in instant noodles. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the steps in an embodiment of the present invention; Figure 2 This is a diagram showing the effect of neighborhood noise reduction and neighborhood image enhancement processing before processing in an embodiment of the present invention; Figure 3 This is a diagram showing the effect of neighborhood noise reduction and neighborhood image enhancement processing in an embodiment of the present invention. Figure 4 This is a diagram showing the effect of averaging the black hole as a whole through the interference removal step in an embodiment of the present invention. Detailed Implementation

[0014] Example: Please refer to Figure 1 This embodiment presents a method for detecting foreign objects on the surface of instant noodles based on deep learning, which includes the following steps: S1) Use a camera to acquire images of instant noodle samples to obtain sample data. The instant noodle samples include good products and various defective products, including instant noodle cakes with black spots and instant noodle cakes with yellow spots. This sample data is mainly used for subsequent training of the deep learning model.

[0015] S2) Preprocess the sample data. Specifically, this includes: S21) Calculate the inverse normalization results corresponding to the 256 pixel values, construct a Gamma correction table, and perform Gamma correction on each pixel in the sample data image. By generating a Gamma correction table, the computational efficiency can be effectively improved, thereby improving the preprocessing efficiency.

[0016] S211) Normalize the sample data images according to Formula 1): Formula 1) Where p is the actual grayscale value of the pixel in the sample data image, and S is the result after normalization. S212) Pre-compensation: Pre-compensate the result after normalization according to formula 2): Formula 2) Where B is the pre-compensation result, S is the normalization result, and G is the Gamma value, which is the input set value. S213) Inverse Normalization: Perform inverse normalization on the result after pre-compensation processing according to formula 3): Formula 3) Where B is the pre-compensation result and R is the inverse normalization result; S214) The Gamma correction result for each pixel can be calculated using formulas 1)-3) above. However, since the exponential calculation is involved, the process is time-consuming. Therefore, a Gamma correction table can be generated for faster calculation. The generation method is to calculate the inverse normalization results corresponding to 256 pixel values ​​from 0 to 255 respectively, and then integrate them to obtain formula 4). Formula 4) Save the generated dataset dictionary, and perform a dictionary lookup on all pixels to complete the Gamma calibration. The Gamma-corrected image will expand the shadow areas in the original image captured by the camera, revealing clearer actual details.

[0017] After applying Gamma correction to the instant noodle sample image, the details of the blind holes and black holes in the noodle cake can be clearly defined and the black holes can be faded, thus avoiding the phenomenon of misjudging foreign objects at the blind holes and black holes.

[0018] S22) Neighborhood noise reduction and neighborhood image enhancement: Since the qualified color of instant noodles is generally a light color such as white or yellow, and the blemishes are a darker color, this embodiment uses eight-neighborhood noise reduction to process the image noise and performs eight-neighborhood noise reduction on the result obtained from step S23) Gamma correction.

[0019] (S221) The closer the pixel value is to white in the image, the closer it is to 255; conversely, the closer it is to 0. Calculate the number of non-white pixels in the eight-neighborhood. If the number of non-white pixels is less than the set value, then this pixel area is noise. Based on the influence of the actual production environment's light intensity on the lighting effect, this set value is generally set too high.

[0020] (S222) While performing noise reduction, local image enhancement (local image refers to the area composed of pixels within the set gray value threshold) is also performed simultaneously through a set gray value threshold: If the average gray value of the eight neighborhoods is less than the set gray value threshold, the pixels in the area are considered as defects, and then negative gain (gray value decreases, local becomes black) processing is applied to its gray value. That is, the gray value of the local image composed of pixels whose average gray value of the eight neighborhoods is less than the set gray value threshold decreases and appears black. If the average gray value of the eight neighborhoods is greater than the set gray value threshold, positive gain (gray value increases, local becomes whiter) processing is applied to its gray value. That is, the gray value of the local image composed of pixels whose average gray value of the eight neighborhoods is greater than the set gray value threshold increases and appears white. In addition, the parts with uneven lighting will also become whiter and brighter. The instant noodle image obtained through this processing method will have the contrast difference between the defects and the surrounding pixels amplified, making the defects more prominent and easier to identify, thus effectively avoiding the phenomenon of missed detection.

[0021] The coordinate matrix formed by each point (x, y) in the specific neighborhood image and its eight surrounding pixels during enhancement processing is as follows: .

[0022] The grayscale value at point (x, y) is p(x, y). When the average grayscale value of all points in the eight-neighborhood (excluding noise, i.e., the eight pixels other than point (x, y) in the coordinate matrix above) is greater than a preset maximum grayscale value threshold, the grayscale value of point p(x, y) is the average grayscale value of the other points plus a preset gain value. When the average grayscale value of all points in the eight-neighborhood (excluding noise, i.e., less than a preset minimum grayscale value threshold), the grayscale value of point p(x, y) is the average grayscale value of the other points minus the gain value. This achieves the purpose of noise reduction, enhancing the appearance of defects, increasing some brightness, and hiding useless details in the image. During this process, the minimum grayscale value of the pixels in the overall instant noodle image can be calculated. Adding this to the known minimum grayscale value threshold of the pixels yields the grayscale range of the defects.

[0023] Through the neighborhood denoising and neighborhood image enhancement processes in step S22), pixel areas with fewer than a set number of non-white points are first removed. Then, image enhancement is performed, turning blemishes black and non-blemishes white to facilitate further processing and blemish identification. The comparison image before and after step S22) is shown below. Figures 2-3 As shown.

[0024] S23) Perform image separation to obtain an image containing only pie image information: Using binarization, based on the result of neighborhood image enhancement processing, and depending on the minimum gray value and minimum gray threshold, set a gray threshold T. This gray threshold T is close to the minimum gray threshold and greater than the minimum gray value, and is calculated according to the following formula: ;Formula 5) Where f(x,y) represents the binarized result, p(x,y) represents the gray value of the current point, and T represents the set gray value threshold that is close to and greater than the minimum gray value.

[0025] Image separation processing mainly involves removing the background from the instant noodle cake image, separating the background from the noodle cake itself, and extracting the noodle cake itself, thereby reducing interference from impurities outside the noodle cake in subsequent foreign object identification.

[0026] S24) Removing interference: After image separation processing, use color image conversion to separate the three channels, extract the RGB three-color channel images, use one of the separated channel images to perform image type conversion, and use the CIELab color space conversion algorithm to make the black holes in the pancake more uniform, so that they are not disturbed during detection.

[0027] The main purpose of interference removal processing is to even out the black holes created by blind holes in the dough, displaying all black holes at the same gray level. The effect after processing is as follows: Figure 4 As shown.

[0028] S25) Improve the contrast between foreign objects and the pancake using Fourier transform: Convert the frequency domain image of the image after interference removal into a spatial domain image to obtain the preprocessed final sample data.

[0029] S3) The preprocessed sample data is manually sorted into good and defective products, and the location of foreign objects is marked on the sample images. If there are foreign objects in the sample images, the label is set to 0; otherwise, the label is set to 1.

[0030] S4) Construct a differential convolutional neural network foreign object recognition model, and train it using the data processed in step S3) to obtain the network connection weight matrix; S5) Take the data obtained in step S3), randomly select 70% of the data as the training set, 15% of the data as the validation set, and the remaining 15% of the data as the test set.

[0031] S6) Expand the 70% training set randomly selected in step S5) by using methods such as rotation, mirroring, increasing brightness within the range, cropping, and absolute brightness peak values ​​of random positioning points to obtain expanded training set data, which can reduce the workload of collecting a large amount of sample data.

[0032] S7) Use the extended training set obtained in step S6) to train the differential convolutional neural network foreign object recognition model constructed in step S4) to obtain the trained instant noodle foreign object recognition model. S8) Set up cameras directly above and below the instant noodle cake to be detected, acquire images of the instant noodle cake, and then preprocess it according to the preprocessing method described in step 2). Then, use the instant noodle foreign object recognition model described in step S7) to detect foreign objects in the preprocessed noodle cake image and determine whether it is a good product.

[0033] This embodiment also provides a foreign object detection device for instant noodles based on deep learning, which includes: The image data acquisition module uses multiple cameras to collect image data of instant noodles and form a dataset. The image preprocessing module performs Gamma calibration, neighborhood noise reduction, and neighborhood image enhancement on the dataset in sequence, removes image background and black hole interference, and finally performs Fourier transform on it to obtain the processed data. The image data classification module classifies the processed data into defective products and good products, and assigns different labels. If it is a defective product, it is assigned a label of 0; otherwise, it is assigned a label of 1. The image data expansion module expands the labeled processed data by means of rotation, mirroring, increasing brightness within a range, cropping, and absolute brightness peak values ​​of random positioning points to obtain expanded data; The differential convolutional neural network module uses the expanded data from the image data expansion module for training to obtain the network connection weight matrix; The image data training module uses the extended data to train the differential convolutional neural network module to obtain a trained instant noodle foreign object recognition model. The image recognition and comparison module uses the image to be detected, which has been preprocessed by the image preprocessing module, to perform recognition and detection, and to determine whether it is a good product.

[0034] The above descriptions are merely some embodiments of the present invention. Those skilled in the art can make various modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the scope of protection of the present invention.

Claims

1. A method for detecting foreign objects on the surface of instant noodles based on deep learning, characterized in that: It includes the following steps: S1) Use a camera to capture images of instant noodle samples to obtain sample data; S2) Preprocess the sample data: S21) Perform Gamma correction on the instant noodle sample image to extend the shadow area in the sample image; S22) Perform neighborhood noise reduction and neighborhood image enhancement processing on the image after Gamma correction: This includes calculating the number of non-white points in the eight neighborhoods of each pixel region. If the number of non-white points is less than a set value, then all pixels in the pixel region are determined to be noise points. Calculate the average gray value of all pixels remaining after excluding noise points in the eight neighborhoods of each pixel (x,y). If the average gray value is greater than a preset maximum gray value threshold, then the gray value of the pixel (x,y) is the average gray value plus a preset gain value. When the average gray value is less than a preset minimum gray value threshold, then the gray value of the pixel (x,y) is the average gray value minus the gain value. S23) Perform image separation processing on the image processed in step S22) to remove the background; S24) Perform black hole interference removal processing on the image processed in step S23), which includes using a color image to output three-channel separation, extracting the RGB three-color channel image, using one of the channel images after channel separation to perform image type conversion, using the CIELab color space conversion algorithm to make the black holes in the pancake uniform, so that all the black holes in the pancake are displayed with the same gray level. S25) Use Fourier transform to improve the contrast between foreign objects and the dough; S3) Classify and label the preprocessed sample data as good or bad products, and mark the location of foreign objects on the sample images; S4) Construct a differential convolutional neural network foreign object recognition model, and train it using the sample data processed in step S3) to obtain the network connection weight matrix; S5) Randomly select 70% of the data obtained in step S3) as the training set, 15% as the validation set, and the remaining 15% as the test set. S6) Expand the 70% training set randomly selected in step S5) using samples; S7) Use the extended training set obtained in step S6) to train the differential convolutional neural network foreign object recognition model constructed in step S4) to obtain the trained instant noodle foreign object recognition model. S8) Set up cameras directly above and below the instant noodle cake to be detected, acquire images of the instant noodle cake, and then preprocess it according to the preprocessing method described in step 2). Then, use the instant noodle foreign object recognition model described in step S7) to detect foreign objects in the preprocessed noodle cake image and determine whether it is a good product.

2. The method for detecting foreign objects on the surface of instant noodles based on deep learning as described in claim 1, characterized in that: The Gamma correction process in S21 includes: S211) Normalize the sample data images according to Formula 1), where Formula 1) is: Where p is the actual grayscale value of the pixel in the sample data image, and S is the result after normalization. S212) Pre-compensation: Pre-compensation is performed on the normalized result according to formula 2), where formula 2) is: Where B is the pre-compensation result and G is the Gamma value, which is the input setting value; S213) Inverse Normalization: The result after pre-compensation is inversely normalized according to formula 3), where formula 3) is: , where R is the result of inverse normalization; S214) Integrate Equations 1)-3) to obtain Equation 4): ； Save the generated dataset dictionary, and perform a dictionary lookup on all pixels to complete the Gamma correction process.

3. The method for detecting foreign objects on the surface of instant noodles based on deep learning as described in claim 1, characterized in that: The image separation process in step S23) includes calculating the minimum grayscale value of the pixels in the entire instant noodle image; setting a grayscale threshold T based on the minimum grayscale value and a set minimum grayscale threshold, wherein the grayscale threshold T is close to the minimum grayscale threshold and greater than the minimum grayscale value; and calculating the binarized image according to formula 5), ​​wherein formula 5) is: ； Where f(x,y) represents the binarized result, and p(x,y) represents the gray value of the current point.

4. The method for detecting foreign objects on the surface of instant noodles based on deep learning as described in claim 1, characterized in that: The sample expansion method in step S6) includes one or more of the following methods: rotation, mirroring, increasing brightness within a range, cropping, and absolute brightness peak values ​​of random positioning points.

5. A foreign object detection system for instant noodles, characterized in that: The method for detecting foreign objects on the surface of instant noodles according to claim 1, wherein the detection system comprises: The image data acquisition module uses multiple cameras to collect image data of instant noodles and form a dataset. The image preprocessing module performs Gamma calibration, neighborhood noise reduction, and neighborhood image enhancement on the dataset in sequence, removes image background and black hole interference, and finally performs Fourier transform on it to obtain the processed data. The image data classification module classifies the processed data into defective products and good products, and assigns different labels. If it is a defective product, it is assigned a label of 0; otherwise, it is assigned a label of 1. The image data expansion module expands the labeled processed data using one or more of the following methods: rotation, mirroring, increasing brightness within a range, cropping, and absolute brightness peak values ​​at random positioning points, to obtain expanded data. The differential convolutional neural network module uses the expanded data from the image data expansion module for training to obtain the network connection weight matrix; The image data training module uses the extended data to train the differential convolutional neural network module to obtain a trained instant noodle foreign object recognition model. The image recognition and comparison module uses the image to be detected, which has been preprocessed by the image preprocessing module, to perform recognition and detection, and to determine whether it is a good product.

Images