Intelligent milk powder packaging bag sorting method based on deep learning

Abstract

The invention relates to the technical field of image data processing, and provides a milk powder packaging bag intelligent sorting method based on deep learning, which comprises the following steps: acquiring a plurality of milk powder packaging bag images, and classifying to obtain a plurality of packaging requirement types; acquiring a plurality of detection images of the milk powder packaging bag templates of the same packaging requirement type; constructing an optical information distribution map and taking the optical information distribution map as a plurality of template images of each packaging requirement type; marking a plurality of abnormal edges in the real-time milk powder packaging bag image; obtaining a plurality of sealing influence points and influence weights thereof of each abnormal edge; determining the allowable change range of the optical information value of each pixel point in the real-time milk powder packaging bag image; and the light information value of each pixel point in the real-time milk powder packaging bag image is judged according to the allowable change range of the light information value, the risk probability of the real-time milk powder packaging bag image is obtained, and intelligent sorting is carried out. The invention aims to solve the problem that the quality of a milk powder packaging bag is influenced by subsequent air leakage caused by a leakage point at a sealing part in the milk powder packaging bag process.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and particularly to an intelligent sorting method for milk powder packaging bags based on deep learning. Background Art

[0002] As a processed powdered dairy product, milk powder has excellent storage performance for dairy products, such as reducing moisture content, being light in weight, small in volume and easy to transport. At the same time, the dried milk powder can preserve nutrients and inhibit the growth of microorganisms, thereby extending the storage period. Since milk powder is prone to moisture absorption and oxidation, it relies on packaging to protect its quality during the circulation process. Professional packaging can not only block the intrusion of moisture, oxygen and microorganisms, prevent milk powder from caking or oxidizing and deteriorating, but also avoid microbial contamination to ensure the shelf life of milk powder. Modified atmosphere packaging (MAP) forms a low-oxygen environment mainly composed of nitrogen (N2) and carbon dioxide (CO2) (the O2 concentration is usually <1%) by replacing the air in the package, so as to inhibit the growth of microorganisms and delay the oxidation reaction, and at the same time avoid caking of milk powder products.

[0003] However, in the case of milk powder packaging bags with modified atmosphere packaging, although the packaging machine has been automated and can ensure airtightness, air leakage still occurs from time to time. If there is a leak at the packaging seal, air leakage will occur in the subsequent transportation process of the milk powder packaging bag. Therefore, it is necessary to conduct a risk assessment of air leakage after packaging is completed; deep learning can, on the basis of combining computer vision, evaluate the air leakage of milk powder packaging bags through template learning, and achieve seamless integration with the production line control system, and intelligently sort the milk powder packaging bags with air leakage risks to avoid the problem that air leakage of milk powder packaging bags affects the shelf life of milk powder. Summary of the Invention

[0004] The present invention provides an intelligent sorting method for milk powder packaging bags based on deep learning to solve the problem that if there is a leak at the seal in the existing milk powder packaging bag process, it will cause subsequent air leakage and affect the quality of milk powder packaging bags. The specific technical solutions adopted are as follows: The present invention proposes an intelligent sorting method for milk powder packaging bags based on deep learning, and the method includes the following steps: Obtain a number of milk powder packaging bag images and classify them to obtain a number of packaging requirement types; Obtain a number of milk powder packaging bag images of the milk powder packaging bag template of the same packaging requirement type as its detection images; based on the pixel value performance of the neighborhood pixel points in each color channel in the detection images, obtain the light information values of each pixel point in the detection images, and construct a light information distribution map as a number of template images of each packaging requirement type; Set calibration boxes for several template images of the same packaging requirement type, extract the edges in the calibration boxes, obtain the angle features and length features of each edge of each template image of each packaging requirement type, as well as the angle features and length features of the real-time milk powder packaging bag image, and compare them with each edge of the corresponding packaging requirement type to mark several abnormal edges in the real-time milk powder packaging bag image; Obtain the light information distribution map of the real-time milk powder packaging bag image, and obtain several sealing influence points and their influence weights of each abnormal edge based on the change of the light information value of the pixel points along the extension direction of the abnormal edge in the light information distribution map; combine the template image of the corresponding packaging requirement type and the fluctuation of the light information value of the pixel points at the same position in different template images to determine the allowable change range of the light information value of each pixel point in the real-time milk powder packaging bag image; Judge the light information value of each pixel point in the real-time milk powder packaging bag image according to its allowable change range to obtain the risk probability of the real-time milk powder packaging bag image, and perform intelligent sorting on the real-time milk powder packaging bag according to the risk probability.

[0005] Optionally, the specific method for obtaining the light information value of each pixel point in the detection image based on the pixel value performance of the neighboring pixel points in each color channel of the detection image includes: Based on the pixel value performance of the neighboring pixel points in each color channel of the detection image, obtain the local window of each pixel point in the detection image; Obtain the minimum value of the pixel values of each pixel point in the local window of any pixel point in any color channel of any detection image as the information value of this pixel point in this color channel, and take the minimum value of the information values of this pixel point in all color channels as the light information value of this pixel point.

[0006] Optionally, the specific method for obtaining the local window of each pixel point in the detection image is: For any pixel point in any detection image of any packaging requirement type, preset the initial window size and expansion step length; obtain the absolute value of the difference between the pixel value of this pixel point in any color channel of this detection image and the pixel values of other pixel points in the initial window centered on it as the pixel difference between this pixel point and other pixel points in its initial window in this color channel; If the number of pixel points with pixel differences less than the pixel difference threshold within the initial window is greater than the window expansion threshold, expand the initial window. If the number of pixel points with pixel differences less than the pixel difference threshold within the initial window is less than or equal to the window expansion threshold, do not expand the initial window. And so on, perform expansion judgment on the initial window of this pixel point until the expansion stops when the condition is met. Take the window when the expansion stops as the neighborhood window of this pixel point under this color channel; take the window composed of several pixel points that are all neighborhood windows of this pixel point under each color channel as the local window of this pixel point.

[0007] Optionally, the method for constructing the light information distribution map and using it as several template images for each packaging requirement type specifically includes: Obtain the light information values of each pixel point in any detection image of any packaging requirement type, and take the image composed of the light information values of each pixel point as a template image for the corresponding packaging requirement type of this detection image.

[0008] Optionally, the method for obtaining the angle features and length features of each edge of each template image for each packaging requirement type specifically includes: For any template image of any packaging requirement type, set a calibration box with a fixed size for this template image; Perform edge detection on any calibration box to obtain several edges; obtain the angle and length of any edge of this calibration box, and perform normalization processing on the angle and length respectively. The obtained normalization results are used as the angle feature and length feature of this edge of this template image.

[0009] Optionally, the method for marking several abnormal edges in the real-time milk powder packaging bag image specifically includes: For the real-time milk powder packaging bag image, extract several edges through the calibration box and obtain the angle features and length features of each edge; Construct a three-dimensional sample space of angle features, length features, and edge center positions, where the edge center position is the coordinate corresponding to each edge in the corresponding image. Map each edge of the real-time milk powder packaging bag image to the three-dimensional sample space according to its angle features and length features to obtain sample points, and map each edge of each template image corresponding to the packaging requirement type of the real-time milk powder packaging bag image to the three-dimensional sample space to obtain data points; For any sample point, obtain the Euclidean distance between this sample point and each data point, and take the minimum value among the Euclidean distances between this sample point and several data points of the same template image as the matching edge difference between this sample point and this template image; The mean of the matching edge differences between the sample point and each template image is used as the degree of matching abnormality of the edge corresponding to the sample point in the real-time milk powder packaging bag image. If the degree of matching abnormality is greater than the matching abnormality threshold, the edge is marked as an abnormal edge in the milk powder packaging bag image.

[0010] Optionally, the specific method for obtaining several sealing influence points and their influence weights of each abnormal edge includes: For any abnormal edge in the real-time milk powder packaging bag image, obtain the endpoint on the side of the packaging bag among the two endpoints on both sides of the abnormal edge as the starting endpoint of the extension of the abnormal edge. Along the angle of the abnormal edge, extend from the starting endpoint of the extension into the interior of the packaging bag in the milk powder packaging bag image, and calculate the absolute value of the difference between the light information values of each pixel point passed through and the starting endpoint of the extension during the extension process. During the extension process, judge each pixel point one by one. When the first pixel point with an absolute value of the difference between the light information values greater than the information difference threshold from the starting endpoint of the extension appears, take this pixel point as a sealing influence point of the abnormal edge, and continue to traverse backward. Similarly, calculate the absolute value of the difference between the light information values with the starting endpoint of the extension. If it is greater than the information difference threshold, it is used as a sealing influence point. When the first pixel point with an absolute value of the difference between the light information values less than or equal to the information difference threshold appears after obtaining several sealing influence points, stop the subsequent traversal and obtaining of sealing influence points. This pixel point is not used as a sealing influence point, and several sealing influence points of the abnormal edge are obtained. For any sealing influence point, the inverse proportional normalization result of the distance between the sealing influence point and its corresponding starting endpoint of the extension is used as the influence weight of the sealing influence point.

[0011] Optionally, the specific method for obtaining the allowable change range of the light information values of each pixel point in the real-time milk powder packaging bag image is as follows: Based on the template images corresponding to the packaging requirement types of the real-time milk powder packaging bag image and the fluctuation conditions of the light information values of the pixel points at the same positions in different template images, obtain the allowable light information values and allowable adjustment ranges at each position in each template image corresponding to the packaging requirement types of the real-time milk powder packaging bag image. If a pixel point corresponding to the same position in any of the template images corresponding to the packaging requirement types of the real-time milk powder packaging bag image is a sealing influence point, the product of the difference obtained by subtracting the influence weight of the sealing influence point from 1 and its allowable adjustment range is used as the allowable influence range of the sealing influence point. Correspond the real-time milk powder packaging bag image to each position under the corresponding packaging requirement type. For any pixel point in the real-time milk powder packaging bag image, if it is not a sealing influence point, the closed interval obtained by adding and subtracting the allowable adjustment range of the allowable light information value at the corresponding position is used as the allowable change range of the light information value of this pixel point; If this pixel point is a sealing influence point, the closed interval obtained by adding and subtracting the allowable influence range of this sealing influence point to the allowable light information value at its corresponding position is used as the allowable change range of the light information value of this sealing influence point.

[0012] Optionally, the specific method for obtaining the allowable light information value and the allowable adjustment range of each position in each template image corresponding to the packaging requirement type of the real-time milk powder packaging bag image includes: For each template image corresponding to the packaging requirement type of the real-time milk powder packaging bag image, obtain the standard deviation of the light information values of each pixel point at any same position in each template image, and perform linear normalization on the standard deviations of each position. The obtained result is used as the adjustment weight of this position; Take the mean value of the light information values of each pixel point at any same position in each template image as the allowable light information value of this position. Obtain the absolute value of the difference between the allowable light information value and the maximum value of the light information values of each pixel point at this position, and the absolute value of the difference between the allowable light information value and the minimum value of the light information values of each pixel point at this position. Take the minimum value of the two obtained absolute differences as the allowable fluctuation range of this position; Multiply the allowable fluctuation range by the adjustment weight of this position as the allowable adjustment range of this position.

[0013] Optionally, the specific method for obtaining the risk probability of the real-time milk powder packaging bag image includes: For any pixel point in the real-time milk powder packaging bag image, if its light information value is within its allowable change range, mark this pixel point as 0. If its light information value exceeds its allowable change range, mark this pixel point as 1; Statistically calculate the proportion of the number of pixel points marked as 1 in the real-time milk powder packaging bag image in the total number of all pixel points in the entire milk powder packaging bag image, and use the obtained ratio as the risk probability of the real-time milk powder packaging bag image.

[0014] The beneficial effects of the present invention are as follows: The present invention obtains the optical information values of the milk powder packaging bag templates of various packaging requirement types to obtain the template images, and analyzes the fluctuations of the optical information values between the template images to reflect the positions with random changes in the packaging, reducing the influence of its reflection; at the same time, considering the angle characteristics and length characteristics of the sealing edges of the template images, the extraction of the sealing influence points of the real-time milk powder packaging bag images is carried out accordingly, that is, the change of the optical information value of the pixel points in the extension direction where the corresponding edges may be abnormal; among them, the optical information distribution map of each milk powder discharging image is obtained through the dark channel analysis method to reflect the optical information distribution of each position therein, laying a foundation for the subsequent reflection distribution analysis; at the same time, the detection images are selected for each packaging requirement type, and the template images are obtained based on their optical information distribution; the sealing position edges are extracted by setting calibration frames for the milk powder packaging bag images, and analyzed based on their angle characteristics, length characteristics and distribution positions. By comparing the differences with the edges in the template images under the corresponding packaging requirement types, the abnormal edges are reflected through the edge distribution and morphological differences; through the change of the optical information value of the pixel points in the extension direction of the abnormal edges, the sealing influence points and their influence weights are obtained, reflecting the influence of the change of their optical information value on the displacement of the sealing position edge. Combining the standard deviation of the optical information values of the pixel points at the same position in different template images under the corresponding packaging requirement types, the allowable change range is adjusted to reduce the influence of the distribution of the corresponding positions of each pixel point in the real-time milk powder packaging bag image on the packaging bag form; finally, based on the optical information values of each pixel point and their allowable change ranges, the risk probability judgment of the milk powder packaging bag image is realized, and the risk discharging control of the milk powder packaging bag is carried out accordingly, realizing the pipeline detection of the milk powder packaging bag based on deep learning combined with computer vision. The deep learning unit is seamlessly integrated into the production line control system to realize intelligent sorting, reducing the influence of quality problems caused by air leakage in subsequent transportation and packaging. Description of the Drawings

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0016] Figure 1 Schematic flow chart of the intelligent sorting method for milk powder packaging bags based on deep learning provided by an embodiment of the present invention. Detailed Embodiments

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0018] Please refer to Figure 1 , which shows a flowchart of an intelligent sorting method for milk powder packaging bags based on deep learning provided by an embodiment of the present invention. The method includes the following steps: Step S001: Obtain a number of milk powder packaging bag images and classify them into several packaging requirement types.

[0019] Specifically, during the packaging process of milk powder, a packaging machine is used for packaging (the packaging machine is composed of a feeding panel, a packaging film running device, a packaging film paper core shaft, an inflation device, an end sealing knife seat, a discharging panel and a discharging port (two discharging ports, one is a normal discharging port and the other is a risk discharging port), a control device and an integrated electric box. Among them, in order to ensure the fixed position of the packaging printing picture and the time shelf life during transportation, there is a fixing device for each batch of milk powder packaging bags on the feeding panel and the discharging panel.

[0020] Furthermore, a deep learning unit is integrated behind the discharging panel. The image acquisition module in it is used to collect images. By arranging a polarized light source and a high-definition industrial camera, the collected images are used as milk powder packaging bag images corresponding to the milk powder packaging bags. The acquisition time interval is based on the transportation of milk powder on the production line for acquisition, and the milk powder packaging bag images are input into the deep learning unit. The deep learning unit combines the packaging requirements of the milk powder packaging bags and the corresponding template images, analyzes the air leakage risk of the milk powder packaging bags and obtains the risk probability, outputs the risk classification result of the milk powder packaging bags, and at the same time connects to the screening module in the control device. The screening module controls the moving swing arm to switch between the normal discharging port and the risk discharging port according to the output result of the deep learning unit.

[0021] It should be noted that the air leakage risk is determined by analyzing the shape of the packaging bag. Therefore, it is necessary to quantify the air leakage risk by analyzing the difference between the shape distribution of the packaging under non-air leakage conditions and the shape distribution of the real-time milk powder packaging bags. Therefore, it is necessary to obtain the distribution of the packaging bags, that is, it is necessary to obtain the distribution template under normal conditions; during the image acquisition process, the packaging bags are irregular objects, and there will be different curvatures at different positions on the surface, and different degrees of curvature on the surface will present different degrees of reflectivity due to the influence of light.

[0022] Further, when the packaging machine is packaging milk powder bags, it is necessary to determine parameters such as the packaging length and packaging speed, and classify the templates according to the parameters of the packaging machine. In this embodiment, an existing classification method is adopted: for example, in the process of packaging milk powder bags with a packaging length less than 5 mm and a packaging speed not exceeding 1 bag / min, the milk powder bags are of the same packaging requirement type. In this embodiment, classification is carried out based on every 5 mm of packaging length and every 1 bag / min of packaging speed, and several packaging requirement types are obtained.

[0023] Step S002: Obtain a number of milk powder bag images of the milk powder bag template of the same packaging requirement type as its detection images; based on the pixel value performance of the neighboring pixel points in each color channel in the detection images, obtain the light information values of each pixel point in the detection images, and construct a light information distribution map and use it as several template images of each packaging requirement type.

[0024] Preferably, in an embodiment of the present invention, obtaining a number of milk powder bag images of the milk powder bag template of the same packaging requirement type as its detection images includes the following specific method: It should be noted that under each packaging requirement type, it is necessary to obtain milk powder bags that can be used as templates, and use the milk powder bag images obtained after them on the discharge panel as the detection images of the corresponding packaging requirement type for subsequent processing.

[0025] Specifically, the requirements for the milk powder bags used as templates in this embodiment are: milk powder bags that are placed in a packaging box filled with milk powder bags and do not leak air after being squeezed for 24 hours. For any demand model, 30 milk powder bags used as templates need to be obtained; the milk powder discharge images of the milk powder bags used as templates are obtained through image acquisition after the discharge panel and used as the detection images of this demand model.

[0026] Preferably, in an embodiment of the present invention, based on the pixel value performance of the neighboring pixel points in each color channel in the detection images, obtaining the light information values of each pixel point in the detection images, and constructing a light information distribution map and using it as several template images of each packaging requirement type includes the following specific method: It should be noted that according to the laws of light scattering and object surface reflection, the optical model of the reflective area can be decomposed into a diffuse reflection component and a specular reflection component. The reflective area is a strong specular reflection generated at a certain illumination angle. Therefore, for high-reflectivity situations, the specular reflection is dominant. For the milk powder discharging image, the reflection component can show information differences in the multi-channel image of the image. Therefore, there are significant differences between the reflective area and the non-reflective area in the multi-channel. At the same time, if there are wrinkles caused by possible air leakage in the packaging bag, it will also cause changes in the information in the multi-channel. Therefore, to obtain the reflective distribution by analyzing the change differences in multi-channel information, it is first necessary to obtain the light information distribution map for the milk powder discharging image and the detection image, that is, to construct the light information distribution map through the pixel values of each color channel in the neighborhood range, similar to the fog image formation model in the field of computer vision.

[0027] Specifically, for any pixel point in any detection image of any packaging requirement type, the preset initial window size is , which is described in this embodiment using . And the dilation step size is set to 2, that is, the side length increases by 2 each time the window is dilated; the absolute value of the difference between the pixel value of this pixel point in any color channel of the detection image and the pixel values of other pixel points in the initial window centered on it is used as the pixel difference between this pixel point and other pixel points in its initial window in this color channel; preset the pixel difference threshold and the window dilation threshold. In this embodiment, the pixel difference threshold is described using 10, and the window dilation threshold is described using 2 / 3; if the number of pixel points with pixel differences less than the pixel difference threshold in the initial window is greater than the window dilation threshold, the initial window needs to be dilated. If the number of pixel points with pixel differences less than the pixel difference threshold in the initial window is less than or equal to the window dilation threshold, the initial window does not need to be dilated. And so on, the dilation judgment of the initial window of this pixel point is carried out until the condition is met and the dilation stops. The window when the dilation stops is used as the neighborhood window of this pixel point in this color channel; the window composed of several pixel points that are all the neighborhood windows of this pixel point in each color channel is used as the local window of this pixel point; it should be especially noted that if this pixel point is close to the boundary of the detection image, resulting in its (initial) window exceeding the range of the detection image, then the (initial) window is composed of the actually existing pixel points, and subsequent dilation judgments are carried out based on the actually existing pixel points.

[0028] Further, obtain the minimum value of the pixel values of each pixel in the local window of the pixel under any color channel as the information value of the pixel under this color channel, and take the minimum value of the information values of the pixel under all color channels as the light information value of the pixel; obtain the light information values of each pixel in the detection image, and use the image composed of the light information values of each pixel as a template image of the packaging requirement type corresponding to the detection image; obtain several template images of each packaging requirement type according to the above method.

[0029] Further, obtain the light information value for each pixel of each milk powder discharge image according to the above method, so as to obtain the light information distribution map of each milk powder discharge image.

[0030] So far, the light information distribution map of each milk powder discharge image is obtained through the dark channel analysis method to reflect the light information distribution of each position therein, laying a foundation for the subsequent analysis of the reflection distribution; at the same time, detection images are selected for each packaging requirement type, and template images are obtained based on their light information distribution.

[0031] Step S003: Set calibration frames for several template images of the same packaging requirement type and extract the edges in the calibration frames to obtain the angle features and length features of each edge of each template image of each packaging requirement type; obtain the angle features and length features of the real-time milk powder packaging bag image through the calibration frames and the extracted edges, and compare them with each edge of the corresponding packaging requirement type to mark several abnormal edges in the real-time milk powder packaging bag image.

[0032] It should be noted that since the shape of the packaging bag is irregular and the gas flows inside the packaging bag, the influence of air leakage on different positions inside the packaging bag is different; at the same time, during the process of air leakage assessment, if the packaging bag bulges or leaks, it will cause a change in the sealing shape of the packaging bag. Therefore, the change at the sealing part of each packaging bag can reflect the gas distribution inside the packaging bag; for all images, since the milk powder packaging bag is fixed on the feeding panel and the discharging panel when the images are collected, by comparing the differences between all template images and the real-time collected images, the position where the difference is located corresponds to the position where the sealing shape changes; and since the illumination angle is fixed and the characteristics of the packaging bag make the direction of each reflective area along a direction, and these directions may often be related to the deformation at the sealing part, it is necessary to exclude the influence of some deformations at the sealing part.

[0033] Preferably, in an embodiment of the present invention, setting calibration frames for several template images of the same packaging requirement type and extracting the edges in the calibration frames to obtain the angle features and length features of each edge of each template image of each packaging requirement type includes the following specific method: Specifically, for any template image of any packaging requirement type, a calibration box of a fixed size is manually set for the template image, where the calibration box is used to frame the sealing positions on both sides of the packaging bag. The calibration process is not described in this embodiment. Edge detection is performed on any calibration box to obtain a number of edges. It should be noted that the obtained edges are all truncated straight edges, that is, the angles of the same edge are ensured to be consistent. The angle and length of any edge of the calibration box are obtained, and the angle and length are respectively normalized. Among them, the angle size is normalized with 360°, and the length size is normalized with the maximum value of the lengths of each edge in the two calibration boxes in the template image. The obtained normalized results are used as the angle feature and length feature of this edge of the template image.

[0034] Further, the edges of each template image of each packaging requirement type and their angle features and length features are obtained.

[0035] It should be noted that edge extraction is performed on the sealing position through the calibration box, and then feature comparison is performed based on the edges under the same packaging requirement type. The greater the feature difference, the greater the possibility of edge abnormality, that is, it is more likely that the edge displacement occurs due to air leakage. At the same time, multiple template images are involved in the comparison to reduce the influence of accidental feature differences.

[0036] Preferably, in an embodiment of the present invention, the angle feature and length feature of the real-time milk powder packaging bag image are obtained through the calibration box and the extracted edges, and compared with the edges of each template image of the corresponding packaging requirement type. The specific method for marking a number of abnormal edges in the real-time milk powder packaging bag image includes: For the real-time milk powder packaging bag image, a number of edges are extracted through the calibration box and the angle feature and length feature of each edge are obtained. A three-dimensional sample space of the angle feature, length feature, and edge center position is constructed, where the edge center position is the coordinate corresponding to each edge in the corresponding image. The edges of the real-time milk powder packaging bag image are mapped to the three-dimensional sample space according to their angle features and length features to obtain sample points. At the same time, the edges of each template image of the corresponding packaging requirement type of the real-time milk powder packaging bag image are mapped to the three-dimensional sample space to obtain data points, where each data point has a corresponding template image and a corresponding edge.

[0037] Further, for any sample point, obtain the Euclidean distance between the sample point and each data point. Take the minimum value among the Euclidean distances between the sample point and several data points of the same template image as the matching edge difference between the sample point and the template image. Take the mean value of the matching edge differences between the sample point and each template image as the matching abnormality degree of the edge corresponding to the sample point in the real-time milk powder packaging bag image. Preset a matching abnormality threshold. In this embodiment, the matching abnormality threshold is described as 0.3. If the matching abnormality degree is greater than the matching abnormality threshold, mark the edge as an abnormal edge in the milk powder packaging bag image, and then obtain several abnormal edges in the milk powder packaging bag image.

[0038] It should be noted that the matching edge difference is used to screen the edges in each template image with similar angular features, length features, and distribution positions to the corresponding edge of the sample point. And the matching edge differences in different template images are more likely to be the same edges at the sealing position. Analyze the possibility of abnormal edges in the real-time milk powder packaging bag image through the mean value.

[0039] So far, by setting a calibration frame for the milk powder packaging bag image to extract the edge of the sealing position, and analyzing based on its angular features, length features, and distribution positions, and comparing the differences with each edge in the template image under the corresponding packaging requirement type, the abnormal edges are obtained by reflecting the edge distribution and morphological differences.

[0040] Step S004: Obtain the light information distribution map of the real-time milk powder packaging bag image. According to the change of the light information value of the pixel points along the extension direction of the abnormal edge in the light information distribution map, obtain several sealing influence points and their influence weights of each abnormal edge. Combine the template image corresponding to the packaging requirement type and the fluctuation of the light information value of the pixel points at the same position in different template images to determine the allowable change range of the light information value of each pixel point in the real-time milk powder packaging bag image.

[0041] It should be noted that the light information value of each pixel point is obtained from the packaging bag morphology distribution in all template images of the same type of milk powder packaging bag requirement, and the standard deviation of the light information value of each position is obtained. The larger the standard deviation of each position, the more random the reflection of the light information distribution information at that position. Then, when comparing the differences between the template image and the real-time collected image, the comparison degree corresponding to that position needs to be reduced to prevent misjudgment, so as to adjust the allowable change range. At the same time, if the change of the light information value of each pixel point in the collected image is more consistent with the position where the sealing morphology changes, it indicates that the position where the pixel point is located has a greater risk when evaluating the air leakage risk. That is, by the change of the light information value of the pixel points along the extension direction of the abnormal edge, further obtain the sealing influence points and quantify the sealing influence weight.

[0042] Preferably, in an embodiment of the present invention, for a real-time image of a milk powder packaging bag, its light information distribution map is obtained. Based on the change of the light information values of the pixel points along the extension direction of the abnormal edge in the light information distribution map, several sealing influence points and their influence weights of each abnormal edge are obtained. The specific method includes: For any abnormal edge in the real-time image of the milk powder packaging bag, obtain the endpoint on the side of the packaging bag among the two endpoints on both sides of the abnormal edge (one side of the edge is close to the packaging bag, and the other side is close to the outside) as the starting endpoint of the extension of the abnormal edge; along the angle of the abnormal edge, extend from the starting endpoint of the extension into the interior of the packaging bag in the image of the milk powder packaging bag. During the extension process, calculate the absolute value of the difference between the light information value of each pixel point passed and the light information value of the starting endpoint of the extension. A preset information difference threshold is set. In this embodiment, the information difference threshold is described using 10; during the extension process, judge each pixel point one by one. When the first pixel point whose absolute value of the difference between the light information value and the starting endpoint of the extension is greater than the information difference threshold appears, take this pixel point as a sealing influence point of the abnormal edge, and continue to traverse backward. Similarly, calculate the absolute value of the difference between the light information values with the starting endpoint of the extension. If it is greater than the information difference threshold, it is also taken as a sealing influence point. When the first pixel point whose absolute value of the difference between the light information value and the starting endpoint of the extension is less than or equal to the information difference threshold appears after obtaining several sealing influence points, stop the subsequent traversal and obtaining of sealing influence points, and at the same time, this pixel point is not taken as a sealing influence point, then several sealing influence points of the abnormal edge are obtained.

[0043] Furthermore, for any sealing influence point, the inverse proportional normalization result of the distance between the sealing influence point and its corresponding starting endpoint of the extension is used as the influence weight of the sealing influence point; it should be noted that in this embodiment, a model is used to present the inverse proportional relationship and normalization process, which is an exponential function with the natural constant as the base, is the input of the model. The implementer can set the inverse proportional function and normalization function according to the actual situation. Among them, to avoid the inverse proportional normalization result from being too small, the distance is multiplied by the hyperparameter 0.1 in the input model, and then the inverse proportional normalization result is obtained.

[0044] Further, it should be noted that after obtaining the sealing influence points and their influence weights, it is necessary to adjust the allowable change range of the light information value of the corresponding pixel points according to the influence weight. That is, since it will cause obvious sealing influence in the real-time milk powder packaging bag image, it is necessary to reduce the pixel value fluctuation range to avoid affecting the edge change of the sealing position due to changes. And each pixel point also needs to adjust the allowable change range according to the light information value fluctuation of the corresponding pixel points at the corresponding positions in the template images of different corresponding packaging requirement types. The larger the standard deviation, the more random the light information value, and the larger the allowable change range is required. Similarly, the smaller the standard deviation, the smaller the allowable change range is required to avoid the influence of the change of the light information value.

[0045] Preferably, in an embodiment of the present invention, in combination with the template images of different corresponding packaging requirement types and the light information value fluctuations of the pixel points at the same positions in different template images, the allowable change range of the light information value of each pixel point in the real-time milk powder packaging bag image is determined. The specific method includes: For each template image of the real-time milk powder packaging bag image corresponding to the packaging requirement type, obtain the standard deviation of the light information value of each pixel point at any same position in each template image, and perform linear normalization on the standard deviation of each position. The obtained result is used as the adjustment weight of this position; take the mean value of the light information value of each pixel point at any same position in each template image as the allowable light information value of this position, obtain the absolute value of the difference between the allowable light information value and the maximum value of the light information value of each pixel point at this position, and the absolute value of the difference between the allowable light information value and the minimum value of the light information value of each pixel point at this position. Take the minimum value of the two obtained absolute values of the differences as the allowable fluctuation range of this position; multiply the allowable fluctuation range by the adjustment weight of this position to obtain the allowable adjustment range of this position; if the pixel point corresponding to this position in the real-time milk powder packaging bag image is a sealing influence point, multiply the difference obtained by subtracting the influence weight of this sealing influence point from 1 by the allowable adjustment range to obtain the allowable influence range of this sealing influence point.

[0046] Furthermore, make the real-time milk powder packaging bag image correspond to each position under the corresponding packaging requirement type. For any pixel point in the real-time milk powder packaging bag image, if it is not a sealing influence point, take the closed interval obtained by adding and subtracting the allowable adjustment range from the allowable light information value of the corresponding position as the allowable change range of the light information value of this pixel point; if this pixel point is a sealing influence point, take the closed interval obtained by adding and subtracting the allowable influence range of this sealing influence point from the allowable light information value of its corresponding position as the allowable change range of the light information value of this sealing influence point. Thus, the allowable change range of the light information value of each pixel point in the real-time milk powder packaging bag image is obtained.

[0047] So far, by the change of the light information value of the pixel points in the abnormal edge extension direction, the sealing influence points and their influence weights are obtained, reflecting the influence of the change of their light information value on the edge displacement of the sealing position. Combining the standard deviation of the light information values of the pixel points at the same position of different template images under the corresponding packaging requirement types, the allowable change range is adjusted to reduce the influence of the distribution of the corresponding positions of each pixel point in the real-time milk powder packaging bag image on the shape of the packaging bag.

[0048] Step S005: Judge the light information value of each pixel point in the real-time milk powder packaging bag image according to its allowable change range, obtain the risk probability of the real-time milk powder packaging bag image, and perform intelligent sorting on the real-time milk powder packaging bag according to the risk probability.

[0049] Specifically, for each pixel point in the real-time milk powder packaging bag image, its light information value and the corresponding allowable change range have been obtained. If the light information value of any pixel point is within its allowable change range, mark this pixel point as 0. If its light information value exceeds its allowable change range, mark this pixel point as 1; count the proportion of the number of pixel points marked as 1 in the real-time milk powder packaging bag image in the total number of all pixel points in the entire milk powder packaging bag image, and use the obtained ratio as the risk probability of the real-time milk powder packaging bag image. The risk reminder threshold in this embodiment is set to 1 / 2 of the total number of all pixel points. If the risk probability is greater than the risk reminder threshold, there may be leaks in the milk powder packaging bags discharged by the discharge panel in real time. Then the risk classification result output by the deep learning unit is that there is a risk, and the intelligent control moving swing arm transfers this milk powder packaging bag to the risk discharge port; if the risk probability is less than or equal to the risk reminder threshold, the risk classification result output by the deep learning unit is that there is no risk, and the intelligent control moving swing arm inputs this milk powder packaging bag into the normal discharge port, where the risk probability judgment result is the risk classification result and is input to the control device for intelligent sorting and controlling the moving swing arm.

[0050] So far, by obtaining the light information value for the milk powder packaging bag templates of each packaging requirement type to obtain the template images, and analyzing the light information value fluctuations between the template images to reflect the positions with random changes in the packaging, reducing its reflection influence; at the same time, considering the angular features and length features at the sealing edge of the template images, so as to extract the sealing influence points of the real-time milk powder packaging bag image, that is, the change of the light information value of the pixel points on the corresponding edge that may be abnormal and reflected in the extension direction. Finally, based on the light information value of each pixel point and its allowable change range, the risk probability judgment of the milk powder packaging bag image is realized, and the risk discharge control of the milk powder packaging bag is carried out based on this, realizing the pipeline detection of the milk powder packaging bag based on deep learning combined with computer vision. The deep learning unit is seamlessly integrated into the production line control system to realize intelligent sorting, reducing the influence of quality problems caused by air leakage in subsequent transportation and packaging.

[0051] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An intelligent sorting method for milk powder packaging bags based on deep learning, characterized in that, The method includes the following steps: Obtain a number of milk powder packaging bag images and classify them to obtain several packaging requirement types; Obtain several milk powder packaging bag images of the milk powder packaging bag template of the same packaging requirement type as its detection images; based on the pixel value performance of neighboring pixel points in each color channel in the detection images, obtain the light information values of each pixel point in the detection images, construct a light information distribution map and use it as several template images of each packaging requirement type; Set calibration frames for several template images of the same packaging requirement type and extract the edges in the calibration frames to obtain the angle features and length features of each edge of each template image of each packaging requirement type, as well as the angle features and length features of the real-time milk powder packaging bag image, and compare them with each edge of the corresponding packaging requirement type to mark several abnormal edges in the real-time milk powder packaging bag image; Obtain the light information distribution map of the real-time milk powder packaging bag image, and obtain several sealing influence points and their influence weights of each abnormal edge according to the change of the light information values of the pixel points along the extension direction of the abnormal edge; combine the template images of the corresponding packaging requirement type and the fluctuation of the light information values of the pixel points at the same position of different template images to determine the allowable change range of the light information values of each pixel point in the real-time milk powder packaging bag image; Judge the light information values of each pixel point in the real-time milk powder packaging bag image according to its allowable change range to obtain the risk probability of the real-time milk powder packaging bag image, and perform intelligent sorting on the real-time milk powder packaging bag according to the risk probability.

2. The intelligent sorting method for milk powder packaging bags based on deep learning according to claim 1, wherein The specific method for obtaining the light information values of each pixel point in the detection images based on the pixel value performance of neighboring pixel points in each color channel includes: Based on the pixel value performance of neighboring pixel points in each color channel in the detection images, obtain the local window of each pixel point in the detection images; Obtain the minimum value of the pixel values of each pixel point in the local window of any pixel point in any color channel of any detection image as the information value of this pixel point in this color channel, and use the minimum value of the information values of this pixel point in all color channels as the light information value of this pixel point.

3. The intelligent sorting method for milk powder packaging bags based on deep learning according to claim 2, wherein, The specific method for obtaining the local window of each pixel point in the detection images is: For any pixel point in any detection image of any packaging requirement type, preset the initial window size and expansion step length; obtain the absolute value of the difference between the pixel value of this pixel point in any color channel of this detection image and the pixel values of other pixel points in the initial window centered on it as the pixel difference between this pixel point and other pixel points in its initial window in this color channel; If the number of pixel points with pixel differences less than the pixel difference threshold within the initial window is greater than the window expansion threshold, expand the initial window. If the number of pixel points with pixel differences less than the pixel difference threshold within the initial window is less than or equal to the window expansion threshold, do not expand the initial window. And so on, perform expansion judgment on the initial window of this pixel point until the expansion stops when the condition is met. Take the window at the stop of expansion as the neighborhood window of this pixel point under this color channel; Take the window composed of several pixel points that are all neighborhood windows of this pixel point under each color channel as the local window of this pixel point.

4. The intelligent sorting method for milk powder packaging bags based on deep learning according to claim 1, wherein, The specific method for constructing the optical information distribution map and using it as several template images for each packaging requirement type includes: Obtain the optical information values of each pixel point in any detection image of any packaging requirement type, and take the image composed of the optical information values of each pixel point as a template image corresponding to the packaging requirement type of this detection image.

5. The intelligent sorting method for milk powder packaging bags based on deep learning according to claim 1, characterized in that The specific method for obtaining the angle features and length features of each edge of each template image of each packaging requirement type includes: For any template image of any packaging requirement type, set a calibration box with a fixed size for this template image; Perform edge detection on any calibration box to obtain several edges; Obtain the angle and length of any edge of this calibration box, and perform normalization processing on the angle and length respectively. Take the obtained normalization results as the angle feature and length feature of this edge of this template image.

6. The intelligent sorting method for milk powder packaging bags based on deep learning according to claim 1, characterized in that The specific method for marking several abnormal edges in the real-time milk powder packaging bag image includes: For the real-time milk powder packaging bag image, extract several edges through the calibration box and obtain the angle features and length features of each edge; Construct a three-dimensional sample space of angle features, length features, and edge center positions, where the edge center position is the coordinate corresponding to each edge in the corresponding image. Map each edge of the real-time milk powder packaging bag image to the three-dimensional sample space according to its angle feature and length feature to obtain sample points, and map each edge of each template image corresponding to the packaging requirement type of the real-time milk powder packaging bag image to the three-dimensional sample space to obtain data points; For any sample point, obtain the Euclidean distance between this sample point and each data point, and take the minimum value among the Euclidean distances between this sample point and several data points of the same template image as the matching edge difference between this sample point and this template image; Take the average value of the matching edge differences between this sample point and each template image as the matching abnormal degree of the edge corresponding to this sample point in the real-time milk powder packaging bag image; If the matching abnormal degree is greater than the matching abnormal threshold, mark this edge as an abnormal edge in the milk powder packaging bag image.

7. The intelligent sorting method for milk powder packaging bags based on deep learning according to claim 1, characterized in that, The specific method for obtaining several sealing influence points and their influence weights of each abnormal edge includes: For any abnormal edge in the real-time milk powder packaging bag image, obtain the endpoint located on one side of the packaging bag among the two endpoints on both sides of this abnormal edge as the extension starting endpoint of this abnormal edge; Along the angle of the abnormal edge, extend from the extended starting endpoint into the interior of the milk powder packaging bag in the milk powder packaging bag image. During the extension process, calculate the absolute value of the difference between the light information values of each pixel point passed through and the extended starting endpoint. During the extension process, judge each pixel point one by one. When the first pixel point with an absolute value of the difference between its light information value and that of the extended starting endpoint is greater than the information difference threshold appears, take this pixel point as a sealing influence point of the abnormal edge, and continue to traverse backward. Similarly, calculate the absolute value of the difference between the light information values and the extended starting endpoint. If it is greater than the information difference threshold, it is used as a sealing influence point. When a number of sealing influence points are obtained and the first pixel point with an absolute value of the difference between its light information value and that of the extended starting endpoint is less than or equal to the information difference threshold appears, stop the subsequent traversal and obtaining of sealing influence points. This pixel point is not used as a sealing influence point, and a number of sealing influence points of the abnormal edge are obtained. For any sealing influence point, take the inverse proportional normalization result of the distance between this sealing influence point and its corresponding extended starting endpoint as the influence weight of this sealing influence point.

8. The intelligent sorting method for milk powder packaging bags based on deep learning according to claim 1, wherein, The allowable change range of the light information values of each pixel point in the real-time milk powder packaging bag image is obtained specifically as follows: Based on the template images corresponding to the packaging requirement types of the real-time milk powder packaging bag image and the fluctuation conditions of the light information values of the pixel points at the same positions in different template images, obtain the allowable light information values and allowable adjustment ranges at each position in each template image corresponding to the packaging requirement types of the real-time milk powder packaging bag image. If a pixel point at any same position in each template image corresponding to the packaging requirement type of the real-time milk powder packaging bag image is a sealing influence point, take the product of the difference obtained by subtracting the influence weight of this sealing influence point from 1 and its allowable adjustment range as the allowable influence range of this sealing influence point. Correspond the real-time milk powder packaging bag image to each position under the corresponding packaging requirement type. For any pixel point in the real-time milk powder packaging bag image, if it is not a sealing influence point, take the closed interval obtained by adding and subtracting its allowable adjustment range to the allowable light information value at the corresponding position as the allowable change range of the light information value of this pixel point. If this pixel point is a sealing influence point, take the closed interval obtained by adding and subtracting the allowable influence range of this sealing influence point to the allowable light information value at its corresponding position as the allowable change range of the light information value of this sealing influence point.

9. The intelligent sorting method for milk powder packaging bags based on deep learning according to claim 8, characterized in that The specific method included in obtaining the allowable light information values and allowable adjustment ranges at each position in each template image corresponding to the packaging requirement types of the real-time milk powder packaging bag image is as follows: For each template image corresponding to the packaging requirement type of the real-time milk powder packaging bag image, obtain the standard deviation of the light information values of each pixel point at any same position in each template image, and perform linear normalization on the standard deviations at each position. The obtained result is used as the adjustment weight at this position. The mean value of the light information values of each pixel at any same position in each template image is used as the allowable light information value at that position. The absolute value of the difference between the allowable light information value and the maximum value of the light information values of each pixel at that position, and the absolute value of the difference between the allowable light information value and the minimum value of the light information values of each pixel at that position are obtained. The minimum value of the two obtained absolute values of the differences is used as the allowable fluctuation range at that position; the product of the allowable fluctuation range and the adjustment weight at that position is used as the allowable adjustment range at that position.

10. The intelligent sorting method for milk powder packaging bags based on deep learning according to claim 1, characterized in that The specific method for obtaining the risk probability of the real-time milk powder packaging bag image includes: For any pixel in the real-time milk powder packaging bag image, if its light information value is within its allowable change range, the pixel is marked as 0; if its light information value exceeds its allowable change range, the pixel is marked as 1. The proportion of the number of pixels marked as 1 in the real-time milk powder packaging bag image to the total number of pixels in the entire milk powder packaging bag image is statistically calculated, and the obtained ratio is used as the risk probability of the real-time milk powder packaging bag image.

Images