Intelligent sorting method for milk powder packaging bags based on deep learning

Abstract

The present invention relates to the field of image data processing technology, and proposes a method for intelligent sorting of milk powder packaging bags based on deep learning, comprising: obtaining a plurality of milk powder packaging bag images and classifying them into a plurality of packaging requirement types; obtaining a plurality of detection images of milk powder packaging bag templates of the same packaging requirement type; constructing a light information distribution map and using it as a plurality of template images of each packaging requirement type; marking a plurality of abnormal edges in the real-time milk powder packaging bag images; obtaining a plurality of sealing influence points of each abnormal edge and their influence weights; determining the allowable variation range of the light information value of each pixel point in the real-time milk powder packaging bag images; judging the light information value of each pixel point in the real-time milk powder packaging bag images based on their allowable variation range, obtaining the risk probability of the real-time milk powder packaging bag images, and performing intelligent sorting. The present invention aims to solve the problem that if leaks occur at the sealing point during the milk powder packaging process, subsequent air leakage will occur, thereby affecting the quality of the milk powder packaging bags.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to an intelligent sorting method for milk powder packaging bags based on deep learning. Background Art

[0002] Milk powder, a processed powdered dairy product, offers excellent storage properties, including reduced moisture content, light weight, compact size, and ease of transportation. Furthermore, dried milk powder preserves nutrients and inhibits microbial growth, thereby extending its shelf life. Because milk powder is susceptible to moisture absorption and oxidation, packaging is crucial for quality protection during distribution. Professional packaging not only blocks the intrusion of moisture, oxygen, and microorganisms, preventing the powder from clumping or oxidative deterioration, but also prevents microbial contamination, thereby ensuring its shelf life. Modified atmosphere packaging (MAP) replaces the air within the package, creating a low-oxygen environment (O2 concentration typically <1%) dominated by nitrogen (N2) and carbon dioxide (CO2). This inhibits microbial growth, slows oxidation reactions, and prevents the powder from clumping.

[0003] However, for modified atmosphere packaging of milk powder bags, although the packaging machines are automated and can ensure airtightness, air leakage still occurs from time to time. If there is a leak at the packaging seal, the milk powder bags will leak during subsequent transportation. Therefore, it is necessary to conduct an air leakage risk assessment after the packaging is completed. Deep learning can conduct air leakage assessment on milk powder bags through template learning based on computer vision, and achieve seamless integration with the production line control system. It can intelligently sort milk powder bags with leakage risks to avoid the problem of air leakage in milk powder bags affecting the shelf life of milk powder. Summary of the Invention

[0004] The present invention provides a deep learning-based intelligent sorting method for milk powder packaging bags to solve the problem that leaks at the seals of existing milk powder packaging bags can lead to subsequent air leakage and affect the quality of milk powder packaging bags. The technical solutions adopted are as follows:

[0005] The present invention proposes a method for intelligent sorting of milk powder packaging bags based on deep learning, which includes the following steps:

[0006] Obtain several milk powder packaging bag images and classify them into several packaging requirement types;

[0007] Acquire several images of milk powder packaging bags of the same packaging requirement type as test images; obtain the light information value of each pixel in the test image based on the pixel value representation of the neighborhood pixels in each color channel, construct a light information distribution map and use it as several template images of each packaging requirement type;

[0008] A calibration frame is set for several template images of the same packaging requirement type and edges within the calibration frame are extracted. The angle features and length features of each edge of each template image of each packaging requirement type, as well as the angle features and length features of the real-time milk powder packaging bag image, are obtained. The edges are then compared with the edges of the corresponding packaging requirement type, and several abnormal edges in the real-time milk powder packaging bag image are marked.

[0009] Obtaining a light information distribution map from a real-time milk powder packaging bag image, and determining several sealing influence points and their influence weights for each abnormal edge based on the changes in light information values ​​of pixels along the direction of the abnormal edge in the light information distribution map. Determining the allowable variation range of the light information value of each pixel in the real-time milk powder packaging bag image by combining the fluctuations in light information values ​​of pixels at the same position in template images corresponding to the packaging requirement type and different template images;

[0010] The light information value of each pixel in the real-time milk powder packaging bag image is judged according to its allowable variation range to obtain the risk probability of the real-time milk powder packaging bag image, and the real-time milk powder packaging bags are intelligently sorted according to the risk probability.

[0011] Optionally, the light information value of each pixel in the detection image is obtained based on the pixel value representation of the neighborhood pixel in each color channel in the detection image, including the specific method of:

[0012] Based on the pixel value representation of the neighborhood pixel points in each color channel in the detection image, a local window of each pixel point in the detection image is obtained;

[0013] Obtain the minimum value of the pixel values ​​of each pixel in the local window of any pixel under any color channel in any detection image as the information value of the pixel under the color channel, and the minimum value of the information value of the pixel under all color channels as the light information value of the pixel.

[0014] Optionally, the local window of each pixel in the detection image is obtained by:

[0015] For any pixel point in any inspection image of any packaging requirement type, preset the initial window size and expansion step size; obtain the absolute value of the difference between the pixel value of the pixel point and the other pixels in the initial window centered on it in any color channel of the inspection image as the pixel difference between the pixel point and the other pixels in the initial window in that color channel;

[0016] If the number of pixels in the initial window whose corresponding pixel differences are less than the pixel difference threshold is greater than the window expansion threshold, the initial window is expanded. If the number of pixels in the initial window whose corresponding pixel differences are less than the pixel difference threshold is less than or equal to the window expansion threshold, the initial window is not expanded. Similarly, the expansion of the initial window of the pixel is judged until the expansion is stopped when the conditions are met. The window at the time of stopping expansion is used as the neighborhood window of the pixel under the color channel; the window composed of several pixels that belong to the neighborhood window of the pixel under each color channel is used as the local window of the pixel.

[0017] Optionally, the light information distribution map is constructed and used as a plurality of template images for each packaging requirement type, including the specific method of:

[0018] The light information value of each pixel in any detection image of any packaging requirement type is obtained, and an image composed of the light information values ​​of each pixel is used as a template image of the packaging requirement type corresponding to the detection image.

[0019] Optionally, the specific method of obtaining the angle feature and length feature of each edge of each template image of each packaging requirement type includes:

[0020] For any template image of any packaging requirement type, a fixed-size calibration frame is set for the template image;

[0021] Perform edge detection on any calibration frame to obtain several edges; obtain the angle and length of any edge of the calibration frame, and normalize the angle and length respectively. The normalized results are used as the angle feature and length feature of the edge of the template image.

[0022] Optionally, the marking of several abnormal edges in the real-time milk powder packaging bag image includes the following specific methods:

[0023] For real-time milk powder packaging bag images, several edges are extracted through calibration frames and the angle and length features of each edge are obtained;

[0024] Constructing a three-dimensional sample space of angle features, length features, and edge center positions, where the edge center positions are the coordinates of each edge in the corresponding image. Mapping each edge of the real-time milk powder packaging bag image to the three-dimensional sample space based on its angle features and length features to obtain sample points. Mapping each edge of each template image corresponding to the packaging requirement type of the real-time milk powder packaging bag image to the three-dimensional sample space to obtain data points.

[0025] For any sample point, obtain the Euclidean distance between the sample point and each data point, and use the minimum value of the Euclidean distance between the sample point and several data points of the same template image as the matching edge difference between the sample point and the template image;

[0026] The average of the differences between the matching edges of the sample point and each template image is used as the matching abnormality degree of the edge corresponding to the sample point in the real-time milk powder packaging bag image; if the matching abnormality degree is greater than the matching abnormality threshold, the edge is marked as an abnormal edge in the milk powder packaging bag image.

[0027] Optionally, the specific method of obtaining a number of sealing influence points and their influence weights of each abnormal edge includes:

[0028] For any abnormal edge in a real-time milk powder packaging bag image, obtain the endpoint on the packaging bag side of the abnormal edge's two end points as the starting end point for extending the abnormal edge; extend the edge from the starting end point toward the interior of the packaging bag in the milk powder packaging bag image along the angle of the abnormal edge, and calculate the absolute value of the difference between the light information value of each pixel passed through and the starting end point during the extension process;

[0029] During the extension process, each pixel is judged one by one. When the first pixel whose absolute value of the difference between the light information value and the starting endpoint of the extension is greater than the information difference threshold appears, the pixel is used as a sealing influence point of the abnormal edge, and the traversal is continued backward. The absolute value of the difference between the light information value and the starting endpoint of the extension is also calculated. If it is greater than the information difference threshold, it is used as a sealing influence point. After obtaining several sealing influence points, the first pixel whose absolute value of the difference between the light information value and the starting endpoint of the extension is less than or equal to the information difference threshold appears. The subsequent traversal and the acquisition of sealing influence points are stopped, and the pixel is not used as a sealing influence point. Several sealing influence points of the abnormal edge are obtained.

[0030] For any sealing influence point, the inverse proportional normalization result of the distance between the sealing influence point and its corresponding extension start endpoint is used as the influence weight of the sealing influence point.

[0031] Optionally, the allowable variation range of the light information value of each pixel in the real-time milk powder packaging bag image is obtained by:

[0032] Based on the real-time milk powder packaging bag image corresponding to the template image of the packaging requirement type and the fluctuation of the light information value of the pixel point at the same position in different template images, the allowable light information value and the allowable adjustment range of each position in each template image corresponding to the packaging requirement type of the real-time milk powder packaging bag image are obtained;

[0033] If the pixel corresponding to any identical position in the template images of the real-time milk powder packaging bag image corresponding to the packaging requirement type is a sealing influence point, the difference obtained by subtracting the influence weight of the sealing influence point from 1 and multiplying the difference by the allowable adjustment range is used as the allowable influence range of the sealing influence point;

[0034] The real-time milk powder packaging bag image is mapped to each position under the corresponding packaging requirement type. For any pixel point in the real-time milk powder packaging bag image, if it is not a sealing-affecting point, the closed interval obtained by adding or subtracting the allowable adjustment range from the allowable light information value of the corresponding position is used as the allowable variation range of the light information value of the pixel point;

[0035] If the pixel point is a sealing influence point, the closed interval obtained by adding the allowable light information value at its corresponding position to the allowable influence range of the sealing influence point and subtracting it from the allowable influence range of the sealing influence point is used as the allowable variation range of the light information value of the sealing influence point.

[0036] Optionally, the specific method of obtaining the allowable light information value and the allowable adjustment range of each position in each template image corresponding to the packaging requirement type of the real-time milk powder packaging bag image includes:

[0037] For each template image of the packaging type corresponding to the real-time milk powder packaging bag image, obtain the standard deviation of the light information value of each pixel at any identical position in each template image, and linearly normalize the standard deviation at each position, and use the obtained result as the adjustment weight for that position;

[0038] The average of the light information values ​​of each pixel at any identical position in each template image is used as the allowable light information value of the position. The absolute value of the difference between the allowable light information value and the maximum value of the light information value of each pixel at the position is obtained, as is the absolute value of the difference between the allowable light information value and the minimum value of the light information value of each pixel at the position. The minimum value of the two absolute values ​​of the difference is used as the allowable fluctuation range of the position. The product of the allowable fluctuation range and the adjustment weight of the position is used as the allowable adjustment range of the position.

[0039] Optionally, the specific method of obtaining the real-time risk probability of the milk powder packaging bag image includes:

[0040] For any pixel in the real-time milk powder packaging bag image, if its light information value is within its allowable range, the pixel is marked as 0; if its light information value exceeds its allowable range, the pixel is marked as 1;

[0041] The ratio of the number of pixels marked as 1 in the real-time milk powder packaging bag image to the total number of pixels in the entire milk powder packaging bag image is counted, and the obtained ratio is used as the risk probability of the real-time milk powder packaging bag image.

[0042] The beneficial effects of the present invention are as follows: the present invention obtains light information values ​​of milk powder packaging bag templates of each packaging requirement type to obtain template images, and analyzes light information value fluctuations between template images to reflect the position of random changes in the packaging, thereby reducing its reflection effect; at the same time, the angle characteristics and length characteristics of the sealing edge of the template image are considered to extract the sealing influence points of the real-time milk powder packaging bag image, that is, the corresponding edge may be abnormal and reflected in the light information value changes of the pixel points in the extension direction; wherein the light information distribution map of each milk powder discharge image is obtained by the dark channel analysis method to reflect the light information distribution of each position therein, paving the way for subsequent reflection distribution analysis; at the same time, the detection image is selected for each packaging requirement type, and a template image is obtained based on its light information distribution; the sealing position edge is extracted by setting a calibration frame for the milk powder packaging bag image, and based on its angle characteristics, length characteristics and distribution position, the sealing position edge is analyzed by comparing with the corresponding packaging The edges in the template image under the demand type are compared to reflect the abnormal edges through edge distribution and morphological differences; the sealing influencing points and their influence weights are obtained through the changes in the light information values ​​of the pixels in the extension direction of the abnormal edges, reflecting the influence of the changes in their light information values ​​on the edge displacement of the sealing position. Combined with the standard deviation of the light information values ​​of the pixels at the same position in different template images under the corresponding packaging demand type, the allowable variation range is adjusted to reduce the influence of the distribution of the corresponding positions of each pixel in the real-time milk powder packaging bag image on the packaging bag morphology; finally, the risk probability judgment of the milk powder packaging bag image is realized based on the light information value of each pixel and its allowable variation range, and the risk discharge control of the milk powder packaging bag is carried out based on this, realizing the milk powder packaging bag assembly line detection based on deep learning combined with computer vision. The deep learning unit is seamlessly integrated into the production line control system to realize intelligent sorting, reducing the impact of quality problems caused by subsequent transportation packaging leakage. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 A schematic flow chart of a deep learning-based intelligent sorting method for milk powder packaging bags provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0046] See also Figure 1 , which shows a flow chart of a method for intelligent sorting of milk powder packaging bags based on deep learning provided by one embodiment of the present invention, the method comprising the following steps:

[0047] Step S001: Acquire several milk powder packaging bag images and classify them into several packaging requirement types.

[0048] Specifically, in the process of packaging milk powder, a packaging machine is used for packaging (the packaging machine consists of a feed panel, a packaging film running device, a packaging film paper tube core shaft, an inflation device, an end sealing knife seat, a discharge panel and a discharge port (two discharge ports, one for normal discharge port and the other for risk discharge port), a control device and an integrated electrical box. In order to ensure that the position of the packaging printing image and the shelf life is fixed during transportation, the feed panel and the discharge panel will have a fixing device for each batch of milk powder packaging bags.

[0049] Furthermore, a deep learning unit is integrated behind the discharge panel, and images are collected through the image acquisition module therein. By arranging a polarized light source and a high-definition industrial camera, the collected images are used as milk powder packaging bag images of the corresponding milk powder packaging bags, wherein the collection time interval is based on the transportation of milk powder on the assembly line, and the milk powder packaging bag images are input into the deep learning unit. The deep learning unit analyzes the leakage risk of the milk powder packaging bag and obtains the risk probability based on the packaging requirements of the milk powder packaging bag and the corresponding template image, and outputs the risk classification result of the milk powder packaging bag. At the same time, the screening module in the control device is connected, and the screening module controls the moving swing arm according to the output result of the deep learning unit to switch between the normal discharge port and the risk discharge port.

[0050] It should be noted that the leakage risk is determined by analyzing the shape of the packaging bag. Therefore, it is necessary to quantify the leakage risk by analyzing the difference between the shape distribution of the packaging under non-leakage and the shape distribution of the real-time milk powder packaging bag. Therefore, it is necessary to obtain the distribution of the packaging bag, that is, it is necessary to obtain the distribution template under normal conditions; and in the image acquisition process, the packaging bag is an irregular object with different curvatures at different positions on the surface, and different degrees of curvature of the surface will show different degrees of reflection due to the influence of light.

[0051] Furthermore, when the packaging machine is packaging milk powder bags, it is necessary to determine parameters such as packaging length and packaging speed, and classify the templates according to the parameters of the packaging machine. This embodiment adopts the existing classification method: for example, in the process of packaging milk powder bags with a packaging length less than 5 mm and a packaging speed not exceeding 1 bag / min, the milk powder bags are regarded as the same packaging requirement type. This embodiment classifies them by packaging length every 5 mm and packaging speed every 1 bag / min, thereby obtaining several packaging requirement types.

[0052] Step S002: Acquire several images of milk powder packaging bags of the same packaging requirement type as test images; obtain the light information value of each pixel in the test image based on the pixel value representation of the neighborhood pixel points in each color channel, construct a light information distribution map and use it as several template images of each packaging requirement type.

[0053] Preferably, in one embodiment of the present invention, a plurality of milk powder packaging bag images of milk powder packaging bag templates of the same packaging requirement type are obtained as the detection images, and the specific method includes:

[0054] It should be noted that, under each packaging requirement type, it is necessary to obtain a milk powder packaging bag that can be used as a template, so as to use the milk powder packaging bag image obtained after the discharge panel as the detection image of the corresponding packaging requirement type for subsequent processing.

[0055] Specifically, the requirements of this embodiment for the milk powder packaging bags used as templates are: the milk powder packaging bags are placed in a packaging box full of milk powder packaging bags and are squeezed for 24 hours without leakage. For any demand model, 30 milk powder packaging bags are required to serve as templates; the milk powder packaging bags used as templates are placed behind the discharge panel through image acquisition to obtain the milk powder discharge image, and this is used as the detection image of the demand model.

[0056] Preferably, in one embodiment of the present invention, based on the pixel value representation of the neighborhood pixel points in each color channel in the detection image, the light information value of each pixel point in the detection image is obtained, and a light information distribution map is constructed and used as a plurality of template images for each packaging requirement type. The specific method includes:

[0057] It should be noted that, according to the law of light scattering and surface reflection of objects, the optical model of the reflective area can be decomposed into diffuse reflection component and specular reflection component. The reflective area is a strong specular reflection generated under a certain lighting angle. Therefore, for high-reflectivity conditions, the specular reflection is dominant; for the milk powder discharge image, the reflection component can reflect the information difference in the multi-channel image. Therefore, there is a large gap between the reflective area in the image and the non-reflective area in the multi-channel. At the same time, if the packaging bag has wrinkles due to possible air leakage, it will also cause differences in information in the multi-channel. Therefore, in order to obtain the reflective distribution by analyzing the changes in multi-channel information, it is first necessary to obtain the light information distribution map of the milk powder discharge image and the detection image, that is, through the pixel value representation of each color channel in the neighborhood range, similar to the fog map formation model in the field of computer vision, to construct a light information distribution map.

[0058] Specifically, for any pixel point in any detection image of any packaging requirement type, the preset initial window size is , this embodiment adopts The description is made, and the expansion step is set to 2, that is, the side length increases by 2 each time the window is expanded; the absolute value of the difference between the pixel value of the pixel point and the other pixel points in the initial window centered on it under any color channel of the detection image is obtained as the pixel difference between the pixel point and the other pixel points in the initial window under the color channel; a pixel difference threshold and a window expansion threshold are preset. In this embodiment, the pixel difference threshold is described as 10, and the window expansion threshold is described as 2 / 3; if the number of pixels in the initial window whose corresponding pixel difference is less than the pixel difference threshold is greater than the window expansion threshold, the initial window needs to be expanded. If the corresponding pixel difference in the initial window is less than the pixel difference threshold, the initial window needs to be expanded. If the number of pixels less than the pixel difference threshold is less than or equal to the window expansion threshold, the initial window does not need to be expanded. Similarly, the expansion judgment of the initial window of the pixel is performed until the expansion is stopped when the condition is met. The window at the time of stopping the expansion is used as the neighborhood window of the pixel under the color channel; the window composed of several pixels belonging to the neighborhood window of the pixel under each color channel is used as the local window of the pixel; it is particularly noted that if the pixel is close to the boundary of the detection image and its (initial) window exceeds the range of the detection image, the actual existing pixels are used to constitute its (initial) window, and subsequent expansion judgments are made based on the actual existing pixels.

[0059] Furthermore, the minimum value of the pixel values ​​of each pixel point in the local window of the pixel point under any color channel is obtained as the information value of the pixel point under the color channel, and the minimum value of the information value of the pixel point under all color channels is used as the light information value of the pixel point; the light information value of each pixel point in the detection image is obtained, and the image composed of the light information values ​​of each pixel point is used as a template image of the packaging requirement type corresponding to the detection image; and several template images of each packaging requirement type are obtained according to the above method.

[0060] Furthermore, according to the above method, light information values ​​are obtained for each pixel point of each milk powder output image, thereby obtaining a light information distribution map of each milk powder output image.

[0061] At this point, the dark channel analysis method is used to obtain the light information distribution map of each milk powder output image to reflect the light information distribution at each position, paving the way for subsequent reflection distribution analysis; at the same time, the detection image is selected for each packaging requirement type, and the template image is obtained based on its light information distribution.

[0062] Step S003: Set a calibration frame for several template images of the same packaging requirement type and extract the edges in the calibration frame to obtain the angle characteristics and length characteristics of each edge of each template image of each packaging requirement type; obtain the angle characteristics and length characteristics of the real-time milk powder packaging bag image through the calibration frame and edge extraction, and compare them with the edges of the corresponding packaging requirement type, and mark several abnormal edges in the real-time milk powder packaging bag image.

[0063] It should be noted that since the shape of the packaging bag is irregular and the gas flows inside the packaging bag, different positions inside the packaging bag are affected differently by leakage. At the same time, during the leakage assessment process, if the packaging bag bulges or leaks, the sealing shape of the packaging bag will change. Therefore, the changes in the seal of each packaging bag can reflect the gas distribution inside the packaging bag. For all images, since the milk powder packaging bags are fixed on the feed panel and the discharge panel when the images are collected, by comparing the differences between all template images and the images collected in real time, the location of the difference corresponds to the location where the sealing shape changes. Since the illumination angle is fixed, and the characteristics of the packaging bag make the direction of each reflective area along one direction, and these directions may often be related to the deformation of the seal, it is necessary to exclude the influence of some deformations of the seal.

[0064] Preferably, in one embodiment of the present invention, a calibration frame is set for several template images of the same packaging requirement type and edges in the calibration frame are extracted to obtain angle features and length features of each edge of each template image of each packaging requirement type. The specific method includes:

[0065] Specifically, for any template image of any packaging requirement type, a fixed-size calibration frame is manually set for the template image, wherein the calibration frame is used to select the sealing positions on both sides of the packaging bag. The calibration process will not be described in this embodiment; edge detection is performed on any calibration frame to obtain several edges. It should be noted that the obtained edges are all truncated straight line edges, that is, the angle of the same edge is guaranteed to be consistent; the angle and length of any edge of the calibration frame are obtained, and the angle and length are normalized respectively, wherein the angle size is normalized with 360°, and the length size is normalized with the maximum length of each edge in the two calibration frames in the template image, and the normalized result obtained is used as the angle feature and length feature of the edge of the template image.

[0066] Furthermore, each edge of each template image of each packaging requirement type and its angle features and length features are obtained.

[0067] It should be noted that the edge of the sealing position is extracted through the calibration frame, and then the feature comparison is performed based on the edge under the same packaging requirement type. The greater the feature difference, the greater the possibility of edge abnormality, that is, the more likely it is that the edge will be displaced due to leakage; at the same time, multiple template images are used in the comparison to reduce the impact of occasional feature differences.

[0068] Preferably, in one embodiment of the present invention, the angle features and length features of the real-time milk powder packaging bag image are obtained by calibrating the frame and extracting the edge, and are compared with the edges of each template image of the corresponding packaging requirement type, and several abnormal edges in the real-time milk powder packaging bag image are marked. The specific method includes:

[0069] For real-time milk powder packaging bag images, several edges are extracted through calibration frames and the angle features and length features of each edge are obtained; a three-dimensional sample space of angle features, length features and edge center positions is constructed, where the edge center position is the corresponding coordinate of each edge in the corresponding image, and each edge of the real-time milk powder packaging bag image is mapped to the three-dimensional sample space according to its angle features and length features to obtain sample points. At the same time, each edge of each template image corresponding to the packaging requirement type of the real-time milk powder packaging bag image is mapped to the three-dimensional sample space to obtain data points, where each data point has a corresponding template image and a corresponding edge.

[0070] Furthermore, for any sample point, the Euclidean distance between the sample point and each data point is obtained, and the minimum value of the Euclidean distance between the sample point and several data points of the same template image is used as the matching edge difference between the sample point and the template image; the average value of the matching edge difference between the sample point and each template image is used as the matching abnormality degree of the edge corresponding to the sample point in the real-time milk powder packaging bag image; a matching abnormality threshold is preset, and the matching abnormality threshold in this embodiment is described as 0.3. If the matching abnormality degree is greater than the matching abnormality threshold, the edge is marked as an abnormal edge in the milk powder packaging bag image, and several abnormal edges in the milk powder packaging bag image are obtained.

[0071] It should be noted that the matching edge difference is used to screen the edges in each template image that have similar angle features, length features, and distribution positions to the edges corresponding to the sample points. The matching edge differences in different template images correspond to the same edges at the sealing position with a greater probability. The possibility of abnormal edges in the real-time milk powder packaging bag image is analyzed by the mean.

[0072] At this point, a calibration frame is set for the milk powder packaging bag image to extract the sealing position edge, and the edge is analyzed based on its angle characteristics, length characteristics and distribution position. The difference is compared with the edges in the template image under the corresponding packaging requirement type, so as to reflect the abnormal edge through the edge distribution and morphological differences.

[0073] Step S004: Obtain a light information distribution map for the real-time milk powder packaging bag image, and obtain a number of sealing influencing points and their influencing weights for each abnormal edge based on the changes in the light information values ​​of the pixels along the extension direction of the abnormal edge in the light information distribution map; and determine the allowable variation range of the light information value of each pixel in the real-time milk powder packaging bag image by combining the fluctuations in the light information values ​​of the pixels at the same position in the template image of the corresponding packaging requirement type and different template images.

[0074] It should be noted that the light information value of each pixel point is obtained according to the distribution of packaging bag shapes in all template images of the same type of milk powder packaging bag requirements, and the standard deviation of the light information value at each position is obtained. The larger the standard deviation value of each position, the more random the reflection of the light information distribution information at that position is. In the subsequent difference between the template image and the real-time collected image, the degree of comparison at the corresponding position needs to be reduced to prevent misjudgment, so as to adjust the allowable variation range. At the same time, if the change in the light information value of each pixel point in the collected image is more consistent with the position where the sealing shape changes, it indicates that the position of the pixel point has a greater risk when assessing the leakage risk, that is, through the change in the light information value of the pixel point in the extension direction of the abnormal edge, the sealing influence point is further obtained and the sealing influence weight is quantified.

[0075] Preferably, in one embodiment of the present invention, a light information distribution map is obtained for a real-time image of a milk powder packaging bag, and a number of sealing influencing points and their influencing weights of each abnormal edge are obtained based on the changes in light information values ​​of pixels along the extension direction of the abnormal edge in the light information distribution map. The specific method includes:

[0076] For any abnormal edge in the real-time milk powder packaging bag image, obtain the endpoint on one side of the packaging bag among the two end points of the abnormal edge (one side of the edge is close to the packaging bag, and the other side is close to the outside) as the extension starting endpoint of the abnormal edge; extend from the extension starting endpoint to the inside of the packaging bag in the milk powder packaging bag image along the angle of the abnormal edge, and calculate the absolute value of the difference between the light information value of each pixel passed by and the extension starting endpoint during the extension process, and preset the information difference threshold. In this embodiment, the information difference threshold is described as 10; judge pixel by pixel during the extension process, and when the first pixel that is different from the extension value appears, the pixel is judged. For pixels where the absolute value of the difference between the light information values ​​of the starting and ending points of the extension is greater than the information difference threshold, the pixel point is used as a sealing influence point of the abnormal edge, and the traversal continues backward. The absolute value of the difference between the light information values ​​and the starting and ending points of the extension is also calculated. If the absolute value is greater than the information difference threshold, it is also used as a sealing influence point. After obtaining several sealing influence points, the first pixel point whose absolute value of the difference between the light information value and the starting and ending points of the extension is less than or equal to the information difference threshold appears. The subsequent traversal and the acquisition of sealing influence points are stopped. At the same time, the pixel point is not used as a sealing influence point, and several sealing influence points of the abnormal edge are obtained.

[0077] Furthermore, for any sealing influence point, the inverse proportional normalization result of the distance between the sealing influence point and its corresponding extension starting end point is used as the influence weight of the sealing influence point; it should be noted that this embodiment adopts Model to present inverse proportional relationship and normalization processing, is an exponential function with a natural constant as its base, As the input of the model, the implementer can set the inverse proportional function and normalization function according to the actual situation. In order to avoid the inverse proportional normalization result being too small, the distance is multiplied by the hyperparameter 0.1 in the input model, and then the inverse proportional normalization result is obtained.

[0078] It should be further explained that after obtaining the sealing influence point and its influence weight, the allowable variation range of the light information value of the corresponding pixel point needs to be reduced according to the influence weight, that is, since it will have an obvious sealing influence in the real-time milk powder packaging bag image, the pixel value fluctuation range needs to be reduced to avoid changes that affect the edge changes of the sealing position; and each pixel point also needs to adjust the allowable variation range according to the fluctuation of the light information value of the pixel point at the corresponding position in the template image of the corresponding packaging requirement type. The larger the standard deviation, the more random the light information value, and the larger the allowable variation range needs to be. Similarly, the smaller the standard deviation, the smaller the allowable variation range needs to be to avoid the impact of changes in the light information value.

[0079] Preferably, in one embodiment of the present invention, the allowable variation range of the light information value of each pixel in the real-time milk powder packaging bag image is determined by combining the template image corresponding to the packaging requirement type and the fluctuation of the light information value of the pixel at the same position in different template images, and the specific method includes:

[0080] For each template image of the packaging requirement type corresponding to the real-time milk powder packaging bag image, the standard deviation of the light information value of each pixel at any identical position in each template image is obtained, and the standard deviation at each position is linearly normalized, and the result obtained is used as the adjustment weight of the position; the mean of the light information values ​​of each pixel at any identical position in each template image is used as the allowable light information value of the position, the absolute value of the difference between the allowable light information value and the maximum value of the light information value of each pixel at the position, and the absolute value of the difference between the allowable light information value and the minimum value of the light information value of each pixel at the position are obtained, and the minimum of the two obtained absolute values ​​of the difference is used as the allowable fluctuation range of the position; the product of the allowable fluctuation range and the adjustment weight of the position is used as the allowable adjustment range of the position; if the pixel corresponding to the position in the real-time milk powder packaging bag image is a sealing influence point, the difference obtained by subtracting the influence weight of the sealing influence point from 1 is multiplied by the allowable adjustment range as the allowable influence range of the sealing influence point.

[0081] Furthermore, the real-time milk powder packaging bag image is corresponded to each position under the corresponding packaging requirement type. For any pixel point in the real-time milk powder packaging bag image, if it is not a sealing influence point, the closed interval obtained by adding or subtracting the allowable light information value of the corresponding position from its allowable adjustment range is used as the allowable variation range of the light information value of the pixel point; if the pixel point is a sealing influence point, the closed interval obtained by adding or subtracting the allowable influence range of the sealing influence point from the allowable light information value of the corresponding position is used as the allowable variation range of the light information value of the sealing influence point. In this way, the allowable variation range of the light information value of each pixel point in the real-time milk powder packaging bag image is obtained.

[0082] At this point, the sealing influencing points and their influence weights are obtained by analyzing the changes in the light information values ​​of the pixels in the extension direction of the abnormal edge, reflecting the impact of the changes in their light information values ​​on the edge displacement of the sealing position. Combined with the standard deviation of the light information values ​​of the pixels at the same position in different template images under the corresponding packaging requirement type, the allowable variation range is adjusted to reduce the influence of the corresponding positions of each pixel in the real-time milk powder packaging bag image on the distribution of the packaging bag morphology.

[0083] Step S005: The light information value of each pixel in the real-time milk powder packaging bag image is judged according to its allowable variation range to obtain the risk probability of the real-time milk powder packaging bag image, and the real-time milk powder packaging bags are intelligently sorted according to the risk probability.

[0084] Specifically, for each pixel in the real-time milk powder packaging bag image, the light information value and the corresponding allowable variation range have been obtained. If the light information value of any pixel is within its allowable variation range, the pixel is marked as 0; if its light information value exceeds its allowable variation range, the pixel is marked as 1; the number of pixels marked as 1 in the real-time milk powder packaging bag image is counted as a proportion of the number of all pixels in the entire milk powder packaging bag image, and the obtained ratio is used as the risk probability of the real-time milk powder packaging bag image; the risk warning threshold in this embodiment is set to the number of all pixels 1 / 2 of the total risk. If the risk probability is greater than the risk warning threshold, the milk powder packaging bag discharged in real time by the discharge panel may have a leak, and the risk classification result output by the deep learning unit is that there is a risk, and the intelligently controlled mobile swing arm transfers the milk powder packaging bag to the risk discharge port; if the risk probability is less than or equal to the risk warning threshold, the risk classification result output by the deep learning unit is that there is no risk, and the intelligently controlled mobile swing arm transfers the milk powder packaging bag to the normal discharge port, where the risk probability judgment result is the risk classification result, and is input into the control device for intelligent sorting and control of the mobile swing arm.

[0085] At this point, the light information value of the milk powder packaging bag template of each packaging requirement type is obtained to obtain the template image, and the light information value fluctuation between the template images is analyzed to reflect the position of random changes in the packaging, thereby reducing its reflection effect; at the same time, the angle characteristics and length characteristics at the sealing edge of the template image are considered to extract the sealing influence points of the real-time milk powder packaging bag image, that is, the corresponding edge may have anomalies and is reflected in the light information value changes on the pixel points in the extension direction. Finally, the risk probability judgment of the milk powder packaging bag image is realized based on the light information value of each pixel point and its allowable variation range, and the risk discharge control of the milk powder packaging bag is carried out based on this, realizing the milk powder packaging bag assembly line detection based on deep learning combined with computer vision. The deep learning unit is seamlessly integrated into the production line control system to realize intelligent sorting, reducing the impact of quality problems caused by subsequent transportation packaging leakage.

[0086] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An intelligent sorting method for milk powder packaging bags based on deep learning, characterized in that: The method comprises the following steps: Obtain several milk powder packaging bag images and classify them into several packaging requirement types; Acquire several images of milk powder packaging bags of the same packaging requirement type as test images; obtain the light information value of each pixel in the test image based on the pixel value representation of the neighborhood pixels in each color channel, construct a light information distribution map and use it as several template images of each packaging requirement type; A calibration frame is set for several template images of the same packaging requirement type and edges in the calibration frame are extracted. The angle characteristics and length characteristics of each edge of each template image of each packaging requirement type, as well as the angle characteristics and length characteristics of each edge of the real-time milk powder packaging bag image, are obtained. The edges are compared with the edges of each template image of the corresponding packaging requirement type, and several abnormal edges in the real-time milk powder packaging bag image are marked; Obtaining a light information distribution map from a real-time milk powder packaging bag image, and determining several sealing influence points and their influence weights for each abnormal edge based on the changes in light information values ​​of pixels along the direction of the abnormal edge in the light information distribution map. Determining the allowable variation range of the light information value of each pixel in the real-time milk powder packaging bag image by combining fluctuations in light information values ​​of pixels at the same position in template images corresponding to the packaging requirement type and different template images; The light information value of each pixel in the real-time milk powder packaging bag image is judged according to its allowable variation range to obtain the risk probability of the real-time milk powder packaging bag image, and the real-time milk powder packaging bags are intelligently sorted according to the risk probability.

2. The method for intelligent sorting of milk powder packaging bags based on deep learning according to claim 1, characterized in that: The light information value of each pixel in the detection image is obtained based on the pixel value representation of the neighborhood pixel in each color channel, including the specific method of: Based on the pixel value representation of the neighborhood pixel points in each color channel in the detection image, a local window of each pixel point in the detection image is obtained; Obtain the minimum value of the pixel values ​​of each pixel in the local window of any pixel under any color channel in any detection image as the information value of the pixel under the color channel, and the minimum value of the information value of the pixel under all color channels as the light information value of the pixel.

3. The method for intelligent sorting of milk powder packaging bags based on deep learning according to claim 2, characterized in that: The specific method for obtaining the local window of each pixel in the detection image is as follows: For any pixel point in any inspection image of any packaging requirement type, preset the initial window size and expansion step size; obtain the absolute value of the difference between the pixel value of the pixel point and the other pixels in the initial window centered on it in any color channel of the inspection image as the pixel difference between the pixel point and the other pixels in the initial window in that color channel; If the number of pixels in the initial window whose corresponding pixel differences are less than the pixel difference threshold is greater than the window expansion threshold, the initial window is expanded. If the number of pixels in the initial window whose corresponding pixel differences are less than the pixel difference threshold is less than or equal to the window expansion threshold, the initial window is not expanded. Similarly, the expansion of the initial window of the pixel is judged until the expansion is stopped when the conditions are met. The window at the time of stopping expansion is used as the neighborhood window of the pixel under the color channel; the window composed of several pixels that belong to the neighborhood window of the pixel under each color channel is used as the local window of the pixel.

4. The method for intelligent sorting of milk powder packaging bags based on deep learning according to claim 1, characterized in that: The specific method of constructing the light information distribution map and using it as a plurality of template images for each packaging requirement type includes: The light information value of each pixel in any detection image of any packaging requirement type is obtained, and an image composed of the light information values ​​of each pixel is used as a template image of the packaging requirement type corresponding to the detection image.

5. The method for intelligent sorting of milk powder packaging bags based on deep learning according to claim 1, characterized in that: The specific method of obtaining the angle characteristics and length characteristics of each edge of each template image of each packaging requirement type includes: For any template image of any packaging requirement type, a fixed-size calibration frame is set for the template image; Perform edge detection on any calibration frame to obtain several edges; The angle and length of any edge of the calibration frame are obtained, and the angle and length are normalized respectively. The normalized results are used as the angle feature and length feature of the edge of the template image.

6. The method for intelligent sorting of milk powder packaging bags based on deep learning according to claim 1, characterized in that: The specific method of marking several abnormal edges in the real-time milk powder packaging bag image includes: For real-time milk powder packaging bag images, several edges are extracted through calibration frames and the angle and length features of each edge are obtained; Constructing a three-dimensional sample space of angle features, length features, and edge center positions, where the edge center positions are the coordinates of each edge in the corresponding image. Mapping each edge of the real-time milk powder packaging bag image to the three-dimensional sample space based on its angle features and length features to obtain sample points. Mapping each edge of each template image corresponding to the packaging requirement type of the real-time milk powder packaging bag image to the three-dimensional sample space to obtain data points. For any sample point, obtain the Euclidean distance between the sample point and each data point, and use the minimum value of the Euclidean distance between the sample point and several data points of the same template image as the matching edge difference between the sample point and the template image; The average of the differences between the matching edges of the sample point and each template image is used as the matching abnormality degree of the edge corresponding to the sample point in the real-time milk powder packaging bag image; if the matching abnormality degree is greater than the matching abnormality threshold, the edge is marked as an abnormal edge in the milk powder packaging bag image.

7. The method for intelligent sorting of milk powder packaging bags based on deep learning according to claim 1, characterized in that: The specific method for obtaining the several sealing influence points and their influence weights of each abnormal edge includes: For any abnormal edge in the real-time milk powder packaging bag image, obtain the endpoint located on the packaging bag side among the two end points of the abnormal edge as the extension starting endpoint of the abnormal edge; Extending from the extension start point toward the interior of the milk powder packaging bag in the milk powder packaging bag image along the angle of the abnormal edge, calculating the absolute value of the difference between the light information value of each pixel passed through and the extension start point during the extension process; During the extension process, each pixel is judged one by one. When the first pixel whose absolute value of the difference between the light information value and the starting endpoint of the extension is greater than the information difference threshold appears, the pixel is used as a sealing influence point of the abnormal edge, and the traversal is continued backward. The absolute value of the difference between the light information value and the starting endpoint of the extension is also calculated. If it is greater than the information difference threshold, it is used as a sealing influence point. After obtaining several sealing influence points, the first pixel whose absolute value of the difference between the light information value and the starting endpoint of the extension is less than or equal to the information difference threshold appears. The subsequent traversal and the acquisition of sealing influence points are stopped, and the pixel is not used as a sealing influence point. Several sealing influence points of the abnormal edge are obtained. For any sealing influence point, the inverse proportional normalization result of the distance between the sealing influence point and its corresponding extension start endpoint is used as the influence weight of the sealing influence point.

8. The method for intelligent sorting of milk powder packaging bags based on deep learning according to claim 1, characterized in that: The specific method for obtaining the allowable variation range of the light information value of each pixel in the real-time milk powder packaging bag image is as follows: Based on the real-time milk powder packaging bag image corresponding to the template image of the packaging requirement type and the fluctuation of the light information value of the pixel point at the same position in different template images, the allowable light information value and the allowable adjustment range of each position in each template image corresponding to the packaging requirement type of the real-time milk powder packaging bag image are obtained; If the pixel corresponding to any identical position in the template images of the real-time milk powder packaging bag image corresponding to the packaging requirement type is a sealing influence point, the difference obtained by subtracting the influence weight of the sealing influence point from 1 and multiplying the difference by the allowable adjustment range is used as the allowable influence range of the sealing influence point; The real-time milk powder packaging bag image is mapped to each position under the corresponding packaging requirement type. For any pixel point in the real-time milk powder packaging bag image, if it is not a sealing-affecting point, the closed interval obtained by adding or subtracting the allowable adjustment range from the allowable light information value of the corresponding position is used as the allowable variation range of the light information value of the pixel point; If the pixel point is a sealing influence point, the closed interval obtained by adding the allowable light information value at its corresponding position to the allowable influence range of the sealing influence point and subtracting it from the allowable influence range of the sealing influence point is used as the allowable variation range of the light information value of the sealing influence point.

9. The method for intelligent sorting of milk powder packaging bags based on deep learning according to claim 8, characterized in that: The specific method of obtaining the allowable light information value and the allowable adjustment range of each position in each template image corresponding to the packaging requirement type of the real-time milk powder packaging bag image includes: For each template image of the packaging type corresponding to the real-time milk powder packaging bag image, obtain the standard deviation of the light information value of each pixel at any identical position in each template image, and linearly normalize the standard deviation at each position, and use the obtained result as the adjustment weight for that position; The average of the light information values ​​of each pixel at any identical position in each template image is used as the allowable light information value of the position. The absolute value of the difference between the allowable light information value and the maximum value of the light information value of each pixel at the position is obtained, as is the absolute value of the difference between the allowable light information value and the minimum value of the light information value of each pixel at the position. The minimum value of the two obtained absolute values ​​of the difference is used as the allowable fluctuation range of the position. The product of the allowable fluctuation range and the adjustment weight of the position is used as the allowable adjustment range of the position.

10. The method for intelligent sorting of milk powder packaging bags based on deep learning according to claim 1, characterized in that: The specific method for obtaining the risk probability of the real-time milk powder packaging bag image is as follows: For any pixel in the real-time milk powder packaging bag image, if its light information value is within its allowable range, the pixel is marked as 0; if its light information value exceeds its allowable range, the pixel is marked as 1; The ratio of the number of pixels marked as 1 in the real-time milk powder packaging bag image to the total number of pixels in the entire milk powder packaging bag image is counted, and the obtained ratio is used as the risk probability of the real-time milk powder packaging bag image.

Images