Packaging visual inspection method and platform for product conveyor belt

Abstract

The invention discloses a packaging visual detection method and platform for a product conveyor belt, and relates to the technical field of visual detection, and the method comprises the steps: carrying out the lateral lighting and image collection of the product conveyor belt at a plurality of positions according to a plurality of angles in a process of conveying a product through the product conveyor belt, and obtaining a plurality of side light images; performing image quality analysis to obtain a plurality of pieces of image quality information; performing integrated packaging identification on the plurality of gain images to obtain a plurality of packaging identification results; and performing matching degree identification on the plurality of gain images to obtain a matching degree, and obtaining a total packaging identification result. The technical problem that in the prior art, when the product conveying belt changes from black to white, a transparent film is packaged on the white product conveying belt, and whether packaging is completed or not is difficult to judge on the packaging edge in a visual detection mode is solved, accurate detection and judgment on the packaging condition of the product conveying belt are achieved, and the packaging efficiency is improved. And the accuracy and reliability of packaging detection are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of visual inspection, and in particular to a packaging visual inspection method and platform for a product conveyor belt. Background Art

[0002] In the field of packaging inspection of product conveyor belts, traditional visual inspection methods encounter difficulties when facing color changes of product conveyor belts. When the product conveyor belt changes from black to white, it is difficult to determine whether the packaging is completed by visual inspection of the packaging edge when a transparent film is packaged on the white product conveyor belt. This is because in the existing technology, factors such as the lighting direction may cause the edge of the transparent film to be insufficiently distinguished from the edge of the conveyor belt against the background of the white conveyor belt, thus affecting the accurate judgment of the packaging situation. This situation will lead to unstable product quality in some production scenarios with high requirements for packaging quality, affecting production efficiency and product qualification rate.

[0003] In the prior art, when the product conveyor belt changes from black to white, a transparent film is packaged on the white product conveyor belt, and it is difficult to determine whether the packaging is completed by visual inspection of the packaging edge. Summary of the invention

[0004] The present application provides a packaging visual inspection method and platform for a product conveyor belt, which is used to solve the technical problem in the prior art that when the product conveyor belt changes from black to white, a transparent film is packaged on the white product conveyor belt, and it is difficult to determine whether the packaging is completed by visual inspection of the packaging edge.

[0005] In view of the above problems, the present application provides a packaging visual inspection method and platform for product conveyor belts.

[0006] In a first aspect of the present application, a packaging visual inspection method for a product conveyor belt is provided, the method comprising: In the process of conveying products by using a product conveyor belt, the product conveyor belt is laterally illuminated and images are collected at multiple positions and angles to obtain multiple side-lit images; the multiple side-lit images are input into an image gain model for processing, and a corresponding multiple gain images are output; image quality analysis is performed on the multiple gain images to obtain multiple image quality information; based on the multiple image quality information, integrated packaging recognition is performed on the multiple gain images to obtain multiple packaging recognition results, wherein each packaging recognition result includes whether it is packaged; matching degree recognition is performed on the multiple gain images to obtain matching degrees, and the multiple packaging recognition results are summarized and corrected in combination with the multiple image quality information to obtain a total packaging recognition result.

[0007] The second aspect of the present application provides a packaging visual inspection platform for a product conveyor belt, the platform comprising: A side light image acquisition module, wherein the side light image acquisition module is used to perform side lighting and image acquisition on the product conveyor belt at multiple positions and at multiple angles during the process of conveying products using the product conveyor belt, so as to obtain multiple side light images; a gain image acquisition module, wherein the gain image acquisition module is used to input the multiple side light images into an image gain model for processing, and output corresponding multiple gain images; an image quality information acquisition module, wherein the image quality information acquisition module is used to perform image quality analysis on the multiple gain images, so as to obtain multiple image quality information; a packaging recognition result acquisition module, wherein the packaging recognition result acquisition module is used to perform integrated packaging recognition on the multiple gain images according to the multiple image quality information, so as to obtain multiple packaging recognition results, wherein each packaging recognition result includes whether the product is packaged; and a total packaging recognition result acquisition module, wherein the total packaging recognition result acquisition module is used to perform matching degree recognition on the multiple gain images, so as to obtain matching degrees, and to summarize and correct the multiple packaging recognition results in combination with the multiple image quality information, so as to obtain a total packaging recognition result.

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages: In the process of using a product conveyor belt to convey products, the product conveyor belt is illuminated from the side and images are collected at multiple positions and angles to obtain multiple side-light images; image quality analysis is performed on the multiple gain images to obtain multiple image quality information; integrated packaging recognition is performed on the multiple gain images to obtain multiple packaging recognition results; matching degree recognition is performed on the multiple gain images to obtain matching degrees, and the multiple packaging recognition results are summarized and corrected to obtain the total packaging recognition result. The technical effect of accurately detecting and judging the packaging situation of the product conveyor belt and improving the accuracy and reliability of packaging detection is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A schematic diagram of a packaging visual inspection method for a product conveyor belt provided in an embodiment of the present application; Figure 2 Schematic diagram of the structure of the packaging visual inspection platform for the product conveyor belt provided in an embodiment of the present application.

[0011] Explanation of reference numerals: side light image acquisition module 10 , gain image acquisition module 20 , image quality information acquisition module 30 , package recognition result acquisition module 40 , total package recognition result acquisition module 50 . DETAILED DESCRIPTION

[0012] The present application provides a product conveyor belt packaging visual inspection method and platform, which is used to solve the technical problem in the prior art that when the product conveyor belt changes from black to white, a transparent film is packaged on the white product conveyor belt, and it is difficult to determine whether the packaging is completed by visual inspection of the packaging edge.

[0013] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0014] Embodiment 1, as Figure 1 As shown, the present application provides a packaging visual inspection method for a product conveyor belt, the method comprising: Step S100: When the product conveyor belt is used to convey the product, the product conveyor belt is illuminated from the side and images are captured at multiple positions and angles to obtain multiple side-lit images.

[0015] Specifically, in the process of product transmission, in order to solve the problem that the transparent packaging film is difficult to accurately identify against the background of the white conveyor belt, the method of side lighting and image acquisition is adopted. When the product is transported on the white conveyor belt, the conveyor belt is illuminated from the side at multiple specific angles at multiple different positions of the conveyor belt. When lighting, blue light is selected as the light source, which is determined based on optical principles and a large number of experiments. When blue light is irradiated on white products, it can produce a good optical reaction with the material and surface characteristics of the product, making the edge image of the white product clearer, highlighting the edge details, and providing a high-quality image basis for subsequent image analysis and recognition. Because the white conveyor belt is illuminated from the side, the wrinkles, deformation and other characteristics of the packaging film will produce unique shadow effects. These shadows become an important basis for judging whether it is encapsulated. At the same time, at each position where the conveyor belt is illuminated from the side, an image acquisition operation is performed at an angle perpendicular to the product conveyor belt. Through this method of side lighting and corresponding image acquisition at multiple positions and multiple angles, multiple side-lit images are finally obtained. These side-lit images contain rich characteristic information about the packaging film under side lighting, providing a key data basis for subsequent further image quality analysis and packaging recognition operations.

[0016] Step S200: inputting the plurality of side-light images into an image gain model for processing, and outputting a plurality of corresponding gain images.

[0017] Specifically, these side-light images are sequentially input into an image gain model based on a convolutional neural network. The convolutional neural network includes multiple convolutional layers, pooling layers, and fully connected layers. In the convolutional layer, many convolution kernels slide on the side-light image, which can keenly capture local features such as edges and textures in the image. For example, for white products, the convolution kernel can accurately lock its edge information and enhance the originally weak or blurred edge signals. After continuous processing by multiple layers of convolutional layers, the features of the image are gradually refined and enhanced. The pooling layer downsamples the convolutional image. Taking the maximum pooling as an example, it greatly reduces the amount of data without losing key features, improves the computational efficiency of the model, and further highlights the extracted important features, such as the key pixel information of the edge of the white product. Finally, the fully connected layer integrates the features processed by convolution and pooling, performs operations according to the trained parameters, maps the processed image to a specific output space, and then outputs multiple gain images. Compared with the original side-lit images, these gain images have clearer images of the edges of white products, providing a higher-quality and more accurate image basis for subsequent image quality analysis and product packaging identification, significantly improving the accuracy and reliability of the entire inspection process.

[0018] Step S300: performing image quality analysis on the multiple gain images to obtain multiple pieces of image quality information.

[0019] Specifically, image quality analysis is performed on multiple gain images to obtain multiple image quality information, and the speed and vibration amplitude data of the product conveyor belt when it is conveyed at multiple positions are obtained through speed sensors and vibration sensors. Since image quality is closely related to the conveying speed and vibration amplitude, the higher the conveying speed and the larger the vibration amplitude, the lower the image quality. Based on this principle, the ratio of the standard conveying speed to the conveying speed at each position is calculated respectively, thereby obtaining multiple speed quality information; at the same time, the ratio of the standard vibration amplitude to the vibration amplitude at each position is calculated respectively, thereby obtaining multiple vibration quality information. Finally, these speed quality information and vibration quality information are weighted and calculated to obtain multiple image quality information that can reflect the quality status of the gain image.

[0020] Step S400: performing integrated packaging identification on the multiple gain images according to the multiple image quality information to obtain multiple packaging identification results, wherein each packaging identification result includes whether it is packaged.

[0021] Specifically, multiple gain images are integrated for package recognition based on the multiple image quality information obtained above to obtain multiple package recognition results. First, based on the package detection data of the product conveyor belt, a package recognition channel is constructed with the help of integrated learning and machine vision, and this channel contains multiple package recognition paths. Then, the number of paths used for integrated recognition is determined according to the image quality information. When the image quality is higher, in order to improve the recognition accuracy, the number of recognition paths set is larger; conversely, if the image quality is lower, in order to save computing power, the number of recognition paths is smaller. The image quality information is multiplied by the total number of package recognition paths and rounded to obtain the number of recognition paths corresponding to each gain image. Then, according to the number of recognition paths obtained, the corresponding number of package recognition path sets are randomly selected. After that, the multiple gain images are respectively input into the corresponding package recognition path sets for recognition, thereby obtaining multiple path package recognition result sets. Finally, these path package recognition result sets are respectively subjected to majority voting, and finally multiple package recognition results are obtained, each of which contains information on whether it is encapsulated, where 1 indicates that there is a package film and 0 indicates that there is no package film.

[0022] Step S500: performing matching degree identification on the multiple gain images to obtain matching degrees, and combining the multiple image quality information to summarize and correct the multiple package identification results to obtain a total package identification result.

[0023] Specifically, the matching degree of multiple gain images is identified and the package recognition results are summarized and corrected in combination with the image quality information to obtain the total package recognition result. First, the matching degree of multiple gain images is identified. Since there is a matching relationship between the gain images at different positions, for example, the shadow formed by a wrinkle of the package film at the first position and the first angle should be symmetrical with the shadow formed at the second angle corresponding to the second position. Based on this feature, the matching degree of multiple gain images is analyzed, and the matching degree can reflect the accuracy of image acquisition and recognition. Combined with multiple image quality information, the multiple package recognition results are summarized and corrected, and the ratio of each image quality information to the sum of multiple image quality information is calculated to obtain multiple image weights. These image weights are used to perform weighted calculation on multiple package recognition results to obtain a summary package recognition result. Then, the difference between 1 and the matching degree is used as the correction coefficient to compensate and correct the summary package recognition result to obtain the package recognition result interval. Finally, it is determined whether the package recognition result interval is greater than or equal to the package recognition result threshold. If so, the total package recognition result of the package is obtained, and if not, the total package recognition result of the unpackaged package is obtained.

[0024] In a possible implementation, step S100 further includes: Step S110: When the product conveyor belt is used to convey the product, the product conveyor belt is laterally illuminated at multiple positions according to multiple lateral angles, wherein the angles between the multiple lateral angles and the direction perpendicular to the product conveyor belt are less than 90 degrees.

[0025] Step S120: capturing an image of the product conveyor belt at an angle perpendicular to the product conveyor belt to obtain a plurality of gain images.

[0026] Specifically, in the process of the product being conveyed by the product conveyor belt, in order to better obtain the characteristic information related to the conveyor belt for subsequent detection and analysis, it is necessary to implement lateral lighting operations on the product conveyor belt at multiple positions. The multiple positions referred to here are several points that are evenly distributed along the length direction of the conveyor belt or have a specific distribution pattern according to actual needs. For the angle of lateral lighting, it is required that the angle between multiple lateral angles and the direction perpendicular to the product conveyor belt is less than 90 degrees. This angle setting is carefully considered, because when the light is irradiated on the conveyor belt at such a lateral angle, it can highlight some detailed features of the conveyor belt surface and the packaging film that may exist. For example, the wrinkles and deformation of the packaging film will produce unique shadow effects under the irradiation of this lateral light. These shadows are of great reference value for subsequent judgment of whether the packaging is completed and the quality of the packaging. At the same time, multiple different lateral angles can observe the conveyor belt and the packaging film from different sides, further enriching the information obtained and providing guarantees for more accurate detection and identification.

[0027] Image acquisition is performed in a direction perpendicular to the product conveyor belt. This is because the angle perpendicular to the conveyor belt can most accurately present the actual status of the conveyor belt surface and related items thereon (such as products, packaging films, etc.). At each position where side lighting is performed, images are acquired in a vertical direction. Since side lighting is performed at multiple positions in step S110, an image can be acquired at each corresponding position. These images acquired from multiple positions have unique light and shadow characteristics due to the previous side lighting, and together they constitute multiple gain images. These gain images will serve as the basic data for subsequent image analysis and processing, which is crucial for accurately judging the packaging condition of the product conveyor belt.

[0028] In a possible implementation, step S300 further includes: Step S310: collecting multiple conveying speeds and multiple conveying vibration amplitudes of the product conveyor belt passing through the multiple positions through a speed sensor and a vibration sensor.

[0029] Step S320: Calculate the ratios of the standard transmission speed and the multiple transmission speeds respectively to obtain multiple speed quality information.

[0030] Step S330: Calculate the ratios of the standard vibration amplitude and the multiple transmission vibration amplitudes respectively to obtain multiple vibration quality information.

[0031] Step S340: weighted calculation of the plurality of velocity quality information and the plurality of vibration quality information to obtain a plurality of image quality information.

[0032] Specifically, in order to obtain relevant data that can reflect the impact of the running status of the product conveyor belt on the image quality, speed sensors and vibration sensors are used for collection. The speed sensor is installed at the relevant position of the product conveyor belt. When the conveyor belt passes through multiple set positions in the process of conveying products, the speed sensor can accurately measure the conveying speed corresponding to each position. These conveying speed data are one of the important bases for analyzing image quality, because different conveying speeds may cause image blurring, deformation and other quality problems. At the same time, the vibration sensor is also reasonably placed near the conveyor belt. When the conveyor belt passes through multiple positions during operation, the vibration sensor will record the conveying vibration amplitude corresponding to each position. The conveying vibration amplitude also has an important impact on image quality. A larger vibration amplitude may cause the image to produce adverse effects such as shaking and noise. Through the coordinated work of the speed sensor and the vibration sensor, multiple conveying speeds and multiple conveying vibration amplitudes of the product conveyor belt passing through multiple positions can be fully and accurately collected, providing the necessary data basis for subsequent analysis of image quality.

[0033] A standard conveying speed is determined, which is set based on the speed of the product conveyor belt under ideal operating conditions. Then, for multiple conveying speeds of the product conveyor belt passing through multiple positions previously collected by the speed sensor, the ratio of the standard conveying speed to each conveying speed is calculated respectively. By performing such calculations for each position, multiple speed quality information is finally obtained. These speed quality information can reflect the degree of deviation of the conveying speed at each position from the standard conveying speed, and can be used to evaluate the speed-related quality of the image collected at that position.

[0034] Define a standard vibration amplitude, which represents the vibration condition of the product conveyor belt under ideal conditions. Then, for the multiple transmission vibration amplitudes of the product conveyor belt passing through multiple positions collected by the vibration sensor, calculate the ratio of the standard vibration amplitude to each transmission vibration amplitude. By performing such calculations for each position, multiple vibration quality information can be obtained in the end. These vibration quality information can reflect the degree of deviation of the vibration amplitude at each position from the standard vibration amplitude, and are thus used to evaluate the vibration-related quality of the image collected at that position.

[0035] In order to comprehensively consider the impact of speed and vibration on image quality, it is necessary to perform weighted calculation on multiple speed quality information and multiple vibration quality information to obtain multiple image quality information. The weights of speed quality information and vibration quality information in image quality assessment are determined based on a large amount of experimental data and the understanding of the operating characteristics of the product conveyor belt. Then, for the speed quality information and vibration quality information corresponding to each position, weighted calculation is performed according to the established weights. For example, let the speed quality information be V q , whose weight is W v , the vibration quality information is a q , whose weight is W a , then the image quality information I at this position q Formula I q =W v ×V q +W a ×a q By performing such weighted calculations on each position, multiple image quality information is finally obtained. These image quality information can comprehensively reflect the image quality of the product conveyor belt at each position, providing an important basis for subsequent integrated packaging recognition and other operations.

[0036] In a possible implementation, step S400 further includes: Step S410: Based on the packaging detection data of the product conveyor belt, based on integrated learning and machine vision, a packaging identification channel is constructed, wherein the packaging identification channel includes a plurality of packaging identification paths.

[0037] Step S420: multiplying the plurality of image quality information by the total number of the plurality of package identification paths and rounding the result to obtain the plurality of package identification path numbers.

[0038] Step S430: According to the number of the multiple package identification paths, a plurality of package identification path sets of the multiple package identification path numbers are randomly selected, the plurality of gain images are respectively input into the plurality of package identification path sets, and a plurality of path package identification result sets are identified to obtain.

[0039] Step S440: performing majority voting processing on the plurality of path encapsulation identification result sets respectively to obtain a plurality of encapsulation identification results.

[0040] Specifically, for the ensemble learning part, a large amount of product conveyor belt packaging inspection data covering various packaging conditions and corresponding conveyor belt feature information needs to be collected. For example, the data may contain packaging records under different product types, different packaging materials, different conveyor belt speeds and vibration conditions. These data are preprocessed, including removing erroneous data and outliers, and standardizing the data to ensure that different feature data are in the same dimension so that subsequent algorithms can better process them. The decision tree ensemble method is used to randomly extract multiple sub-datasets from the preprocessed data set, and a decision tree is built for each sub-dataset. When building a decision tree, the best split feature is selected from a random subset of all features each time to increase the diversity of the decision tree, and finally a random forest model composed of multiple decision trees is formed. Each decision tree can give a prediction result for the packaging condition, and the combination of these results can obtain a more accurate and stable prediction. For the machine vision part, a convolutional neural network (CNN) is used for feature extraction. The convolution layer of CNN slides the convolution kernel on the image to perform convolution operations and automatically extract local features in the image, such as edges and textures. The pooling layer compresses and reduces the dimension of the features extracted by the convolution layer, reducing the amount of data while retaining key features. After multiple layers of convolution and pooling operations, a feature vector containing rich image features is finally obtained. Then, the feature vector extracted from CNN is combined with the model obtained by the ensemble learning algorithm. For example, the feature vector is used as input and input into the random forest model, allowing the model to classify and predict the packaging situation based on these feature vectors. Through the comprehensive application of the above ensemble learning and machine vision technology, a package recognition channel containing multiple package recognition paths is constructed, and each path can accurately identify the package situation from different angles.

[0041] The obtained multiple image quality information are obtained, which comprehensively reflects the quality status of the image, including the impact of factors such as conveyor belt speed and vibration on the image. Then, each image quality information is multiplied by the total number of preset package recognition paths. This total number is determined when building the package recognition channel, and it represents the sum of all paths that can be used for recognition. Finally, the product result is rounded, and the integer value obtained is the number of package recognition paths corresponding to the image. In this way, according to the different image quality, different numbers of package recognition paths are reasonably allocated to each gain image. More paths may be allocated to images with good image quality to improve recognition accuracy, while fewer paths are allocated to images with poor image quality to save computing power.

[0042] For each gain image, the operation is performed according to the number of corresponding package recognition paths. For example, if the number of package recognition paths for a certain gain image is n, n paths are randomly selected from all available package recognition path sets, and these paths constitute a package recognition path set for the gain image. Then, the gain image is input into this specific package recognition path set, in which each path has its own unique recognition method and algorithm model. For example, some paths are based on convolutional neural network (CNN) algorithms, which can automatically extract various features in the image, such as edges, textures, etc., and judge whether the packaging of the product conveyor in the image meets the requirements based on these features. Other paths are based on decision tree algorithms, which classify and judge based on some predefined feature attributes of the image, such as color distribution, shape features, etc., to determine whether it is packaged. Through the independent recognition processing of the gain image by each path, a package recognition result of the image under this path will be obtained. Since there are n paths, a set of n recognition results will be obtained in the end, which is the path package recognition result set for the gain image. The above process is repeated for each gain image, that is, determining the number of its package recognition paths, randomly selecting a path set, inputting the image and obtaining the corresponding path package recognition result set. In this way, we will eventually obtain multiple path package recognition result sets, which will provide basic data for the subsequent majority voting process to further determine the final package recognition result.

[0043] A majority vote is performed on each set of path package recognition results. Since each set is composed of the results obtained by recognizing the gain image on multiple paths, the number of times each result appears in the set must be counted. For example, if there are three results in a path package recognition result set, namely "packaged", "unpackaged", and "packaged", then the result "packaged" appears the most times, and it is determined as the package recognition result corresponding to the set. All path package recognition result sets are processed in this way, and multiple package recognition results are finally obtained. These results will provide a basis for further judging the packaging status of the product conveyor belt.

[0044] In a possible implementation, step S410 further includes: Step S411: Based on the packaging detection data of the product conveyor belt, a set of sample polarization images is collected, and whether the product conveyor belt is packaged in each sample polarization image is marked to obtain a set of sample packaging recognition results, where the sample packaging recognition results include 1 or 0, 1 for packaged and 0 for unpackaged.

[0045] Step S412: Based on ensemble learning, the sample polarization image set and the sample package recognition result set are divided to obtain multiple groups of package recognition supervision training data.

[0046] Step S413: constructing a plurality of package identification paths in the package identification channel based on a convolutional neural network in machine vision.

[0047] Step S414: Based on ensemble learning, the plurality of groups of package identification supervision training data are respectively used to perform supervised training on the plurality of package identification paths until convergence, and package identification channels are obtained by combination.

[0048] Specifically, we first need to use the existing product conveyor belt packaging inspection data, which covers conveyor belt related information in different environments, different product types, and different packaging states. From these rich data, according to certain collection rules and methods, a set of sample polarized images is obtained. These images are collected from different angles, different time points, and different product placement conditions. For each sample polarized image collected, it is necessary to carefully observe and analyze it to determine whether the product conveyor belt is packaged. If it can be clearly seen in the image that the product conveyor belt is completely packaged without any openings or unsealed parts, then the sample polarized image is marked as 1, indicating the packaged state; conversely, if the product conveyor belt is found to have an unsealed area or incomplete packaging in the image, it is marked as 0, indicating the unpackaged state. By performing such an annotation operation on all the collected sample polarized images, a set containing each sample polarized image and its corresponding packaging state annotation is finally obtained, that is, the sample packaging recognition result set. This set provides important basic data for subsequent model training and algorithm optimization.

[0049] The ensemble learning algorithm plays a key role. First, it takes the sample polarization image set and the corresponding sample package recognition result set as input data. Then, based on the principles and methods of ensemble learning, the data is divided. Ensemble learning aims to improve the learning effect by combining multiple learners. By adopting a random division method, each group contains a part of the sample polarization images and their corresponding sample package recognition results. These grouped data constitute multiple groups of package recognition supervision training data. These data will be used for subsequent training of package recognition paths, so that each path can learn more accurate recognition patterns from different data sets, thereby improving the accuracy and reliability of the entire package recognition system.

[0050] With the help of convolutional neural network (CNN) in machine vision, multiple package recognition paths in the package recognition channel are constructed. CNN has powerful feature extraction capabilities and can automatically learn various feature patterns in images. The basic structure of the convolutional neural network is determined, including convolution layer, pooling layer and fully connected layer. The convolution layer performs convolution operation by sliding the convolution kernel on the image to extract local features of the image, such as edges and textures. The pooling layer compresses and reduces the dimension of the features extracted by the convolution layer, reducing the amount of data while retaining key features. Then, according to the specific needs of product conveyor belt package recognition, the parameters of the convolutional neural network are adjusted and optimized. For example, set the appropriate convolution kernel size, step size, padding method, etc., and determine the pooling method and window size of the pooling layer. Through continuous training and adjustment, the convolutional neural network can learn the feature patterns of the product conveyor belt package image, thereby constructing multiple package recognition paths. Each package recognition path has a unique network structure and parameter settings, which can identify the packaging situation of the product conveyor belt from different angles. These package recognition paths together constitute the package recognition channel, providing a variety of recognition methods for subsequent package recognition.

[0051] The ensemble learning method is applied again. For each package recognition path, corresponding multiple sets of package recognition supervised training data are used. These training data contain sample polarized images and their corresponding package recognition result annotations. Then, the supervised training process begins. Each package recognition path learns and adjusts its own parameters based on the input training data. During the training process, by comparing the difference between the predicted results and the actual annotation results, the model structure and parameters of the path are continuously optimized to improve the recognition accuracy of the package situation. This training process will continue until the model converges. Convergence means that the performance of the model no longer improves significantly with the increase in the number of training times, that is, the model has learned enough features and patterns to accurately identify the package situation. When all package recognition paths have completed training and converged, they are combined together to form the final package recognition channel, which can combine the advantages of multiple paths and accurately identify the packaging situation of the product conveyor belt from different angles.

[0052] In a possible implementation, step S500 further includes: Step S510: Calculate the ratio of each image quality information to the sum of the multiple image quality information to obtain multiple image weights.

[0053] Step S520: performing matching degree identification on the plurality of gain images to obtain a matching degree, wherein the matching degree includes a matching percentage.

[0054] Step S530: performing weighted calculation on the multiple package recognition results using the multiple image weights to obtain a summary package recognition result.

[0055] Step S540: using 1-the difference of the matching degree as a correction coefficient, compensating and correcting the summarized package recognition result to obtain a package recognition result interval.

[0056] Step S550: determine whether the package identification result intervals are all greater than or equal to the package identification result threshold, if so, obtain the packaged total package identification result, if not, obtain the unpackaged total package identification result.

[0057] Specifically, we need to identify all the image quality information, which is obtained by a series of analyses of the gain image, and comprehensively reflects the impact of various image characteristics on the final result. Then, for each image quality information, divide it by the sum of all image quality information. For example, suppose there are three image quality information, namely q1, q2, and q3, and their sum is Q=q1+q2+q3. Then the weight W1 corresponding to the first image quality information q1 is , the weight W2 corresponding to the second image quality information q2 is , the weight W3 corresponding to the third image quality information q3 is Through this calculation method, a relative weight in the overall image quality information is determined for each image quality information. These weights will be used in subsequent calculations to weight the package recognition results, so as to more reasonably consider the impact of image quality on the final result.

[0058] The gain image is preprocessed, such as resizing and normalizing, to meet the requirements of network input. The CNN network consists of multiple convolutional layers, pooling layers, and fully connected layers. The convolution layer performs convolution operations by sliding the convolution kernel on the image to automatically extract local features of the image, such as edges and textures. The pooling layer compresses and reduces the dimension of the features extracted by the convolution layer, reducing the amount of data while retaining key features. In the training stage, the sample gain image combination set and the corresponding sample matching degree set are used as input data. Through the back propagation algorithm, the network continuously adjusts the parameters such as the weight and bias of the convolution kernel to minimize the error between the predicted matching degree and the actual matching degree. In the testing stage, the gain image to be matched is input into the trained CNN network. The network will calculate the matching degree between the images based on the learned feature patterns and parameters, and output it in the form of matching percentage. This algorithm can automatically learn the feature representation of the image and has good matching ability for different gain images, thereby providing accurate matching degree information for subsequent packaging recognition result processing.

[0059] The weighted calculation of multiple package recognition results is performed using the multiple image weights obtained previously. First, it is clear that each image weight is obtained by calculating the ratio of each image quality information to the sum of all image quality information, which reflects the relative importance of each image quality information in the whole. Then, for each package recognition result, it is multiplied by the corresponding image weight. Finally, all weighted results are added together to obtain the summary package recognition result. This summary package recognition result comprehensively considers the impact of image quality information on each package recognition result, so that the final result more accurately reflects the packaging situation of the product conveyor belt.

[0060] The correction coefficient is determined according to the matching degree, which is presented in percentage form. For example, if the matching degree is 80%, then the correction coefficient is 1 minus the matching degree, that is, 1-80%=20%. Next, the summary package recognition result is compensated and corrected. The summary package recognition result is calculated through the previous steps, for example, the weighted calculation result is 0.8. The compensation correction process is to multiply the summary package recognition result by (1±correction coefficient), which is 0.8×(1±20%) in this example, and get an interval result (0.64, 0.96). This interval is the package recognition result interval. This is because there will inevitably be errors in the image acquisition and recognition process, so the calculated summary package recognition result cannot always be an absolute 1 (completely encapsulated) or 0 (completely unencapsulated). Therefore, it is necessary to set a threshold for judgment, and manual sampling can be added later to further ensure the quality of product conveyor belt packaging. In this way, the packaging situation of the product conveyor belt can be judged more accurately, providing a more reasonable basis for quality control.

[0061] The package recognition result interval is judged. The package recognition result interval comprehensively considers the impact of factors such as image quality information and matching degree on the package recognition result. Then, all values ​​in this interval are compared with the pre-set package recognition result threshold. If all values ​​in the interval are greater than or equal to this threshold, it means that the packaging of the product conveyor belt meets the requirements from multiple angles and factors. At this time, the total package recognition result of the package is obtained. On the contrary, if some or all values ​​are less than the threshold, it means that the packaging of the product conveyor belt does not meet the requirements, then the total package recognition result of the unpackaged package is obtained. This threshold is set based on actual production needs, product standards, experience and other factors. It is an important basis for judging whether the product conveyor belt is packaged qualified.

[0062] In a possible implementation, step S520 further includes: Step S521: based on the packaging inspection data of the product conveyor belt, a set of sample gain image combinations is collected, wherein each sample gain image combination includes a plurality of sample gain images.

[0063] Step S522: performing matching degree identification on multiple sample gain images in each sample gain image combination to obtain a sample matching degree set.

[0064] Step S523: using the sample gain image combination set and the sample matching degree set as supervised training data, and training a matching degree recognition channel based on a convolutional neural network.

[0065] Step S524: input the plurality of gain images into the trained matching degree recognition channel, and output the obtained matching degree.

[0066] Specifically, operations are carried out based on the packaging inspection data of the product conveyor belt, which covers relevant information of the product conveyor belt in different working environments, different operating states, and different product placements. Through data collection, appropriate sample gain images are selected from these rich data. Then, the selected sample gain images are combined according to certain rules to form a sample gain image combination set. In each sample gain image combination, multiple sample gain images are included. These images may be collected at different time points, different positions, or different angles. They reflect the characteristics of the product conveyor belt from multiple dimensions and provide sufficient data basis for subsequent matching degree identification and analysis.

[0067] For multiple sample gain images in each sample gain image combination, the structural similarity index (SSIM) algorithm is used to identify the matching degree. First, for each pair of sample gain images in the combination, it is treated as a whole. Calculate the brightness, contrast and structural information of the image. The brightness is measured by calculating the average gray value of the image, the contrast is reflected by calculating the standard deviation of the image, and the structural information is obtained by calculating the local correlation of the image. Then, according to the calculation formula of SSIM, the calculated brightness, contrast and structural information are substituted into the formula to obtain the SSIM value of each pair of images. This value is between 0 and 1, and the closer it is to 1, the higher the matching degree. By performing the above calculations on all possible image pairs in the combination, a sample matching degree set containing all SSIM values ​​is finally obtained. Each value in this set corresponds to the matching degree of a pair of sample gain images in the combination, which provides an important data basis for subsequent supervised training based on convolutional neural networks.

[0068] The sample gain image combination set and the corresponding sample matching degree set are clearly used as supervised training data. The sample gain image combination set contains multiple sample gain image combinations. Each combination contains multiple related images. These images reflect the characteristics of different aspects of the product conveyor belt. The sample matching degree set records the matching degree between the images in each sample gain image combination. Then, training is performed based on the convolutional neural network (CNN). CNN has a strong feature learning ability. During the training process, the sample gain image combination set is input into CNN as input data, and the convolution layer of the network automatically extracts various features of the image, such as edges, textures, shapes, etc. At the same time, the sample matching degree set is used as a supervisory label and compared with the output result of the network. By continuously adjusting the weights and parameters of CNN, the output result of the network is made as close as possible to the true value in the sample matching degree set. This process will go through multiple iterations. As the number of iterations increases, the network's ability to recognize image matching degrees gradually improves, and finally a matching degree recognition channel that can accurately recognize the matching degree of gain images is trained.

[0069] Get the trained matching recognition channel. This channel is obtained by training based on the convolutional neural network using the sample gain image combination set and the sample matching set as supervised training data in the previous steps. It already has the ability to identify the matching of multiple gain images. Then, the multiple gain images that need to be matched are input into this trained channel in sequence. The channel will analyze and process each input gain image combination according to the feature patterns and parameters learned internally. During the processing, the channel will extract various features of the image combination and compare and calculate them with the previously learned patterns. Finally, according to the internal algorithm and logic, the matching degree corresponding to each gain image combination is output. This matching degree is presented in the form of a numerical value, which can reflect the similarity between the input gain image combination and the relevant pattern in the training data, and provide an important basis for the subsequent processing of the package recognition results.

[0070] Embodiment 2, based on the same inventive concept as the packaging visual inspection method of the product conveyor belt in the above embodiment, Figure 2 As shown, the present application provides a packaging visual inspection platform for product conveyor belts, and the platform in the embodiment of the present application and the method embodiment are based on the same inventive concept. Among them, the platform includes: The side-light image acquisition module 10 is used to perform side lighting and image acquisition on the product conveyor belt at multiple positions and at multiple angles during the process of conveying products using the product conveyor belt to obtain multiple side-light images.

[0071] A gain image acquisition module 20, wherein the gain image acquisition module 20 is used to input the plurality of side light images into an image gain model for processing and output a plurality of corresponding gain images; The image quality information acquisition module 30 is used to perform image quality analysis on the multiple gain images to obtain multiple image quality information.

[0072] The package recognition result acquisition module 40 is used to perform integrated package recognition on the multiple gain images according to the multiple image quality information to obtain multiple package recognition results, wherein each package recognition result includes whether it is packaged.

[0073] The total package recognition result acquisition module 50 is used to perform matching degree recognition on the multiple gain images to obtain the matching degree, and to summarize and correct the multiple package recognition results in combination with the multiple image quality information to obtain the total package recognition result.

[0074] Furthermore, the side light image acquisition module 10 further includes: A side lighting unit is used to perform side lighting on the product conveyor belt at multiple positions according to multiple side angles during the process of conveying products using the product conveyor belt, wherein the angles between the multiple side angles and the direction perpendicular to the product conveyor belt are less than 90 degrees.

[0075] A conveyor belt image acquisition unit is used to acquire an image of the product conveyor belt at an angle perpendicular to the product conveyor belt to obtain a plurality of side light images.

[0076] Furthermore, the image quality information acquisition module 30 also includes: A transmission data acquisition unit is used to collect multiple transmission speeds and multiple transmission vibration amplitudes of the product conveyor belt passing through the multiple positions through a speed sensor and a vibration sensor.

[0077] A speed quality information acquisition unit is used to respectively calculate the ratio of the standard transmission speed to the multiple transmission speeds to obtain multiple speed quality information.

[0078] A vibration quality information acquisition unit is used to respectively calculate the ratios of the standard vibration amplitude and the multiple transmission vibration amplitudes to obtain multiple vibration quality information.

[0079] The image quality information acquisition unit is used for weighted calculation of the plurality of velocity quality information and the plurality of vibration quality information to obtain a plurality of image quality information.

[0080] Furthermore, the package identification result acquisition module 40 further includes: A package identification channel construction unit is provided, wherein the package identification channel construction unit constructs a package identification channel based on package detection data of a product conveyor belt, based on integrated learning and machine vision, wherein the package identification channel includes a plurality of package identification paths.

[0081] A package identification path quantity acquisition unit is used to obtain multiple package identification path quantities by multiplying the multiple image quality information by the total quantity of the multiple package identification paths and rounding the result.

[0082] A path encapsulation recognition result set acquisition unit is used to randomly select multiple package identification path sets according to the multiple package identification path numbers, input the multiple gain images into the multiple package identification path sets, and identify and obtain multiple path encapsulation recognition result sets.

[0083] A voting processing unit is used to perform majority voting processing on multiple path encapsulation identification result sets respectively to obtain multiple encapsulation identification results.

[0084] Furthermore, the encapsulation identification channel construction unit also includes: A sample packaging recognition result set acquisition unit, which collects a sample polarization image set based on the packaging detection data of the product conveyor belt, and marks whether the product conveyor belt in each sample polarization image is packaged, to obtain a sample packaging recognition result set, wherein the sample packaging recognition result includes 1 or 0, 1 for packaged, and 0 for unpackaged.

[0085] A package recognition supervision training data acquisition unit is configured to divide the sample polarization image set and the sample package recognition result set based on ensemble learning to obtain multiple groups of package recognition supervision training data.

[0086] A package identification path construction unit is used to construct a plurality of package identification paths in the package identification channel based on a convolutional neural network in machine vision.

[0087] The package identification channel combination unit uses the multiple groups of package identification supervision training data based on ensemble learning to perform supervised training on the multiple package identification paths until convergence, and combines them to obtain package identification channels.

[0088] Furthermore, the total package recognition result acquisition module 50 further includes: An image weight acquisition unit is used to calculate the ratio of each image quality information to the sum of the multiple image quality information to obtain multiple image weights.

[0089] A matching degree obtaining unit is used to perform matching degree identification on the plurality of gain images to obtain a matching degree, wherein the matching degree includes a matching percentage.

[0090] A summary package recognition result acquisition unit is used to perform weighted calculation on the multiple package recognition results using the multiple image weights to obtain a summary package recognition result.

[0091] A compensation correction unit is used to use 1-the difference of the matching degree as a correction coefficient to compensate and correct the summary package identification result to obtain a package identification result interval.

[0092] The package identification result interval judgment unit is used to judge whether the package identification result intervals are all greater than or equal to the package identification result threshold. If so, the total package identification result of the package is obtained; if not, the total package identification result of the unpackage is obtained.

[0093] Furthermore, the matching degree obtaining unit further includes: A sample gain image combination set acquisition unit is used to acquire a sample gain image combination set based on packaging detection data of a product conveyor belt, wherein each sample gain image combination includes a plurality of sample gain images.

[0094] A matching degree identification unit is used to identify the matching degrees of multiple sample gain images in each sample gain image combination to obtain a sample matching degree set.

[0095] A matching degree identification channel training unit is used to train a matching degree identification channel based on a convolutional neural network by using the sample gain image combination set and the sample matching degree set as supervised training data.

[0096] A matching degree output unit is used to input the plurality of gain images into a trained matching degree recognition channel and output the obtained matching degree.

[0097] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0098] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

[0099] This specification and the drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.

Claims

1. A packaging visual inspection method for a product conveyor belt, characterized in that: The method comprises: In the process of conveying products by using a product conveyor belt, the product conveyor belt is illuminated from the side and images are collected at multiple positions and angles to obtain multiple side-light images; Inputting the multiple side-light images into an image gain model for processing, and outputting corresponding multiple gain images; Performing image quality analysis on the multiple gain images to obtain multiple pieces of image quality information; According to the plurality of image quality information, performing integrated package identification on the plurality of gain images to obtain a plurality of package identification results, wherein each package identification result includes whether the package is packaged; The matching degree is identified on the multiple gain images to obtain the matching degree, and the multiple package identification results are summarized and corrected in combination with the multiple image quality information to obtain the total package identification result.

2. The packaging visual inspection method of a product conveyor belt according to claim 1, characterized in that: When the product is conveyed by the product conveyor belt, the product conveyor belt is illuminated from the side and images are collected at multiple positions and angles to obtain multiple side-light images, including: In the process of conveying the product by the product conveyor belt, the product conveyor belt is laterally illuminated at multiple positions according to multiple lateral angles, wherein the angles between the multiple lateral angles and the direction perpendicular to the product conveyor belt are less than 90 degrees; The image of the product conveyor belt is captured at an angle perpendicular to the product conveyor belt to obtain a plurality of side light images.

3. The packaging visual inspection method of a product conveyor belt according to claim 1, characterized in that: Performing image quality analysis on the multiple gain images to obtain multiple image quality information includes: By means of a speed sensor and a vibration sensor, a plurality of conveying speeds and a plurality of conveying vibration amplitudes of the product conveyor belt passing through the plurality of positions are collected; respectively calculating ratios of a standard transmission speed to the plurality of transmission speeds to obtain a plurality of speed quality information; respectively calculating the ratios of the standard vibration amplitude and the plurality of transmission vibration amplitudes to obtain a plurality of vibration quality information; The plurality of velocity quality information and the plurality of vibration quality information are weighted and calculated to obtain a plurality of image quality information.

4. The packaging visual inspection method of a product conveyor belt according to claim 1, characterized in that: According to the plurality of image quality information, integrated package identification is performed on the plurality of gain images to obtain a plurality of package identification results, including: Based on the package detection data of the product conveyor belt, based on integrated learning and machine vision, a package identification channel is constructed, wherein the package identification channel includes multiple package identification paths; According to the plurality of image quality information, multiplying the total number of the plurality of package identification paths and rounding the result to obtain a plurality of package identification path numbers; According to the number of the plurality of package identification paths, a plurality of package identification path sets of the plurality of package identification paths are randomly selected respectively, the plurality of gain images are input into the plurality of package identification path sets respectively, and a plurality of path package identification result sets are identified and obtained; A plurality of path encapsulation identification result sets are respectively subjected to majority voting to obtain a plurality of encapsulation identification results.

5. The packaging visual inspection method of a product conveyor belt according to claim 4, characterized in that: Based on the packaging inspection data of the product conveyor belt, a packaging recognition channel is built based on integrated learning and machine vision, including: Based on the packaging detection data of the product conveyor belt, a set of sample polarization images is collected, and whether the product conveyor belt is packaged in each sample polarization image is marked to obtain a set of sample packaging recognition results, where the sample packaging recognition results include 1 or 0, 1 for packaging, and 0 for unpackaging; Based on ensemble learning, the sample polarization image set and the sample package recognition result set are divided to obtain multiple groups of package recognition supervision training data; Based on the convolutional neural network in the machine vision, construct multiple package identification paths in the package identification channel; Based on ensemble learning, the multiple groups of package identification supervision training data are respectively used to perform supervised training on the multiple package identification paths until convergence, and then the package identification channels are obtained by combination.

6. The packaging visual inspection method of a product conveyor belt according to claim 1, characterized in that: Performing matching degree identification on the multiple gain images to obtain matching degrees, and summarizing and correcting the multiple package identification results in combination with the multiple image quality information to obtain a total package identification result, including: Calculating a ratio of each image quality information to a sum of the plurality of image quality information to obtain a plurality of image weights; Performing matching degree identification on the multiple gain images to obtain a matching degree, wherein the matching degree includes a matching percentage; Performing weighted calculation on the multiple package recognition results using the multiple image weights to obtain a summary package recognition result; Using 1-the difference of the matching degree as a correction coefficient, compensating and correcting the summarized package recognition result to obtain a package recognition result interval; It is determined whether the package identification result intervals are all greater than or equal to the package identification result threshold. If so, the packaged total package identification result is obtained; if not, the unpackaged total package identification result is obtained.

7. The packaging visual inspection method of a product conveyor belt according to claim 6, characterized in that: Performing matching degree identification on the multiple gain images to obtain matching degrees includes: Based on the packaging inspection data of the product conveyor belt, a sample gain image combination set is collected, wherein each sample gain image combination includes a plurality of sample gain images; Performing matching degree identification on multiple sample gain images in each sample gain image combination to obtain a sample matching degree set; Using the sample gain image combination set and the sample matching degree set as supervised training data, and training a matching degree recognition channel based on a convolutional neural network; The multiple gain images are input into a trained matching degree recognition channel, and the matching degree is output.

8. Product conveyor belt packaging visual inspection platform, characterized in that, The platform is used to implement the packaging visual inspection method of the product conveyor belt according to any one of claims 1 to 7, and the platform comprises: A side-light image acquisition module, which is used to perform side lighting and image acquisition on the product conveyor belt at multiple positions and at multiple angles during the process of conveying products using the product conveyor belt to obtain multiple side-light images; A gain image acquisition module, the gain image acquisition module is used to input the multiple side light images into the image gain model for processing, and output a corresponding multiple gain images; An image quality information acquisition module, the image quality information acquisition module is used to perform image quality analysis on the multiple gain images to obtain multiple image quality information; A package recognition result acquisition module, the package recognition result acquisition module is used to perform integrated package recognition on the multiple gain images according to the multiple image quality information to obtain multiple package recognition results, wherein each package recognition result includes whether it is packaged; The total package recognition result acquisition module is used to perform matching degree identification on the multiple gain images to obtain the matching degree, and to summarize and correct the multiple package recognition results in combination with the multiple image quality information to obtain the total package recognition result.

Images