Vision Inspection Method and Platform for the Encapsulation of Product Conveyor Belt

Abstract

The present invention discloses a method and platform for visual inspection of packaging of a product conveyor belt, which relates to the field of visual inspection technology. The method comprises: during the process of conveying products using a product conveyor belt, performing side lighting and image acquisition on the product conveyor belt at multiple positions and angles to obtain multiple side-lit images; performing image quality analysis to obtain multiple image quality information; performing integrated packaging recognition on the multiple gain images to obtain multiple packaging recognition results; performing matching degree recognition on the multiple gain images to obtain matching degrees and obtain an overall packaging recognition result. The present invention solves the technical problem in the prior art that when a product conveyor belt changes from black to white, it is difficult to determine whether the packaging is completed by visual inspection when a transparent film is packaged on a white product conveyor belt. This achieves the technical effect of accurately detecting and judging the packaging status of the product conveyor belt, and improving the accuracy and reliability of packaging detection.

Description

Technical Field

[0001] The present invention relates to the field of visual inspection technology, and in particular to a packaging visual inspection method and platform for a product conveyor belt. Background Art

[0002] When inspecting product packaging on conveyor belts, traditional visual inspection methods face challenges with color changes. For example, when a product conveyor belt changes from black to white, it's difficult to visually determine the package's integrity by encapsulating a transparent film on the white conveyor belt. This is because existing technologies, such as lighting direction, can make it difficult to distinguish the edge of the transparent film from the edge of the conveyor belt against the white background, hindering accurate assessment of the packaging. This situation can lead to unstable product quality in production scenarios with high packaging quality requirements, impacting production efficiency and product qualification rates.

[0003] In the prior art, when the product conveyor belt changes from black to white, a transparent film is packaged on the white product conveyor belt, and it is difficult to determine whether the packaging is completed by visual inspection of the packaging edge. Summary of the Invention

[0004] The present application provides a packaging visual inspection method and platform for a product conveyor belt, which is used to solve the technical problem in the prior art that when the product conveyor belt changes from black to white, a transparent film is packaged on a white product conveyor belt, and it is difficult to determine whether the packaging is completed through visual inspection of the packaging edge.

[0005] In view of the above problems, the present application provides a packaging visual inspection method and platform for product conveyor belts.

[0006] In a first aspect of the present application, a method for visually inspecting packaging of a product conveyor belt is provided, the method comprising:

[0007] During the process of conveying products using a product conveyor belt, the product conveyor belt is side-illuminated and image acquisition is performed at multiple positions and angles to obtain multiple side-lit images; the multiple side-lit images are input into an image gain model for processing, and corresponding multiple gain images are output; image quality analysis is performed on the multiple gain images to obtain multiple image quality information; based on the multiple image quality information, integrated packaging recognition is performed on the multiple gain images to obtain multiple packaging recognition results, wherein each packaging recognition result includes whether the product is packaged; matching degree recognition is performed on the multiple gain images to obtain matching degrees, and the multiple packaging recognition results are summarized and corrected in combination with the multiple image quality information to obtain an overall packaging recognition result.

[0008] A second aspect of the present application provides a packaging visual inspection platform for a product conveyor belt, the platform comprising:

[0009] A side light image acquisition module, which is used to perform side lighting and image capture on the product conveyor belt at multiple positions and at multiple angles during the process of conveying products using the product conveyor belt, thereby obtaining multiple side light images; a gain image acquisition module, which is used to input the multiple side light images into an image gain model for processing and output corresponding multiple gain images; an image quality information acquisition module, which is used to perform image quality analysis on the multiple gain images to obtain multiple image quality information; a packaging recognition result acquisition module, which is used to perform integrated packaging recognition on the multiple gain images based on the multiple image quality information to obtain multiple packaging recognition results, wherein each packaging recognition result includes whether the product is packaged; and a total packaging recognition result acquisition module, which is used to perform matching degree recognition on the multiple gain images to obtain a matching degree, and to summarize and correct the multiple packaging recognition results in combination with the multiple image quality information to obtain a total packaging recognition result.

[0010] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0011] During product conveyance on a product conveyor belt, the conveyor belt is illuminated from the side and images are captured at multiple locations and angles to obtain multiple side-lit images. Image quality analysis is performed on these multiple gain images to obtain multiple pieces of image quality information. Integrated package recognition is performed on these multiple gain images to obtain multiple package recognition results. Matching degree recognition is performed on these multiple gain images to obtain matching degrees, and the multiple package recognition results are aggregated and corrected to obtain an overall package recognition result. This achieves the technical effect of accurately detecting and judging the packaging condition of product conveyor belts, improving the accuracy and reliability of package detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 A schematic flow chart of a method for visually inspecting product conveyor belt packaging according to an embodiment of the present application;

[0014] Figure 2 Schematic diagram of the structure of the packaging visual inspection platform for the product conveyor belt provided in an embodiment of the present application.

[0015] Explanation of reference numerals: side light image acquisition module 10 , gain image acquisition module 20 , image quality information acquisition module 30 , package recognition result acquisition module 40 , total package recognition result acquisition module 50 . DETAILED DESCRIPTION

[0016] This application provides a packaging visual inspection method and platform for product conveyor belts, which is used to solve the technical problem in the prior art that when the product conveyor belt changes from black to white, a transparent film is packaged on a white product conveyor belt, and it is difficult to determine whether the packaging is completed through visual inspection of the packaging edge.

[0017] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative work are within the scope of protection of this application.

[0018] Example 1, as Figure 1 As shown, the present application provides a packaging visual inspection method for a product conveyor belt, the method comprising:

[0019] Step S100: When the product conveyor belt is used to convey the product, the product conveyor belt is illuminated from the side and images are captured at multiple positions and angles to obtain multiple side-lit images.

[0020] Specifically, during product transport, to address the difficulty of accurately identifying transparent packaging film against a white conveyor belt, a side-lighting and image acquisition method was implemented. As the product moved along the white conveyor belt, side-lighting was applied at multiple locations and specific angles. Blue light was selected as the lighting source, a technique determined based on optical principles and extensive experimentation. Blue light illuminating the white product creates a favorable optical interaction with the product's material and surface characteristics, resulting in clearer images of the product's edges and highlighting edge details, providing a high-quality foundation for subsequent image analysis and recognition. Due to the side-lighting, wrinkles and deformations in the packaging film create unique shadow effects. These shadows serve as a crucial indicator for determining whether the product is packaged. Furthermore, at each location where the side-lighting was applied, an image was captured perpendicular to the product conveyor belt. This method of side-lighting and image acquisition at multiple locations and angles ultimately yields multiple side-lighting images. These images contain rich information about the packaging film's characteristics under side-lighting, providing critical data for subsequent image quality analysis and package recognition.

[0021] Step S200: inputting the plurality of side-light images into an image gain model for processing, and outputting a plurality of corresponding gain images.

[0022] Specifically, these side-lit images are sequentially input into an image gain model built using a convolutional neural network. This convolutional neural network consists of multiple convolutional layers, pooling layers, and fully connected layers. In the convolutional layers, numerous convolution kernels slide across the side-lit image, keenly capturing local features such as edges and textures. For example, for white products, the convolution kernels can precisely lock onto edge information, enhancing previously weak or blurred edge signals. Through continuous processing by multiple convolutional layers, the image features are gradually refined and enhanced. The pooling layer downsamples the convolved image. Taking maximum pooling as an example, it significantly reduces the amount of data without losing key features, improving the model's computational efficiency while further highlighting important features already extracted, such as key pixel information on the edges of white products. Finally, the fully connected layer integrates the features processed by convolution and pooling, performs operations according to the trained parameters, maps the processed image to a specific output space, and outputs multiple gain images. Compared with the original side-lit images, these gain images show clearer images of the edges of white products, providing a higher-quality and more accurate image basis for subsequent image quality analysis and product packaging identification, significantly improving the accuracy and reliability of the entire inspection process.

[0023] Step S300: performing image quality analysis on the plurality of gain images to obtain a plurality of image quality information.

[0024] Specifically, image quality analysis is performed on multiple gain images to obtain multiple image quality information. Speed and vibration amplitude data are collected from the product conveyor belt at multiple locations using speed sensors and vibration sensors. Image quality is closely related to conveyor speed and vibration amplitude. Higher conveyor speeds and larger vibration amplitudes result in lower image quality. Based on this principle, the ratio of the standard conveyor speed to the conveyor speed at each location is calculated to obtain multiple speed quality information. Simultaneously, the ratio of the standard vibration amplitude to the vibration amplitude at each location is calculated to obtain multiple vibration quality information. Finally, these speed quality and vibration quality information are weighted and calculated to obtain multiple image quality information reflecting the quality of the gain image.

[0025] Step S400: performing integrated package identification on the multiple gain images according to the multiple image quality information to obtain multiple package identification results, wherein each package identification result includes whether the package is packaged.

[0026] Specifically, multiple gain images are integrated for package recognition based on the previously acquired image quality information to obtain multiple package recognition results. First, based on the package inspection data from the product conveyor, a package recognition channel is constructed using ensemble learning and machine vision. This channel contains multiple package recognition paths. The number of paths used for integrated recognition is then determined based on the image quality information. Higher image quality results in a larger number of recognition paths to improve recognition accuracy; conversely, lower image quality results in a smaller number of recognition paths to conserve computing power. The image quality information is multiplied by the total number of package recognition paths and rounded to the nearest integer to determine the number of recognition paths corresponding to each gain image. Next, a corresponding number of package recognition path sets are randomly selected based on the obtained number of recognition paths. The multiple gain images are then input into their corresponding package recognition path sets for recognition, resulting in multiple sets of path package recognition results. Finally, a majority vote is performed on these path package recognition result sets to obtain multiple package recognition results. Each package recognition result contains information on whether the package is encapsulated, with 1 indicating the presence of a film and 0 indicating the absence of a film.

[0027] Step S500: performing matching degree identification on the multiple gain images to obtain matching degrees, and summarizing and correcting the multiple package identification results in combination with the multiple image quality information to obtain an overall package identification result.

[0028] Specifically, multiple gain images are evaluated for matching, and the package recognition results are aggregated and corrected based on image quality information to obtain a total package recognition result. First, the multiple gain images are evaluated for matching. Since matching relationships exist between gain images at different positions—for example, the shadow formed by a wrinkle in the packaging film at a first position and a first angle should be symmetrical with the shadow formed at a second position and a second angle—the matching degree of the multiple gain images is analyzed based on this characteristic. This matching degree can reflect the accuracy of image acquisition and recognition. The multiple package recognition results are aggregated and corrected based on the image quality information. The ratio of each image quality information to the sum of the multiple image quality information is calculated to obtain multiple image weights. These image weights are then used to perform a weighted calculation on the multiple package recognition results to obtain a total package recognition result. The aggregate package recognition result is then compensated and corrected using the difference between 1 and the matching degree as a correction coefficient to obtain a package recognition result interval. Finally, a determination is made as to whether the package recognition result intervals are all greater than or equal to a package recognition result threshold. If so, a total package recognition result for the packaged package is obtained. If not, a total package recognition result for the unpackaged package is obtained.

[0029] In one possible implementation, step S100 further includes:

[0030] Step S110: When the product conveyor belt is used to convey the product, the product conveyor belt is laterally illuminated at multiple positions according to multiple lateral angles, wherein the angles between the multiple lateral angles and the direction perpendicular to the product conveyor belt are less than 90 degrees.

[0031] Step S120: capturing an image of the product conveyor belt at an angle perpendicular to the product conveyor belt to obtain a plurality of gain images.

[0032] Specifically, while products are being transported along a conveyor belt, lateral lighting is applied to the belt at multiple locations to better capture belt-related feature information for subsequent inspection and analysis. These multiple locations are either evenly distributed along the length of the belt or distributed in a specific pattern based on actual needs. The angles of the lateral lighting must be less than 90 degrees from the perpendicular to the belt. This angle is carefully considered because when light strikes the belt at these lateral angles, it highlights details on the belt surface and any packaging film. For example, wrinkles and deformations in the packaging film create unique shadows under this lateral lighting. These shadows are crucial for determining the packaging's completeness and quality. Furthermore, multiple lateral angles provide a diverse perspective on the belt and packaging film, further enriching the information captured and ensuring more accurate inspection and identification.

[0033] Image capture is performed perpendicular to the product conveyor belt. This is because an angle perpendicular to the conveyor belt provides the most accurate representation of the actual condition of the conveyor belt surface and associated items (such as products, packaging films, etc.) on it. At each location where side lighting is applied, an image is captured perpendicularly. Because side lighting is applied at multiple locations in step S110, an image is captured at each corresponding location. These images captured from multiple locations each possess unique lighting characteristics due to the previous side lighting. Together, they constitute multiple gain images, which serve as the basis for subsequent image analysis and processing and are crucial for accurately determining the packaging condition of the product conveyor belt.

[0034] In one possible implementation, step S300 further includes:

[0035] Step S310: collecting multiple conveying speeds and multiple conveying vibration amplitudes of the product conveyor belt passing through the multiple positions through a speed sensor and a vibration sensor.

[0036] Step S320: Calculate the ratios of the standard transmission speed to the multiple transmission speeds respectively to obtain multiple speed quality information.

[0037] Step S330: Calculating ratios of the standard vibration amplitude to the multiple transmission vibration amplitudes to obtain multiple vibration quality information.

[0038] Step S340: performing weighted calculation on the plurality of velocity quality information and the plurality of vibration quality information to obtain a plurality of image quality information.

[0039] Specifically, to capture data that reflects the impact of the conveyor belt's operating status on image quality, speed sensors and vibration sensors were used. Speed sensors were installed at relevant locations on the conveyor belt. As the conveyor belt passed through multiple designated locations while transporting products, the speed sensors accurately measured the conveyor speed at each location. This speed data is crucial for analyzing image quality, as varying speeds can cause image blurring, distortion, and other quality issues. Furthermore, vibration sensors were strategically placed near the conveyor belt. As the conveyor belt passed through multiple locations, the sensors recorded the corresponding vibration amplitude. This vibration amplitude also significantly impacted image quality; large vibration amplitudes can cause undesirable effects such as image shake and noise. The coordinated operation of the speed and vibration sensors comprehensively and accurately captured multiple conveyor speeds and vibration amplitudes at each location, providing the necessary data foundation for subsequent image quality analysis.

[0040] A standard conveyor speed is determined, based on the ideal speed of the product conveyor belt. Then, for each of the multiple conveyor speeds of the product conveyor belt at various locations, previously captured by the speed sensor, the ratio between the standard conveyor speed and each conveyor speed is calculated. This calculation is repeated for each location, ultimately yielding multiple speed quality information points. This speed quality information indicates the degree of deviation from the standard conveyor speed at each location and can be used to assess the speed-related quality of the images captured at that location.

[0041] A standard vibration amplitude is defined, representing the ideal vibration condition of the conveyor belt. Next, the ratio of the standard vibration amplitude to each conveyor vibration amplitude at various locations, as captured by the vibration sensor, is calculated. By performing this calculation for each location, multiple vibration quality indicators are ultimately obtained. This vibration quality indicator reflects the degree of deviation from the standard vibration amplitude at each location, allowing for the assessment of the vibration-related quality of the images captured at that location.

[0042] In order to comprehensively consider the impact of speed and vibration on image quality, it is necessary to perform weighted calculation on multiple speed quality information and multiple vibration quality information to obtain multiple image quality information. The weight of speed quality information and vibration quality information in image quality assessment is determined based on a large amount of experimental data and understanding of the operating characteristics of the product conveyor belt. Then, for the speed quality information and vibration quality information corresponding to each position, weighted calculation is performed according to the established weight. For example, if the speed quality information is V q , whose weight is W v , the vibration quality information is a q , whose weight is W a , then the image quality information I at this position q Formula I q =W v ×V q +W a ×a q By performing this weighted calculation for each position, multiple image quality information is ultimately obtained. This image quality information can comprehensively reflect the image quality of the product conveyor belt at each position, providing an important basis for subsequent operations such as integrated packaging recognition.

[0043] In one possible implementation, step S400 further includes:

[0044] Step S410: Based on the package detection data of the product conveyor belt, based on integrated learning and machine vision, a package identification channel is constructed, wherein the package identification channel includes multiple package identification paths.

[0045] Step S420: multiplying the plurality of image quality information by the total number of the plurality of package identification paths and rounding the result to obtain the number of plurality of package identification paths.

[0046] Step S430: randomly selecting a plurality of package identification path sets according to the plurality of package identification path numbers, inputting the plurality of gain images into the plurality of package identification path sets, and identifying and obtaining a plurality of path package identification result sets.

[0047] Step S440: performing majority voting on the plurality of path encapsulation identification result sets respectively to obtain a plurality of encapsulation identification results.

[0048] Specifically, the ensemble learning component requires collecting a large amount of conveyor belt packaging inspection data covering a variety of packaging conditions and corresponding conveyor belt feature information. For example, the data may include packaging records for different product types, packaging materials, conveyor belt speeds, and vibration conditions. This data is preprocessed, including removing erroneous data and outliers, and normalizing the data to ensure that different feature data are on the same scale for better processing by the subsequent algorithm. A decision tree ensemble approach is used to randomly extract multiple sub-datasets from the preprocessed dataset, and a decision tree is constructed for each sub-dataset. When constructing the decision tree, the best split feature is selected from a random subset of all features each time, increasing the diversity of the decision trees. Ultimately, a random forest model is formed, consisting of multiple decision trees. Each decision tree can produce a prediction for the packaging condition, and the combined results can produce more accurate and stable predictions. For the machine vision component, a convolutional neural network (CNN) is used for feature extraction. The convolutional layer of a CNN performs a convolution operation by sliding a kernel across the image, automatically extracting local features such as edges and textures. The pooling layer compresses and reduces the dimensionality of the features extracted by the convolutional layer, reducing the amount of data while retaining key features. After multiple layers of convolution and pooling, a feature vector containing rich image features is ultimately obtained. The feature vector extracted from the CNN is then combined with the model generated by the ensemble learning algorithm. For example, the feature vector is fed into a random forest model, which uses this feature vector to classify and predict the package condition. Through the combined application of ensemble learning and machine vision technologies, a package recognition pipeline is constructed that includes multiple package recognition paths, each capable of accurately identifying the package from a different perspective.

[0049] The obtained multiple image quality information are obtained, which comprehensively reflects the quality status of the image, including the impact of factors such as conveyor belt speed and vibration on the image. Then, each image quality information is multiplied by the total number of pre-set package recognition paths. This total number is determined when the package recognition channel is constructed, and it represents the sum of all paths that can be used for recognition. Finally, the product result is rounded, and the integer value obtained is the number of package recognition paths corresponding to the image. In this way, according to the different image quality, different numbers of package recognition paths are reasonably allocated to each gain image. Images with good quality may be allocated more paths to improve recognition accuracy, while images with poor quality may be allocated fewer paths to save computing power.

[0050] For each gain image, operations are performed based on the number of corresponding package recognition paths. For example, if a gain image has n package recognition paths, n paths are randomly selected from the set of all available package recognition paths to form a package recognition path set for that gain image. The gain image is then input into this specific package recognition path set. Within this set, each path has its own unique recognition method and algorithm model. For example, some paths are based on convolutional neural networks (CNNs), which automatically extract various image features such as edges and textures and use these features to determine whether the package on the conveyor belt in the image meets the requirements. Other paths are based on decision tree algorithms, which classify and determine whether the product is packaged based on predefined image attributes such as color distribution and shape characteristics. Each path independently processes the gain image, generating a package recognition result for that path. Since there are n paths, the final result is a set of n recognition results, which is the path package recognition result set for that gain image. The above process is repeated for each gain image, determining the number of package identification paths, randomly selecting a set of paths, inputting the image, and obtaining the corresponding set of path package identification results. This ultimately results in multiple sets of path package identification results, which serve as the basis for subsequent majority voting to determine the final package identification result.

[0051] A majority vote is performed on each set of path package recognition results. Since each set is composed of the results obtained by recognizing the gain image across multiple paths, the number of times each result appears in the set must be counted. For example, if a path package recognition result set contains three results: "packaged," "unpackaged," and "packaged," then the result "packaged" appears the most times and is determined to be the package recognition result corresponding to that set. All path package recognition result sets are processed in this manner, ultimately obtaining multiple package recognition results. These results will provide a basis for further determining the packaging status of the product conveyor belt.

[0052] In one possible implementation, step S410 further includes:

[0053] Step S411: Based on the packaging detection data of the product conveyor belt, a set of sample polarization images is collected, and whether the product conveyor belt is packaged in each sample polarization image is marked to obtain a set of sample packaging recognition results, where the sample packaging recognition results include 1 or 0, 1 for packaged and 0 for unpackaged.

[0054] Step S412: Based on ensemble learning, the sample polarization image set and the sample package recognition result set are divided to obtain multiple groups of package recognition supervision training data.

[0055] Step S413: constructing multiple package identification paths in the package identification channel based on the convolutional neural network in the machine vision.

[0056] Step S414: Based on ensemble learning, the plurality of groups of package identification supervision training data are respectively used to perform supervised training on the plurality of package identification paths until convergence, and the package identification channels are obtained by combination.

[0057] Specifically, the process first utilizes existing packaging inspection data for product conveyor belts, covering conveyor belts in various environments, product types, and packaging states. Based on this rich data, a set of sample polarization images is acquired according to specific collection rules and methods. These images are collected from different angles, at different times, and under different product placement conditions. Each sample polarization image is carefully observed and analyzed to determine whether the product conveyor belt is sealed. If the image clearly shows the product conveyor belt is completely sealed, without any openings or unsealed areas, the sample polarization image is labeled as 1, indicating a sealed state. Conversely, if the image shows unsealed areas or incomplete packaging, the sample polarization image is labeled as 0, indicating an unsealed state. By performing this labeling operation on all collected sample polarization images, a set of sample polarization images and their corresponding packaging state labels is obtained, known as the sample packaging recognition result set. This set provides important foundational data for subsequent model training and algorithm optimization.

[0058] The ensemble learning algorithm plays a key role. First, it takes as input a set of sample polarization images and their corresponding sample package recognition results. Then, based on the principles and methods of ensemble learning, it performs a partitioning operation on this data. Ensemble learning aims to improve learning effectiveness by combining multiple learners. Using a random partitioning approach, each group contains a subset of sample polarization images and their corresponding sample package recognition results. These grouped data constitute multiple sets of supervised training data for package recognition. This data is then used to train the package recognition pathways, enabling each pathway to learn more accurate recognition patterns from diverse datasets, thereby improving the accuracy and reliability of the entire package recognition system.

[0059] The convolutional neural network (CNN) within machine vision is used to construct multiple package recognition paths within the package recognition pipeline. CNNs possess powerful feature extraction capabilities and can automatically learn various characteristic patterns within images. The basic structure of the CNN is defined, including convolutional layers, pooling layers, and fully connected layers. The convolutional layer performs convolution operations by sliding a convolution kernel across the image, extracting local features such as edges and textures. The pooling layer compresses and reduces the dimensionality of the features extracted by the convolutional layer, reducing the data volume while preserving key features. The parameters of the CNN are then adjusted and optimized based on the specific requirements of package recognition on product conveyor belts. For example, appropriate convolution kernel size, stride, and padding are set, and the pooling method and window size of the pooling layer are determined. Through continuous training and adjustment, the CNN learns the characteristic patterns of package images on product conveyor belts, thereby constructing multiple package recognition paths. Each package recognition path has a unique network structure and parameter settings, enabling it to identify package conditions on product conveyor belts from different angles. Together, these package recognition paths constitute the package recognition pipeline, providing a diverse range of recognition methods for subsequent package recognition.

[0060] An ensemble learning approach is again applied. For each package recognition path, multiple sets of supervised package recognition training data are used. These training data include sample polarized images and their corresponding package recognition result annotations. Then, the supervised training process begins. Each package recognition path learns and adjusts its parameters based on the input training data. During training, by comparing the differences between the predicted results and the actual annotation results, the model structure and parameters of the path are continuously optimized to improve the accuracy of package recognition. This training process continues until the model converges. Convergence means that the model's performance no longer improves significantly with increasing training cycles, meaning that the model has learned enough features and patterns to accurately identify the package. Once all package recognition paths have completed training and converged, they are combined to form the final package recognition channel. This package recognition channel can combine the advantages of multiple paths to accurately identify the package status of the product conveyor belt from different angles.

[0061] In one possible implementation, step S500 further includes:

[0062] Step S510: Calculate the ratio of each piece of image quality information to the sum of the plurality of image quality information to obtain a plurality of image weights.

[0063] Step S520: performing matching degree identification on the plurality of gain images to obtain a matching degree, wherein the matching degree includes a matching percentage.

[0064] Step S530: performing weighted calculation on the multiple package recognition results using the multiple image weights to obtain a summary package recognition result.

[0065] Step S540: using 1-the difference in the matching degree as a correction coefficient to perform compensation correction on the summarized package recognition result to obtain a package recognition result interval.

[0066] Step S550: determining whether the package identification result intervals are all greater than or equal to the package identification result threshold; if so, obtaining the packaged total package identification result; if not, obtaining the unpackaged total package identification result.

[0067] Specifically, we need to identify all the image quality information, which is obtained by a series of analyses on the gain image, and comprehensively reflects the impact of various image characteristics on the final result. Then, for each image quality information, divide it by the sum of all image quality information. For example, suppose there are three image quality information, namely q1, q2 and q3, and their sum is Q=q1+q2+q3. Then the weight W1 corresponding to the first image quality information q1 is , the weight W2 corresponding to the second image quality information q2 is , the weight W3 corresponding to the third image quality information q3 is Through this calculation method, a relative weight is determined for each piece of image quality information in the overall picture. These weights will be used in subsequent calculations to weight the package recognition results, so as to more reasonably consider the impact of image quality on the final result.

[0068] The gain image undergoes preprocessing, such as resizing and normalization, to meet network input requirements. A CNN network consists of multiple convolutional layers, pooling layers, and fully connected layers. The convolutional layers perform convolution operations by sliding convolution kernels across the image, automatically extracting local features such as edges and textures. The pooling layers compress and reduce the dimensionality of the features extracted by the convolutional layers, reducing the data size while preserving key features. During the training phase, a set of sample gain images and a corresponding set of sample matching scores are used as input. Using a backpropagation algorithm, the network continuously adjusts parameters such as the convolution kernel weights and biases to minimize the error between the predicted and actual matching scores. During the testing phase, the gain image to be matched is input into the trained CNN network. Based on the learned feature patterns and parameters, the network calculates the matching scores between the images and outputs them as matching percentages. This algorithm automatically learns image feature representations and achieves good matching performance for different gain images, thus providing accurate matching information for subsequent package recognition processing.

[0069] The package recognition results are weighted using the previously obtained image weights. First, each image weight is calculated by calculating the ratio of each image's quality information to the sum of all image quality information, reflecting its relative importance within the overall image. Then, for each package recognition result, the corresponding image weight is multiplied by it. Finally, all weighted results are summed to produce a summary package recognition result. This summary package recognition result comprehensively considers the impact of image quality information on each package recognition result, ensuring that the final result more accurately reflects the packaging conditions on the product conveyor belt.

[0070] A correction factor is determined based on the degree of match. The degree of match is expressed as a percentage. For example, if the degree of match is 80%, the correction factor is 1 minus the degree of match: 1 minus 80% = 20%. Next, compensation is applied to the aggregated package recognition result. The aggregated package recognition result is calculated using the previous steps. For example, the weighted calculation result is 0.8. The compensation process multiplies the aggregated package recognition result by (1 ± the correction factor)—in this example, 0.8 × (1 ± 20%)—to obtain a range of results (0.64, 0.96). This range is the package recognition result range. This is done because errors are inevitable during the image acquisition and recognition process, so the calculated aggregated package recognition result cannot always be an absolute 1 (fully encapsulated) or 0 (completely unencapsulated). Therefore, setting a threshold for discrimination is necessary, and manual spot checks can be added later to further ensure the quality of conveyor belt packaging. This approach allows for more accurate assessment of conveyor belt packaging conditions, providing a more robust basis for quality control.

[0071] The package recognition result interval is judged, taking into account the impact of factors such as image quality information and matching degree on the package recognition result. Then, all values within this interval are compared with a pre-set package recognition result threshold. If all values within the interval are greater than or equal to this threshold, it means that the packaging of the product conveyor belt meets the requirements based on a comprehensive assessment of multiple angles and factors. At this time, the packaged total package recognition result is obtained. Conversely, if some or all values are less than the threshold, it indicates that the packaging of the product conveyor belt does not meet the requirements, and the unpackaged total package recognition result is obtained. This threshold is set based on actual production needs, product standards, and experience, and is an important basis for determining whether the product conveyor belt is packaged properly.

[0072] In one possible implementation, step S520 further includes:

[0073] Step S521: Based on the packaging inspection data of the product conveyor belt, a sample gain image combination set is collected, wherein each sample gain image combination includes a plurality of sample gain images.

[0074] Step S522: performing matching degree identification on multiple sample gain images in each sample gain image combination to obtain a sample matching degree set.

[0075] Step S523: using the sample gain image combination set and the sample matching degree set as supervised training data, and training a matching degree recognition channel based on a convolutional neural network.

[0076] Step S524: inputting the plurality of gain images into the trained matching degree recognition channel, and outputting the obtained matching degree.

[0077] Specifically, the system operates based on packaging inspection data from product conveyor belts. This data covers various conditions, including conveyor belt operating environments, operating states, and product placement. Through data collection, appropriate sample gain images are selected from this rich data set. These selected sample gain images are then combined according to specific rules to form a sample gain image combination set. Each sample gain image combination contains multiple sample gain images. These images may have been acquired at different times, locations, or angles. They reflect the characteristics of the product conveyor belt from multiple dimensions, providing a sufficient data foundation for subsequent match identification and analysis.

[0078] For each sample gain image combination, the Structural Similarity Index (SSIM) algorithm is used to identify matching scores for multiple sample gain images. First, each pair of sample gain images within the combination is treated as a whole. Image brightness, contrast, and structural information are calculated. Brightness is measured by calculating the image's mean grayscale value, contrast is reflected by calculating the image's standard deviation, and structural information is obtained by calculating the image's local correlation. Then, according to the SSIM calculation formula, the calculated brightness, contrast, and structural information are substituted into the formula to obtain the SSIM value for each pair of images. This value ranges from 0 to 1, with a closer value to 1 indicating a higher degree of matching. By performing this calculation for all possible image pairs within the combination, a sample matching score set containing all SSIM values is obtained. Each value in this set corresponds to the matching score between a pair of sample gain images within the combination, providing an important data foundation for subsequent supervised training based on convolutional neural networks.

[0079] The supervised training data is explicitly defined as a set of sample gain image combinations and their corresponding sample matching scores. The sample gain image combination set contains multiple sample gain image combinations, each containing multiple related images that reflect different aspects of the conveyor belt. The sample matching score set records the matching scores between the images within each sample gain image combination. Training is then performed using a convolutional neural network (CNN), which has powerful feature learning capabilities. During training, the sample gain image combination set is fed into the CNN as input data, and the network's convolutional layers automatically extract various image features, such as edges, texture, and shape. Simultaneously, the sample matching score set serves as a supervisory label and is compared with the network's output. By continuously adjusting the CNN's weights and parameters, the network's output is made as close as possible to the true values in the sample matching score set. This process is repeated multiple times. As the number of iterations increases, the network's ability to identify image matching scores gradually improves, ultimately training a matching score recognition channel that can accurately identify gain image matching scores.

[0080] Obtain a trained match recognition channel. This channel was trained using a convolutional neural network using a set of sample gain image combinations and a set of sample match values as supervised training data in the previous steps. It now possesses the ability to identify matches for multiple gain images. Multiple gain images for match recognition are then sequentially input into this trained channel. The channel analyzes and processes each input gain image combination based on its learned feature patterns and parameters. During this processing, the channel extracts various features from the image combination and compares and calculates them against previously learned patterns. Finally, based on its internal algorithms and logic, it outputs the matching score for each gain image combination. This matching score, presented as a numerical value, reflects the degree of similarity between the input gain image combination and the relevant pattern in the training data, providing an important basis for subsequent processing of the package recognition results.

[0081] The second embodiment is based on the same inventive concept as the packaging visual inspection method of the product conveyor belt in the above embodiment. Figure 2 As shown, the present application provides a packaging visual inspection platform for product conveyor belts. The platform in the embodiments of the present application and the method embodiment are based on the same inventive concept. The platform includes:

[0082] The side light image acquisition module 10 is used to perform side lighting and image acquisition on the product conveyor belt at multiple positions and at multiple angles during the process of conveying products using the product conveyor belt to obtain multiple side light images.

[0083] a gain image acquisition module 20, configured to input the plurality of side-light images into an image gain model for processing and output a plurality of corresponding gain images;

[0084] The image quality information acquisition module 30 is configured to perform image quality analysis on the plurality of gain images to obtain a plurality of image quality information.

[0085] The package recognition result acquisition module 40 is used to perform integrated package recognition on the multiple gain images according to the multiple image quality information to obtain multiple package recognition results, wherein each package recognition result includes whether it is packaged.

[0086] The total package recognition result acquisition module 50 is used to perform matching degree identification on the multiple gain images to obtain matching degrees, and to summarize and correct the multiple package recognition results in combination with the multiple image quality information to obtain a total package recognition result.

[0087] Furthermore, the side light image acquisition module 10 further includes:

[0088] A side lighting unit is used to perform side lighting on the product conveyor belt at multiple positions according to multiple side angles during the process of conveying products using the product conveyor belt, wherein the angle between the multiple side angles and the direction perpendicular to the product conveyor belt is less than 90 degrees.

[0089] A conveyor belt image acquisition unit is used to acquire images of the product conveyor belt at an angle perpendicular to the product conveyor belt to obtain multiple side light images.

[0090] Furthermore, the image quality information acquisition module 30 further includes:

[0091] A conveying data acquisition unit is used to acquire a plurality of conveying speeds and a plurality of conveying vibration amplitudes of the product conveyor belt passing through the plurality of positions through a speed sensor and a vibration sensor.

[0092] The speed quality information acquisition unit is used to respectively calculate the ratios of the standard transmission speed and the multiple transmission speeds to obtain multiple pieces of speed quality information.

[0093] The vibration quality information acquisition unit is used to respectively calculate the ratios of the standard vibration amplitude and the multiple transmission vibration amplitudes to obtain multiple vibration quality information.

[0094] The image quality information acquisition unit is configured to perform weighted calculation on the plurality of velocity quality information and the plurality of vibration quality information to obtain a plurality of image quality information.

[0095] Furthermore, the package recognition result acquisition module 40 further includes:

[0096] A package identification channel construction unit is configured to construct a package identification channel based on package detection data of a product conveyor belt, integrated learning, and machine vision, wherein the package identification channel includes a plurality of package identification paths.

[0097] The package identification path number acquisition unit is configured to obtain multiple package identification path numbers by multiplying the multiple image quality information by the total number of the multiple package identification paths and rounding the result.

[0098] A path encapsulation recognition result set acquisition unit is used to randomly select multiple package identification path sets according to the number of the multiple package identification paths, input the multiple gain images into the multiple package identification path sets, and identify and obtain multiple path encapsulation recognition result sets.

[0099] A voting processing unit is used to perform majority voting on multiple path encapsulation identification result sets to obtain multiple encapsulation identification results.

[0100] Furthermore, the package identification channel construction unit further includes:

[0101] A sample packaging recognition result set acquisition unit is provided. The sample packaging recognition result set acquisition unit collects a sample polarization image set based on the packaging detection data of the product conveyor belt, and marks whether the product conveyor belt in each sample polarization image is packaged, thereby obtaining a sample packaging recognition result set, wherein the sample packaging recognition result includes 1 or 0, 1 means packaged, and 0 means not packaged.

[0102] A package recognition supervision training data acquisition unit is configured to divide the sample polarization image set and the sample package recognition result set based on ensemble learning to obtain multiple groups of package recognition supervision training data.

[0103] A package identification path construction unit is configured to construct a plurality of package identification paths within the package identification channel based on a convolutional neural network within machine vision.

[0104] The package identification channel combination unit uses the multiple groups of package identification supervision training data based on ensemble learning to perform supervised training on the multiple package identification paths until convergence, and combines them to obtain package identification channels.

[0105] Furthermore, the total package identification result acquisition module 50 further includes:

[0106] An image weight acquisition unit is configured to calculate a ratio of each piece of image quality information to the sum of the plurality of image quality information to obtain a plurality of image weights.

[0107] A matching degree obtaining unit is configured to perform matching degree identification on the plurality of gain images to obtain a matching degree, wherein the matching degree includes a matching percentage.

[0108] The summarized package recognition result obtaining unit is configured to perform weighted calculation on the multiple package recognition results using the multiple image weights to obtain a summarized package recognition result.

[0109] A compensation correction unit is configured to use 1-the difference in the matching degree as a correction coefficient to perform compensation correction on the summary package recognition result to obtain a package recognition result interval.

[0110] The package identification result interval judgment unit is used to judge whether the package identification result intervals are all greater than or equal to the package identification result threshold. If so, the packaged total package identification result is obtained; if not, the unpackaged total package identification result is obtained.

[0111] Furthermore, the matching degree obtaining unit further includes:

[0112] The sample gain image combination set acquisition unit acquires a sample gain image combination set based on packaging detection data of a product conveyor belt, wherein each sample gain image combination includes a plurality of sample gain images.

[0113] A matching degree identification unit is used to identify the matching degrees of multiple sample gain images in each sample gain image combination to obtain a sample matching degree set.

[0114] A matching degree identification channel training unit is used to use the sample gain image combination set and the sample matching degree set as supervised training data to train the matching degree identification channel based on a convolutional neural network.

[0115] A matching degree output unit is configured to input the plurality of gain images into a trained matching degree recognition channel and output the obtained matching degree.

[0116] It should be noted that the order in which the embodiments of the present application are presented is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. Furthermore, the foregoing descriptions of specific embodiments of this specification are provided. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential sequence shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0117] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

[0118] This specification and drawings are merely illustrative of the present application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Obviously, those skilled in the art may make various modifications and variations to this application without departing from the scope of this application. Thus, this application is intended to include such modifications and variations as fall within the scope of this application and its equivalents.

Claims

1. A packaging visual inspection method for a product conveyor belt, characterized in that: The method comprises: When the product is conveyed by the product conveyor belt, the product conveyor belt is illuminated from the side and images are captured at multiple positions and angles to obtain multiple side-light images; performing image quality analysis on the plurality of side-lit images to obtain a plurality of image quality information; performing integrated package identification on the plurality of side-light images according to the plurality of image quality information to obtain a plurality of package identification results, wherein each package identification result includes whether the package is encapsulated; Performing matching degree identification on the multiple side-light images to obtain a matching degree, and summarizing and correcting the multiple package identification results in combination with the multiple image quality information to obtain an overall package identification result; Performing matching degree identification on the multiple side-light images to obtain a matching degree, and summarizing and correcting the multiple package identification results in combination with the multiple image quality information to obtain an overall package identification result, including: calculating a ratio of each piece of image quality information to a sum of the plurality of pieces of image quality information to obtain a plurality of image weights; Performing matching degree identification on the plurality of side-light images to obtain a matching degree, wherein the matching degree includes a matching percentage; performing weighted calculation on the multiple package recognition results using the multiple image weights to obtain a summary package recognition result; Using 1-the difference in the matching degree as a correction coefficient, compensating and correcting the summarized package recognition result to obtain a package recognition result interval; Determine whether the package identification result intervals are all greater than or equal to the package identification result threshold, if so, obtain the packaged total package identification result, if not, obtain the unpackaged total package identification result; Performing matching degree identification on the multiple side-light images to obtain a matching degree includes: Based on the packaging inspection data of the product conveyor belt, a sample side light image combination set is collected, wherein each sample side light image combination includes multiple sample side light images; Performing matching degree identification on multiple sample side-light images in each sample side-light image combination to obtain a sample matching degree set; Using the sample side-light image combination set and the sample matching degree set as supervised training data, and training a matching degree recognition channel based on a convolutional neural network; Combining the multiple side-light images and inputting them into a trained matching recognition channel, and outputting the obtained matching degree; Performing image quality analysis on the multiple side-lit images to obtain multiple pieces of image quality information, including: collecting, by means of a speed sensor and a vibration sensor, a plurality of conveying speeds and a plurality of conveying vibration amplitudes of the product conveyor belt passing through the plurality of positions; respectively calculating ratios of the standard transmission speed to the plurality of transmission speeds to obtain a plurality of speed quality information; respectively calculating ratios of the standard vibration amplitude to the plurality of transmission vibration amplitudes to obtain a plurality of vibration quality information; weighted calculation of the plurality of velocity quality information and the plurality of vibration quality information to obtain a plurality of image quality information; Performing integrated package recognition on the multiple side-light images according to the multiple image quality information to obtain multiple package recognition results, including: Based on the package detection data of the product conveyor belt, a package identification channel is constructed based on integrated learning and machine vision, wherein the package identification channel includes multiple package identification paths; multiplying the plurality of image quality information by the total number of the plurality of package identification paths and rounding the result to obtain a plurality of package identification path numbers; According to the number of the plurality of package identification paths, randomly selecting a plurality of package identification path sets of the plurality of package identification paths, inputting the plurality of side light images into the plurality of package identification path sets, and identifying to obtain a plurality of path package identification result sets; A majority voting process is performed on the plurality of path encapsulation identification result sets to obtain a plurality of encapsulation identification results.

2. The packaging visual inspection method of a product conveyor belt according to claim 1, characterized in that: When using a product conveyor belt to transport products, the product conveyor belt is illuminated from the side and images are captured at multiple positions and angles to obtain multiple side-lit images, including: During the process of conveying the product on the product conveyor belt, the product conveyor belt is lighted at multiple locations according to multiple lateral angles, wherein the angle between the multiple lateral angles and the direction perpendicular to the product conveyor belt is less than 90 degrees; The image of the product conveyor belt is captured at an angle perpendicular to the product conveyor belt to obtain a plurality of side-light images.

3. The packaging visual inspection method of a product conveyor belt according to claim 1, characterized in that: Based on the package inspection data of the product conveyor belt, a package recognition channel is built based on integrated learning and machine vision, including: Based on the packaging inspection data of the product conveyor belt, a set of sample polarized images is collected, and whether the product conveyor belt is packaged in each sample polarized image is marked to obtain a set of sample packaging recognition results, where the sample packaging recognition results include 1 or 0, where 1 indicates packaged and 0 indicates unpackaged; Based on ensemble learning, the sample polarization image set and the sample package recognition result set are divided to obtain multiple groups of package recognition supervision training data; Constructing multiple package identification paths within the package identification channel based on a convolutional neural network within machine vision; Based on ensemble learning, the multiple groups of package identification supervision training data are respectively used to perform supervised training on the multiple package identification paths until convergence, and then the package identification channels are obtained by combination.

4. Product conveyor belt packaging visual inspection platform, characterized by: The platform is used to implement the packaging visual inspection method of the product conveyor belt according to any one of claims 1 to 3, and the platform includes: A side-light image acquisition module, configured to perform side-lighting and image acquisition on the product conveyor belt at multiple positions and angles during product conveyance using the product conveyor belt, thereby obtaining multiple side-light images; an image quality information acquisition module, configured to perform image quality analysis on the plurality of side-lit images to obtain a plurality of image quality information; a package recognition result acquisition module, configured to perform integrated package recognition on the plurality of side-light images based on the plurality of image quality information to obtain a plurality of package recognition results, wherein each package recognition result includes whether the package is encapsulated; The total package recognition result acquisition module is used to perform matching degree identification on the multiple side-light images to obtain the matching degree, and to summarize and correct the multiple package recognition results in combination with the multiple image quality information to obtain the total package recognition result.

Images