Intelligent sorting method and system for printed products based on machine vision

Abstract

The present application relates to the technical field of visual sorting, and a printed product intelligent sorting method and system based on machine vision, which comprises the following steps: confirming an intelligent sorting environment based on an intelligent sorting instruction, wherein an image acquisition unit is used to acquire images of the printed products, the image acquisition unit is used to pre-process the image set of the printed products, an image processing unit is used to extract features from the pre-processed silhouette images, the image processing unit is used to confirm a plurality of defect areas in the pre-processed light and dark texture images, the image processing unit is used to calculate features of the plurality of defect areas, and the printed products are sorted and classified based on the defect evaluation index, a preset classification level and a sorting execution unit. The present application can improve the accuracy, efficiency and automation level of the intelligent sorting of printed products.

Description

Technical Field

[0001] This invention relates to the field of visual sorting technology, and in particular to an intelligent sorting method and system for printed products based on machine vision. Background Technology

[0002] As the printing industry rapidly transforms towards intelligent and refined processes, traditional quality sorting methods are no longer sufficient to meet the demands of high-precision production. Currently, the industry generally relies on manual visual inspection or automated equipment based on a single dimension. Manual methods are inefficient, inconsistent in standards, and costly. Existing automation technologies have significant limitations: they lack a single detection dimension, failing to simultaneously and accurately quantify multiple defects in printed materials, such as geometric deformation and microscopic scratches; secondly, they lack intelligent fusion decision-making, resulting in isolated detection results from each dimension. The shortcomings of existing technologies in terms of efficiency, accuracy, and intelligence necessitate an automated sorting system capable of intelligent fusion of multi-dimensional features and real-time decision-making. Summary of the Invention

[0003] This invention provides a machine vision-based intelligent sorting method and system for printed products, the main purpose of which is to improve the accuracy, efficiency and automation level of intelligent sorting of printed products.

[0004] To achieve the above objectives, the present invention provides an intelligent sorting method for printed materials based on machine vision, comprising: The system confirms receipt of the intelligent sorting instruction and confirms the intelligent sorting environment based on the instruction. The intelligent sorting environment includes an intelligent sorting system and printed products. The intelligent sorting system includes an image acquisition unit, an image processing unit, an identification and classification unit, and a sorting execution unit. The image acquisition unit includes a color photo acquisition device, a line scanning device, and a light source device. The printed product image set is obtained by acquiring images of the printed product based on the image acquisition unit, wherein the printed product image set includes high-definition color images, light and dark texture images and silhouette images; Based on the image acquisition unit, the printed product image set is preprocessed to obtain a preprocessed high-definition color image, a preprocessed light and dark texture image, and a preprocessed silhouette image. Based on the image processing unit, feature extraction is performed on the preprocessed silhouette image to obtain a silhouette image parameter set, and a geometric feature vector is obtained based on the silhouette image parameter set. Based on the image processing unit, multiple defect regions are identified in the preprocessed light and dark texture image. Based on the image processing unit, feature calculation is performed on the multiple defect regions to obtain defect vectors. Based on the preprocessed high-definition color image and multiple preset key regions, color difference calculation is performed to obtain color difference value and text similarity; Based on the identification and classification unit, geometric feature vector, defect vector, color difference value and text similarity, a defect assessment index is obtained. Based on the defect assessment index, the preset classification level and the sorting execution unit, the printed products are sorted and classified to obtain classified printed products. Based on the classified printed products, intelligent sorting of the printed products is realized.

[0005] Optionally, the step of acquiring images of the printed product based on the image acquisition unit to obtain a set of printed product images includes: The printed product is tracked and positioned based on a pre-built conveying device and encoder to obtain a real-time positioning area. A visual detection area is obtained. If the real-time positioning area reaches the visual detection area, an image acquisition command is obtained. Based on the image acquisition command and the color photo acquisition device, a color photo is acquired on the printed product located in the visual detection area to obtain a high-definition color image. Based on the line scanning device, a light and dark texture image is obtained; based on the line scanning device and the light source device, a silhouette image is obtained by capturing the silhouette of the printed product in the visual detection area. By combining the high-definition color images, light and dark texture images, and silhouette images, a set of printed product images is obtained.

[0006] Optionally, the step of performing image preprocessing on the printed product image set based on the image acquisition unit to obtain a preprocessed high-definition color image, a preprocessed light and dark texture image, and a preprocessed silhouette image includes: Based on the preprocessed image set and the pre-constructed white balance algorithm, the high-definition color image is color corrected to obtain a preprocessed high-definition color image; Based on the image acquisition unit, the contrast of the light and dark texture image is enhanced to obtain a preprocessed light and dark texture image; The silhouette image is binarized using the image acquisition unit to obtain a preprocessed silhouette image.

[0007] Optionally, the step of extracting features from the preprocessed silhouette image based on the image processing unit to obtain a silhouette image parameter set, and obtaining a geometric feature vector based on the silhouette image parameter set, includes: Based on the image processing unit set and the preset contour tracking algorithm, the contour pixels of the preprocessed silhouette image are extracted to obtain the contour pixel set; The outer contour point set is divided based on the contour pixel point set, and the minimum outer bounding matrix is ​​obtained based on the outer contour point set. The actual length, actual width and actual contour area are calculated based on the minimum outer bounding matrix. Obtain a standard outer contour point set, and based on the standard outer contour point set, obtain the standard length, standard width, and standard contour area. Based on the actual length, actual width, actual contour area, standard length, standard width, and standard contour area, calculate the length deviation, width deviation, and area deviation. The right-angle deviation between the contour pixel set and the standard external contour point set is calculated using a preset contour analysis algorithm. The length deviation, width deviation, area deviation, and right angle deviation are summarized to obtain the silhouette image parameter set. The silhouette image parameter set is then mapped according to a preset dimensional order to obtain the geometric feature vector.

[0008] Optionally, the image processing unit identifies multiple defect regions in the preprocessed light and dark texture image, and performs feature calculations on the multiple defect regions to obtain a defect vector, including: Based on the image processing unit, a difference analysis is performed on the preprocessed light and dark texture image and the preset defect-free template image to obtain multiple candidate defect regions; Based on a preset minimum defect area threshold, the multiple candidate defect regions are filtered to obtain multiple defect regions. For each of the multiple defect regions, the following operation is performed: Pixel statistics are performed on the defective region to obtain the region's pixel area. The minimum bounding matrix is ​​obtained based on the defective region, and the aspect ratio is calculated based on the minimum bounding matrix. The average gray value of the pixels and the average gray value of the background are obtained based on the defect area. The gray value contrast is calculated based on the average gray value of the pixels and the average gray value of the background. The consistency of the pixel gradient direction within the defect area is calculated based on the preset histogram algorithm to obtain the direction consistency. The pixel area, aspect ratio, grayscale contrast, and orientation consistency of the region are correlated to obtain multiple defect sub-vectors. The multiple defect sub-vectors are then normalized to obtain a defect vector.

[0009] Optionally, the step of calculating color difference based on the preprocessed high-definition color image and multiple preset key regions to obtain color difference values ​​and text similarity includes: Based on the key regions, multiple detection areas were identified in the preprocessed high-definition color image. Multidimensional color detection was performed on the multiple detection areas to obtain the average hue, average brightness, and average vividness. Obtain standard hue, standard brightness, and standard vividness; and calculate hue difference, brightness difference, and vividness difference based on the average hue, average brightness, average vividness, standard hue, standard brightness, and standard vividness. The color difference value is calculated based on the hue difference, brightness difference, and vividness difference, using the following formula:

[0010] in, Indicates the color difference value. Indicates the difference in brightness. Indicates the difference in vibrancy. Indicates the hue difference. , and These represent the brightness correction, vividness correction, and hue correction items, respectively. , and All represent weighting coefficients. Indicates compensation item; Based on a preset text recognition algorithm, text is extracted from the multiple detection areas to obtain image-extracted text. The image-extracted text is then compared with a preset standard benchmark text to obtain text similarity.

[0011] Optionally, the step of verifying the extracted text from the image with a preset standard benchmark text to obtain text similarity includes: The extracted text from the image and the standard reference text are compared character by character using a preset text matching algorithm to obtain the number of correctly matched characters, the number of incorrectly inserted characters, the number of incorrectly deleted characters, and the number of incorrectly replaced characters. The text recall rate is calculated based on the number of correctly matched characters, the number of incorrectly deleted characters, and the number of incorrectly replaced characters. The calculation formula is as follows:

[0012] in, Indicates text recall. Indicates the number of correctly matched characters. Indicates the number of characters deleted incorrectly. Indicates the number of characters replaced incorrectly; The text accuracy is calculated based on the number of correctly matched characters, the number of incorrectly inserted characters, and the number of incorrectly replaced characters. The calculation formula is as follows:

[0013] in, Indicates text precision. Indicates the number of characters inserted incorrectly; Text similarity is calculated based on the text recall and text precision, using the following formula:

[0014] in, Indicates text similarity.

[0015] Optionally, obtaining the defect assessment index based on the identification and classification unit, geometric feature vector, defect vector, color difference value, and text similarity includes: Perform a normalization operation on the geometric eigenvectors to obtain the geometric eigenvector norm; Based on the aforementioned identification and classification unit, geometric feature vector norm, defect vector, color difference value, and text similarity, the defect assessment index is calculated using the following formula:

[0016] in, Indicates the defect assessment index. Represents the norm of geometric eigenvectors. Represents the defect vector. Represents a nonlinear mapping function. , , and All of these represent weighting coefficients.

[0017] Optionally, the sorting and classification of the printed products based on the defect assessment index, the preset classification level, and the sorting execution unit to obtain classified printed products includes: A classification threshold is determined based on the classification level, wherein the classification threshold includes a first-level threshold, a second-level threshold, and a third-level threshold. If the defect assessment index is greater than or equal to the first-level threshold, the printed product is determined to be a high-quality product. If the defect assessment index is less than the first-level threshold and greater than or equal to the second-level threshold, the printed product is confirmed as a qualified product. If the defect assessment index is less than the second-level threshold and greater than or equal to the third-level threshold, the printed product is identified as a slightly defective product. If the defect assessment index is less than the level three threshold, the printed product will be identified as a defective product. The sorting unit sorts and classifies the high-quality products, qualified products, slightly defective products, or substandard products to obtain classified printed products.

[0018] To achieve the above objectives, the present invention also provides a machine vision-based intelligent sorting system for printed materials, comprising: An environment confirmation module is used to confirm the receipt of intelligent sorting instructions and to confirm the intelligent sorting environment based on the intelligent sorting instructions. The intelligent sorting environment includes an intelligent sorting system and printed products. The intelligent sorting system includes an image acquisition unit, an image processing unit, a recognition and classification unit, and a sorting execution unit. The image acquisition unit includes a color photo acquisition device, a line scanning device, and a light source device. An image acquisition module is used to acquire images of the printed product based on the image acquisition unit to obtain a set of printed product images, wherein the set of printed product images includes high-definition color images, light and dark texture images, and silhouette images; The image processing module is used to perform image preprocessing on the printed product image set based on the image acquisition unit to obtain a preprocessed high-definition color image, a preprocessed light and dark texture image, and a preprocessed silhouette image. Based on the image processing unit, feature extraction is performed on the preprocessed silhouette image to obtain a silhouette image parameter set, and a geometric feature vector is obtained based on the silhouette image parameter set. Based on the image processing unit, multiple defect regions are identified in the preprocessed light and dark texture image. Based on the image processing unit, feature calculation is performed on the multiple defect regions to obtain defect vectors. Based on the preprocessed high-definition color image and multiple preset key regions, color difference calculation is performed to obtain color difference value and text similarity; The classification and sorting module is used to obtain a defect assessment index based on the identification and classification unit, geometric feature vector, defect vector, color difference value and text similarity. Based on the defect assessment index, the preset classification level and the sorting execution unit, the printed products are sorted and classified to obtain classified printed products. Based on the classified printed products, intelligent sorting of the printed products is realized.

[0019] To address the above problems, the present invention also provides an electronic device, the electronic device comprising: Memory, storing at least one instruction; The processor executes the instructions stored in the memory to implement the machine vision-based intelligent sorting method for printed materials described above.

[0020] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the aforementioned intelligent sorting method for printed materials based on machine vision.

[0021] To address the problems described in the background art, this invention confirms the receipt of intelligent sorting instructions and, based on these instructions, confirms the intelligent sorting environment. The intelligent sorting environment includes an intelligent sorting system and printed products. The intelligent sorting system includes an image acquisition unit, an image processing unit, a recognition and classification unit, and a sorting execution unit. The image acquisition unit includes a color image acquisition device, a line scanning device, and a light source device. Therefore, this invention considers the diversity and real-time requirements of complex printing environments during the sorting process of printed products. By confirming the intelligent sorting environment, modular collaboration of the system is ensured, providing a reliable foundation for subsequent visual processing, thereby improving the overall stability and adaptability of the sorting process. Based on the image acquisition unit… The image acquisition unit performs image acquisition on the printed product to obtain an image set of the printed product, which includes high-definition color images, light and dark texture images, and silhouette images. This demonstrates that the present invention employs a multi-modal image acquisition mechanism to capture the multi-dimensional features of the printed product, avoiding the limitations of a single image, and thus laying a comprehensive data foundation for defect detection. Based on the image acquisition unit, the printed product image set is preprocessed to obtain preprocessed high-definition color images, preprocessed light and dark texture images, and preprocessed silhouette images. This shows that the present invention performs preprocessing immediately after acquisition to optimize image quality and reduce noise interference, thereby improving the accuracy and efficiency of subsequent analysis. Based on the image processing unit, the preprocessed silhouette images are further processed... Feature extraction yields a silhouette image parameter set, and geometric feature vectors are obtained based on this parameter set. This demonstrates that the present invention extracts geometric features from silhouette images, achieving precise quantification of shape and contour, thus avoiding errors associated with traditional manual inspection. Based on the image processing unit's identification of multiple defect regions in the preprocessed light and dark texture image, feature calculations are performed on these defect regions to obtain defect vectors. This embodiment of the present invention identifies and quantifies defect regions through texture analysis, forming vector descriptions, thereby improving the objectivity and fine granularity of defect assessment. Color difference calculations are performed based on the preprocessed high-definition color image and multiple preset key regions to obtain color difference values ​​and text similarity. This demonstrates that the present invention combines... High-definition images are used to calculate color difference and text similarity to evaluate core indicators of printing quality, thereby ensuring the comprehensiveness and reliability of sorting standards. A defect assessment index is obtained based on the identification and classification unit, geometric feature vectors, defect vectors, color difference values, and text similarity. Based on the defect assessment index, preset classification levels, and the sorting execution unit, the printed products are sorted and classified to obtain categorized printed products. Based on these categorized printed products, intelligent sorting of the printed products is achieved. It is evident that this invention integrates multi-source features to calculate a comprehensive index during the classification stage, forming a closed-loop sorting execution, and adapts to different needs through dynamic levels. Therefore, this invention can improve the accuracy, efficiency, and automation level of intelligent sorting of printed products. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating a machine vision-based intelligent sorting method for printed materials according to an embodiment of the present invention. Figure 2 A functional block diagram of a machine vision-based intelligent sorting system for printed products provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device for implementing the machine vision-based intelligent sorting method for printed materials, according to an embodiment of the present invention.

[0023] Explanation of reference numerals in the attached figures: 10. Electronic device; 11. Processor; 12. Memory; 13. Bus.

[0024] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0025] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0026] This application provides a machine vision-based intelligent sorting method for printed materials. The executing entity of the machine vision-based intelligent sorting method for printed materials includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the machine vision-based intelligent sorting method for printed materials can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0027] Reference Figure 1 The diagram shown is a flowchart illustrating a machine vision-based intelligent sorting method for printed materials according to an embodiment of the present invention. In this embodiment, the machine vision-based intelligent sorting method for printed materials includes: S1. Confirm receipt of intelligent sorting instruction, and confirm intelligent sorting environment based on intelligent sorting instruction. The intelligent sorting environment includes intelligent sorting system and printed products. The intelligent sorting system includes image acquisition unit, image processing unit, recognition and classification unit and sorting execution unit. The image acquisition unit includes color photo acquisition device, line scanning device and light source device.

[0028] It should be explained that the intelligent sorting instruction refers to the instruction issued by the personnel who want to achieve intelligent sorting; the intelligent sorting environment refers to the necessary environment for achieving intelligent sorting; the intelligent sorting system refers to a system capable of achieving intelligent sorting; and the intelligent sorting system includes an image acquisition unit, an image processing unit, a recognition and classification unit, and a sorting execution unit. The image acquisition unit includes a color photo acquisition device, a line scanning device, and a light source device. For the specific application of the units, please refer to the following embodiments. The printed products refer to various finished products produced using printing technology, such as books, newspapers, or flat advertising images. The purpose of this invention is to improve the level of intelligence, efficiency, and accuracy of printed product sorting.

[0029] For example, Xiao Zhang is a worker in a printing factory. In order to achieve intelligent sorting of printed products and improve the level of intelligence, efficiency and accuracy of sorting, Xiao Zhang issued an intelligent sorting instruction and confirmed the zero-discharge environment for wastewater.

[0030] S2. Based on the image acquisition unit, images of the printed product are acquired to obtain a set of printed product images, wherein the set of printed product images includes high-definition color images, light and dark texture images, and silhouette images.

[0031] Furthermore, the step of acquiring images of the printed product based on the image acquisition unit to obtain a set of printed product images includes: The printed product is tracked and positioned based on a pre-built conveying device and encoder to obtain a real-time positioning area. A visual detection area is obtained. If the real-time positioning area reaches the visual detection area, an image acquisition command is obtained. Based on the image acquisition command and the color photo acquisition device, a color photo is acquired on the printed product located in the visual detection area to obtain a high-definition color image. Based on the line scanning device, a light and dark texture image is obtained; based on the line scanning device and the light source device, a silhouette image is obtained by capturing the silhouette of the printed product in the visual detection area. By combining the high-definition color images, light and dark texture images, and silhouette images, a set of printed product images is obtained.

[0032] It should be understood that the method for tracking and positioning the printed products refers to placing the printed products on a conveyor device for batch conveying, and using an encoder installed on the conveyor device to detect the position of the printed products on the conveyor device in real time. The conveyor device refers to a mechanical device capable of conveying printed products in batches. Optionally, the conveyor device can be constructed using a motor roller, a servo driver, and a conveyor belt. The encoder refers to a rotary encoder installed on the drive shaft of the conveyor device. The real-time positioning area refers to the current position of the printed products on the conveyor device. The visual detection area refers to a preset area for image acquisition on the conveyor device. If the real-time positioning area reaches the visual detection area, it indicates that the printed products have reached the area for image acquisition, and subsequent image acquisition can be performed. The image acquisition command refers to the instruction issued by the image acquisition unit to drive its sub-modules to perform image acquisition after the printed product arrives at the designated visual inspection area. The image acquisition unit is a functional module capable of acquiring images of the printed product. The image acquisition unit includes a color image acquisition device, a line scanning device, and a light source device. The color image acquisition device is a functional sub-module within the image acquisition unit capable of acquiring color images; optionally, a high-resolution industrial camera can be used. The line scanning device is a functional sub-module within the image acquisition unit capable of accurately capturing surface details and textures; optionally, a line scanning camera and a linear array sensor can be used. The light source device is a functional sub-module within the image acquisition unit that provides directional illumination for the image acquisition process of the printed product; optionally, LED strip lights, ring lights, or backlights can be used. The method of performing color image acquisition on the printed product in the visual inspection area refers to using the image acquisition command to drive the color image acquisition device to acquire images of the printed product. The high-resolution color image refers to the high-resolution color image acquired by the color image acquisition device. The method for acquiring light and dark texture images based on the line scanning device refers to using the line scanning device, in conjunction with directional lighting (such as oblique light source or backlighting) of the light source device, to continuously scan the surface of the printed product entering the visual inspection area line by line, acquiring images that can reflect the light and dark distribution characteristics of the surface texture of the printed product. The method for acquiring silhouette images of the printed product based on the line scanning device and the light source device refers to using the line scanning device, in conjunction with backlighting of the light source device, to scan the printed product entering the visual inspection area and acquire silhouette images. The silhouette image refers to an image acquired by the line scanning device in conjunction with backlighting of the light source device that clearly shows the overall outline of the printed product. The printed product image set refers to a collection of high-definition color images, light and dark texture images, and silhouette images.

[0033] S3. Based on the image acquisition unit, perform image preprocessing on the printed product image set to obtain a preprocessed high-definition color image, a preprocessed light and dark texture image, and a preprocessed silhouette image.

[0034] It should be explained that the image preprocessing of the printed product image set based on the image acquisition unit to obtain preprocessed high-definition color images, preprocessed light and dark texture images, and preprocessed silhouette images includes: Based on the preprocessed image set and the pre-constructed white balance algorithm, the high-definition color image is color corrected to obtain a preprocessed high-definition color image; Based on the image acquisition unit, the contrast of the light and dark texture image is enhanced to obtain a preprocessed light and dark texture image; The silhouette image is binarized using the image acquisition unit to obtain a preprocessed silhouette image.

[0035] Furthermore, the method for color correction of the high-definition color image refers to adjusting the white balance of the high-definition color image using the white balance algorithm built into the image acquisition unit to correct the color temperature deviation of the light source and ensure true color reproduction. Optionally, the grayscale world algorithm or the perfect reflection algorithm can be used as the white balance algorithm, and the preprocessed high-definition color image refers to the high-definition color image after color correction, which is used for subsequent color difference calculation. The method for contrast enhancement of the bright and dark texture image refers to improving the local contrast of the image using histogram equalization or the CLAHE algorithm to make the texture details clearer, and the image processing unit refers to the functional module for image analysis and feature extraction. Optionally, the image processing unit can be constructed using the OpenCV library or a similar image processing framework, and the preprocessed bright and dark texture image refers to the bright and dark texture image after contrast enhancement. The method for binarizing the silhouette image refers to converting the silhouette image into a black and white binary image using the Otsu thresholding method or an adaptive thresholding algorithm to highlight the contour edges and facilitate geometric feature extraction, and the preprocessed silhouette image refers to the silhouette image after binarization.

[0036] S4. Based on the image processing unit, feature extraction is performed on the preprocessed silhouette image to obtain a silhouette image parameter set, and geometric feature vectors are obtained based on the silhouette image parameter set.

[0037] It should be understood that the step of extracting features from the preprocessed silhouette image based on the image processing unit to obtain a silhouette image parameter set, and obtaining a geometric feature vector based on the silhouette image parameter set, includes: Based on the image processing unit set and the preset contour tracking algorithm, the contour pixels of the preprocessed silhouette image are extracted to obtain the contour pixel set; The outer contour point set is divided based on the contour pixel point set, and the minimum outer bounding matrix is ​​obtained based on the outer contour point set. The actual length, actual width and actual contour area are calculated based on the minimum outer bounding matrix. Obtain a standard outer contour point set, and based on the standard outer contour point set, obtain the standard length, standard width, and standard contour area. Based on the actual length, actual width, actual contour area, standard length, standard width, and standard contour area, calculate the length deviation, width deviation, and area deviation. The right-angle deviation between the contour pixel set and the standard external contour point set is calculated using a preset contour analysis algorithm. The length deviation, width deviation, area deviation, and right angle deviation are summarized to obtain the silhouette image parameter set. The silhouette image parameter set is then mapped according to a preset dimensional order to obtain the geometric feature vector.

[0038] It should be explained that the method of extracting contour pixels of the preprocessed silhouette image based on the image processing unit and the preset contour tracking algorithm refers to using the image processing unit and the contour tracking algorithm to scan the binary pixels of the preprocessed silhouette image, identify and extract the boundary pixel sequence, and form a contour pixel set. Optionally, Moore's neighborhood tracking or the Suzuki algorithm can be used as the contour tracking algorithm. The contour pixel set refers to the set of extracted boundary pixel coordinates, which is used for subsequent geometric calculations. The method of dividing the outer contour point set based on the contour pixel set refers to selecting the outermost closed contour (ignoring internal holes) from the contour pixel set to obtain the outer contour point set. The outer contour point set refers to the boundary point sequence of the overall shape of the printed product. The method of obtaining the minimum bounding matrix based on the outer contour point set refers to calculating the minimum rectangular bounding box that can enclose the outer contour point set through the rotating caliper algorithm. The minimum bounding matrix refers to the rectangular bounding box connected by the outer contour points in the outer contour point set. The method for calculating the actual length, actual width, and actual contour area based on the minimum outer matrix refers to extracting the long side as the actual length and the short side as the actual width from the matrix, and using the polygon area formula to calculate the area enclosed by the set of outer contour points as the actual contour area. The actual length, actual width, and actual contour area refer to the geometric dimension indicators of the printed product in the silhouette view.

[0039] Furthermore, the method for obtaining the standard external contour point set refers to extracting the point set of the ideal contour from a preset standard printed product template or database. The standard external contour point set refers to the reference contour coordinates of the defect-free printed product. The method for obtaining the standard length, standard width, and standard contour area based on the standard external contour point set is similar to the method for calculating the actual value described above, and will not be elaborated here. The standard length, standard width, and standard contour area refer to the reference dimensions of the ideal printed product. The method for calculating the length deviation, width deviation, and area deviation based on the actual length, actual width, actual contour area, standard length, standard width, and standard contour area refers to calculating the relative difference between the actual value and the standard value respectively. Optionally, the length deviation can be obtained by subtracting the absolute value of the standard length from the actual length and then dividing by the standard length. The method of calculating the right-angle deviation of the contour pixel set and the standard external contour point set using a preset contour analysis algorithm refers to using a corner detection algorithm (such as Harris or Shi-Tomasi) to identify right-angle points in the contour and calculating the average deviation of the actual right angle from the standard 90°. This right-angle deviation refers to the deformation at the right angle of the printed product edge, used to detect corner or edge defects. The silhouette image parameter set refers to the set of length deviation, width deviation, area deviation, and right-angle deviation. The method of mapping the silhouette image parameter set according to a preset dimensional order refers to arranging them in a preset order (e.g., length, width, area, right angle) to generate a geometric feature vector. The geometric feature vector is a one-dimensional vector that maps the parameters in the silhouette image parameter set according to a preset dimensional order and is used for subsequent defect assessment.

[0040] For example, assuming the preprocessed silhouette image is traced to obtain a set of contour pixels, after separating the outer contour point set, the minimum bounding matrix is ​​calculated to obtain an actual length of 100mm, an actual width of 50mm, and an actual contour area of ​​4500mm²; the standard value is a length of 98mm, a width of 48mm, and an area of ​​4600mm², with calculated deviations such as a length deviation of 2%, a width deviation of 4%, and an area deviation of 2%; the right angle deviation is 3°; the parameters obtained above are arranged in a preset order (length deviation, width deviation, area deviation, right angle deviation) to obtain a geometric feature vector (0.02, 0.04, 0.02, 3).

[0041] S5. Based on the image processing unit, multiple defect regions are identified in the preprocessed light and dark texture image. Based on the image processing unit, feature calculation is performed on the multiple defect regions to obtain defect vectors.

[0042] It should be understood that, based on the image processing unit identifying multiple defect regions in the preprocessed light and dark texture image, and based on the image processing unit performing feature calculations on the multiple defect regions to obtain a defect vector, including: Based on the image processing unit, a difference analysis is performed on the preprocessed light and dark texture image and the preset defect-free template image to obtain multiple candidate defect regions; Based on a preset minimum defect area threshold, the multiple candidate defect regions are filtered to obtain multiple defect regions. For each of the multiple defect regions, the following operation is performed: Pixel statistics are performed on the defective region to obtain the region's pixel area. The minimum bounding matrix is ​​obtained based on the defective region, and the aspect ratio is calculated based on the minimum bounding matrix. The average gray value of the pixels and the average gray value of the background are obtained based on the defect area. The gray value contrast is calculated based on the average gray value of the pixels and the average gray value of the background. The consistency of the pixel gradient direction within the defect area is calculated based on the preset histogram algorithm to obtain the direction consistency. The pixel area, aspect ratio, grayscale contrast, and orientation consistency of the region are correlated to obtain multiple defect sub-vectors. The multiple defect sub-vectors are then normalized to obtain a defect vector.

[0043] It should be explained that the method of performing difference analysis on the preprocessed light and dark texture image and the preset defect-free template image based on the image processing unit refers to the image processing unit calculating the difference between the preprocessed light and dark texture image and the defect-free template image (the texture reference image of the ideal printed product) by pixel-level subtraction, highlighting abnormal areas, forming a difference image, and obtaining multiple candidate defect areas. The multiple candidate defect areas refer to image blocks that may have defects (such as scratches, stains, or uneven texture), and the defect-free template image refers to a pre-stored texture image of a perfect printed product used for benchmark comparison. The method of filtering the multiple candidate defect areas based on a preset minimum defect area threshold refers to calculating the pixel area of ​​each candidate defect area. If it is greater than the threshold (e.g., 10 pixels), it is retained; otherwise, it is filtered to eliminate noise interference. The minimum defect area threshold refers to a preset minimum pixel area used to filter the candidate defect areas, and the multiple defect areas refer to image locations that have been confirmed to have significant defects.

[0044] Furthermore, the method of obtaining the region pixel area by performing pixel statistics based on the defect region refers to counting the number of pixels within the defect region as the region pixel area, where the region pixel area refers to the pixel area of ​​the defect region. The method of obtaining the minimum bounding matrix based on the defect region is similar to the above operation of separating the outer contour point set based on the contour pixel point set and obtaining the minimum bounding matrix based on the separated outer contour point set, and will not be elaborated here. The method of calculating the aspect ratio based on the minimum bounding matrix refers to dividing the longer side of the minimum bounding matrix by the shorter side to obtain the aspect ratio, where the aspect ratio refers to the degree of stretching of the defect shape and is used to distinguish between strip-shaped and block-shaped defects. The method for obtaining the average grayscale value of pixels and the average grayscale value of the background based on the defect area refers to first using the OTSU algorithm to binarize the defect area and its surrounding area, and then using pixels to divide the defect area and the surrounding background. Next, the grayscale values ​​of the defect area and the surrounding background are extracted using the YCbCr brightness conversion method. Finally, the average grayscale value of the pixels within the defect area and the average grayscale value of the surrounding background area are calculated using a weighted average. The average grayscale value of pixels and the average grayscale value of the background refer to the brightness levels of the defect and the background, respectively. The method for calculating grayscale contrast based on the average grayscale value of pixels and the average grayscale value of the background refers to subtracting the average grayscale value of the background from the average grayscale value of pixels and then dividing by the average grayscale value of the background to obtain the grayscale contrast. The grayscale contrast refers to the visibility of the defect area relative to the background. The method for calculating the consistency of pixel gradient directions within the defect area based on a preset histogram algorithm to obtain directional consistency refers to using the Sobel operator to calculate the gradient direction histogram and calculating the peak sharpness of the histogram as directional consistency. This directional consistency refers to the degree of orderliness of the defect texture direction and is used to identify directional defects such as scratches. The method of associating the pixel area, aspect ratio, grayscale contrast, and orientation consistency of the region refers to concatenating these feature values ​​into a one-dimensional array to form a defect sub-vector. These multiple defect sub-vectors refer to the feature array corresponding to each defect region. The method of normalizing these multiple defect sub-vectors refers to using a Min-Max or Z-score normalization algorithm to make the feature values ​​fall within [0,1] or a standard distribution, facilitating subsequent evaluation and obtaining a defect vector. This defect vector is the average vector of all defect sub-vectors, used to characterize the overall defect.

[0045] For example, suppose that the preprocessed light and dark texture image is analyzed to obtain 3 candidate defect regions, and after screening, 2 defect regions are retained; the first defect region is analyzed to obtain the following values: region pixel area = 200, aspect ratio = 2.5, grayscale contrast = 0.3, orientation consistency = 0.8, which are sorted to form a defect sub-vector [200, 2.5, 0.3, 0.8].

[0046] S6. Based on the preprocessed high-definition color image and multiple preset key regions, perform color difference calculation to obtain color difference value and text similarity.

[0047] It should be understood that the step of calculating color difference based on the preprocessed high-definition color image and multiple preset key regions to obtain color difference values ​​and text similarity includes: Based on the key regions, multiple detection areas were identified in the preprocessed high-definition color image. Multidimensional color detection was performed on the multiple detection areas to obtain the average hue, average brightness, and average vividness. Obtain standard hue, standard brightness, and standard vividness; and calculate hue difference, brightness difference, and vividness difference based on the average hue, average brightness, average vividness, standard hue, standard brightness, and standard vividness. The color difference value is calculated based on the hue difference, brightness difference, and vividness difference, using the following formula:

[0048] in, Indicates the color difference value. Indicates the difference in brightness. Indicates the difference in vibrancy. Indicates the hue difference. , and These represent the brightness correction, vividness correction, and hue correction items, respectively. , and All represent weighting coefficients. Indicates compensation item; Based on a preset text recognition algorithm, text is extracted from the multiple detection areas to obtain image-extracted text. The image-extracted text is then compared with a preset standard benchmark text to obtain text similarity.

[0049] It should be explained that the method of identifying multiple detection areas in a preprocessed high-definition color image based on the key regions refers to locating and cropping these areas in the preprocessed high-definition color image according to preset key regions (e.g., coordinates of color blocks, pattern areas, or text areas on printed materials), forming multiple detection areas. These multiple detection areas refer to image sub-blocks used for color and text analysis. The method of performing multi-dimensional color detection on the multiple detection areas refers to calculating the average hue (average value of H channel), average brightness (average value of V or L channel), and average vividness (average value of S or ab channel modulus) of each detection area in the HSV or Lab color space. These average hue, average brightness, and average vividness refer to the color statistical indicators of the detection areas, used to quantify printing color deviation. The method of obtaining standard hue, standard brightness, and standard vividness refers to extracting the reference values ​​of the corresponding key regions from a defect-free template image or database. These standard hue, standard brightness, and standard vividness refer to the hue, brightness, and vividness of an ideal printed product. The method for calculating hue difference, luminance difference, and luminance difference based on the average hue, average luminance, average vividness, standard hue, standard luminance, and standard luminance refers to calculating the difference between the average value and the standard value, respectively. The hue difference, luminance difference, and luminance difference refer to the differences between the average hue, average luminance, and average luminance of key areas in a high-definition color image and the standard hue, standard luminance, and standard luminance. The luminance correction term is a dynamic correction coefficient used to "amplify" or "reduce" the contribution of the luminance difference to the color difference value; its calculation formula is as follows:

[0050] in, Indicates the weighting coefficient. Indicates standard brightness.

[0051] The This refers to the preset weighting coefficient, with a default value of 1. The vividness correction item refers to the correction coefficient used to "enlarge" or "reduce" the vividness difference in the color difference value, and its calculation formula is as follows:

[0052] in, Indicates the weighting coefficient. Indicates standard vividness.

[0053] The This refers to the preset weighting coefficient, with a default value of 1. The brightness correction item refers to the correction coefficient used to "magnify" or "reduce" the contribution of the hue difference to the color difference value, and its calculation formula is as follows:

[0054] Among them, the These are the weighting coefficients. This is the hue correction factor.

[0055] The This refers to the preset weighting coefficient, with a default value of 1. The hue correction factor is a fixed value calculated based on the standard hue. For example, when the standard hue... When the value is 0° (corresponding to the red area), substitute it into the following formula: Calculation yields ≈1.0, when the standard hue =240° (corresponding to the blue area), substituting into the formula, we can obtain... ≈0.5. The compensation term refers to the compensation coefficient used to correct the color difference calculation deviation in high-vividity hue areas and ensure the accuracy of color difference assessment for key hues such as red and yellow. Optionally, the compensation term can be calculated using the CIEDE2000 international standard formula. The color difference value refers to the overall color deviation; the larger the color difference value, the more significant the color difference, used for subsequent overall evaluation. The weighting coefficient refers to the preset coefficient used to control the weight of different indicators; its default value is 1, but it can be adjusted according to the actual scenario. The method of extracting text from the multiple detection areas based on the preset text recognition algorithm refers to using an OCR algorithm (such as Tesseract or EasyOCR) to recognize and extract printed text from the detection areas. The extracted text refers to the string sequence extracted from the detection areas, used for subsequent similarity calculation.

[0056] Furthermore, the step of verifying the extracted text from the image with a preset standard benchmark text to obtain text similarity includes: The extracted text from the image and the standard reference text are compared character by character using a preset text matching algorithm to obtain the number of correctly matched characters, the number of incorrectly inserted characters, the number of incorrectly deleted characters, and the number of incorrectly replaced characters. The text recall rate is calculated based on the number of correctly matched characters, the number of incorrectly deleted characters, and the number of incorrectly replaced characters. The calculation formula is as follows:

[0057] in, Indicates text recall. Indicates the number of correctly matched characters. Indicates the number of characters deleted incorrectly. Indicates the number of characters replaced incorrectly; The text accuracy is calculated based on the number of correctly matched characters, the number of incorrectly inserted characters, and the number of incorrectly replaced characters. The calculation formula is as follows:

[0058] in, Indicates text precision. Indicates the number of characters inserted incorrectly; Text similarity is calculated based on the text recall and text precision, using the following formula:

[0059] in, Indicates text similarity.

[0060] It should be understood that the method of comparing the extracted text from the image with the standard reference text character by character refers to using a text matching algorithm to compare two strings, identifying matching, insertion, deletion, and replacement operations, and obtaining the number of correctly matched characters (completely identical characters), the number of incorrectly inserted characters (redundant characters), the number of incorrectly deleted characters (missing characters), and the number of incorrectly replaced characters (replaced characters). Optionally, the Levenshtein distance algorithm can be used as the text matching algorithm. The method of calculating the text recall based on the number of correctly matched characters, the number of incorrectly deleted characters, and the number of incorrectly replaced characters refers to using a formula to calculate the recall rate, which is the proportion of the extracted text covering the standard text, used to assess completeness. The text precision is the proportion of the correct parts in the extracted text, used to assess accuracy. The text similarity is the harmonic mean of the combined recall and precision, used to comprehensively assess the text printing quality.

[0061] For example, suppose the extracted text is "Hello Wrad", the standard baseline text is "Hello World", the number of correctly matched characters obtained by character-by-character comparison is 10, the insertion error is 0, the deletion error is 1 (missing 'o'), and the replacement error is 1 ('a' replaced by 'l'); the recall rate is 10 / (10+1+1)=0.833, the precision rate is 10 / (10+0+1)=0.909, and the text similarity F=20.8330.909 / (0.833+0.909)=0.87.

[0062] S7. Based on the identification and classification unit, geometric feature vector, defect vector, color difference value and text similarity, obtain the defect assessment index. Based on the defect assessment index, the preset classification level and the sorting execution unit, sort and classify the printed products to obtain classified printed products. Based on the classified printed products, intelligent sorting of the printed products is realized.

[0063] It should be explained that the method of obtaining the defect assessment index based on the identification and classification unit, geometric feature vector, defect vector, color difference value, and text similarity includes: Perform a normalization operation on the geometric eigenvectors to obtain the geometric eigenvector norm; Based on the aforementioned identification and classification unit, geometric feature vector norm, defect vector, color difference value, and text similarity, the defect assessment index is calculated using the following formula:

[0064] in, Indicates the defect assessment index. Represents the norm of geometric eigenvectors. Represents the defect vector. Represents a nonlinear mapping function. , , and All of these represent weighting coefficients.

[0065] Furthermore, the method of performing a normalization operation on the geometric feature vector refers to calculating the Euclidean norm (L2 norm) of the geometric feature vector, that is, taking the square root of the sum of the squares of each element in the vector to obtain a scalar value. The geometric feature vector norm is a comprehensive measure of geometric features; the larger the norm, the more significant the geometric deviation. The method of calculating the defect assessment index based on the recognition and classification unit, the geometric feature vector norm, the defect vector, the color difference value, and the text similarity refers to inputting these parameters into an exponential function for nonlinear fusion to obtain a comprehensive defect score. The defect assessment index is a quantitative indicator of the overall quality of the printed product; the smaller the defect assessment index, the more serious the defect, which is used for subsequent classification decisions. The nonlinear mapping function is a preset activation function used to enhance the nonlinear expression of the defect vector. Optionally, the Sigmoid function can be used as the nonlinear mapping function. The weight coefficients are preset adjustment parameters used to balance the contribution of each factor in the defect assessment index. They can be adjusted according to the actual printing scenario, for example, 0.1, 0.2, 0.3, and 0.4 respectively. The identification and classification unit refers to a functional module that can integrate the above-mentioned parameters and vectors and calculate the defect assessment index in order to classify the printed products in the future.

[0066] It should be understood that the sorting and classification of the printed products based on the defect assessment index, the preset classification level, and the sorting execution unit to obtain classified printed products includes: A classification threshold is determined based on the classification level, wherein the classification threshold includes a first-level threshold, a second-level threshold, and a third-level threshold. If the defect assessment index is greater than or equal to the first-level threshold, the printed product is determined to be a high-quality product. If the defect assessment index is less than the first-level threshold and greater than or equal to the second-level threshold, the printed product is confirmed as a qualified product. If the defect assessment index is less than the second-level threshold and greater than or equal to the third-level threshold, the printed product is identified as a slightly defective product. If the defect assessment index is less than the level three threshold, the printed product will be identified as a defective product. The sorting unit sorts and classifies the high-quality products, qualified products, slightly defective products, or substandard products to obtain classified printed products.

[0067] It should be explained that the method of determining the classification threshold based on the classification level refers to extracting threshold boundaries from an empirical database or training model according to a preset classification level (e.g., four levels: excellent, qualified, slightly defective, and substandard). The classification threshold refers to the critical value used to divide the defect assessment index. The first-level threshold, the second-level threshold, and the third-level threshold represent the quality dividing points from high to low, respectively. For example, the first-level threshold can be set to 0.9, the second-level threshold can be set to 0.7, and the third-level threshold can be set to 0.5, which can be dynamically adjusted according to the type of printed product. The method of determining the printed product as excellent means that when the defect assessment index is higher than the first-level threshold, it is judged as a high-quality product without obvious defects. Excellent products refer to printed products suitable for direct shipment or high-end applications. The method of determining the printed product as qualified means that when the defect assessment index is between the first-level and second-level thresholds, it is judged as a product that basically meets the standards. Qualified products refer to acceptable normal printed products. The method of determining the printed product as slightly defective means that when the defect assessment index is between the second-level and third-level thresholds, it is judged as a product with minor defects. Slightly defective products refer to printed products that can be used after simple repair. The method for identifying the printed products as defective products refers to determining a product as having a serious defect when the defect assessment index is below the level three threshold. Defective products refer to printed products that need to be discarded or reprinted. The method for sorting and classifying the high-quality, qualified, slightly defective, or defective products based on the sorting execution unit refers to using the sorting execution unit to convey the printed products to different output channels according to the classification results. The sorting execution unit refers to a hardware execution module in an intelligent sorting system used to achieve physical separation. Optionally, the sorting execution unit can be constructed using a robotic arm, conveyor belt, and programmable logic controller (PLC).

[0068] To address the problems described in the background art, this invention confirms the receipt of intelligent sorting instructions and, based on these instructions, confirms the intelligent sorting environment. The intelligent sorting environment includes an intelligent sorting system and printed products. The intelligent sorting system includes an image acquisition unit, an image processing unit, a recognition and classification unit, and a sorting execution unit. The image acquisition unit includes a color image acquisition device, a line scanning device, and a light source device. Therefore, this invention considers the diversity and real-time requirements of complex printing environments during the sorting process of printed products. By confirming the intelligent sorting environment, modular collaboration of the system is ensured, providing a reliable foundation for subsequent visual processing, thereby improving the overall stability and adaptability of the sorting process. Based on the image acquisition unit… The image acquisition unit performs image acquisition on the printed product to obtain an image set of the printed product, which includes high-definition color images, light and dark texture images, and silhouette images. This demonstrates that the present invention employs a multi-modal image acquisition mechanism to capture the multi-dimensional features of the printed product, avoiding the limitations of a single image, and thus laying a comprehensive data foundation for defect detection. Based on the image acquisition unit, the printed product image set is preprocessed to obtain preprocessed high-definition color images, preprocessed light and dark texture images, and preprocessed silhouette images. This shows that the present invention performs preprocessing immediately after acquisition to optimize image quality and reduce noise interference, thereby improving the accuracy and efficiency of subsequent analysis. Based on the image processing unit, the preprocessed silhouette images are further processed... Feature extraction yields a silhouette image parameter set, and geometric feature vectors are obtained based on this parameter set. This demonstrates that the present invention extracts geometric features from silhouette images, achieving precise quantification of shape and contour, thus avoiding errors associated with traditional manual inspection. Based on the image processing unit's identification of multiple defect regions in the preprocessed light and dark texture image, feature calculations are performed on these defect regions to obtain defect vectors. This embodiment of the present invention identifies and quantifies defect regions through texture analysis, forming vector descriptions, thereby improving the objectivity and fine granularity of defect assessment. Color difference calculations are performed based on the preprocessed high-definition color image and multiple preset key regions to obtain color difference values ​​and text similarity. This demonstrates that the present invention combines... High-definition images are used to calculate color difference and text similarity to evaluate core indicators of printing quality, thereby ensuring the comprehensiveness and reliability of sorting standards. A defect assessment index is obtained based on the identification and classification unit, geometric feature vectors, defect vectors, color difference values, and text similarity. Based on the defect assessment index, preset classification levels, and the sorting execution unit, the printed products are sorted and classified to obtain categorized printed products. Based on these categorized printed products, intelligent sorting of the printed products is achieved. It is evident that this invention integrates multi-source features to calculate a comprehensive index during the classification stage, forming a closed-loop sorting execution, and adapts to different needs through dynamic levels. Therefore, this invention can improve the accuracy, efficiency, and automation level of intelligent sorting of printed products.

[0069] like Figure 2 The diagram shown is a functional block diagram of a machine vision-based intelligent sorting system for printed products provided in an embodiment of the present invention.

[0070] The machine vision-based intelligent sorting system 100 for printed materials described in this invention can be installed in an electronic device. Depending on the functions implemented, the machine vision-based intelligent sorting system 100 for printed materials may include an environment verification module 101, an image acquisition module 102, an image processing module 103, and a classification and sorting module 104. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device.

[0071] The environment confirmation module 101 is used to confirm the receipt of the intelligent sorting instruction and confirm the intelligent sorting environment based on the intelligent sorting instruction. The intelligent sorting environment includes an intelligent sorting system and printed products. The intelligent sorting system includes an image acquisition unit, an image processing unit, an identification and classification unit, and a sorting execution unit. The image acquisition unit includes a color photo acquisition device, a line scanning device, and a light source device. The image acquisition module 102 is used to acquire images of the printed product based on the image acquisition unit to obtain a set of printed product images, wherein the set of printed product images includes high-definition color images, light and dark texture images and silhouette images; The image processing module 103 is used to perform image preprocessing on the printed product image set based on the image acquisition unit to obtain a preprocessed high-definition color image, a preprocessed light and dark texture image and a preprocessed silhouette image. Based on the image processing unit, feature extraction is performed on the preprocessed silhouette image to obtain a silhouette image parameter set, and a geometric feature vector is obtained based on the silhouette image parameter set. Based on the image processing unit, multiple defect regions are identified in the preprocessed light and dark texture image. Based on the image processing unit, feature calculation is performed on the multiple defect regions to obtain defect vectors. Based on the preprocessed high-definition color image and multiple preset key regions, color difference calculation is performed to obtain color difference value and text similarity; The classification and sorting module 104 is used to obtain a defect assessment index based on the identification and classification unit, geometric feature vector, defect vector, color difference value and text similarity, and to sort and classify the printed products based on the defect assessment index, the preset classification level and the sorting execution unit to obtain classified printed products. Based on the classified printed products, intelligent sorting of the printed products is realized.

[0072] In detail, the modules in the machine vision-based intelligent sorting system 100 for printed products described in this embodiment of the invention employ the same methods as described above. Figure 1 The method uses the same technical means as the intelligent sorting method for printed products based on machine vision described in the article, and can produce the same technical effect, so it will not be repeated here.

[0073] like Figure 3 The diagram shown is a structural schematic of an electronic device for implementing a machine vision-based intelligent sorting method for printed materials, according to an embodiment of the present invention.

[0074] The electronic device 1 may include a processor 10, a memory 11 and a bus 12, and may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as a machine vision-based intelligent sorting method program for printed materials.

[0075] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as the portable hard drive of the electronic device 1. In other embodiments, the memory 11 can be an external storage device of the electronic device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 1. Furthermore, the memory 11 includes both internal storage units and external storage devices of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code of a machine vision-based intelligent sorting method for printed materials, but also to temporarily store data that has been output or will be output.

[0076] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., a machine vision-based intelligent sorting method for printed materials) and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.

[0077] The bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 12 can be divided into an address bus, a data bus, a control bus, etc. The bus 12 is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc.

[0078] Figure 3 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0079] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0080] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 1 and other electronic devices.

[0081] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.

[0082] The machine vision-based intelligent sorting method program for printed materials stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When run in the processor 10, it can achieve the following: The system confirms receipt of the intelligent sorting instruction and confirms the intelligent sorting environment based on the instruction. The intelligent sorting environment includes an intelligent sorting system and printed products. The intelligent sorting system includes an image acquisition unit, an image processing unit, an identification and classification unit, and a sorting execution unit. The image acquisition unit includes a color photo acquisition device, a line scanning device, and a light source device. The printed product image set is obtained by acquiring images of the printed product based on the image acquisition unit, wherein the printed product image set includes high-definition color images, light and dark texture images and silhouette images; Based on the image acquisition unit, the printed product image set is preprocessed to obtain a preprocessed high-definition color image, a preprocessed light and dark texture image, and a preprocessed silhouette image. Based on the image processing unit, feature extraction is performed on the preprocessed silhouette image to obtain a silhouette image parameter set, and a geometric feature vector is obtained based on the silhouette image parameter set. Based on the image processing unit, multiple defect regions are identified in the preprocessed light and dark texture image. Based on the image processing unit, feature calculation is performed on the multiple defect regions to obtain defect vectors. Based on the preprocessed high-definition color image and multiple preset key regions, color difference calculation is performed to obtain color difference value and text similarity; Based on the identification and classification unit, geometric feature vector, defect vector, color difference value and text similarity, a defect assessment index is obtained. Based on the defect assessment index, the preset classification level and the sorting execution unit, the printed products are sorted and classified to obtain classified printed products. Based on the classified printed products, intelligent sorting of the printed products is realized.

[0083] Specifically, the processor 10's implementation method for the above instructions can be found in [reference needed]. Figures 1 to 3 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0084] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0085] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following: The system confirms receipt of the intelligent sorting instruction and confirms the intelligent sorting environment based on the instruction. The intelligent sorting environment includes an intelligent sorting system and printed products. The intelligent sorting system includes an image acquisition unit, an image processing unit, an identification and classification unit, and a sorting execution unit. The image acquisition unit includes a color photo acquisition device, a line scanning device, and a light source device. The printed product image set is obtained by acquiring images of the printed product based on the image acquisition unit, wherein the printed product image set includes high-definition color images, light and dark texture images and silhouette images; Based on the image acquisition unit, the printed product image set is preprocessed to obtain a preprocessed high-definition color image, a preprocessed light and dark texture image, and a preprocessed silhouette image. Based on the image processing unit, feature extraction is performed on the preprocessed silhouette image to obtain a silhouette image parameter set, and a geometric feature vector is obtained based on the silhouette image parameter set. Based on the image processing unit, multiple defect regions are identified in the preprocessed light and dark texture image. Based on the image processing unit, feature calculation is performed on the multiple defect regions to obtain defect vectors. Based on the preprocessed high-definition color image and multiple preset key regions, color difference calculation is performed to obtain color difference value and text similarity; Based on the identification and classification unit, geometric feature vector, defect vector, color difference value and text similarity, a defect assessment index is obtained. Based on the defect assessment index, the preset classification level and the sorting execution unit, the printed products are sorted and classified to obtain classified printed products. Based on the classified printed products, intelligent sorting of the printed products is realized.

[0086] In the embodiments provided by this invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and actual implementations may have other classification methods.

[0087] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0088] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0089] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A machine vision-based intelligent sorting method for printed materials, characterized in that, The method includes: The system confirms receipt of the intelligent sorting instruction and confirms the intelligent sorting environment based on the instruction. The intelligent sorting environment includes an intelligent sorting system and printed products. The intelligent sorting system includes an image acquisition unit, an image processing unit, an identification and classification unit, and a sorting execution unit. The image acquisition unit includes a color photo acquisition device, a line scanning device, and a light source device. The printed product image set is obtained by acquiring images of the printed product based on the image acquisition unit, wherein the printed product image set includes high-definition color images, light and dark texture images and silhouette images; Based on the image acquisition unit, the printed product image set is preprocessed to obtain a preprocessed high-definition color image, a preprocessed light and dark texture image, and a preprocessed silhouette image. Based on the image processing unit, feature extraction is performed on the preprocessed silhouette image to obtain a silhouette image parameter set, and a geometric feature vector is obtained based on the silhouette image parameter set. Based on the image processing unit, multiple defect regions are identified in the preprocessed light and dark texture image. Based on the image processing unit, feature calculation is performed on the multiple defect regions to obtain defect vectors. Based on the preprocessed high-definition color image and multiple preset key regions, color difference calculation is performed to obtain color difference value and text similarity; Based on the identification and classification unit, geometric feature vector, defect vector, color difference value and text similarity, a defect assessment index is obtained. Based on the defect assessment index, the preset classification level and the sorting execution unit, the printed products are sorted and classified to obtain classified printed products. Based on the classified printed products, intelligent sorting of the printed products is realized.

2. The intelligent sorting method for printed materials based on machine vision as described in claim 1, characterized in that, The step of acquiring images of the printed product based on the image acquisition unit to obtain a set of printed product images includes: The printed product is tracked and positioned based on a pre-built conveying device and encoder to obtain a real-time positioning area. A visual detection area is obtained. If the real-time positioning area reaches the visual detection area, an image acquisition command is obtained. Based on the image acquisition command and the color photo acquisition device, a color photo is acquired on the printed product located in the visual detection area to obtain a high-definition color image. Based on the line scanning device, a light and dark texture image is obtained; based on the line scanning device and the light source device, a silhouette image is obtained by capturing the silhouette of the printed product in the visual detection area. By combining the high-definition color images, light and dark texture images, and silhouette images, a set of printed product images is obtained.

3. The intelligent sorting method for printed products based on machine vision as described in claim 2, characterized in that, The image preprocessing of the printed product image set based on the image acquisition unit to obtain a preprocessed high-definition color image, a preprocessed light and dark texture image, and a preprocessed silhouette image includes: Based on the preprocessed image set and the pre-constructed white balance algorithm, the high-definition color image is color corrected to obtain a preprocessed high-definition color image; Based on the image acquisition unit, the contrast of the light and dark texture image is enhanced to obtain a preprocessed light and dark texture image; The silhouette image is binarized using the image acquisition unit to obtain a preprocessed silhouette image.

4. The intelligent sorting method for printed materials based on machine vision as described in claim 3, characterized in that, The step of extracting features from the preprocessed silhouette image using the image processing unit to obtain a silhouette image parameter set, and obtaining a geometric feature vector based on the silhouette image parameter set, includes: Based on the image processing unit set and the preset contour tracking algorithm, the contour pixels of the preprocessed silhouette image are extracted to obtain the contour pixel set; The outer contour point set is divided based on the contour pixel point set, and the minimum outer bounding matrix is ​​obtained based on the outer contour point set. The actual length, actual width and actual contour area are calculated based on the minimum outer bounding matrix. Obtain a standard outer contour point set, and based on the standard outer contour point set, obtain the standard length, standard width, and standard contour area. Based on the actual length, actual width, actual contour area, standard length, standard width, and standard contour area, calculate the length deviation, width deviation, and area deviation. The right-angle deviation between the contour pixel set and the standard external contour point set is calculated using a preset contour analysis algorithm. The length deviation, width deviation, area deviation, and right angle deviation are summarized to obtain the silhouette image parameter set. The silhouette image parameter set is then mapped according to a preset dimensional order to obtain the geometric feature vector.

5. The intelligent sorting method for printed materials based on machine vision as described in claim 4, characterized in that, The image processing unit identifies multiple defect regions in the preprocessed light and dark texture image, and performs feature calculations on these defect regions to obtain defect vectors, including: Based on the image processing unit, a difference analysis is performed on the preprocessed light and dark texture image and the preset defect-free template image to obtain multiple candidate defect regions; Based on a preset minimum defect area threshold, the multiple candidate defect regions are filtered to obtain multiple defect regions. For each of the multiple defect regions, the following operation is performed: Pixel statistics are performed on the defective region to obtain the region's pixel area. The minimum bounding matrix is ​​obtained based on the defective region, and the aspect ratio is calculated based on the minimum bounding matrix. The average gray value of the pixels and the average gray value of the background are obtained based on the defect area. The gray value contrast is calculated based on the average gray value of the pixels and the average gray value of the background. The consistency of the pixel gradient direction within the defect area is calculated based on the preset histogram algorithm to obtain the direction consistency. The pixel area, aspect ratio, grayscale contrast, and orientation consistency of the region are correlated to obtain multiple defect sub-vectors. The multiple defect sub-vectors are then normalized to obtain a defect vector.

6. The intelligent sorting method for printed materials based on machine vision as described in claim 5, characterized in that, The step of calculating color difference based on the preprocessed high-definition color image and multiple preset key regions to obtain color difference values ​​and text similarity includes: Based on the key regions, multiple detection areas were identified in the preprocessed high-definition color image. Multidimensional color detection was performed on the multiple detection areas to obtain the average hue, average brightness, and average vividness. Obtain standard hue, standard brightness, and standard vividness; and calculate hue difference, brightness difference, and vividness difference based on the average hue, average brightness, average vividness, standard hue, standard brightness, and standard vividness. The color difference value is calculated based on the hue difference, brightness difference, and vividness difference, using the following formula: in, Indicates the color difference value. Indicates the difference in brightness. Indicates the difference in vibrancy. Indicates the hue difference. , and These represent the brightness correction, vividness correction, and hue correction items, respectively. , and All represent weighting coefficients. Indicates compensation item; Based on a preset text recognition algorithm, text is extracted from the multiple detection areas to obtain image-extracted text. The image-extracted text is then compared with a preset standard benchmark text to obtain text similarity.

7. The intelligent sorting method for printed materials based on machine vision as described in claim 6, characterized in that, The step of verifying the extracted text from the image against a preset standard benchmark text to obtain text similarity includes: The extracted text from the image and the standard reference text are compared character by character using a preset text matching algorithm to obtain the number of correctly matched characters, the number of incorrectly inserted characters, the number of incorrectly deleted characters, and the number of incorrectly replaced characters. The text recall rate is calculated based on the number of correctly matched characters, the number of incorrectly deleted characters, and the number of incorrectly replaced characters. The calculation formula is as follows: in, Indicates text recall. Indicates the number of correctly matched characters. Indicates the number of characters deleted incorrectly. Indicates the number of characters replaced incorrectly; The text accuracy is calculated based on the number of correctly matched characters, the number of incorrectly inserted characters, and the number of incorrectly replaced characters. The calculation formula is as follows: in, Indicates text precision. Indicates the number of characters inserted incorrectly; Text similarity is calculated based on the text recall and text precision, using the following formula: in, Indicates text similarity.

8. The intelligent sorting method for printed materials based on machine vision as described in claim 7, characterized in that, The process of obtaining the defect assessment index based on the identification and classification unit, geometric feature vector, defect vector, color difference value, and text similarity includes: Perform a normalization operation on the geometric eigenvectors to obtain the geometric eigenvector norm; Based on the aforementioned identification and classification unit, geometric feature vector norm, defect vector, color difference value, and text similarity, the defect assessment index is calculated using the following formula: in, Indicates the defect assessment index. Represents the norm of geometric eigenvectors. Represents the defect vector. Represents a nonlinear mapping function. , , and All of these represent weighting coefficients.

9. The intelligent sorting method for printed materials based on machine vision as described in claim 8, characterized in that, The sorting and classification of the printed products based on the defect assessment index, the preset classification level, and the sorting execution unit to obtain classified printed products includes: A classification threshold is determined based on the classification level, wherein the classification threshold includes a first-level threshold, a second-level threshold, and a third-level threshold. If the defect assessment index is greater than or equal to the first-level threshold, the printed product is determined to be a high-quality product. If the defect assessment index is less than the first-level threshold and greater than or equal to the second-level threshold, the printed product is confirmed as a qualified product. If the defect assessment index is less than the second-level threshold and greater than or equal to the third-level threshold, the printed product is identified as a slightly defective product. If the defect assessment index is less than the level three threshold, the printed product will be identified as a defective product. The sorting unit sorts and classifies the high-quality products, qualified products, slightly defective products, or substandard products to obtain classified printed products.

10. A machine vision-based intelligent sorting system for printed materials, characterized in that, The device includes: An environment confirmation module is used to confirm the receipt of intelligent sorting instructions and to confirm the intelligent sorting environment based on the intelligent sorting instructions. The intelligent sorting environment includes an intelligent sorting system and printed products. The intelligent sorting system includes an image acquisition unit, an image processing unit, a recognition and classification unit, and a sorting execution unit. The image acquisition unit includes a color photo acquisition device, a line scanning device, and a light source device. An image acquisition module is used to acquire images of the printed product based on the image acquisition unit to obtain a set of printed product images, wherein the set of printed product images includes high-definition color images, light and dark texture images, and silhouette images; The image processing module is used to perform image preprocessing on the printed product image set based on the image acquisition unit to obtain a preprocessed high-definition color image, a preprocessed light and dark texture image, and a preprocessed silhouette image. Based on the image processing unit, feature extraction is performed on the preprocessed silhouette image to obtain a silhouette image parameter set, and a geometric feature vector is obtained based on the silhouette image parameter set. Based on the image processing unit, multiple defect regions are identified in the preprocessed light and dark texture image. Based on the image processing unit, feature calculation is performed on the multiple defect regions to obtain defect vectors. Based on the preprocessed high-definition color image and multiple preset key regions, color difference calculation is performed to obtain color difference value and text similarity; The classification and sorting module is used to obtain a defect assessment index based on the identification and classification unit, geometric feature vector, defect vector, color difference value and text similarity. Based on the defect assessment index, the preset classification level and the sorting execution unit, the printed products are sorted and classified to obtain classified printed products. Based on the classified printed products, intelligent sorting of the printed products is realized.

Images