A fusion sorting method based on multi-band multiplexing

By adopting a multi-band multiplexing fusion sorting method in the color sorting machine, combined with deep learning and color sorting algorithm, the problem that a single light source and algorithm are difficult to sort foreign objects in a specific shape is solved, and high accuracy of large-particle materials is achieved Sorting significantly improves the sorting effect.

CN114441444BActive Publication Date: 2024-11-26CHINA HEFEI TAIHE OPTOELECTRONICS TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210095829.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-26
Publication Date
2024-11-26
Estimated Expiration
2042-01-26

AI Technical Summary

Technical Problem

It is difficult for existing color sorting machines to sort foreign objects with specific shape characteristics using a single light source, and the single color sorting algorithm is not effective in sorting the shape, texture, and type information of large-particle materials.

Method used

Using a fusion sorting method based on multi-band multiplexing, the materials to be screened are photographed by light sources of different bands, and images matching the photosensitive properties of qualified materials and foreign matter impurities are determined. The data fusion is combined with deep learning algorithms and color sorting algorithms to improve the Selection accuracy.

Benefits of technology

Determining the type of materials to be screened from multiple dimensions significantly improves the accuracy of sorting and can more effectively identify and remove foreign matter impurities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114441444B_ABST
    Figure CN114441444B_ABST
Patent Text Reader

Abstract

The present invention discloses a fusion sorting method based on multi-band multiplexing, and relates to the technical field of color sorters. The method is applied to a color sorter. When it is detected that a material to be screened enters a target area, light sources of different bands are used in sequence to take photos of the material to be screened, and multiple images containing the material to be screened are obtained as the first image; the material type of the material to be screened includes qualified materials and foreign impurities; an image matching the photosensitive properties of the qualified material is determined from the first image as the second image, and an image matching the photosensitive properties of the foreign impurities is determined from the first image as the third image; the material type of the material to be screened is determined according to the image features of the second image and the third image; if the material type of the material to be screened is a foreign impurity, the material to be screened is removed. The type of the material to be screened is determined from multiple dimensions, and the material to be screened is sorted, thereby improving the accuracy of sorting.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of color sorters, and in particular to a fusion sorting method based on multi-band multiplexing. Background Art

[0002] Color sorters are mainly used for sorting agricultural products such as rice, walnuts, peanuts, and soybeans. They can remove impurities, foreign bodies, and unqualified materials from the above materials. The specific working principle is: when the material leaves the trough, the material reflects the light emitted by the illumination light source to the material, and the reflected light enters the sensor to obtain an image of the material. The processor processes the image and analyzes the characteristics of the image to determine whether the material is qualified, foreign, or unqualified. The controller then controls the removal system to remove foreign impurities and unqualified materials.

[0003] In the prior art, it is difficult for a color sorter to sort foreign matter with specific shape characteristics using a single light source, and a single color sorting algorithm is not effective in sorting the shape, texture, and type information of large particles of materials. Summary of the invention

[0004] The purpose of the embodiment of the present invention is to provide a fusion sorting method based on multi-band multiplexing, so as to determine the type of materials to be screened from multiple dimensions, sort the materials to be screened, and improve the accuracy of sorting.

[0005] The specific technical solutions are as follows:

[0006] In a first aspect of the present invention, a fusion sorting method based on multi-band multiplexing is first provided, the method is applied to a color sorter, and the method comprises:

[0007] When it is detected that the material to be screened enters the target area, light sources of different wavelengths are used in sequence to take photos of the material to be screened, and multiple images containing the material to be screened are obtained as the first image; the material types of the material to be screened include qualified materials and foreign impurities;

[0008] Determine an image that matches the photosensitive property of the qualified material from the first image as the second image, and determine an image that matches the photosensitive property of the foreign matter from the first image as the third image;

[0009] determining the material type of the material to be screened according to the image features of the second image and the third image;

[0010] If the material type of the material to be screened is foreign matter, the material to be screened is discarded.

[0011] Optionally, the color sorter includes a light source control module, a light source driving module, a light source module and an image acquisition module; wherein the light source module includes N light source sub-modules; the light bands emitted by the N light source sub-modules do not overlap with each other;

[0012] When it is detected that the material to be screened enters the target area, light sources of different wavelengths are used in sequence to take photos of the material to be screened, and a plurality of images containing the material to be screened are obtained as the first image, including:

[0013] When it is detected that the material to be screened enters the target area, the light source control module sends a trigger pulse with a frequency of F / N to the light source driving module, so that the light source driving module controls the N light source sub-modules in the light source module to emit light of N light bands in sequence;

[0014] At the same time, a pulse trigger signal with a frequency of F is sent to the image acquisition module through the light source control module, so that the image acquisition module takes pictures at the rising edge of the pulse, and collects N images containing the material to be screened generated in different bands as the first image.

[0015] Optionally, the trigger pulse frequencies of the N light source sub-modules in the light source module are controlled to be the same, and the phases thereof differ by π / N°.

[0016] Optionally, the method further comprises:

[0017] A synchronization signal is sent to the image processing module through the light source control module, so that the image processing module determines the corresponding relationship between each light band and the first image captured by the light band according to the synchronization signal.

[0018] Optionally, determining an image matching the photosensitive property of the qualified material from the first image as the second image, and determining an image matching the photosensitive property of the foreign matter from the first image as the third image, comprises:

[0019] According to the photosensitivity property of the qualified material, determine the light band with the highest imaging clarity of the qualified material among the N light bands emitted by the light source module, and use the first image corresponding to the light band as the second image;

[0020] According to the photosensitivity property of the foreign impurities, a light band with the highest imaging clarity of the foreign impurities among the N light bands emitted by the light source module is determined, and a first image corresponding to the light band is used as the third image.

[0021] Optionally, the color sorter further includes a deep learning algorithm processing module, a color sorting algorithm processing module and a data fusion module;

[0022] Determining the type of the material to be screened according to the image features of the second image and the third image includes:

[0023] The second image is input into the color sorting algorithm processing module to obtain the probability that the material type of the material to be screened belongs to foreign matter impurities as the first probability; the color sorting algorithm processing module is used to determine the probability that the material type of the material belongs to foreign matter impurities according to the color attribute of the material in the input image;

[0024] The third image is input into the deep learning algorithm processing module to obtain the probability that the material type of the material to be screened belongs to foreign matter impurities as the second probability; the deep learning algorithm processing module is used to determine the probability that the material type of the material belongs to foreign matter impurities according to the shape attributes of the material in the input image;

[0025] The material type of the material to be screened is determined by the data fusion module according to the first probability, the second probability, a preset weight and a preset threshold.

[0026] Optionally, the color sorter further comprises a rejection module;

[0027] If the material type of the material to be screened is a foreign impurity, then the material to be screened is removed, including:

[0028] If the data fusion module determines that the material type of the material to be screened is a foreign impurity, the blow valve data is sent to the rejection module through the data fusion module, so that the rejection module rejects the material to be screened.

[0029] Based on the fusion sorting method based on multi-band multiplexing provided by the embodiment of the present invention, when it is detected that the material to be screened enters the target area, light sources of different bands are used in turn to take pictures of the material to be screened, and multiple images containing the material to be screened are obtained as the first image; the material type of the material to be screened includes qualified materials and foreign impurities; an image matching the photosensitive properties of the qualified material is determined from the first image as the second image, and an image matching the photosensitive properties of the foreign impurities is determined from the first image as the third image; the material type of the material to be screened is determined according to the image features of the second image and the third image; if the material type of the material to be screened is a foreign impurity, the material to be screened is removed. By collecting images of the material to be screened under light sources of different bands, images matching the photosensitive properties of qualified materials and foreign impurities are selected to analyze the characteristics of the material to be screened. The type of the material to be screened can be determined from multiple dimensions, and the material to be screened can be sorted, thereby improving the accuracy of sorting. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The present invention will be further described below in conjunction with the accompanying drawings.

[0031] Figure 1 A flowchart of a fusion sorting method based on multi-band multiplexing provided in an embodiment of the present invention;

[0032] Figure 2 is a flow chart of another fusion sorting method based on multi-band multiplexing provided by an embodiment of the present invention;

[0033] Figure 3 It is a structural block diagram of a fusion sorting system based on multi-band multiplexing provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] In the prior art, it is difficult for a color sorter to sort foreign matter with specific shape characteristics using a single light source, and a single color sorting algorithm is not effective in sorting the shape, texture, and type information of large particles of materials.

[0036] In view of the above problems, an embodiment of the present invention provides a fusion sorting method based on multi-band multiplexing, which is applied to a color sorter.

[0037] See also Figure 1 , Figure 1 A flowchart of a fusion sorting method based on multi-band multiplexing provided in an embodiment of the present invention, the method may include the following steps:

[0038] S101, when it is detected that the material to be screened enters the target area, light sources of different wavelength bands are used to take pictures of the material to be screened in sequence to obtain multiple images containing the material to be screened as the first image.

[0039] S102, determining an image that matches the photosensitive property of the qualified material from the first image as the second image, and determining an image that matches the photosensitive property of the foreign matter from the first image as the third image.

[0040] S103, determining the material type of the material to be screened according to the image features of the second image and the third image.

[0041] S104: If the material type of the material to be screened is foreign matter or impurities, the material to be screened is discarded.

[0042] The material types to be screened include qualified materials and foreign impurities.

[0043] Based on the fusion sorting method based on multi-band multiplexing provided by the embodiment of the present invention, by collecting images of the materials to be screened under light sources of different bands, images matching the photosensitive properties of qualified materials and foreign impurities are selected to analyze the characteristics of the materials to be screened. The type of materials to be screened can be determined from multiple dimensions, and the materials to be screened can be sorted, thereby improving the accuracy of sorting.

[0044] In one implementation, the materials to be screened may be agricultural products such as rice, walnuts, peanuts, and soybeans, which may be mixed with foreign matter such as stones and glass. The types of materials to be screened include qualified materials and foreign matter impurities. For example, the qualified materials may be qualified rice, walnuts, peanuts, and soybeans, and the foreign matter impurities may be foreign matter such as stones and glass, or the foreign matter impurities may be damaged rice, walnuts, peanuts, and soybeans.

[0045] In one implementation, an image matching the photosensitive properties of qualified materials is determined, in which the features of qualified materials are more obvious, so that foreign impurities can be more easily eliminated. An image matching the photosensitive properties of foreign impurities is determined, in which the features of foreign impurities are more obvious, so that foreign impurities can be more easily determined. Therefore, the material type of the material to be screened can be determined from multiple dimensions, and the material to be screened can be sorted, thereby improving the accuracy of sorting.

[0046] In one implementation, multiple images matching the photosensitive properties of multiple foreign matter impurities may be determined from the first image as the third image.

[0047] In one implementation, a corresponding algorithm model is matched according to the photosensitive properties of the qualified material and the photosensitive properties of the foreign impurities, and the foreign impurities in the second image and the third image are identified by the algorithm model.

[0048] In one embodiment, the color sorter includes a light source control module, a light source driving module, a light source module and an image acquisition module, wherein the light source module includes N light source sub-modules, and the light bands emitted by the N light source sub-modules do not overlap with each other.

[0049] Step S101 includes:

[0050] Step 1: When it is detected that the material to be screened enters the target area, a trigger pulse with a frequency of F / N is sent to the light source driving module through the light source control module, so that the light source driving module controls the N light source sub-modules in the light source module to emit light of N light bands in sequence.

[0051] Step 2. At the same time, a pulse trigger signal with a frequency of F is sent to the image acquisition module through the light source control module, so that the image acquisition module takes pictures at the rising edge of the pulse, and collects N images containing the material to be screened generated in different bands as the first image.

[0052] In one implementation, the image acquisition module includes an image sensor and a lens, wherein the image sensor can respond to light sources of multiple bands, or is composed of multiple sensors that respond to a single band. When the material to be screened passes through the slide of the color sorter and enters the field of view of the camera, the light source control module sends a trigger signal to control the 1-band light source for illumination, and at the same time sends a trigger signal to the camera, so that the camera takes a picture of the material to be screened within the illumination window time of the 1-band light source, and completes the image exposure. Then, the light source control module turns off the 1-band light source, turns on the 2-band light source for illumination, and at the same time sends a trigger signal to the camera, so that the camera takes a picture of the material to be screened within the illumination window time of the 2-band light source, and completes the image exposure. In this way, each band light source is used in turn to take a picture of the material to be screened, and the image exposure is completed. Finally, when the material to be screened falls into the entire field of view, N images generated in different illumination bands can be obtained, and appropriate images are selected for image analysis according to the imaging characteristics of the material to be screened in different bands.

[0053] In one embodiment, the trigger pulse frequencies of the N light source sub-modules in the light source module are controlled to be the same, and the phases thereof differ by π / N°.

[0054] In one implementation, the light source control module controls the light source driving board by sending a pulse signal with a phase difference of π / N° through a control signal, thereby realizing the alternating flashing of the band 1 lighting module, the band 2 lighting module to the band N lighting module, so that only one band of light source is lighting at a certain moment.

[0055] In one embodiment, the method further comprises:

[0056] A synchronization signal is sent to the image processing module through the light source control module, so that the image processing module determines the corresponding relationship between each light band and the first image collected by the light band according to the synchronization signal.

[0057] In one embodiment, step S102 includes:

[0058] Step 1: According to the photosensitivity of the qualified material, determine the light band with the highest imaging clarity of the qualified material among the N light bands emitted by the light source module, and use the first image corresponding to the light band as the second image.

[0059] Step 2: According to the photosensitivity of the foreign matter impurities, determine the light band with the highest foreign matter impurity imaging clarity among the N light bands emitted by the light source module, and use the first image corresponding to the light band as the third image.

[0060] In one embodiment, the color sorter further includes a deep learning algorithm processing module, a color sorting algorithm processing module, and a data fusion module. Figure 2 ,exist Figure 1 On the basis of, step S103 comprises:

[0061] S1031, input the second image into the color sorting algorithm processing module to obtain the probability that the material type of the material to be screened is a foreign impurity as the first probability.

[0062] S1032, input the third image into the deep learning algorithm processing module to obtain the probability that the material type of the material to be screened is a foreign impurity as the second probability.

[0063] S1033, determining the material type of the material to be screened according to the first probability, the second probability, the preset weight and the preset threshold through the data fusion module.

[0064] The color sorting algorithm processing module is used to determine the probability that the material type of the material belongs to foreign matter and impurities according to the color attribute of the material in the input image. The deep learning algorithm processing module is used to determine the probability that the material type of the material belongs to foreign matter and impurities according to the shape attribute of the material in the input image.

[0065] In one implementation, the color sorting algorithm processing module can determine the various color components of the material in the image, and the color components of normal materials, abnormal impurities and image background will be significantly different. The material to be screened on the second image is segmented by the connected domain algorithm, and the probability that the material type of the material belongs to foreign impurities is determined by analyzing the various color components of the segmented material to be screened.

[0066] In one implementation, the technician can collect images of foreign impurities under a matching light source according to the photosensitive properties of the foreign impurities, send the collected images containing foreign impurities to the neural network for training, establish an inference model and calculate the configuration parameters, obtain the feature vector T of the image features, and calculate its corresponding weight. The calculated feature vector T and the corresponding weight are stored as an algorithm model in the deep learning algorithm processing module. The deep learning algorithm processing module can store multiple algorithm models, and the deep learning algorithm processing module can match the corresponding algorithm model according to the photosensitive properties of the foreign impurities, and identify the foreign impurities in the third image through the algorithm model.

[0067] In one implementation, the probability that the material type of the material to be screened is a foreign impurity (which may be referred to as a third probability) can be calculated based on the first probability, the second probability, and the preset weight. For example, the preset weight one may be set to 0.5, the preset weight two may be set to 0.5, the first probability may be multiplied by the preset weight one plus the second probability may be multiplied by the preset weight two to obtain the third probability. Alternatively, the preset weight one may be set to 0.6, the preset weight two may be set to 0.4, the first probability may be multiplied by the preset weight one plus the second probability may be multiplied by the preset weight two to obtain the third probability. If the third probability is greater than the preset threshold, it is determined that the material type of the material to be screened is a foreign impurity. Among them, the preset weight and the preset threshold may be set according to the experience of the technician and are not limited here.

[0068] In one embodiment, the color sorter further includes a rejection module, and step S104 is specifically:

[0069] If the data fusion module determines that the material type of the material to be screened is a foreign impurity, the data fusion module sends the blow valve data to the rejection module so that the rejection module rejects the material to be screened.

[0070] See also Figure 3 , Figure 3 A structural block diagram of a fusion sorting system based on multi-band multiplexing provided in an embodiment of the present invention.

[0071] In one embodiment, the color sorter uses a linear array RGB (Red, Green, Blue, red, green and blue color mode) + NIR (Near Infrared, near infrared spectroscopy technology) sensor, and the light source module is composed of an RGB light source and a NIR light source.

[0072] After the color sorter is started, the light source control module sends a pulse signal with a phase difference of 90° through a control signal to control the light source drive board, so that the RGB light source and the NIR light source flash alternately. At the same time, when the lighting of the RGB light source and the NIR light source is switched alternately, the light source control module sends a trigger pulse to control the image acquisition module to take pictures, so that the corresponding photosensitive area of ​​the sensor is exposed when the lighting of the corresponding band is turned on.

[0073] In one lighting cycle, three lines of RGB data can be collected using RGB light source, and in the next cycle, one line of infrared data can be collected using NIR light source. The collected RGB image data and infrared image data are sent to the color sorting algorithm processing module and the deep learning algorithm processing module respectively, and the corresponding algorithms are adapted according to the characteristics of the materials to be sorted.

[0074] The color sorting algorithm processing module analyzes three consecutive lines of RGB image data. The three RGB color components of normal materials, abnormal materials and background will have significant differences. The materials to be screened on the image are segmented through the connected domain algorithm, and the probability of the material type belonging to foreign impurities is determined by analyzing the RGB color components of the segmented materials to be screened. For example, the normal color components of rice are R=139, G=133, B=116, and an interval R[130, 150], G[123, 143], R[106, 126] can be set. If the color RGB components of the material to be screened are not in the above interval, the probability of determining that the material type of the material is a foreign impurity is 1, if two of the color RGB components of the material to be screened are not in the above interval, then the probability of determining that the material type of the material is a foreign impurity is 0.8, if one of the color RGB components of the material to be screened is not in the above interval, then the probability of determining that the material type of the material is a foreign impurity is 0.6, and if all the color RGB components of the material to be screened are in the above interval, then the probability of determining that the material type of the material is a foreign impurity is 0.1.

[0075] Using deep learning algorithms to judge targets requires the use of complete materials to be screened. Therefore, it is first necessary to cache the collected infrared images, cache 64 lines of infrared data, and preprocess the 64 lines of data to find the position and shape of the materials in the image and segment the materials to be screened.

[0076] Since the image data collected by the linear array sensor is continuous data without a beginning or an end, the intercepted 64 lines of image data may contain part of the material to be screened. The data processing method in the deep learning algorithm processing module uses a rolling shutter method, that is, 64 lines are calculated each time, and the next calculation adds a line to the first line, and then removes the last line of data, ensuring that only 64 lines of data are calculated each time and the material to be screened appears in a complete form.

[0077] The color information obtained by calculating the RGB image in the color selection algorithm processing module and the shape information obtained by calculating the infrared image in the deep learning algorithm processing module are sent to the data fusion module.

[0078] Since the time it takes for the color selection algorithm processing module to calculate the RGB image is not completely consistent with the time it takes for the deep learning algorithm processing module to calculate the infrared image, it is necessary to synchronize the results given by the two images in the data fusion module.

[0079] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A fusion sorting method based on multi-band multiplexing, characterized in that: Applied to a color sorter, the method comprises: When it is detected that the material to be screened enters the target area, light sources of different wavelengths are used in sequence to take photos of the material to be screened, and multiple images containing the material to be screened are obtained as the first image; the material types of the material to be screened include qualified materials and foreign impurities; Determine an image that matches the photosensitive property of the qualified material from the first image as the second image, and determine an image that matches the photosensitive property of the foreign matter from the first image as the third image; determining the material type of the material to be screened according to the image features of the second image and the third image; The color sorter comprises a light source control module, a light source driving module, a light source module and an image acquisition module; wherein the light source module comprises N light source sub-modules; the light bands emitted by the N light source sub-modules do not overlap with each other; When it is detected that the material to be screened enters the target area, light sources of different wavelengths are used in sequence to take photos of the material to be screened, and multiple images containing the material to be screened are obtained as the first image, including: When it is detected that the material to be screened enters the target area, the light source control module sends a trigger pulse with a frequency of F / N to the light source driving module, so that the light source driving module controls the N light source sub-modules in the light source module to emit light of N light bands in sequence; At the same time, a pulse trigger signal with a frequency of F is sent to the image acquisition module through the light source control module, so that the image acquisition module takes pictures at the rising edge of the pulse, and collects N images containing the material to be screened generated in different bands as the first image; Controlling the trigger pulse frequencies of the N light source submodules in the light source module to be the same and to have a phase difference of π / N°; The method further comprises: Sending a synchronization signal to the image processing module through the light source control module; so that the image processing module determines the corresponding relationship between each light band and the first image collected by the light band according to the synchronization signal; Determining an image that matches the photosensitive property of the qualified material from the first image as the second image, and determining an image that matches the photosensitive property of the foreign matter from the first image as the third image, including: According to the photosensitivity property of the qualified material, determine the light band with the highest imaging clarity of the qualified material among the N light bands emitted by the light source module, and use the first image corresponding to the light band as the second image; According to the photosensitive properties of the foreign matter impurities, determine the light band with the highest imaging clarity of the foreign matter impurities among the N light bands emitted by the light source module, and use the first image corresponding to the light band as the third image; The color sorter also includes a deep learning algorithm processing module, a color sorting algorithm processing module and a data fusion module; Determining the type of the material to be screened according to the image features of the second image and the third image includes: The second image is input into the color sorting algorithm processing module to obtain the probability that the material type of the material to be screened belongs to foreign matter impurities as the first probability; the color sorting algorithm processing module is used to determine the probability that the material type of the material belongs to foreign matter impurities according to the color attribute of the material in the input image; The third image is input into the deep learning algorithm processing module to obtain the probability that the material type of the material to be screened belongs to foreign matter impurities as the second probability; the deep learning algorithm processing module is used to determine the probability that the material type of the material belongs to foreign matter impurities according to the shape attributes of the material in the input image; The material type of the material to be screened is determined by the data fusion module according to the first probability, the second probability, a preset weight and a preset threshold.

2. The fusion sorting method based on multi-band multiplexing according to claim 1 is characterized in that: The color sorter also includes a rejection module; If the material type of the material to be screened is a foreign impurity, the material to be screened is removed, including: If the data fusion module determines that the material type of the material to be screened is a foreign impurity, the blow valve data is sent to the rejection module through the data fusion module; So that the rejection module can reject the material to be screened.