A tobacco leaf quality monitoring system and method based on multi-source data fusion
By adopting multi-source data fusion technology in the tobacco leaf quality monitoring system, optimizing the light environment and automated control, combined with multi-source data feature extraction, the problems of poor light environment and insufficient automation identification in the existing technology are solved, and the tobacco leaf quality monitoring is improved Efficiency and accuracy.
Patent Information
- Application Number
- CN202411094171.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-08-09
AI Technical Summary
The existing tobacco leaf quality monitoring technology has problems such as poor light environment, lack of intelligence and automation, light source attenuation and incomplete feature extraction, resulting in low efficiency and insufficient accuracy.
Adopting a tobacco leaf quality monitoring system based on multi-source data fusion, the CCD camera, light source, baffle and photoelectric sensor are set in the image acquisition box, combined with the transmission controller and the image processing host, the light environment in the image acquisition box is optimized and automated. Control, extract geometric, color, texture and luminous flux features in multi-source data, and perform image compensation and processing.
It improves image quality, realizes industrial and automated tobacco leaf transmission and recognition, reduces labor costs, improves monitoring efficiency, and improves the accuracy of tobacco leaf quality grading through comprehensive extraction of multi-source data.
Smart Images

Figure CN118624610B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of tobacco leaf quality monitoring, and in particular to a tobacco leaf quality monitoring system and method based on multi-source data fusion. Background Art
[0002] Tobacco leaf quality grading is one of the important application hotspots of artificial intelligence in the tobacco industry. Tobacco leaf quality grading is to classify tobacco leaf quality according to seven factors: identity, oil content, color, length, damage, structure and maturity according to national standards. At present, tobacco leaf quality grading still mostly adopts manual grading, which has problems such as low efficiency, inconsistent standards, subjectivity and limitations. Machine vision technology can identify and classify targets by collecting target images, using computer recognition technology and combining intelligent algorithms, and has great potential in tobacco leaf quality monitoring and grading.
[0003] There are still some problems in the existing methods of image recognition, monitoring and grading of tobacco leaves. First, the basis of image recognition is to accurately collect tobacco leaf images. When collecting tobacco leaf images, due to the complex environment of tobacco leaf manufacturing site, light interference, occlusion and other situations often occur, resulting in a poor light environment for image acquisition, resulting in low accuracy of image recognition. Secondly, most of the existing technologies for tobacco leaf image recognition are in the scientific research stage, and the tobacco leaves need to be placed manually at the image acquisition device, which cannot achieve unmanned and intelligent automatic transmission and recognition, and the efficiency is low. There is a lack of a quality inspection system that can automatically transmit and identify tobacco leaves in batches. In addition, the camera for image acquisition and recognition usually requires light source irradiation, and as the use time becomes longer and longer, the light source will inevitably attenuate, which will reduce the accuracy of image acquisition and recognition. In addition, for the image recognition of tobacco leaves, it is necessary to process the image to extract features, and the existing methods are to obtain images and extract features by setting a light source above the tobacco leaves and collecting the reflected light of the tobacco leaves, which cannot fully and comprehensively obtain the overall state of the tobacco leaves.
[0004] In the prior art, the invention patent application CN113469233A proposes a method and system for automatic tobacco leaf grading based on deep learning, which includes the following steps: obtaining tobacco leaf images, performing data cleaning on the obtained images, and constructing an image data set; performing preprocessing of the image data set, the preprocessing including image enhancement, data deblurring and data enhancement; introducing a convolution residual block in the process of downsampling and extracting features of the convolutional neural network, constructing a deep learning neural network model, training the preprocessed image data set, and obtaining weights of different groups; obtaining the optimal weight corresponding to the optimal test data; inputting the tobacco leaf image to be graded into the deep learning neural network model, directly loading the obtained optimal weight, and obtaining the detection and grading results of the tobacco leaf. The invention uses tobacco leaf images to grade tobacco leaf quality through a neural network algorithm, but does not solve the problems of poor light environment for image acquisition, failure to achieve intelligence and automation, light source attenuation, and insufficient feature extraction. Summary of the invention
[0005] Purpose of the invention: In view of the above problems, the present invention proposes a tobacco quality monitoring system and method based on multi-source data fusion.
[0006] Technical solution:
[0007] In a first aspect, the present invention provides a tobacco leaf quality monitoring system based on multi-source data fusion, including a tobacco leaf conveying system, a tobacco leaf placing plate, a photoelectric sensor, an image acquisition box, a CCD camera, a light source, a baffle controller, a light source controller, a conveying controller, an image processing host, and a monitoring and grading host computer;
[0008] Preferably, the CCD camera is arranged at a first preset distance above the tobacco leaf placement plate;
[0009] The first light source is arranged at a second preset distance above the tobacco placement plate;
[0010] The second light source is arranged at a third preset distance below the tobacco placement plate;
[0011] The first baffle is arranged at the conveyor belt inlet side of the image acquisition box;
[0012] The second baffle is arranged at the conveyor belt outlet side of the image acquisition box;
[0013] A plurality of reference marks are provided on the tobacco leaf placement plate;
[0014] The first photoelectric sensor is arranged below the first baffle, the second photoelectric sensor is arranged below the CCD camera, and the third photoelectric sensor is arranged below the second baffle;
[0015] The baffle controller is used to control the opening and closing of the first baffle and the second baffle;
[0016] The light source controller is used to control the first light source and the second light source to be turned on and off alternately;
[0017] The CCD camera is used to capture a first tobacco leaf image when the first light source is turned on and the second light source is turned off, and to capture a second tobacco leaf image when the second light source is turned on and the first light source is turned off;
[0018] The image processing host is used to perform image processing on the collected tobacco leaf images;
[0019] The monitoring and grading host computer is used to identify the processed images and output tobacco leaf quality grading results through a neural network model.
[0020] Preferably, the conveying controller is used to configure the conveying speed of the tobacco conveying system and control the conveying pause according to the trigger signal of the second photoelectric sensor;
[0021] The baffle controller is used to control the first baffle to open according to the trigger signal generated by the first photoelectric sensor;
[0022] Also according to the trigger signal generated by the second photoelectric sensor, the first baffle is controlled to be closed and the second baffle is controlled to be closed;
[0023] The second baffle is also controlled to open according to a trigger signal generated by the third photoelectric sensor.
[0024] Preferably, the image processing host includes an image filtering module, a grayscale module, a binarization module, an image segmentation module, and a color restoration module;
[0025] The image filtering module is used to reduce the high-frequency noise components in the image through image filtering processing, and reduce the grayscale fluctuations in the edge area of the image to make the image smooth;
[0026] The grayscale module is used to perform image grayscale processing by weighted average method;
[0027] The binarization module is used to perform binarization processing on the image by using a minimum error threshold method;
[0028] The image segmentation module is used to implement image segmentation through a threshold segmentation method;
[0029] The color restoration module is used to restore the color of the tobacco leaf part in the image.
[0030] Preferably, the monitoring and grading host computer includes a first feature extraction module and a second feature extraction module;
[0031] The first feature extraction module is used to extract a first feature in the first tobacco leaf image;
[0032] The second feature extraction module is used to extract the second feature in the second tobacco leaf image;
[0033] The first feature includes a geometric feature, a color feature, and / or a texture feature;
[0034] The geometric features include length, width, aspect ratio, area, and / or rectangularity;
[0035] The color features include H component, I component and S component of the HIS color space model;
[0036] The texture feature is an energy feature after wavelet decomposition;
[0037] The second characteristic comprises a light flux characteristic.
[0038] Preferably, the image processing host is further used to perform partition compensation on the image according to the reflection values of multiple reference marks in the image; including acquiring a first tobacco leaf image containing multiple reference marks;
[0039] Preliminarily identifying the tobacco leaf boundary in the image, and obtaining a first background image after removing the tobacco leaf image;
[0040] According to the reference mark on the first background image, the first background image is divided into a plurality of sub-areas, each sub-area including a reference mark;
[0041] Obtaining a pixel reflection value and a theoretical reflection value of a reference mark in each sub-region, and calculating a first attenuation coefficient;
[0042] Convert the RGB value of each pixel in each sub-region into a HIS value; obtain the standard deviation of the I component of the HIS value of each sub-region;
[0043] Calculate a compensation coefficient according to the first attenuation coefficient and the I component of the HIS value of each sub-region;
[0044] The I component of each pixel in the removed tobacco leaf image is compensated to obtain a compensated I component.
[0045] In a second aspect, the present invention further provides a tobacco leaf quality monitoring method based on multi-source data fusion, the method comprising:
[0046] S1. Arrange the light source, CCD camera, baffle and reference mark in the image acquisition box; including: setting the CCD camera at a first preset distance above the tobacco placement board; setting the first light source at a second preset distance above the tobacco placement board; setting the second light source at a third preset distance below the tobacco placement board; setting the first baffle at the conveyor belt inlet side of the image acquisition box; setting the second baffle at the conveyor belt outlet side of the image acquisition box; setting multiple reference marks on the tobacco placement board;
[0047] S2, start the tobacco leaf conveying system, and control the opening and closing of the baffle of the image acquisition box according to the trigger signal generated by the photoelectric sensor;
[0048] S3, controlling the first light source and the second light source to turn on and off alternately within the preset image acquisition time to collect tobacco leaf images; including: S31, within the first image acquisition period of the preset image acquisition time, the light source controller controls the first light source component to turn on, controls the second light source component to turn off, and starts the CCD camera to collect the first tobacco leaf image; S32, within the second image acquisition period of the preset image acquisition time, the light source controller controls the first light source component to turn off, controls the second light source component to turn on, and starts the CCD camera to collect the second tobacco leaf image;
[0049] S4, performing partition compensation on the image according to the reflection values of multiple reference marks in the image;
[0050] S5, the image processing host performs image preprocessing on the compensated image;
[0051] S6, extracting a first feature from the first tobacco leaf image, and extracting a second feature from the second tobacco leaf image;
[0052] S7. Input the first feature and the second feature into the neural network model, and output the tobacco leaf quality grading result.
[0053] Preferably, S2 includes:
[0054] S21, setting the conveying speed of the tobacco conveying system to v, setting the spacing between the tobacco placing plates on the conveying system to L, setting 0.5D<L<D, where D is the length of the image acquisition box in the conveying direction;
[0055] S22, the first photoelectric sensor of the transmission system generates a trigger signal, which triggers the baffle controller to control the first baffle to open;
[0056] S23, the second photoelectric sensor of the conveying system generates a trigger signal, triggering the baffle controller to control the first baffle to close, the second baffle to close, and triggering the conveying controller to control the suspension of conveying;
[0057] S24, after the preset image acquisition time has passed, trigger the transmission controller to start transmission;
[0058] S25. The third photoelectric sensor of the transmission system generates a trigger signal, which triggers the baffle controller to control the second baffle to open.
[0059] Preferably, the S4 includes:
[0060] S41, acquiring a first tobacco leaf image including a plurality of reference marks;
[0061] S42, preliminarily identifying the tobacco leaf boundary in the image, and obtaining a first background image after removing the tobacco leaf image;
[0062] S43, dividing the first background image into a plurality of sub-regions according to the reference mark on the first background image, each sub-region including a reference mark;
[0063] S44, obtaining a pixel reflection value r and a theoretical reflection value R of a reference mark in each sub-region, and calculating a first attenuation coefficient γ1;
[0064] S45, converting the RGB value of each pixel in each sub-region into an HIS value;
[0065] S46, obtaining the standard deviation I of the I component of the HIS value of each sub-region D ;
[0066] S46, calculating a compensation coefficient β according to the first attenuation coefficient and the I component of the HIS value of each sub-region;
[0067] S47, compensating the I component of each pixel in the tobacco leaf image removed in S42 to obtain a compensated I component I0.
[0068] Preferably, the S5 includes:
[0069] S51, reducing the high-frequency noise component in the image by image filtering processing, and reducing the grayscale fluctuation in the edge area of the image to make the image smooth;
[0070] S52, using a weighted average method to perform image grayscale processing;
[0071] S53, using the minimum error threshold method to perform binarization processing on the image;
[0072] S54, using a threshold segmentation method to implement image segmentation processing;
[0073] S55, restoring the color of the tobacco leaf part in the image.
[0074] Preferably, the S6 includes:
[0075] The first features include geometric features, color features, and texture features;
[0076] The geometric features include length, width, aspect ratio, area, and / or rectangularity;
[0077] The color features include H component, I component and S component of the HIS color space model;
[0078] The texture feature is an energy feature after wavelet decomposition;
[0079] The second characteristic comprises a light flux characteristic Q;
[0080] The luminous flux characteristic Q is calculated based on the I component of the pixels in the second tobacco leaf image.
[0081] In a third aspect, the present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor. Preferably, when the processor executes the computer program, the steps in the tobacco quality monitoring method based on multi-source data fusion are implemented.
[0082] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon. Preferably, when the computer program is executed by a processor, the steps in the tobacco leaf quality monitoring method based on multi-source data fusion are implemented.
[0083] Compared with the prior art, the present invention has the following beneficial effects:
[0084] 1. The present invention provides an image acquisition box, and provides a first baffle and a second baffle on both sides of the box. The opening and closing of the first baffle and the second baffle are controlled by a trigger signal of a photoelectric sensor in a transmission system, thereby ensuring the quality of the light environment inside the box during the image acquisition period, avoiding the influence of light leakage, interference, occlusion and other factors, and improving the image quality.
[0085] 2. The present invention can realize industrialized and automated tobacco leaf transmission and identification through linkage control of the transmission controller, the baffle controller, and the light source controller, thereby reducing labor costs and improving monitoring efficiency;
[0086] 3. The present invention reduces the error of image recognition by setting multiple reference marks in the image acquisition box, analyzing the light attenuation and compensating for it;
[0087] 4. When performing feature extraction on tobacco leaf images, the present invention takes multi-source data into consideration, and not only extracts the geometric features, color features, and texture features of the first tobacco leaf image, but also extracts the luminous flux features in the second tobacco leaf image, thereby more comprehensively collecting tobacco leaf information and improving the accuracy of tobacco leaf quality grading. BRIEF DESCRIPTION OF THE DRAWINGS
[0088] Figure 1 A schematic diagram of the structure of a tobacco leaf quality monitoring system based on multi-source data fusion provided by an embodiment of the present invention;
[0089] Figure 2 A flow chart of a tobacco leaf quality monitoring method based on multi-source data fusion provided by an embodiment of the present invention;
[0090] Figure 3 A schematic diagram of an image acquisition box structure provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0091] Obviously, many modifications and changes made by those skilled in the art based on the purpose of the present invention belong to the protection scope of the present invention.
[0092] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when an element or component is said to be "connected" to another element or component, it may be directly connected to the other element or component, or there may be an intermediate element or component. The term "and / or" used herein includes any unit and all combinations of one or more associated listed items.
[0093] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0094] Embodiment 1:
[0095] The embodiment of the present invention provides a tobacco leaf quality monitoring system based on multi-source data fusion. Figure 1 , Figure 1 A schematic diagram of the structure of a tobacco leaf quality monitoring system based on multi-source data fusion provided by an embodiment of the present invention, the system includes:
[0096] Tobacco leaf conveying system, tobacco leaf placement board, photoelectric sensor, image acquisition box, CCD camera, light source, baffle controller, light source controller, conveying controller, image processing host, monitoring and grading host computer;
[0097] The CCD camera is arranged at a first preset distance above the tobacco leaf placement plate;
[0098] The first light source is arranged at a second preset distance above the tobacco placement plate;
[0099] The second light source is arranged at a third preset distance below the tobacco placement plate;
[0100] The first baffle is arranged at the conveyor belt inlet side of the image acquisition box;
[0101] The second baffle is arranged at the conveyor belt outlet side of the image acquisition box;
[0102] A plurality of reference marks are provided on the tobacco leaf placement plate;
[0103] The first photoelectric sensor is arranged below the first baffle, the second photoelectric sensor is arranged below the CCD camera, and the third photoelectric sensor is arranged below the second baffle;
[0104] The baffle controller is used to control the opening and closing of the first baffle and the second baffle;
[0105] The light source controller is used to control the first light source and the second light source to be turned on and off alternately;
[0106] The CCD camera is used to capture a first tobacco leaf image when the first light source is turned on and the second light source is turned off, and to capture a second tobacco leaf image when the second light source is turned on and the first light source is turned off;
[0107] The image processing host is used to perform image processing on the collected tobacco leaf images;
[0108] The monitoring and grading host computer is used to identify the processed images and output tobacco leaf quality grading results through a neural network model.
[0109] Preferably, the conveying controller is used to configure the conveying speed of the tobacco conveying system and control the conveying pause according to the trigger signal of the second photoelectric sensor;
[0110] The baffle controller is used to control the first baffle to open according to the trigger signal generated by the first photoelectric sensor;
[0111] Also according to the trigger signal generated by the second photoelectric sensor, the first baffle is controlled to be closed and the second baffle is controlled to be closed;
[0112] The second baffle is also controlled to open according to a trigger signal generated by the third photoelectric sensor.
[0113] Preferably, the image processing host includes an image filtering module, a grayscale module, a binarization module, an image segmentation module, and a color restoration module;
[0114] The image filtering module is used to reduce the high-frequency noise components in the image through image filtering processing, and reduce the grayscale fluctuations in the edge area of the image to make the image smooth;
[0115] The grayscale module is used to perform image grayscale processing by weighted average method;
[0116] The binarization module is used to perform binarization processing on the image by using a minimum error threshold method;
[0117] The image segmentation module is used to implement image segmentation through a threshold segmentation method;
[0118] The color restoration module is used to restore the color of the tobacco leaf part in the image.
[0119] Preferably, the monitoring and grading host computer includes a first feature extraction module and a second feature extraction module;
[0120] The first feature extraction module is used to extract a first feature in the first tobacco leaf image;
[0121] The second feature extraction module is used to extract the second feature in the second tobacco leaf image;
[0122] The first feature includes a geometric feature, a color feature, and / or a texture feature;
[0123] The geometric features include length, width, aspect ratio, area, and / or rectangularity;
[0124] The color features include H component, I component and S component of the HIS color space model;
[0125] The texture feature is an energy feature after wavelet decomposition;
[0126] The second characteristic comprises a light flux characteristic.
[0127] The light flux feature Q is calculated based on the I component of the pixel in the second tobacco leaf image; the light flux feature can reflect the thickness information of the tobacco leaf;
[0128] in, , A is the adjustment coefficient, I Q is the mean value of the I component of the tobacco leaf pixels in the second tobacco leaf image, I Y is the preset threshold.
[0129] Preferably, the image processing host is further used to perform partition compensation on the image according to the reflection values of multiple reference marks in the image;
[0130] The method comprises obtaining a first tobacco leaf image including a plurality of reference marks; preliminarily identifying tobacco leaf boundaries in the image, and obtaining a first background image after removing the tobacco leaf image; dividing the first background image into a plurality of sub-regions according to the reference marks on the first background image, each sub-region including a reference mark; obtaining a pixel reflection value r and a theoretical reflection value R of the reference mark in each sub-region, and calculating a first attenuation coefficient γ1;
[0131] in, ;
[0132] Wherein, the pixel reflectance value r of the reference mark and the theoretical reflectance value R are RGB values;
[0133] Convert the RGB value of each pixel in each sub-region to a HIS value; obtain the standard deviation I of the I component of the HIS value of each sub-region D ;
[0134] The calculation formula of the I component is: ;
[0135] Calculate a compensation coefficient β according to the first attenuation coefficient and the I component of the HIS value of each sub-region;
[0136] Among them, the calculation formula of the compensation coefficient β is: ;ω is the adjustment coefficient;
[0137] Compensate the I component of each pixel in the removed tobacco leaf image to obtain a compensated I component I0;
[0138] Among them, the calculation formula of the compensated I component I0 is: ; I1 is the I component before compensation.
[0139] Embodiment 2:
[0140] The embodiment of the present invention also provides a tobacco leaf quality monitoring method based on multi-source data fusion, for details, please refer to Figure 2 , Figure 2 A flow chart of a tobacco leaf quality monitoring method based on multi-source data fusion provided in an embodiment of the present invention, the method comprising the steps of:
[0141] S1. Arrange the light source, CCD camera, baffle and reference mark in the image acquisition box;
[0142] The invention comprises: setting a CCD camera at a first preset distance above the tobacco placement plate; setting a first light source at a second preset distance above the tobacco placement plate; setting a second light source at a third preset distance below the tobacco placement plate; setting a first baffle at the conveyor belt inlet side of the image acquisition box; setting a second baffle at the conveyor belt outlet side of the image acquisition box; setting multiple reference marks on the tobacco placement plate; please refer to the detailed description. Figure 3 , Figure 3 A schematic diagram of an image acquisition box structure provided in an embodiment of the present invention.
[0143] S2, start the tobacco leaf conveying system, and control the opening and closing of the baffle of the image acquisition box according to the trigger signal generated by the photoelectric sensor;
[0144] Preferably, the S2 includes:
[0145] S21, setting the conveying speed of the tobacco conveying system to v, setting the spacing between the tobacco placing plates on the conveying system to L, setting 0.5D<L<D, where D is the length of the image acquisition box in the conveying direction;
[0146] S22, the first photoelectric sensor of the transmission system generates a trigger signal, which triggers the baffle controller to control the first baffle to open;
[0147] S23, the second photoelectric sensor of the conveying system generates a trigger signal, triggering the baffle controller to control the first baffle to close, the second baffle to close, and triggering the conveying controller to control the suspension of conveying;
[0148] S24, after the preset image acquisition time has passed, trigger the transmission controller to start transmission;
[0149] S25. The third photoelectric sensor of the transmission system generates a trigger signal, which triggers the baffle controller to control the second baffle to open.
[0150] S3, controlling the first light source and the second light source to turn on and off alternately within the preset image acquisition time to collect tobacco leaf images; including: S31, within the first image acquisition period of the preset image acquisition time, the light source controller controls the first light source component to turn on, controls the second light source component to turn off, and starts the CCD camera to collect the first tobacco leaf image; S32, within the second image acquisition period of the preset image acquisition time, the light source controller controls the first light source component to turn off, controls the second light source component to turn on, and starts the CCD camera to collect the second tobacco leaf image;
[0151] The present invention provides an image acquisition box, and provides a first baffle and a second baffle on both sides of the box. The opening and closing of the first baffle and the second baffle are controlled by a trigger signal of a photoelectric sensor in a transmission system, thereby ensuring the quality of the light environment inside the box during the image acquisition period, avoiding the influence of factors such as light leakage, interference, and shielding, and improving the image quality.
[0152] The present invention can realize industrialized and automated tobacco leaf transmission and identification through linkage control of a transmission controller, a baffle controller and a light source controller, thereby reducing labor costs and improving monitoring efficiency.
[0153] S4, performing partition compensation on the image according to the reflection values of multiple reference marks in the image;
[0154] Preferably, the S4 includes:
[0155] S41, acquiring a first tobacco leaf image including a plurality of reference marks;
[0156] S42, preliminarily identifying the tobacco leaf boundary in the image, and obtaining a first background image after removing the tobacco leaf image;
[0157] S43, dividing the first background image into a plurality of sub-regions according to the reference mark on the first background image, each sub-region including a reference mark;
[0158] S44, obtaining a pixel reflection value r and a theoretical reflection value R of a reference mark in each sub-region, and calculating a first attenuation coefficient γ1;
[0159] in, ;
[0160] Wherein, the pixel reflectance value r of the reference mark and the theoretical reflectance value R are RGB values;
[0161] S45, converting the RGB value of each pixel in each sub-region into an HIS value;
[0162] S46, obtaining the standard deviation I of the I component of the HIS value of each sub-region D ;
[0163] The calculation formula of the I component is: ;
[0164] S46, calculating a compensation coefficient β according to the first attenuation coefficient and the I component of the HIS value of each sub-region;
[0165] Among them, the calculation formula of the compensation coefficient β is: ;ω is the adjustment coefficient;
[0166] S47, compensating the I component of each pixel in the tobacco leaf image removed in S42 to obtain a compensated I component I0;
[0167] Among them, the calculation formula of the compensated I component I0 is: ; I1 is the I component before compensation.
[0168] The present invention reduces the error of image recognition by arranging a plurality of reference marks in an image acquisition box, analyzing the light attenuation and performing compensation.
[0169] S5, the image processing host performs image preprocessing on the compensated image;
[0170] Preferably, the S5 includes:
[0171] S51, reducing the high-frequency noise component in the image by image filtering processing, and reducing the grayscale fluctuation in the edge area of the image to make the image smooth;
[0172] S52, using a weighted average method to perform image grayscale processing;
[0173] S53, using the minimum error threshold method to perform binarization processing on the image;
[0174] S54, using a threshold segmentation method to implement image segmentation processing;
[0175] S55, restoring the color of the tobacco leaf part in the image.
[0176] S6, extracting a first feature from the first tobacco leaf image, and extracting a second feature from the second tobacco leaf image;
[0177] Preferably, the S6 includes:
[0178] The first features include geometric features, color features, and texture features;
[0179] The geometric features include length, width, aspect ratio, area, and / or rectangularity;
[0180] The color features include H component, I component and S component of the HIS color space model;
[0181] The texture feature is an energy feature after wavelet decomposition;
[0182] The second characteristic comprises a light flux characteristic Q;
[0183] The light flux characteristic Q is calculated based on the I component of the pixel in the second tobacco leaf image;
[0184] in, , A is the adjustment coefficient, I Q is the mean value of the I component of the tobacco leaf pixels in the second tobacco leaf image, I Y is the preset threshold.
[0185] When performing feature extraction on tobacco leaf images, the present invention takes multi-source data into consideration, and not only extracts the geometric features, color features, and texture features of the first tobacco leaf image, but also extracts the luminous flux features in the second tobacco leaf image, wherein the luminous flux information can reflect the thickness information of the tobacco leaves. Therefore, the present invention collects tobacco leaf information more comprehensively and improves the accuracy of tobacco leaf quality grading.
[0186] S7. Input the first feature and the second feature into the neural network model, and output the tobacco leaf quality grading result.
[0187] In addition, the present invention also includes a data set construction step for the neural network model, which requires collecting a large amount of tobacco leaf data, assigning grading labels, constructing training sets and test sets, and finally forming a neural network model for tobacco leaf quality grading through repeated parameter adjustments.
[0188] Embodiment three:
[0189] An embodiment of the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor. Preferably, when the processor executes the computer program, the steps in the tobacco quality monitoring method based on multi-source data fusion are implemented.
[0190] Embodiment 4:
[0191] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. Preferably, when the computer program is executed by a processor, the steps in the tobacco leaf quality monitoring method based on multi-source data fusion are implemented.
[0192] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0193] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0194] Finally, it should be noted that, in this article, relationships such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
Claims
1. A tobacco leaf quality monitoring method based on multi-source data fusion, applied to a tobacco leaf quality monitoring system based on multi-source data fusion, the tobacco leaf quality monitoring system based on multi-source data fusion comprising a tobacco leaf conveying system, a tobacco leaf placing plate, a photoelectric sensor, an image acquisition box, a CCD camera, a light source, a baffle controller, a light source controller, a conveying controller, an image processing host, and a monitoring and grading host computer; characterized in that: The CCD camera is arranged at a first preset distance above the tobacco placement plate; the first light source is arranged at a second preset distance above the tobacco placement plate; the second light source is arranged at a third preset distance below the tobacco placement plate; and the first baffle is arranged at the conveyor belt entrance side of the image acquisition box; The second baffle is arranged at the conveyor belt outlet side of the image acquisition box; A plurality of reference marks are arranged on the tobacco leaf placement plate; a first photoelectric sensor is arranged below the first baffle, a second photoelectric sensor is arranged below the CCD camera, and a third photoelectric sensor is arranged below the second baffle; the baffle controller is used to control the opening and closing of the first baffle and the second baffle; the light source controller is used to control the alternating opening and closing of the first light source and the second light source; the CCD camera is used to collect the first tobacco leaf image when the first light source is turned on and the second light source is turned off, and collect the second tobacco leaf image when the second light source is turned on and the first light source is turned off; the image processing host is used to perform image processing on the collected tobacco leaf images; the monitoring and grading host computer is used to identify the processed images and output the tobacco leaf quality grading results through the neural network model; The conveying controller is used to configure the conveying speed of the tobacco conveying system and control the conveying pause according to the trigger signal of the second photoelectric sensor; the baffle controller is used to control the opening of the first baffle according to the trigger signal generated by the first photoelectric sensor; and also to control the closing of the first baffle and the closing of the second baffle according to the trigger signal generated by the second photoelectric sensor; Also according to the trigger signal generated by the third photoelectric sensor, the second baffle is controlled to open; Characterized in that the method comprises: S1. Arrange the light source, CCD camera, baffle and reference mark in the image acquisition box; including: setting the CCD camera at a first preset distance above the tobacco placement board; setting the first light source at a second preset distance above the tobacco placement board; setting the second light source at a third preset distance below the tobacco placement board; setting the first baffle at the conveyor belt inlet side of the image acquisition box; setting the second baffle at the conveyor belt outlet side of the image acquisition box; setting multiple reference marks on the tobacco placement board; S2, start the tobacco leaf conveying system, and control the opening and closing of the baffle of the image acquisition box according to the trigger signal generated by the photoelectric sensor; The S2 includes: S21, setting the conveying speed of the tobacco conveying system to v, setting the spacing between the tobacco placing plates on the conveying system to L, setting 0.5D<L<D, where D is the length of the image acquisition box in the conveying direction; S22, the first photoelectric sensor of the transmission system generates a trigger signal, which triggers the baffle controller to control the first baffle to open; S23, the second photoelectric sensor of the conveying system generates a trigger signal, triggering the baffle controller to control the first baffle to close, the second baffle to close, and triggering the conveying controller to control the suspension of conveying; S24, after the preset image acquisition time has passed, trigger the transmission controller to start transmission; S25, the third photoelectric sensor of the transmission system generates a trigger signal, which triggers the baffle controller to control the second baffle to open; S3, controlling the first light source and the second light source to turn on and off alternately within the preset image acquisition time to collect tobacco leaf images; including: S31, within the first image acquisition period of the preset image acquisition time, the light source controller controls the first light source component to turn on, controls the second light source component to turn off, and starts the CCD camera to collect the first tobacco leaf image; S32, within the second image acquisition period of the preset image acquisition time, the light source controller controls the first light source component to turn off, controls the second light source component to turn on, and starts the CCD camera to collect the second tobacco leaf image; S4, performing partition compensation on the image according to the reflection values of multiple reference marks in the image; The S4 includes: S41, acquiring a first tobacco leaf image including a plurality of reference marks; S42, preliminarily identifying the tobacco leaf boundary in the image, and obtaining a first background image after removing the tobacco leaf image; S43, dividing the first background image into a plurality of sub-regions according to the reference mark on the first background image, each sub-region including a reference mark; S44, obtaining a pixel reflection value r and a theoretical reflection value R of a reference mark in each sub-region, and calculating a first attenuation coefficient γ1; in, ; S45, converting the RGB value of each pixel in each sub-region into an HIS value; S46, obtaining the standard deviation I of the I component of the HIS value of each sub-region D ; The calculation formula of the I component is: ; S46, calculating a compensation coefficient β according to the first attenuation coefficient and the I component of the HIS value of each sub-region; Among them, the calculation formula of the compensation coefficient β is: ;ω is the adjustment coefficient; S47, compensating the I component of each pixel in the tobacco leaf image removed in S42 to obtain a compensated I component I0; Among them, the calculation formula of the compensated I component I0 is: ; I1 is the I component before compensation; S5, the image processing host performs image preprocessing on the compensated image; S6, extracting a first feature from the first tobacco leaf image, and extracting a second feature from the second tobacco leaf image; The S6 includes: The first features include geometric features, color features, and texture features; The geometric features include length, width, aspect ratio, area, and / or rectangularity; The color features include H component, I component and S component of the HIS color space model; The texture feature is an energy feature after wavelet decomposition; The second characteristic comprises a light flux characteristic Q; The light flux characteristic Q is calculated based on the I component of the pixel in the second tobacco leaf image; in, , A is the adjustment coefficient, I Q is the mean value of the I component of the tobacco leaf pixels in the second tobacco leaf image, I Y is the preset threshold; S7. Input the first feature and the second feature into the neural network model, and output the tobacco leaf quality grading result.
2. The tobacco leaf quality monitoring method based on multi-source data fusion according to claim 1 is characterized in that: The image processing host includes an image filtering module, a grayscale module, a binarization module, an image segmentation module, and a color restoration module; The image filtering module is used to reduce the high-frequency noise components in the image through image filtering processing, and reduce the grayscale fluctuations in the edge area of the image to make the image smooth; The grayscale module is used to perform image grayscale processing by weighted average method; The binarization module is used to perform binarization processing on the image by using a minimum error threshold method; The image segmentation module is used to implement image segmentation through a threshold segmentation method; The color restoration module is used to restore the color of the tobacco leaf part in the image.
3. The tobacco leaf quality monitoring method based on multi-source data fusion according to claim 2 is characterized in that: The monitoring and grading host computer includes a first feature extraction module and a second feature extraction module; The first feature extraction module is used to extract a first feature in the first tobacco leaf image; The second feature extraction module is used to extract the second feature in the second tobacco leaf image; The first feature includes a geometric feature, a color feature, and / or a texture feature; The geometric features include length, width, aspect ratio, area, and / or rectangularity; The color features include H component, I component and S component of the HIS color space model; The texture feature is an energy feature after wavelet decomposition; The second characteristic comprises a light flux characteristic.
4. The tobacco leaf quality monitoring method based on multi-source data fusion according to claim 3 is characterized in that: The image processing host is also used to perform partition compensation on the image according to the reflection values of multiple reference marks in the image; including obtaining a first tobacco leaf image containing multiple reference marks; preliminarily identifying the tobacco leaf boundary in the image, and obtaining a first background image after removing the tobacco leaf image; According to the reference mark on the first background image, the first background image is divided into several sub-areas, each sub-area contains a reference mark; the pixel reflection value and the theoretical reflection value of the reference mark in each sub-area are obtained, and the first attenuation coefficient is calculated; the RGB value of each pixel in each sub-area is converted into an HIS value; and the standard deviation of the I component of the HIS value of each sub-area is obtained; Calculate a compensation coefficient according to the first attenuation coefficient and the I component of the HIS value of each sub-region; The I component of each pixel in the removed tobacco leaf image is compensated to obtain a compensated I component.
5. The tobacco leaf quality monitoring method based on multi-source data fusion according to claim 4 is characterized in that: The S5 includes: S51, reducing the high-frequency noise component in the image by image filtering processing, and reducing the grayscale fluctuation in the edge area of the image to make the image smooth; S52, using a weighted average method to perform image grayscale processing; S53, using the minimum error threshold method to perform binarization processing on the image; S54, using a threshold segmentation method to implement image segmentation processing; S55, restoring the color of the tobacco leaf part in the image.
