Dual-band image sensing device, image analysis method and applications thereof

Abstract

A dual-band image sensing device comprises a containing unit and a light source and image capturing module positioned above the containing unit, capable of viewing the testing object. The device of the present invention includes at least an infrared light source unit, a visible light source unit, and an image capturing unit for acquiring infrared and visible light images of the testing object. An image analysis and machine learning module is signal-connected to the image capturing unit and includes an image preprocessing unit and an image analysis model unit. The present invention enhances predictive performance by meticulously preprocessing the captured images to effectively eliminate various interferences and noise. The image analysis method not only improves image quality but also facilitates the optimization of subsequent data analysis and applications, thereby achieving more accurate and reliable predictions of the object’s state.

Description

Dual-band image sensing device, its image analysis methods and applications An image sensing device, particularly an image sensing device that uses dual-band sensing and analysis. With rising incomes and Westernized dietary habits, coupled with the introduction of internationally renowned coffee brands into the market in recent years, rows of coffee shops, both domestic and international, have become a common street scene. The professionalism of the coffee industry has also continued to improve; the selection, roasting, and brewing techniques of coffee beans in Taiwan are now comparable to those in coffee-producing countries like the United States and Europe, demonstrating the flourishing state of the coffee industry in our country. Currently, the roasting process mainly relies on baristas to judge the roast level of coffee beans by sight. However, this method has many uncertainties, such as the barista's experience level and the influence of ambient light, which often leads to different roasting conditions for each batch of beans, affecting the taste and making it impossible to maintain the quality of coffee beans consistently. To address the issue of inconsistent coffee quality caused by the high degree of human intervention and environmental lighting factors in the current coffee roasting process, it is necessary to provide a more accurate and objective testing method to improve coffee quality and production efficiency. This invention first provides a method for analyzing dual-band images, the steps of which include: acquiring infrared and / or visible light images of the object under test and transmitting them to an image analysis and machine learning module; an image preprocessing unit in the image analysis and machine learning module preprocesses the infrared and / or visible light images, the steps of which include: step S1-1 image cropping, step S1-2 infrared image threshold selection and setting, and step S1-3 visible light image pixel sampling; then, an image analysis model unit in the image analysis and machine learning module analyzes the preprocessed images, the steps of which include: step S2-1 establishing a training dataset, and step S2-2 inputting the training dataset into a regression training model for comparison to obtain the uniformity distribution of the object under test. In step 1-2, shadow or overexposed areas in infrared and / or visible light images are filtered by setting an effective analysis image area threshold range based on the infrared image. The threshold range is obtained by taking the maximum and minimum values ​​of pixel information in different intervals of the infrared image of the object under test as the threshold range of near-infrared light intensity, and defining the final effective area threshold range. Steps 1-3 involve taking pixel coordinates within the threshold range and comparing these coordinates with the visible light image. Step S2-1 involves extracting and filtering the infrared (IR) or near-infrared (NIR) and RGB channels (R, G, B) of the infrared and / or visible light images of the object under test, and integrating them with a standard value of the object under test to obtain several data sets calculated by the following formulas (1), (2), (3) and (4): RGB_input = [RGB]n×3…Formula (1); (N)IR_input = [(N)IR]n×1…Formula (2); RGB (N)IR_input = [RGB (N)IR]n×4…Formula (3); RGB (N)IR SV_input = [RGB (N)IR Standard Value]n×5…Formula (4); where n in the above formulas (1) to (4) is the number of images of the object under test obtained. In step S2-2, the regression training model comparison system inputs the several datasets from step 2-1 into at least one regression model to compare the models of the several datasets. Preferably, the regression model comprises one or more combinations of multiple linear regression, multinomial regression, and partial least squares regression. Further, in step S2-3, the mean, mode, and / or median of the uniformity distribution of the test object can be calculated. The present invention also provides a dual-band image sensing device for performing the aforementioned dual-band image analysis method, comprising: a housing unit for holding an object to be measured; a light source and image capturing module disposed above the housing unit and having a view of the object to be measured; the module includes at least an infrared light source unit, a visible light source unit, and an image capturing unit; the infrared light source unit and the visible light source unit emit infrared light and / or visible light to the object to be measured, and the image capturing unit captures infrared light images and / or visible light images; and an image analysis and machine learning module, which is signal-connected to the image capturing unit and includes an image preprocessing unit and an image analysis model unit electrically and signal-connected. In some preferred embodiments, a photomask unit is provided between the accommodating unit and the light source and image capturing module. The invention further provides a coffee roasting apparatus comprising the aforementioned dual-band image sensing device. A plurality of the visible light source units are arranged in a ring to form a visible light lamp ring; and at least one infrared light source unit is disposed adjacent to the visible light source unit. As can be seen from the above description, the present invention has the following beneficial effects and advantages: 1. This invention combines optical sensors, algorithms, and optical system design to develop a highly efficient and reliable coffee bean or coffee powder roasting system. It eliminates uncertainties such as human experience and ambient light during coffee roasting, significantly improving the accuracy of roasting. This invention uses sensors to instantly read the reflection wavelengths of coffee beans / powder, employs algorithms for data analysis and prediction, and combines this with an optimized optical system design to achieve accurate measurement of coffee bean roasting. Furthermore, the method provided by this invention can obtain the uniformity distribution of the state values ​​of the tested sample, allowing for detailed roasting prediction of individual coffee beans within each sample. Compared to existing coffee bean roasting devices, this invention overcomes the roasting deviation caused by sensing only a single area of ​​each sample. Therefore, this invention can promote coffee quality improvement and contribute to the automation and intelligentization of the coffee production process, propelling the coffee industry towards a more technologically advanced future. 2. This invention, through meticulous preprocessing of images captured by image sensors, effectively eliminates various interferences and noise in the images, thereby significantly improving predictive performance and achieving a superior level. The image processing not only improves image quality but also helps optimize subsequent data analysis and applications, resulting in more accurate and reliable results. The present invention will be described in detail below with reference to several preferred embodiments. The accompanying drawings are merely some exemplary representations or embodiments of the present invention. For those skilled in the art, the present invention can be applied to other similar situations based on these drawings without any further effort. The terms “system,” “apparatus,” “unit,” and / or “module” used in this invention are to distinguish different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions. As illustrated in this invention, unless the context clearly indicates otherwise, words such as “a,” “an,” “an,” and / or “the” do not specifically refer to the singular and may also include the plural. Generally, the terms “comprising” and “including” only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements. This invention uses flowcharts to illustrate the operations performed by the system according to embodiments of the invention. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them. < Dual-band image sensing device > Please refer to Figure 1, which is a schematic diagram of a preferred embodiment of the dual-band image sensing device 100 of the present invention. In order to clearly explain and illustrate the dual-band image sensing device 100 of the present invention, this embodiment takes a coffee bean roasting detection device as an example. However, the dual-band image sensing device 100 provided by the present invention can detect other bean-shaped, granular or powdered foods with the same or similar image characteristics in addition to coffee beans, and is not limited thereto. A preferred embodiment of the dual-band image sensing device 100 of the present invention includes at least: a receiving unit 10 for holding a test object O; a light source and image capturing module 20 disposed above the receiving unit 10 and having a view of the test object O; the module includes at least an infrared light source unit 21, a visible light source unit 22, and an image capturing unit 23; the infrared light source unit 21 and the visible light source unit 22 emit infrared light and / or visible light to the test object O, and the image capturing unit 23 captures infrared light images and / or visible light images; and an image analysis and machine learning module 40, which is signal-connected to the image capturing unit 23, and includes an image preprocessing unit 41 and an image analysis model unit 42 that are electrically and signal-connected. Preferably, in some preferred embodiments, a photomask unit 30 is provided between the accommodating unit 10 and the light source and image capturing module 20, so as to better enable the light source and image capturing module 20 to capture the infrared light image and / or visible light image of the test object O state on the accommodating unit 10. The infrared light source unit 21 is preferably an infrared light-emitting diode (LED); the visible light source unit 22 preferably includes a visible light-emitting diode (LED); the image capturing unit 23 can capture images in at least the visible light band and / or the infrared light band, and preferably, when the light intensity is insufficient (i.e., in a dark environment where the visible light band is lacking), the image capturing unit 23 can capture images only in the infrared light band. More preferably, in another preferred embodiment of the present invention, the infrared light source unit 21 can emit infrared light or near-infrared light, such as near-infrared light with a wavelength of 850 nm, which is not limited herein. In addition to at least one visible light source unit 22 being disposed above the accommodating unit 10, as shown in the preferred embodiment of FIG1, several visible light source units 22 may be arranged in a ring to form a visible light lamp ring, and at least one infrared light source unit 21 may be disposed on the side adjacent to the visible light source unit 22. < Image Analysis Methods for Dual-Band Image Sensing Devices > Please refer to Figure 2. The image preprocessing unit 41 of the image analysis and machine learning module 40 preprocesses the infrared light image and / or visible light image of the object O captured by the image capturing unit 23. The steps include: Step S1-1: Image cropping; Step S1-2: Infrared light image threshold selection and setting; and Step S1-3: Visible light image pixel sampling. In step 1-1, image cropping mainly involves removing invalid image areas from the captured infrared and / or visible light images that are not images of the object under test (O), retaining only the analyzable images of the object under test (O). In the infrared image threshold selection and setting of steps 1-2, in order to avoid the analysis results being affected by the shadow areas caused by the overlapping and occlusion between particles or powders in the image of the object under test, as well as the inaccurate color information due to overexposure in the non-overlapping areas, the effective analysis image area threshold is set for filtering these shadow or overexposed areas. Because of infrared light, such as near-infrared light images, pixel information can be used as a reference for threshold selection. Near-infrared light images only store the intensity value of the infrared light at the time of capture, with an intensity range of 0-255, which can be used to filter out pixels in overexposed or shadowed areas. In practice, the maximum and minimum values ​​of pixel information in different intervals of the infrared image of the object under test O are obtained as the threshold range for near-infrared light intensity, and the final effective area threshold range is defined. In steps S1-3, which involve sampling pixels in visible light images and establishing a dataset, after defining an effective area threshold, pixel coordinates within that threshold range are extracted and then applied to the visible light image. Next, please refer to Figures 1 and 3. The image analysis model unit 42 is included in a desktop or portable device such as a computer, laptop, tablet or mobile phone and performs the following analysis and calculation. The image preprocessed by the image preprocessing unit 41 is then analyzed by the image analysis model unit 42. The steps include: Step S2-1: Establish the training dataset; Step S2-2: Input the training dataset into the regression training model for comparison to obtain the uniformity distribution of the test object O; The above step S2-1, which establishes the training dataset, mainly involves extracting and filtering the infrared (IR) or near-infrared (NIR) and RGB channels (R (red), G (green), B (blue)) of the infrared and / or visible light images of the test object O, and integrating them with a standard value of the test object O to obtain several datasets calculated by the following formulas (1), (2), (3) and (4) for subsequent training model establishment and evaluation analysis. RGB_input = [RGB]n×3…Formula (1); (N)IR_input = [(N)IR]n×1…Equation (2); RGB (N)IR_input = [RGB (N)IR]n×4...Equation (3); RGB (N)IR SV_input = [RGB (N)IR Standard Value]n×5…Equation (4); where n in the above equations (1) to (4) is the number of images of the object O to be tested. Step S2-2, regression training model comparison, mainly involves inputting the various datasets from step 2-1 into at least one regression model for model comparison across these datasets. This regression model includes, but is not limited to, one or more combinations of multiple linear regression (MLR), polynomial regression (PR), and partial least squares (PLS) regression models. Further, in steps S2-3, the mean, mode, and / or median of the obtained uniformity distribution can be calculated to verify the accuracy of the O state value of each test object. < Example 1 > In a preferred embodiment of the present invention, the test substance O is coffee beans. In this embodiment, the roasting state of coffee beans with 16 different roasting levels is analyzed. First, the aforementioned dual-band image sensing device 100 acquires infrared and / or visible light images of the 16 different roasting levels of coffee beans. Then, these infrared and / or visible light images are transmitted to the image analysis and machine learning module 40 for analysis. Please refer to Figures 4A and 4B and Table 1 below. After the image preprocessing unit 41 crops the infrared and / or visible light images to obtain the effective image area, it then performs an infrared image threshold selection and setting step to obtain the effective area threshold range of the infrared image pixel intensity values ​​for several preferred embodiments of different coffee beans. In this embodiment, the minimum and maximum values ​​of the infrared image pixel intensity values ​​for 16 different roasting levels of coffee beans are 35-245, and these values ​​are set as the effective area threshold range. Table 1. Next, the image preprocessed by the image preprocessing unit 41 is analyzed by the image analysis model unit 42. Please refer to Figure 5A, the original visible light image of this embodiment, Figure 5B, the near-infrared light intensity filtered image, and Figure 5C, the visible light image coordinate filtered image. The RGB three-channel information and NIR information values ​​obtained through these images are integrated with the standard data set of coffee bean standard roast value (Standard RD). In this embodiment, a total of 1600 images of 16 types of coffee beans as shown in Table 1 above were obtained. The data were integrated using the above formulas (1) to (4) into the following formulas (5) to (8), thus obtaining the complete data set of the 6 types of coffee beans in this embodiment. …Equation (5); …Equation (6); …Equation (7); …Formula (8). After establishing models for different input datasets, this embodiment uses 16 verification images of coffee beans with different roast levels to evaluate the model's predictive accuracy. After pixel-level filtering, the R, G, B, and NIR values ​​of all pixels in these 16 images are sequentially input into at least one regression model in step S2-2 above to infer the roast uniformity distribution of each coffee bean sample. Finally, to verify accuracy, the uniformity distribution can optionally be calculated as a mean, median, and / or mode, and compared with standard roast values ​​to verify the predicted roast value of the coffee beans. Please refer to Figures 6A-6P and Table 2 below. This embodiment uses multiple linear regression, multinomial regression, and partial least squares regression models with an RGB+NIR four-channel input dataset, and verifies the distribution of coffee bean roast uniformity using the mean, calculating estimated roast values ​​for 16 different coffee bean roast levels. It also compares the results with standard roast values ​​detected by commercially available infrared coffee bean roast testing machines, confirming that the device and method provided by this invention can indeed accurately predict the state value of the test object O. While this embodiment uses the mean for verification, the median and mode are also verified to be valid. Table 2. This invention uses flowcharts to illustrate the operations performed by the system according to embodiments of the invention. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them. In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of range in some embodiments of the invention are approximate values, in specific embodiments, such values ​​are set as precisely as feasible. Finally, it should be understood that the embodiments described in this invention are merely illustrative of the principles of the invention. Other modifications may also fall within the scope of this invention. Therefore, alternative configurations of the embodiments of this invention are considered as examples and not limitations, and are regarded as consistent with the teachings of this invention. Accordingly, the embodiments of this invention are not limited to those explicitly described and illustrated herein. 100: Dual-band image sensing device; 10: Containment unit; 20: Light source and image acquisition module; 21: Infrared light source unit; 22: Visible light source unit; 23: Image acquisition unit; 30: Photomask unit; 40: Image analysis and machine learning module; 41: Image preprocessing unit; 42: Image analysis model unit; O: Test object; S1-1~S1-3, S2-1~S2-3: Steps To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are merely some examples or embodiments of the present invention and are not intended to absolutely limit the technical scope of the present invention. Unless obvious from the context or otherwise stated, the same reference numerals in the figures represent the same structures or operations. Wherein: Figure 1 is a schematic diagram of a preferred embodiment of the dual-band image sensing device of the present invention. Figure 2 is a flowchart of the preprocessing steps performed by the image preprocessing unit in the image analysis and machine learning module of the present invention. Figure 3 is a schematic flowchart of the image analysis steps performed by the image analysis model unit of the present invention. Figures 4A and 4B show the effective area value range of images for several coffee bean samples in Embodiment 1 of the present invention. Figures 5A, 5B, and 5C are the original visible light image, near-infrared light intensity screening image, and visible light image coordinate screening image of the coffee bean samples in Embodiment 1 of the present invention, respectively. Figures 6A to 6P are the coffee bean images and roast uniformity distribution diagrams of several coffee bean samples in Embodiment 1 of the present invention. 100: Dual-band image sensing device 10: Compartment Unit 20: Light source and image capture module 21: Infrared light source unit 22: Visible light source unit 23: Image Capture Unit 30: Photomask Unit 40: Image Analysis and Machine Learning Module 41: Image preprocessing unit 42: Image Analysis Model Unit O: Test object

Claims

1. A dual-band image analysis method includes the following steps: acquiring infrared and visible light images of an object under test and transmitting them to an image analysis and machine learning module, which includes an image preprocessing unit and an image analysis model unit connected by signals; the image preprocessing unit in the image analysis and machine learning module preprocesses the infrared and visible light images, including the following steps: step S1-1: image cropping, removing invalid image areas that are not part of the object under test from the infrared and visible light images, retaining only the analyzable image of the object under test; step S1-2: selecting and setting a threshold for the infrared image to filter out shadows or overexposed areas in the infrared image; and step S1-3: sampling pixels in the visible light image; the image analysis model unit analyzes the preprocessed image, including the following steps: step S2-1: establishing a training dataset; and step S2-2: inputting the training dataset into a regression training model for comparison to obtain the uniformity distribution of the state of the object under test. The dual-band image analysis method as described in claim 1, wherein: Steps 1-2 filter the shadow or overexposed areas in the infrared and visible light images by setting an effective analysis image area threshold range based on the infrared image. This threshold range is obtained by taking the maximum and minimum values ​​of pixel information in different intervals of the infrared image of the object under test as the threshold range of near-infrared light intensity, and defining the final effective area threshold range. The dual-band image analysis method as described in claim 1, wherein: Steps 1-3 involve taking the pixel coordinates within the threshold range and comparing these coordinates with the visible light image. The dual-band image analysis method as described in claim 1, wherein: Step S2-1 involves extracting and filtering the infrared (IR) or near-infrared (NIR) and RGB channels (R (red), G (green), B (blue)) of the infrared and visible light images of the object under test, and integrating them with a standard value of the object under test to obtain several data sets calculated by the following formulas (1), (2), (3) and (4): RGB_input = [RGB]n×3…Formula (1); (N)IR_input = [(N)IR]n×1…Formula (2); RGB (N)IR_input = [RGB (N)IR]n×4…Formula (3); RGB (N)IRSV_input = [RGB (N)IR Standard Value]n×5…Formula (4); where n in the above formulas (1) to (4) is the number of images of the object under test obtained. The dual-band image analysis method as described in claim 1, wherein: Step S2-2, the regression training model comparison, involves inputting the various datasets from Step 2-1 into at least one regression model for model comparison across the datasets; and the regression model includes one or more combinations of multiple linear regression, multinomial regression, and partial least squares regression. The dual-band image analysis method as described in any one of claims 1 to 5, wherein: The mean, mode, and / or median of the uniformity distribution of the state of the test object are then calculated. A dual-band image sensing device for performing the dual-band image analysis method as described in claims 1-6, comprising: a housing unit for holding an object under test; a light source and image capturing module disposed above the housing unit and having a view of the object under test; the module includes at least an infrared light source unit, a visible light source unit, and an image capturing unit; the infrared light source unit and the visible light source unit emit infrared light and / or visible light to the object under test, and the image capturing unit captures infrared light images and / or visible light images; and an image analysis and machine learning module signal-connected to the image capturing unit, and including an image preprocessing unit and an image analysis model unit electrically and signal-connected. The dual-band image sensing apparatus as described in claim 7, wherein: A photomask unit is provided between the housing unit and the light source and image capturing module. The dual-band image sensing apparatus as described in claim 7 or 8, wherein: The infrared light source unit emits infrared light and / or near-infrared light. The dual-band image sensing apparatus as described in claim 7 or 8, wherein: Several visible light source units are arranged in a ring to form a visible light lamp ring; and at least one infrared light source unit is disposed adjacent to the visible light source unit. A coffee roasting degree detection device comprising a dual-band image sensing device as described in any one of claims 7-10.