Method and system for performing point-to-point white reference correction on hyperspectral image
Patent Information
- Application Number
- PCT/CN2023/073836
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-12-06
- Filing Date
- 2023-01-30
- Publication Date
- 2024-11-28
AI Technical Summary
Existing point-to-point white reference correction methods for hyperspectral images have problems such as low accuracy and cumbersome processes. Especially when processing samples with high transmittances, equipment stability errors and contamination have a greater impact, and there is a lack of simple and accurate methods. Correction method.
A method is adopted to obtain the hyperspectral data of the standard reference white plate by building a hardware scene, and select the spectral average value of the unobstructed area when shooting the sample, calculate the white reference correction coefficient, and use this coefficient to perform the hyperspectral data of the sample. Correction simplifies the process and avoids repeated shooting and equipment stability errors.
It improves the accuracy and reliability of hyperspectral analysis, simplifies the data collection process, reduces data collection time, enhances analysis efficiency, and effectively avoids errors caused by light source non-uniformity.
Smart Images

Figure CN2023073836_28112024_PF_FP_ABST
Abstract
Description
A method and system for point-to-point white reference correction of hyperspectral images
[0001] Related applications
[0002] This application claims priority to Chinese patent application No. CN202111477742X filed on December 6, 2021, the entire contents of which are incorporated herein by reference. Technical Field
[0003] The present invention relates to the technical field of hyperspectral analysis, and in particular to a method and system for performing point-to-point white reference correction on a hyperspectral image. Background Art
[0004] Hyperspectral imaging technology can simultaneously obtain both image and spectral information. In the image dimension, it can be combined with machine vision technology to identify object images, and in the spectral dimension, it can also analyze material composition. This new technology holds great potential. The spectral analysis capabilities of hyperspectral imaging technology derive from its ability to record spectral information obtained from the interaction between light and matter at different wavelengths. This spectral information corresponds to information about the physical and chemical composition of an object.
[0005] Description of the interaction process between light and matter: When incident light shines on a substance, the components of the substance will absorb, reflect, and scatter the light, thereby changing the spectral shape of the reflected (or transmitted) light; different components of a substance have different effects on light, which leads to different spectral shapes for substances with different components. The absolute spectral shape of reflected or transmitted light will change with changes in the shape of the light source, but the relative reflectivity (transmittance) spectrum has nothing to do with the light source. Calculating the reflectivity spectrum is an indispensable step, especially for some application scenarios with high requirements for analytical accuracy. The general method of obtaining reflectivity is to perform white parameter correction.
[0006] Considering that even a small amount of contamination on the whiteboard can affect the calculation of the entire calibration parameter during use, seriously affecting calibration accuracy, a protective structure for the reference area is designed to ensure that reliable data is collected from the selected reference area, thereby maintaining the accuracy and stability of the whiteboard calibration algorithm described above. Furthermore, for samples with high transmittance, such as jade or crystal, not only does the reference area affect calibration accuracy, but contamination of the whiteboard area where the sample is placed also affects calibration accuracy.
[0007] There are two main white ginseng calibration schemes:
[0008] 1. On the sample shooting surface, first take a photo of the hyperspectral data data_white(x,y,w) of a standard white plate (x,y,w represents the width, height and wavelength of the photo, respectively); record the shape of the light source spectrum at each spatial position, then take the hyperspectral data data_sample(x,y,w) of the sample, and perform a matrix point division between the hyperspectral data of the sample and the hyperspectral data of the white plate to obtain the sample reflectance REFL(x,y,w). This method can perform white parameter correction for each pixel separately, but it requires two shots to obtain the sample reflectance spectrum. The process is complicated, and there may be equipment stability errors between the two shots.
[0009] 2. When capturing the sample's hyperspectral data (data_sample(x, y, w)), place a small whiteboard next to the sample. Obtain the light source spectrum shape (data_white(1, 1, w)) from the whiteboard. Copy this spectrum matrix in one dimension (x copies, data_white(x, 1, w)). Then, copy data_white(x, 1, w) in the second dimension (y copies) to obtain the whiteboard's hyperspectral data (data_white(x, y, w). Perform a matrix dot division between the sample's hyperspectral data and the whiteboard's hyperspectral data to obtain the sample's reflectance (REFL(x, y, w)). This method requires only a single capture and is simple, but it assumes the light source spectrum shape is identical for each pixel in the hyperspectral image dimension, failing to account for errors caused by unevenness in the light source, camera imaging chip, ambient light, and other factors.
[0010] Currently, the industry lacks a simple and highly accurate method for performing point-by-point white spectral correction on each pixel of a hyperspectral image.
[0011] Summary of the Invention
[0012] The present invention proposes a method and system for performing point-to-point white reference correction on hyperspectral images to address the above-mentioned defects of the prior art.
[0013] In one aspect, the present invention provides a method for performing point-to-point white reference correction on a hyperspectral image, the method comprising the following steps:
[0014] S1: Set up the hardware scene used for shooting, take a photo of the standard reference white plate to be used for point-to-point white reference calibration, thereby obtaining hyperspectral data of the standard reference white plate, and store and record the hyperspectral data of the standard reference white plate. The hyperspectral data of the standard reference white plate is represented as white(x, y, w), where x, y, and w represent the width, height, and wavelength of the photo, respectively.
[0015] S2: placing the sample to be tested on the standard reference whiteboard and taking a photo of the sample to be tested, thereby obtaining hyperspectral data sample(x, y, w) of the sample to be tested, where x, y, and w represent the width, height, and wavelength of the photo of the sample to be tested, respectively;
[0016] S3: selecting an area within a certain range of the standard reference white plate that is not blocked and has no shadow from the hyperspectral data sample (x, y, w) of the sample to be tested, and recording the area as area A;
[0017] S4: Calculate the average spectrum value in the region A on the photograph of the sample to be tested, and the obtained average spectrum value is expressed as S A (w), and calculate the spectral average value in the same area as the area A in the hyperspectral data white (x, y, w) of the standard reference white plate, and the obtained spectral average value is expressed as W A (w), W A (w) and S A (w) is divided to obtain the white reference correction coefficient alpha(w) = W A (w). / S A (w);
[0018] S5: Divide the hyperspectral data of the sample to be tested by the hyperspectral data of the standard reference white plate to obtain a reflectance hyperspectral matrix, and then multiply the reflectance hyperspectral matrix by the white reference correction coefficient alpha(w) to obtain a corrected hyperspectral reflectance image matrix:
[0019] REFL(x,y,w)=alpha(w).*sample(x,y,w). / white(x,y,w).
[0020] First, the above method uses a more accurate white spectral correction method for hyperspectral photographs with multiple frames of images. It can restore the light source spectral information of each pixel on the photographed surface from a single hyperspectral photograph of the sample according to the spectral curve of a small fixed white plate area, combined with the white plate correction parameters in the memory; then use the restored white plate spectral information of each pixel to perform pixel-by-pixel white spectral correction on the hyperspectral data of the sample. Secondly, this method not only uses a more accurate white spectral correction method for hyperspectral images (taking into account the possible errors of multi-frame white spectral correction), but also restores the light source spectral information of each pixel on the photographed surface from a single hyperspectral photograph of the sample according to the spectral curve of a small fixed white plate area, combined with the white plate correction parameters in the memory. The restored light source spectral information of each pixel is used to perform accurate pixel-by-pixel white spectral correction on the hyperspectral data of the sample.
[0021] The standard white board used in this method is the reflectivity white board used for instrument calibration. It has high reflectivity, a flat and uniform surface, good diffusion and stability. Its reflection spectrum can be used to represent the light source spectrum and thus be used to calculate the reflection spectrum of the target object.
[0022] The method of the present invention has the following beneficial effects:
[0023] 1. It can simplify the hyperspectral analysis process. Normal hyperspectral analysis mainly includes three parts: sample spectrum acquisition, reference spectrum acquisition, and data processing and analysis. However, since the present invention does not need to separately acquire whiteboard spectra, it can omit the reference spectrum acquisition part, simplifying the process into two parts, reducing data acquisition time and increasing analysis efficiency.
[0024] 2. Using the method in the present invention can increase the accuracy of reflectance spectrum calculation. The technical solution of the present invention does not need to re-shoot the whiteboard every time. The light source spectrum data of each pixel of the whiteboard can be restored according to the pre-stored whiteboard data through an algorithm (the light source spectrum data is based on the data of the light source type and light source brightness at the time of this shooting), so as to realize point-to-point whiteboard correction of the sample. Point-to-point whiteboard correction can avoid errors caused by uneven light sources. For the shooting of transparent or translucent objects such as jade and gemstones, the use of this method and device can effectively improve the accuracy of hyperspectral analysis;
[0025] 3. Using data from the same shooting session for white reference correction can avoid additional errors caused by slight changes in equipment hardware or environment between the two shots (whiteboard shooting and sample shooting), greatly increasing the reliability and repeatability of the measurement.
[0026] In a specific embodiment, the storage records the hyperspectral data of the standard reference white plate, and stores the hyperspectral data of the standard reference white plate as pre-stored hyperspectral data of the standard reference white plate. When the method of claim 1 is executed within a certain time range, the pre-stored hyperspectral data of the standard reference white plate is used as the hyperspectral data of the standard reference white plate to execute S2 to S5.
[0027] In a specific embodiment, the standard reference whiteboard specifically includes a whiteboard base and a protective cover:
[0028] The thickness of the whiteboard base is m millimeters, and the outer shape and size are specifically designed according to the application scenario of the shooting, including square, circle and oval;
[0029] The upper surface of the whiteboard substrate is covered with a coating having isotropically uniform diffuse reflective properties, and a downwardly concave region exists on the upper surface of the whiteboard substrate, the downwardly concave region being the region A. The depth of the downwardly concave region is n millimeters (m>n), and the bottom surface and sidewalls of the downwardly concave region have the same diffuse reflective properties as the upper surface of the standard reference whiteboard, the diffuse reflective properties including the diffuse reflective coating;
[0030] The downwardly recessed area includes one or more different locations of different shapes and sizes on the standard reference whiteboard;
[0031] The protective cover is a layer of spectrally transparent material covering the surface of the standard reference whiteboard;
[0032] The spectrally transparent material is selected based on actual application, including a transparent material with no obvious absorption characteristics within the tested spectral band, and the spectrally transparent material is a material with the characteristics of wear resistance and easy cleaning, and the thickness of the spectrally transparent material ranges from 0 to 10 mm;
[0033] The spectrally transparent material specifically includes quartz glass and sapphire mirror;
[0034] The protective covers are mounted on the downwardly recessed areas on the standard reference whiteboard, wherein the number of the protective covers is the same as the number of the downwardly recessed areas.
[0035] In a specific embodiment, when photographing the standard reference whiteboard to be used for point-to-point white reference calibration, the protective cover is assembled on the whiteboard base to obtain an assembled standard reference whiteboard, and the assembled standard reference whiteboard is then placed on a photographing surface in the hardware scene for photographing the sample to be tested;
[0036] When taking the photo of the sample to be tested, the sample to be tested is placed on an area of the assembled standard reference white plate that is not protected by the protective cover.
[0037] In a specific embodiment, when the whiteboard substrate is contaminated to a certain extent:
[0038] replacing the contaminated whiteboard substrate with a whiteboard substrate having the same structure as the standard reference whiteboard;
[0039] The replacement is performed at fixed intervals or after photographing several samples to be tested.
[0040] In a specific embodiment, the method further includes: based on the corrected hyperspectral reflectance image matrix, the sample to be tested is analyzed using trained chemometric models including K nearest neighbor (KNN), support vector machine (SVM), artificial neural network (ANN) and deep neural network (NN).
[0041] According to a second aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer processor, the above method is implemented.
[0042] According to a third aspect of the present invention, a system for performing point-to-point white reference correction on a hyperspectral image is provided, the system comprising:
[0043] Whiteboard hyperspectral data acquisition module: configured to build the hardware scene used for shooting, take a photo of the standard reference whiteboard to be used for point-to-point white reference calibration, thereby acquiring the hyperspectral data of the standard reference whiteboard, and storing and recording the hyperspectral data of the standard reference whiteboard. The hyperspectral data of the standard reference whiteboard is represented as white(x, y, w), where x, y, and w represent the width, height, and wavelength of the photo, respectively;
[0044] Sample hyperspectral data acquisition module: configured to place the sample to be tested on the standard reference whiteboard and take a photo of the sample to be tested, thereby obtaining the hyperspectral data sample(x, y, w) of the sample to be tested, where x, y, and w represent the width, height, and wavelength of the photo of the sample to be tested, respectively;
[0045] A reference region selection module is configured to select an area within a certain range of the standard reference whiteboard that is not blocked and has no shadow from the hyperspectral data sample (x, y, w) of the sample to be tested, and record the area as region A;
[0046] White reference correction coefficient acquisition module: configured to calculate the spectrum average value in the region A on the photo of the sample to be tested, and the obtained spectrum average value is expressed as S A (w), and calculate the spectral average value in the same area as the area A in the hyperspectral data white (x, y, w) of the standard reference white plate, and the obtained spectral average value is expressed as W A (w), W A (w) and S A (w) is divided to obtain the white reference correction coefficient alpha(w) = W A (w). / S A (w);
[0047] White reference correction module: configured to divide the hyperspectral data of the sample to be tested by the hyperspectral data of the standard reference white plate to obtain a reflectance hyperspectral matrix, and then multiply the reflectance hyperspectral matrix by the white reference correction coefficient alpha(w) to obtain a corrected hyperspectral reflectance image matrix:
[0048] REFL(x,y,w)=alpha(w).*sample(x,y,w). / white(x,y,w).
[0049] The present invention pre-captures the hyperspectral data white(x,y,w) of the standard reference whiteboard and stores the data; then captures the sample data hyperspectral data sample(x,y,w); selects a small unobstructed whiteboard area in the sample hyperspectral data sample(x,y,w), which is recorded as area A; calculates the spectral average value in the ROI area on the sample image, and obtains S A (w); and calculate the average spectrum in the same ROI area in the whiteboard data to obtain W A (w); Divide the two and get the correction coefficient alpha(w) = W A (w). / S A (w); Multiply the white reference correction coefficient alpha by the sample reflectance image matrix after white reference correction to obtain the final hyperspectral reflectance image matrix REFL(x,y,w) = alpha(w).*sample(x,y,w). / white(x,y,w). This method eliminates the need to separately collect whiteboard spectra, reducing data acquisition time and increasing analysis efficiency, effectively improving the accuracy of hyperspectral analysis. Furthermore, using data from the same capture for white reference correction greatly increases measurement reliability and repeatability. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and, together with the description, serve to explain the principles of the invention. Other embodiments and many of the intended advantages of the embodiments will be readily apparent as they become better understood by reference to the following detailed description. Other features, objects, and advantages of the present application will become more apparent upon reading the detailed description of the non-limiting embodiments made with reference to the following drawings:
[0051] FIG1 is a diagram of an exemplary system architecture in which the present application may be applied;
[0052] FIG2 is a flow chart of a method for performing point-to-point white reference correction on a hyperspectral image according to an embodiment of the present invention;
[0053] FIG3 is a structural diagram of a standard reference whiteboard according to a specific embodiment of the present invention;
[0054] FIG4 is a diagram of a hardware configuration for jewelry detection using hyperspectral imaging technology according to a specific embodiment of the present invention;
[0055] FIG5 is a flowchart of jewelry detection using hyperspectral imaging technology according to a specific embodiment of the present invention;
[0056] FIG6 is a schematic diagram of a whiteboard for jewelry inspection using hyperspectral imaging technology according to a specific embodiment of the present invention;
[0057] FIG7 is a flow chart of measuring gemstone color using hyperspectral imaging technology according to a specific embodiment of the present invention;
[0058] FIG8 is a framework diagram of a system for performing point-to-point white reference correction on a hyperspectral image according to an embodiment of the present invention;
[0059] FIG9 is a schematic diagram of the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0060] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.
[0061] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0062] FIG1 shows an exemplary system architecture 100 to which a method for performing point-to-point white reference correction on a hyperspectral image according to an embodiment of the present application can be applied.
[0063] As shown in Figure 1, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.
[0064] Users can use terminal devices 101, 102, 103 to interact with server 105 via network 104 to receive or send messages, etc. Various applications can be installed on terminal devices 101, 102, 103, such as data processing applications, data visualization applications, web browser applications, etc.
[0065] Terminal devices 101, 102, and 103 can be hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices, including but not limited to smartphones, tablet computers, laptop computers, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software or software modules (for example, software or software modules used to provide distributed services), or they can be implemented as a single software or software module. No specific limitations are given here.
[0066] The server 105 may be a server that provides various services, such as a background information processing server that supports displaying photos of the test sample on the terminal devices 101, 102, and 103. The background information processing server may process the acquired hyperspectral data and generate processing results (e.g., a hyperspectral reflectance image matrix).
[0067] It should be noted that the method provided in the embodiment of the present application can be executed by the server 105 or by the terminal devices 101, 102, and 103. The corresponding device is generally set in the server 105 and can also be set in the terminal devices 101, 102, and 103.
[0068] It should be noted that the server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software or software modules (e.g., software or software modules for providing distributed services), or as a single software or software module. No specific limitations are given here.
[0069] It should be understood that the number of terminal devices, networks, and servers in Figure 1 is merely illustrative and any number of terminal devices, networks, and servers may be provided as required.
[0070] FIG2 shows a flow chart of a method for performing point-to-point white reference correction on a hyperspectral image according to an embodiment of the present invention. As shown in FIG2 , the method includes the following steps:
[0071] S1: Set up the hardware scene used for shooting, take a photo of the standard reference white plate to be used for point-to-point white reference calibration, thereby obtaining hyperspectral data of the standard reference white plate, and store and record the hyperspectral data of the standard reference white plate. The hyperspectral data of the standard reference white plate is represented as white(x, y, w), where x, y, and w represent the width, height, and wavelength of the photo, respectively.
[0072] S2: placing the sample to be tested on the standard reference whiteboard and taking a photo of the sample to be tested, thereby obtaining hyperspectral data sample(x, y, w) of the sample to be tested, where x, y, and w represent the width, height, and wavelength of the photo of the sample to be tested, respectively;
[0073] S3: selecting an area within a certain range of the standard reference white plate that is not blocked and has no shadow from the hyperspectral data sample (x, y, w) of the sample to be tested, and recording the area as area A;
[0074] S4: Calculate the average spectrum value in the region A on the photograph of the sample to be tested, and the obtained average spectrum value is expressed as S A (w), and calculate the spectral average value in the same area as the area A in the hyperspectral data white (x, y, w) of the standard reference white plate, and the obtained spectral average value is expressed as W A (w), W A (w) and S A (w) is divided to obtain the white reference correction coefficient alpha(w) = W A (w). / S A (w);
[0075] S5: Divide the hyperspectral data of the sample to be tested by the hyperspectral data of the standard reference white plate to obtain a reflectance hyperspectral matrix, and then multiply the reflectance hyperspectral matrix by the white reference correction coefficient alpha(w) to obtain a corrected hyperspectral reflectance image matrix:
[0076] REFL(x,y,w)=alpha(w).*sample(x,y,w). / white(x,y,w).
[0077] In a specific embodiment, the storage records the hyperspectral data of the standard reference white plate, and stores the hyperspectral data of the standard reference white plate as pre-stored hyperspectral data of the standard reference white plate. When the method of claim 1 is executed within a certain time range, the pre-stored hyperspectral data of the standard reference white plate is used as the hyperspectral data of the standard reference white plate to execute S2 to S5.
[0078] In a specific embodiment, the standard reference whiteboard specifically includes a whiteboard base and a protective cover:
[0079] The thickness of the whiteboard base is m millimeters, and the outer shape and size are specifically designed according to the application scenario of the shooting, including square, circle and oval;
[0080] The upper surface of the whiteboard substrate is covered with a coating having isotropically uniform diffuse reflective properties, and a downwardly concave region exists on the upper surface of the whiteboard substrate, the downwardly concave region being the region A. The depth of the downwardly concave region is n millimeters (m>n), and the bottom surface and sidewalls of the downwardly concave region have the same diffuse reflective properties as the upper surface of the standard reference whiteboard, the diffuse reflective properties including the diffuse reflective coating;
[0081] The downwardly recessed area includes one or more different locations of different shapes and sizes on the standard reference whiteboard;
[0082] The protective cover is a layer of spectrally transparent material covering the surface of the standard reference whiteboard;
[0083] The spectrally transparent material is selected based on actual application, including a transparent material with no obvious absorption characteristics within the tested spectral band, and the spectrally transparent material is a material with the characteristics of wear resistance and easy cleaning, and the thickness of the spectrally transparent material ranges from 0 to 10 mm;
[0084] The spectrally transparent material specifically includes quartz glass and sapphire mirror;
[0085] The protective covers are mounted on the downwardly recessed areas on the standard reference whiteboard, wherein the number of the protective covers is the same as the number of the downwardly recessed areas.
[0086] FIG3 shows a structural diagram of a standard reference whiteboard according to a specific embodiment of the present invention, which mainly includes a whiteboard base 301, a recess 302 on the whiteboard, and a protective cover 303:
[0087] 1. Whiteboard base 301: Its thickness is m millimeters, and its outer contour shape and size are specifically designed according to the application scenario of the shooting (it can be a square as shown in the figure or other circular, elliptical, etc.). Its upper surface is covered with a coating with isotropic uniform diffuse reflection properties. A certain area on the upper surface of the whiteboard is concave downward (this area is the selected reference whiteboard area), and the depth of the concave part is n millimeters (m>n). The bottom surface and side walls of the concave part have the same diffuse reflection properties as the upper surface of the whiteboard (coated with the same diffuse reflection coating). The reference whiteboard area can be selected at different positions on the whiteboard, and can be of different shapes and sizes, or there can be multiple protected reference areas. When the whiteboard base 301 is severely contaminated, it can be replaced with a replacement whiteboard of the same structure (or the whiteboard base 301 can be replaced at fixed intervals or after shooting N samples) to ensure the accuracy of the radiometry when the sample is shot (that is, to ensure the accuracy of the numerator REFL (x, y, w) when calculating the reflectivity);
[0088] 2. Protective cover 303: This is a layer of spectrally transparent material covering the surface of the whiteboard. The material can be selected based on the actual application. It should be transparent, have no significant absorption characteristics within the tested spectral band, be wear-resistant, and be easy to clean. Its thickness is generally several millimeters. Common materials include quartz glass and sapphire mirrors. Protective cover 303 will be installed in the recesses 302 on the whiteboard (the number of protective covers 303 and recesses 302 on the whiteboard will be the same). This protects the reference area from contamination and ensures the accuracy of the whiteboard correction matrix calculation (i.e., the accuracy of the denominator in the reflectivity calculation, REFL(x,y,w) = alpha(w).*sample(x,y,w). / white(x,y,w). Protective cover 303 is easy to clean and can extend the life of the whiteboard.
[0089] After the hardware scene for shooting is set up, the protective cover 303 is assembled on the whiteboard base 301, and the assembled entire whiteboard is placed on the designed sample shooting surface. When shooting, the sample is placed on the area of the whiteboard not protected by the protective cover 303 (or at least ensure that there is a protected reference area not covered by the sample), and data collection can be carried out.
[0090] In a specific embodiment, when photographing the standard reference whiteboard to be used for point-to-point white reference calibration, the protective cover is assembled on the whiteboard base to obtain an assembled standard reference whiteboard, and the assembled standard reference whiteboard is then placed on a photographing surface in the hardware scene for photographing the sample to be tested;
[0091] When taking the photo of the sample to be tested, the sample to be tested is placed on an area of the assembled standard reference white plate that is not protected by the protective cover.
[0092] In a specific embodiment, when the whiteboard substrate is contaminated to a certain extent:
[0093] replacing the contaminated whiteboard substrate with a whiteboard substrate having the same structure as the standard reference whiteboard;
[0094] The replacement is performed at fixed intervals or after photographing several samples to be tested.
[0095] In a specific embodiment, the method further includes: based on the corrected hyperspectral reflectance image matrix, simultaneously using trained chemometric models including K nearest neighbor (KNN), support vector machine (SVM), artificial neural network (ANN) and deep neural network (NN) to analyze the sample to be tested.
[0096] 1) Specific Example 1: Jewelry testing is performed using the method described in the present invention.
[0097] FIG4 shows a hardware configuration diagram for jewelry detection using hyperspectral imaging technology according to a specific embodiment of the present invention, and FIG5 shows a flow chart of this embodiment.
[0098] As shown in FIG4 , the hardware setup includes: a hyperspectral camera 401 ; a light source 402 ; a jewelry inspection black box 403 ; a sample placement table 404 ; quartz glass 405 ; a door switch shaft 406 ; a door panel 407 ; and a door handle 408 .
[0099] Its characteristics are:
[0100] The visual range of the hyperspectral camera 401 is exactly the size of the sample placement platform 404;
[0101] All sides of the jewelry testing black box 403 are sealed and made of black material;
[0102] The detailed structure of the sample placement table 404 and the quartz glass 405 refers to the glass protection scheme of the whiteboard. The area protected by the quartz glass 405 is selected in a corner of the sample placement table 404. When placing the sample, try to place the sample in the center and do not cover the area protected by the quartz glass 405.
[0103] 406, 407, and 408 are door structure combinations. Samples are placed after the doors are opened, and jewelry samples are tested after the doors are closed.
[0104] After the hardware is set up, the calibration parameters are stored: ensure that the sample placement table is clean and free of samples, capture the hyperspectral data of a pure white board (white(x,y,w), and calculate the average spectrum W of the white board within the glass protection area A. A (w), calculate the white parameter correction matrix based on the glass protection area: compare each pixel (x, y) in white(x, y, w) with W A (w) Take the Hadamard quotient: C(x,y,w)=white(x,y,w). / W A (w); stores the correction matrix C(x,y,w) and calls it directly in future applications.
[0105] FIG6 shows a schematic diagram of a whiteboard for jewelry inspection using hyperspectral imaging technology according to a specific embodiment of the present invention. When performing jewelry inspection, refer to the calibration whiteboard schematic diagram 601 and the photographed whiteboard schematic diagram 602 in FIG6 :
[0106] Step 1: Calibration
[0107] As shown in the calibration whiteboard diagram 601, keep the whiteboard shooting surface free of contaminants, and shoot the whiteboard hyperspectral data separately to calculate the whiteboard calibration matrix. This step can be completed when the hyperspectral product leaves the factory or is installed and debugged for the customer, or it can be updated by professional staff after the equipment has been working continuously for a period of time. Specifically, shoot the whiteboard hyperspectral data white (x, y, w), select a fixed initial reference area (an area of n*n pixels, such as the area marked at position A in the figure), and calculate the average spectrum W of the reference area A. A (w)=<white(A,w)> A ,in <m> B Indicates that the average value of m is obtained in area B. To calculate the correction matrix C(x,y,w), the spectrum of each pixel in white(x,y,w) is divided by the average spectrum of reference area A, that is, C(x,y,w) = white(x,y,w). / W A (w), where . / represents the Hadamard quotient of two vectors. The whiteboard hyperspectral data white(x,y,w) and the correction matrix C(x,y,w) are stored in the device memory.
[0108] Step 2: Shoot
[0109] 1. As shown in the whiteboard shooting diagram 602, place a jewelry sample on the whiteboard and shoot the sample's hyperspectral data sample (x, y, w)
[0110] 2. Calculate the average spectrum of the glass protection area A in the hyperspectral data of the jewelry: S A (w)=<sample(A,w)> A ; and read the correction matrix C(x,y,w) in the memory,
[0111] 3. Restore the whiteboard data at this time, recorded as white N (x,y,w), let each pixel (x,y) in C(x,y,w) be respectively A (w) Hadamard product: white N (x,y,w)=C(x,y,w).*S A (w). Where * represents the Hadamard product of two vectors or matrices;
[0112] 4. Calculate the reflectivity, i.e. REFL(x,y,w)=sample(x,y,w). / white N (x,y,w);
[0113] 5. Based on the reflectivity data, use trained chemometric models such as K-nearest neighbor (KNN), support vector machine (SVM), artificial neural network (ANN), and deep neural network (NN) to analyze the type of jewelry, authenticity, and whether optimization treatment has been performed.
[0114] 2) Specific Example 2: The color of a gemstone is measured using the method described in the present invention.
[0115] FIG7 shows a flowchart of measuring the color of a gemstone using hyperspectral imaging technology according to a specific embodiment of the present invention. The specific steps are as follows:
[0116] There is an initial reference area A1. Based on the sample's hyperspectral data, it is determined whether the reference area A1 is blocked by the sample. If not, the reference area A1 is confirmed to be valid, and the average spectrum W of the white plate (which can also be calculated in advance) and the sample in the area is calculated. A1 (w) and S A1 (w), and obtain the white ginseng correction coefficient alpha(w)=W A1 (w) / S A1 (w);
[0117] If blocked, read the pure whiteboard hyperspectral data white (x, y, w) in the memory, reselect the new reference area A2, and calculate the white spectral correction coefficient alpha = W based on the reference area A2 A2 (w) / S A2 (w).
[0118] Extract the ROI area B of the sample and obtain the sample and whiteboard hyperspectral data sample(xB,yB,w) and white(xB,yB,w) of the selected area B;
[0119] Calculate the reflectance spectrum matrix of the sample area:
[0120] REFL(xB,yB,w)=alpha(w).*sample(xB,yB,w). / white(xB,yB,w);
[0121] After obtaining the reflectance spectrum matrix data of the gem area, the specific color value of the gem can be calculated according to the calculation method of color theory and a more in-depth analysis can be performed based on this.
[0122] It should be recognized that, in theory, the initial reference area A1 can be selected anywhere on the sample placement surface; in actual application, the best location A1 is a place on the sample placement surface that is not easily covered by the sample, so a corner of the sample placement surface is preferably selected.
[0123] Figure 8 shows a framework diagram of a system for performing point-to-point white reference correction on hyperspectral images, according to one embodiment of the present invention. The system includes a whiteboard hyperspectral data acquisition module 801, a sample hyperspectral data acquisition module 802, a reference region selection module 803, a white reference correction coefficient acquisition module 804, and a white reference correction module 805.
[0124] In a specific embodiment, the whiteboard hyperspectral data acquisition module 801 is configured to build a hardware scene used for shooting, take a photo of a standard reference whiteboard to be used for point-to-point white reference calibration, thereby acquiring hyperspectral data of the standard reference whiteboard, and storing and recording the hyperspectral data of the standard reference whiteboard. The hyperspectral data of the standard reference whiteboard is represented as white(x, y, w), where x, y, and w represent the width, height, and wavelength of the photo, respectively.
[0125] The sample hyperspectral data acquisition module 802 is configured to place the sample to be tested on the standard reference whiteboard and take a photo of the sample to be tested, thereby obtaining the hyperspectral data sample(x, y, w) of the sample to be tested, where x, y, and w represent the width, height, and wavelength of the photo of the sample to be tested, respectively.
[0126] The reference region selection module 803 is configured to select an area within a certain range of the standard reference whiteboard that is not blocked and has no shadow from the hyperspectral data sample (x, y, w) of the sample to be tested, and record the area as area A;
[0127] The white reference correction coefficient acquisition module 804 is configured to calculate the spectrum average value in the region A on the photograph of the sample to be tested. The obtained spectrum average value is represented by S A (w), and calculate the spectral average value in the same area as the area A in the hyperspectral data white (x, y, w) of the standard reference white plate, and the obtained spectral average value is expressed as W A (w), W A (w) and S A (w) is divided to obtain the white reference correction coefficient alpha(w) = W A (w). / S A (w);
[0128] The white reference correction module 805 is configured to divide the hyperspectral data of the sample to be tested by the hyperspectral data of the standard reference white plate to obtain a reflectance hyperspectral matrix, and then multiply the reflectance hyperspectral matrix by the white reference correction coefficient alpha(w) to obtain a corrected hyperspectral reflectance image matrix: REFL(x,y,w)=alpha(w).*sample(x,y,w). / white(x,y,w).
[0129] The system pre-captures the hyperspectral data white(x,y,w) of the standard reference whiteboard and stores the data; then captures the sample data hyperspectral data sample(x,y,w); selects a small unobstructed whiteboard area in the sample hyperspectral data sample(x,y,w), which is recorded as area A; calculates the spectral average value in the ROI area on the sample image, and obtains S A (w); and calculate the average spectrum in the same ROI area in the whiteboard data to obtain W A (w); Divide the two and get the correction coefficient alpha(w) = W A (w). / S A (w); Multiply the white reference correction coefficient alpha by the sample reflectance image matrix after white reference correction to obtain the final hyperspectral reflectance image matrix REFL(x,y,w) = alpha(w).*sample(x,y,w). / white(x,y,w). This method eliminates the need to separately collect whiteboard spectra, reducing data acquisition time and increasing analysis efficiency, effectively improving the accuracy of hyperspectral analysis. Furthermore, using data from the same capture for white reference correction greatly increases measurement reliability and repeatability.
[0130] 9, which shows a schematic diagram of a computer system 900 suitable for implementing an electronic device according to an embodiment of the present invention. The electronic device shown in FIG9 is merely an example and should not limit the functionality and scope of use of the embodiments of the present invention.
[0131] As shown in FIG9 , a computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage unit 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the system 900 are also stored in the RAM 903. The CPU 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.
[0132] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, and the like; an output section 907 including a liquid crystal display (LCD) and a speaker; a storage section 908 including a hard disk and the like; and a communication section 909 including a network interface card such as a LAN card or a modem. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 910 as needed, so that a computer program read therefrom can be installed in the storage section 908 as needed.
[0133] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above-mentioned functions defined in the method of the present application are executed. It should be noted that the computer-readable storage medium described in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0134] Computer program code for performing the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0135] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.
[0136] The modules involved in the embodiments described in this application may be implemented in software or hardware. The units described may also be provided in a processor, and the names of these units do not, in certain circumstances, constitute limitations on the units themselves.
[0137] Embodiments of the present invention also relate to a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer processor, implements the method described above. The computer program includes program code for executing the method shown in the flowchart. It should be noted that the computer-readable medium of the present application may be a computer-readable signal medium or a computer-readable medium, or any combination thereof.
[0138] The present invention pre-captures the hyperspectral data white(x,y,w) of the standard reference whiteboard and stores the data; then captures the sample data hyperspectral data sample(x,y,w); selects a small unobstructed whiteboard area in the sample hyperspectral data sample(x,y,w), which is recorded as area A; calculates the spectral average value in the ROI area on the sample image, and obtains S A (w); and calculate the average spectrum in the same ROI area in the whiteboard data to obtain W A (w); Divide the two and get the correction coefficient alpha(w) = W A (w). / S A (w); Multiply the white reference correction coefficient alpha by the sample reflectance image matrix after white reference correction to obtain the final hyperspectral reflectance image matrix REFL(x,y,w) = alpha(w).*sample(x,y,w). / white(x,y,w). This method eliminates the need to separately collect whiteboard spectra, reducing data acquisition time and increasing analysis efficiency, effectively improving the accuracy of hyperspectral analysis. Furthermore, using data from the same capture for white reference correction greatly increases measurement reliability and repeatability.
[0139] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.< / m>
Claims
1. A method for performing point-to-point white reference correction on a hyperspectral image, characterized in that: The following steps are involved: S1: Set up the hardware scene used for shooting, take a photo of the standard reference white plate to be used for point-to-point white reference calibration, thereby obtaining hyperspectral data of the standard reference white plate, and store and record the hyperspectral data of the standard reference white plate. The hyperspectral data of the standard reference white plate is represented as white(x, y, w), where x, y, and w represent the width, height, and wavelength of the photo, respectively. S2: placing the sample to be tested on the standard reference whiteboard and taking a photo of the sample to be tested, thereby obtaining hyperspectral data sample(x, y, w) of the sample to be tested, where x, y, and w represent the width, height, and wavelength of the photo of the sample to be tested, respectively; S3: selecting an area within a certain range of the standard reference white plate that is not blocked and has no shadow from the hyperspectral data sample (x, y, w) of the sample to be tested, and recording the area as area A; S4: Calculate the average spectrum value in the region A on the photograph of the sample to be tested, and the obtained average spectrum value is expressed as S A (w), and calculate the spectral average value in the same area as the area A in the hyperspectral data white (x, y, w) of the standard reference white plate, and the obtained spectral average value is expressed as W A (w), W A (w) and S A (w) is divided to obtain the white reference correction coefficient alpha(w) = W A (w). / S A (w); S5: Divide the hyperspectral data of the sample to be tested by the hyperspectral data of the standard reference white plate to obtain a reflectance hyperspectral matrix, and then multiply the reflectance hyperspectral matrix by the white reference correction coefficient alpha(w) to obtain a corrected hyperspectral reflectance image matrix: REFL(x,y,w)=alpha(w).*sample(x,y,w). / white(x,y,w).
2. The method according to claim 1, characterized in that The storage records the hyperspectral data of the standard reference white plate, and stores the hyperspectral data of the standard reference white plate as pre-stored hyperspectral data of the standard reference white plate within a certain time range. When executing the method according to claim 1, the pre-stored hyperspectral data of the standard reference white plate is used as the hyperspectral data of the standard reference white plate to execute S2 to S5.
3. The method according to claim 1, characterized in that The standard reference whiteboard specifically includes a whiteboard base and a protective cover: The thickness of the whiteboard base is m millimeters, and the outer shape and size are specifically designed according to the application scenario of the shooting, including square, circle and oval; The upper surface of the whiteboard substrate is covered with a coating having isotropically uniform diffuse reflective properties, and a downwardly concave region exists on the upper surface of the whiteboard substrate, the downwardly concave region being the region A. The depth of the downwardly concave region is n millimeters (m>n), and the bottom surface and sidewalls of the downwardly concave region have the same diffuse reflective properties as the upper surface of the standard reference whiteboard, the diffuse reflective properties including the diffuse reflective coating; The downwardly recessed area includes one or more different locations of different shapes and sizes on the standard reference whiteboard; The protective cover is a layer of spectrally transparent material covering the surface of the standard reference whiteboard; The spectrally transparent material is selected based on actual application, including a transparent material with no obvious absorption characteristics within the tested spectral band, and the spectrally transparent material is a material with the characteristics of wear resistance and easy cleaning, and the thickness of the spectrally transparent material ranges from 0 to 10 mm; The spectrally transparent material specifically includes quartz glass and sapphire mirror; The protective covers are mounted on the downwardly recessed areas on the standard reference whiteboard, wherein the number of the protective covers is the same as the number of the downwardly recessed areas.
4. The method according to claim 3, characterized in that When photographing the standard reference white plate to be used for point-to-point white reference calibration, the protective cover is assembled on the white plate base to obtain an assembled standard reference white plate, and the assembled standard reference white plate is then placed on a photographing surface in the hardware scene for photographing the sample to be tested; When taking the photo of the sample to be tested, the sample to be tested is placed on an area of the assembled standard reference white plate that is not protected by the protective cover.
5. The method according to claim 3, characterized in that When the whiteboard substrate is contaminated to a certain extent: Replacing the contaminated whiteboard substrate with a whiteboard substrate having the same structure as the standard reference whiteboard in claim 3; The replacement is performed at fixed intervals or after photographing several samples to be tested.
6. The method according to claim 1, characterized in that The method also includes: based on the corrected hyperspectral reflectance image matrix, simultaneously using trained chemometric models including K nearest neighbor (KNN), support vector machine (SVM), artificial neural network (ANN) and deep neural network (NN) to analyze the sample to be tested.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a computer processor, the method according to any one of claims 1 to 6 is implemented.
8. A system for performing point-to-point white reference correction on hyperspectral images, characterized in that: include: Whiteboard hyperspectral data acquisition module: configured to build the hardware scene used for shooting, take a photo of the standard reference whiteboard to be used for point-to-point white reference calibration, thereby acquiring the hyperspectral data of the standard reference whiteboard, and storing and recording the hyperspectral data of the standard reference whiteboard. The hyperspectral data of the standard reference whiteboard is represented as white(x, y, w), where x, y, and w represent the width, height, and wavelength of the photo, respectively; Sample hyperspectral data acquisition module: configured to place the sample to be tested on the standard reference whiteboard and take a photo of the sample to be tested, thereby obtaining the hyperspectral data sample(x, y, w) of the sample to be tested, where x, y, and w represent the width, height, and wavelength of the photo of the sample to be tested, respectively; A reference region selection module is configured to select an area within a certain range of the standard reference whiteboard that is not blocked and has no shadow from the hyperspectral data sample (x, y, w) of the sample to be tested, and record the area as region A; White reference correction coefficient acquisition module: configured to calculate the spectrum average value in the region A on the photo of the sample to be tested, and the obtained spectrum average value is expressed as S A (w), and calculate the spectral average value in the same area as the area A in the hyperspectral data white (x, y, w) of the standard reference white plate, and the obtained spectral average value is expressed as W A (w), W A (w) and S A (w) is divided to obtain the white reference correction coefficient alpha(w) = W A (w). / S A (w); White reference correction module: configured to divide the hyperspectral data of the sample to be tested by the hyperspectral data of the standard reference white plate to obtain a reflectance hyperspectral matrix, and then multiply the reflectance hyperspectral matrix by the white reference correction coefficient alpha(w) to obtain a corrected hyperspectral reflectance image matrix: REFL(x,y,w)=alpha(w).*sample(x,y,w). / white(x,y,w).
