Spectral processing method, spectral processing apparatus, and spectral processing system

Abstract

The application relates to a spectrum processing method, a spectrum processing device and a spectrum processing system. The spectrum recovery method comprises the following steps: acquiring parameter information of a spectrum chip; configuring parameters corresponding to a shot spectrum image according to the parameter information of the spectrum chip, and collecting a spectrum image; and recovering spectrum information of a to-be-detected object based on the spectrum image and a corresponding spectrum recovery algorithm. In this way, a miniature spectrum processing device with small volume, convenient carrying, direct interaction with a terminal, convenience and high recovery precision can be realized.

Description

Technical Field

[0001] This application relates to the field of spectral technology, and more specifically, to a spectral processing method, a spectral processing device, and a spectral processing system. Background Technology

[0002] With the continuous improvement of social informatization, information technology has permeated all aspects of social life. Currently, imaging chips and imaging devices are widely used. Taking a camera as an example, it contains an image sensor (e.g., a CMOS image sensor or a CCD sensor) to acquire image information of the subject, such as the RGB color information of the subject.

[0003] However, regardless of whether a CMOS image sensor or a CCD sensor is used as the imaging chip, it can only acquire image information of the subject, but not its spectral information. In other words, existing imaging chips and devices cannot obtain the spectral information of objects, preventing the resulting images from being widely applicable to scenarios requiring spectral information as data support, such as intelligent AI recognition and qualitative and quantitative analysis of material components.

[0004] Because light interacts with matter through processes such as absorption, scattering, fluorescence, and Raman spectroscopy, it produces specific spectra, and each substance's spectrum is unique. In current methods, the spectral information of the subject requires specialized equipment (e.g., a spectrometer or spectral camera) to acquire. Therefore, when it is necessary to simultaneously obtain the spectral and image information of the subject, multiple camera modules and / or devices are often required to work together, and algorithms are used to integrate the acquired image and spectral information.

[0005] It is understandable that the need for multiple camera modules and / or devices increases costs. More importantly, this solution occupies a relatively large space, resulting in poor portability and integrability; that is, it is difficult to integrate it into terminal devices.

[0006] Therefore, it is desirable to provide an improved spectral processing scheme. Summary of the Invention

[0007] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a spectral processing method, a spectral processing device, and a spectral processing system. These systems acquire spectral images and recover the spectral information of the object under test based on parameter information from a spectral chip. This results in a miniature spectral processing device that is small in size, portable, can directly interact with a terminal, and offers convenient and high-precision recovery.

[0008] According to one aspect of this application, a spectral recovery method is provided, comprising: acquiring parameter information of a spectral chip; configuring parameters corresponding to a captured spectral image based on the parameter information of the spectral chip, and acquiring the spectral image; and recovering the spectral information of the object under test based on the spectral image and the corresponding spectral recovery algorithm.

[0009] In the above spectral processing method, obtaining the parameter information of the spectral chip includes: pre-setting parameter information corresponding to different spectral chip types; determining the type of the current spectral chip; and searching for and obtaining the parameter information corresponding to the type of the current spectral chip.

[0010] In the above spectral processing method, finding and obtaining parameter information corresponding to the type of the current spectral chip includes: determining whether the type of the current spectral chip matches a preset type; in response to the type of the current spectral chip matching a preset type, obtaining parameter information of the spectral chip of the preset type; and in response to the type of the current spectral chip not matching a preset type, determining that the parameter information of the spectral chip cannot be obtained.

[0011] In the above spectral processing method, configuring the parameters corresponding to the captured spectral image according to the parameter information of the spectral chip and obtaining the spectral image includes: obtaining the configuration parameters corresponding to the spectral chip, wherein the configuration parameters correspond to the type of the spectral chip and the configuration parameters are calibration parameters, wherein the configuration parameters include at least one or a combination of exposure parameters and gain parameters.

[0012] The above spectral processing method further includes: optimizing the configuration parameters based on the acquired spectral image; and storing the optimized configuration parameters.

[0013] The above spectral processing method further includes: sending the recovered spectral information to a server, wherein the server optimizes configuration parameters based on the difference between the spectral information and standard data; and receiving feedback information from the server regarding the optimization of the configuration parameters, and storing the optimized configuration parameters.

[0014] In the above spectral processing method, optimizing the configuration parameters based on the acquired spectral images includes: cyclically acquiring and caching the spectral images according to a set frame rate; and performing noise reduction processing on the cached spectral images.

[0015] In the above spectral processing method, recovering the spectral information of the object under test based on the spectral image and the corresponding spectral recovery algorithm includes: recovering the spectral information of the object under test based on the denoised spectral image and the corresponding spectral recovery algorithm.

[0016] In the above spectral processing method, recovering the spectral information of the object under test based on the spectral image and the corresponding spectral recovery algorithm includes: acquiring the spectral information of each modulation unit in the light modulation layer of the spectral chip corresponding to the pixel on the image sensor and the light intensity information of each non-modulation unit in the modulation layer corresponding to the pixel; determining the spectral data of the object based on the spectral information of the pixel corresponding to each modulation unit; and determining the image data of the object based on the light intensity information of the pixel corresponding to each non-modulation unit.

[0017] The above spectral processing method further includes: sending the recovered spectral information to the client side.

[0018] According to another aspect of this application, a spectral processing apparatus is provided, comprising: an acquisition unit for acquiring parameter information of a spectral chip; a collection unit for configuring parameters corresponding to a captured spectral image based on the parameter information of the spectral chip, and collecting the spectral image; and a processing unit for recovering spectral information of a test object based on the spectral image and a corresponding spectral recovery algorithm.

[0019] In the above-mentioned spectral processing device, the acquisition unit is used to: pre-set parameter information corresponding to different spectral chip types; determine the type of the current spectral chip; and search for and acquire parameter information corresponding to the type of the current spectral chip.

[0020] In the above-described spectral processing device, the acquisition unit's process of searching for and acquiring parameter information corresponding to the type of the current spectral chip includes: determining whether the type of the current spectral chip matches a preset type; in response to the current spectral chip type matching a preset type, acquiring parameter information of the preset type of spectral chip; and in response to the current spectral chip type not matching a preset type, determining that the parameter information of the spectral chip cannot be acquired.

[0021] In the above-mentioned spectral processing device, the acquisition unit is used to: acquire configuration parameters corresponding to the spectral chip, the configuration parameters corresponding to the type of the spectral chip, and the configuration parameters being calibration parameters, wherein the configuration parameters include at least one or a combination of exposure parameters and gain parameters.

[0022] The above-described spectral processing apparatus further includes: an optimization unit for optimizing the configuration parameters based on the acquired spectral image; and a storage unit for storing the optimized configuration parameters.

[0023] The aforementioned spectral processing apparatus further includes: a first transmitting unit, configured to transmit the recovered spectral information to a server side, wherein the server side optimizes configuration parameters based on the difference between the spectral information and standard data; and a receiving unit, configured to receive feedback information from the server side regarding the optimization of the configuration parameters, and store the optimized configuration parameters.

[0024] In the above-mentioned spectral processing device, the acquisition unit is used to: cyclically acquire the spectral images according to a set frame rate and cache them; and to perform noise reduction processing on the cached spectral images.

[0025] In the above-mentioned spectral processing device, the processing unit is used to: recover the spectral information of the object under test based on the denoised spectral image and the corresponding spectral recovery algorithm.

[0026] In the above-described spectral processing apparatus, the processing unit is configured to: acquire spectral information of pixels on the image sensor corresponding to each modulation unit in the light modulation layer of the spectral chip and light intensity information of pixels corresponding to each non-modulation unit in the modulation layer; determine spectral data of the subject based on the spectral information of the pixels corresponding to each modulation unit; and determine image data of the subject based on the light intensity information of the pixels corresponding to each non-modulation unit.

[0027] The aforementioned spectral processing apparatus further includes a second transmitting unit for transmitting the recovered spectral information to the client side.

[0028] According to another aspect of this application, a spectral processing system is provided, comprising a spectral processing device that applies the spectral recovery method described above. The spectral processing system further comprises a server and a client. The spectral processing device is connected to the client to send acquired spectral information to the client, and the spectral processing device is connected to the server to download firmware required for updating the spectral processing device from the server.

[0029] The spectral processing method, spectral processing device, and spectral processing system provided in this application can acquire spectral images and recover the spectral information of the object under test based on the parameter information of the spectral chip, thereby realizing a miniature spectral processing device that is small in size, easy to carry, can be directly interacted with the terminal, and is convenient and has high recovery accuracy. Attached Figure Description

[0030] Various other advantages and benefits of this application will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0031] Figure 1 The illustration shows a schematic configuration of a spectral processing apparatus according to an embodiment of this application;

[0032] Figure 2 A schematic flowchart of a spectral processing method according to an embodiment of this application is shown;

[0033] Figure 3 The illustration shows a schematic diagram of an application example of the spectral processing method according to an embodiment of this application;

[0034] Figure 4 The figure shows a schematic block diagram of a spectral processing apparatus according to an embodiment of the present application;

[0035] Figure 5 The diagram shows a schematic of the spectral lines displayed after the information is collected.

[0036] Figure 6 The diagram shows a comparison of the transmission spectra of normal apples and apples at different degrees of decay.

[0037] Figure 7 The illustration shows a schematic diagram of a spectral processing system including a spectral processing apparatus that applies a spectral processing method according to an embodiment of the present application. Detailed Implementation

[0038] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0039] Application Overview

[0040] The spectral processing method according to the embodiments of this application can be implemented by a spectral processing device with a spectral chip, such as a spectrometer or a spectral imaging device.

[0041] The spectral processing apparatus according to embodiments of this application may have, for example: Figure 1 The configuration shown is shown here. Figure 1 A schematic configuration diagram of a spectral processing apparatus according to an embodiment of this application is shown. Figure 1 As shown, in the spectral processing apparatus according to the embodiments of this application, the optical system is optional, and it may be an optical system such as a lens assembly or a light homogenizing assembly. The optical system is located at the front end of the filter structure. The light source is adjusted by the optical system and then modulated by the filter structure before being received by the image sensor to obtain the spectral response.

[0042] For example, the optical system can be implemented as an optical module for acquiring the subject to be photographed, acquiring the incident light of the subject entering the optical module (the spectrum obtained here may be the projection spectrum or the reflection spectrum), and transmitting the acquired incident light to the spectral chip for processing.

[0043] In this embodiment, the spectral chip includes a filter structure and an image sensor. The filter structure is a broadband filter structure in the frequency domain or wavelength domain. The filter structure can be a metasurface, photonic crystal, nanopillar, multilayer film, dye, quantum dot, MEMS (microelectromechanical systems), FP etalon, cavity layer, waveguide layer, diffraction element, or other structures or materials with filtering properties. For example, in this embodiment, the filter structure can be the light modulation layer described in Chinese Patent CN201921223201.2.

[0044] In the optical modulation layer, each modulation unit has a different modulation effect on light of different wavelengths. The modulation methods of the input spectrum of each modulation unit can be the same or different. Different modulation methods may include, but are not limited to, scattering, absorption, transmission, reflection, interference, excitation, resonance enhancement, etc. The final effect of the modulation effect is that the transmission spectrum of light of different wavelengths is different after passing through the modulation unit.

[0045] The image sensor (i.e., photodetector array) can be a CMOS image sensor (CIS), CCD, array photodetector, etc. Additionally, optional data processing units can be MCUs, CPUs, GPUs, FPGAs, NPUs, ASICs, etc., which can export the data generated by the image sensor for external processing.

[0046] In other words, the spectral chip according to the embodiments of this application has a filter structure on the photodetector array, through which image information and / or spectral information of the subject can be obtained. Then, spectral restoration is performed using a spectral restoration algorithm to obtain spectral information, which can realize the restoration of a single spectral line and / or the restoration of the entire spectral image. Furthermore, the spectral processing device according to the embodiments of this application is generally small in size, easy to carry, can directly interact with other terminals, and is convenient and has high restoration accuracy.

[0047] Exemplary methods

[0048] Figure 2 A schematic flowchart of a spectral processing method according to an embodiment of this application is shown.

[0049] like Figure 2 As shown, the spectral recovery method according to an embodiment of this application includes the following steps.

[0050] Step S110: Obtain parameter information of the spectral chip. Here, the parameter information of the spectral chip includes, but is not limited to, the type and size of the spectral chip. Parameter information corresponding to different spectral chip types can be preset, allowing direct searching based on a preset spectral chip type. Furthermore, it can be determined whether the obtained spectral chip information matches a preset type, searching for a type that matches the current spectral chip. For example, if it is determined to be a preset chip type, proceed to the next instruction; if it is determined not to be a preset chip type, a message can be displayed indicating that it cannot be obtained.

[0051] When the acquired chip information is of a preset type, the current spectral chip information can be obtained, which also includes configuration parameters. These configuration parameters can include gain parameters, exposure parameters, etc. Each set of configuration parameters can correspond one-to-one with different chip types, and the configuration parameters corresponding to different chip types can be calibrated and verified in advance. Then, they can be fine-tuned according to different application scenarios or based on the currently acquired real-time spectral information.

[0052] Therefore, in the spectral processing method according to the embodiments of this application, obtaining the parameter information of the spectral chip includes: pre-setting parameter information corresponding to different spectral chip types; determining the type of the current spectral chip; and searching for and obtaining the parameter information corresponding to the type of the current spectral chip.

[0053] Furthermore, in the above-described spectral processing method, finding and obtaining parameter information corresponding to the type of the current spectral chip includes: determining whether the type of the current spectral chip matches a preset type; in response to the type of the current spectral chip matching a preset type, obtaining parameter information of the spectral chip of the preset type; and in response to the type of the current spectral chip not matching a preset type, determining that the parameter information of the spectral chip cannot be obtained.

[0054] Step S120: Configure the parameters corresponding to the captured spectral image according to the parameter information of the spectral chip, and acquire the spectral image. Here, the acquired spectral image can be referred to as RAW (raw) data, which contains the spectral information and image information of the subject. Furthermore, the spectral image can be acquired with different configured parameters according to user needs. For example, the configured parameters can be calibration parameters corresponding to the type of spectral chip, including exposure parameters, gain parameters, etc., or combinations thereof.

[0055] Therefore, in the spectral processing method according to the embodiments of this application, configuring the parameters corresponding to the captured spectral image according to the parameter information of the spectral chip and obtaining the spectral image includes: obtaining the configuration parameters corresponding to the spectral chip, wherein the configuration parameters correspond to the type of the spectral chip and the configuration parameters are calibration parameters, wherein the configuration parameters include at least one or a combination of exposure parameters and gain parameters.

[0056] Furthermore, in this embodiment, the configuration parameters of the spectral chip can be updated and optimized; that is, the spectral processing device according to this embodiment can undergo firmware upgrades. Specifically, the spectral processing device according to this embodiment can optimize the configuration parameters based on the acquired spectral image and can store the optimized configuration parameters.

[0057] In this way, by optimizing the configuration parameters based on the acquired spectral image, the imaging performance of the spectral image can be improved based on the optimized configuration parameters.

[0058] Therefore, the spectral processing method according to the embodiments of this application further includes: optimizing the configuration parameters based on the acquired spectral image; and storing the optimized configuration parameters.

[0059] Furthermore, the spectral processing device according to the embodiments of this application can be connected to a server, for example, via a network. When an upgrade instruction is received from the server, firmware upgrades can be performed for different spectral chip types.

[0060] Furthermore, the configuration parameters can be adjusted and optimized based on the finally recovered spectral information. That is, the spectral processing device according to this application embodiment can send the recovered spectral information to a server. The server then optimizes the configuration parameters based on the difference between the obtained spectral information and standard data, and then feeds the optimization back to the spectral processing device, thereby optimizing the configuration parameters for different spectral chip types. This is also a learning process; the continuously optimized configuration parameters will better fit the corresponding spectral chip type and more accurately obtain the recovered spectral information.

[0061] Therefore, the spectral recovery method according to the embodiments of this application further includes: sending the recovered spectral information to a server side, wherein the server side optimizes configuration parameters based on the difference between the spectral information and standard data; and receiving feedback information from the server side regarding the optimization of the configuration parameters, and storing the optimized configuration parameters.

[0062] Furthermore, in this embodiment, the optimization of configuration parameters can involve cyclically acquiring the RAW data and optimizing the configuration parameters based on different RAW data acquired cyclically, such as optimizing one or more parameters including exposure, gain, and resolution. Moreover, when optimizing configuration parameters, micro / nano structures on the chip can be selected as needed. That is, since different micro / nano structures have different locations and types, the recoverable spectral information varies, allowing for selection based on requirements.

[0063] Here, during the cyclic acquisition of RAW data, the acquired RAW data can be cached. That is, for the cyclically acquired RAW data, when the acquired RAW data reaches a set data packet value, it can be cached in the cache module. For example, every 10 frames can be used as a set data packet value; each time 10 frames of RAW data are acquired, it is cached. Because this allows for the acquisition of more data, the recovered spectral information can be more accurate.

[0064] In addition, the cached RAW data can be further denoised to increase the accuracy of the data through batch denoising.

[0065] That is, in the spectral recovery method according to the embodiments of this application, optimizing the configuration parameters based on the acquired spectral image includes: cyclically acquiring and caching the spectral image according to a set frame rate; and performing noise reduction processing on the cached spectral image.

[0066] Step S130: Recover the spectral information of the object under test based on the spectral image and the corresponding spectral recovery algorithm. Here, a corresponding spectral recovery algorithm can be matched based on the spectral image; for example, matching can be performed according to the needs of different user application scenarios. Since the acquired spectral image can be used for a lot of data analysis, different spectral recovery algorithms can achieve different analysis results, such as recovery of a single spectral line, recovery of multiple spectral lines, or even recovery of the entire spectral image. Furthermore, spectral recovery algorithms may specifically include, but are not limited to, least squares method, non-negative least squares method, simulated annealing method, Tikhonov regularization method, truncated singular value decomposition method, sparse optimization method, etc.

[0067] Thus, after determining the spectral restoration algorithm, the spectral information of the object under test can be recovered. Here, recovering the spectral information can mean recovering the spectral information of the subject at a specified location, or the spectral information of a single pixel. Furthermore, the recovered spectral information can be sent to other terminals, such as AI devices used for object recognition based on spectral data.

[0068] Furthermore, as described above, when the spectral image is repeatedly acquired, cached, and denoised, the spectral information of the object under test can be recovered based on the denoised spectral image and the corresponding spectral recovery algorithm.

[0069] That is, in the spectral processing method according to the embodiments of this application, recovering the spectral information of the object under test based on the spectral image and the corresponding spectral recovery algorithm includes: recovering the spectral information of the object under test based on the denoised spectral image and the corresponding spectral recovery algorithm.

[0070] Furthermore, as described above, in the spectral chip of this application embodiment, after light is modulated by the modulation unit, the light intensity information is detected by the sensing unit of the image sensor corresponding to the modulation unit. Additionally, multiple different modulation units can be distributed at the edge and center positions of the light modulation layer. Each modulation unit can correspond to one or more sensing units. The modulation units can be located at any edge or center position, and can be continuously or discontinuously distributed; their positions can be arbitrarily selected. Each modulation unit can be an array composed of multiple identical modulation sub-units, or an array composed of multiple different modulation sub-units.

[0071] Each modulation unit and the sensing unit below it constitute a pixel. An algorithm can be used to obtain the intensity distribution of each wavelength at a pixel. The modulation unit of the optical modulation layer is configured to modulate the imaging light entering its corresponding sensing unit, and the corresponding sensing unit is adapted to acquire the spectral information of the imaging light. The non-modulation unit of the optical modulation layer is configured not to modulate the imaging light entering its corresponding sensing unit, and the corresponding sensing unit is adapted to acquire the light intensity information of the imaging light.

[0072] Therefore, in the spectral processing method according to the embodiments of this application, recovering the spectral information of the object under test based on the spectral image and the corresponding spectral recovery algorithm includes: acquiring the spectral information of each modulation unit in the light modulation layer of the spectral chip corresponding to the pixel on the image sensor and the light intensity information of each non-modulation unit in the modulation layer corresponding to the pixel; determining the spectral data of the object based on the spectral information of the pixel corresponding to each modulation unit; and determining the image data of the object based on the light intensity information of the pixel corresponding to each non-modulation unit.

[0073] In this way, the spectral response of the incident light in the entire image can be obtained by using the spectral data of each pixel.

[0074] Furthermore, the recovered spectral information can be packaged and sent to the data processing unit. This data processing unit can interconnect with external interfaces and process data via mobile terminals or PCs, configuring it according to user needs to obtain the required parameter information. The data processing unit can be managed by a software control unit, which can simultaneously bind multiple spectral devices and manage the light sources required for the incident light, including but not limited to adjusting the light source type, color temperature, color, or intensity. It can also analyze and process the acquired spectral information for various applications. For example, it can detect the sweetness of fruits, identify the substances in jade, detect the color temperature of the environment, and adjust the entire spectral image and specific information within it. For instance, it can adjust the image's color, obtain high-fidelity images, and adjust the image's white balance.

[0075] Therefore, the spectral processing method according to the embodiments of this application further includes: sending the recovered spectral information to the client side. It can achieve accurate wavelength measurement within a wide spectral range of 400-1000nm, exhibiting good wavelength repeatability and stability. The raster-free overall design also greatly reduces operational difficulty and the need for repeated calibration of traditional spectral sensors, making spectral sensing and analysis as simple as taking a picture.

[0076] Application Examples

[0077] Figure 3 The illustration shows a schematic diagram of an application example of the spectral processing method according to an embodiment of this application.

[0078] like Figure 3 As shown, the spectral processing method according to the embodiments of this application can be implemented as spectral processing software. After the software starts, it first acquires the information of the spectral chip to determine whether it is a supported spectral chip type. If the chip is supported, it determines the configuration parameters for spectral imaging and starts acquiring the spectral image (i.e., RAW data). On the other hand, it starts a network service to modify the configuration parameters based on the control of a host computer.

[0079] After acquiring the spectral image, i.e., RAW data, the configuration parameters, such as exposure and gain parameters, are adjusted based on the RAW data. Simultaneously, the RAW data is cached and processed with noise reduction and other techniques. Then, the RAW data is spectrally restored to obtain the recovered spectral data, i.e., the spectral information as described above.

[0080] In addition, the recovered spectral data is sent to the host computer via network service so that the host computer can optimize the configuration parameters based on the recovered spectral data to improve the imaging performance of the spectral chip.

[0081] Exemplary spectral processing device

[0082] Figure 4 A schematic block diagram of a spectral processing apparatus according to an embodiment of this application is shown.

[0083] like Figure 4 As shown, the spectral processing apparatus 200 according to an embodiment of this application includes: an acquisition unit 210 for acquiring parameter information of a spectral chip; an acquisition unit 220 for configuring parameters corresponding to the captured spectral image according to the parameter information of the spectral chip and acquiring the spectral image; and a processing unit 230 for recovering the spectral information of the object under test based on the spectral image and the corresponding spectral recovery algorithm.

[0084] In one example, in the spectral processing device 200 described above, the acquisition unit 210 is used to: pre-set parameter information corresponding to different spectral chip types; determine the type of the current spectral chip; and search for and acquire parameter information corresponding to the type of the current spectral chip.

[0085] In one example, in the above-described spectral processing apparatus 200, the acquisition unit 210 searches for and acquires parameter information corresponding to the type of the current spectral chip, which includes: determining whether the type of the current spectral chip matches a preset type; in response to the type of the current spectral chip matching a preset type, acquiring parameter information of the spectral chip of the preset type; and in response to the type of the current spectral chip not matching a preset type, determining that the parameter information of the spectral chip cannot be acquired.

[0086] In one example, in the above-described spectral processing device 200, the acquisition unit 220 is used to: acquire configuration parameters corresponding to the spectral chip, the configuration parameters corresponding to the type of the spectral chip, and the configuration parameters being calibration parameters, wherein the configuration parameters include at least one or a combination of exposure parameters and gain parameters.

[0087] In one example, the spectral processing apparatus 200 described above further includes: an optimization unit for optimizing the configuration parameters based on the acquired spectral image; and a storage unit for storing the optimized configuration parameters.

[0088] In one example, the spectral processing apparatus 200 further includes: a first transmitting unit for transmitting the recovered spectral information to a server side, wherein the server side optimizes configuration parameters based on the difference between the spectral information and standard data; and a receiving unit for receiving feedback information from the server side regarding the optimization of the configuration parameters and storing the optimized configuration parameters.

[0089] In one example, in the above-described spectral processing apparatus 200, the acquisition unit 220 is configured to: cyclically acquire the spectral images and cache them according to a set frame rate; and to perform noise reduction processing on the cached spectral images.

[0090] In one example, in the above-mentioned spectral processing device 200, the processing unit 230 is used to: recover the spectral information of the object under test based on the denoised spectral image and the corresponding spectral recovery algorithm.

[0091] In one example, in the aforementioned spectral processing apparatus 200, the processing unit 230 is configured to: acquire spectral information of pixels on the image sensor corresponding to each modulation unit in the light modulation layer of the spectral chip, and light intensity information of pixels corresponding to each non-modulation unit in the modulation layer; determine spectral data of the subject based on the spectral information of the pixels corresponding to each modulation unit; and determine image data of the subject based on the light intensity information of the pixels corresponding to each non-modulation unit. Figure 5 The diagram shows a schematic of the spectrum display. Users can modify the acquisition parameters as needed to obtain and display the measured spectral lines. The X-axis represents wavelength, and the Y-axis represents relative intensity. The spectral lines are plotted as curves based on real-time spectral data. Moving the mouse displays a dashed "+" sign, showing the specific value at the intersection of the spectral curve and the "+", making it easy to view the value at a specific point on the curve when reviewing historical data. Selecting or dragging the historical data file to be viewed into the dashed box in the pop-up window allows for quick redrawing as a spectral graph for viewing. When reviewing historical data, users can view detailed information such as the measurement acquisition parameters and the spectral sensing module used, facilitating the reconstruction of the test scenario. Users can also set the number of spectral data frames output per batch to acquire multiple frames of spectral data at once. The data can be downloaded to the computer's local hard drive using the default save settings. Alternatively, multiple frames of spectral data can be continuously acquired according to the acquisition configuration.

[0092] In one example, the spectral processing apparatus 200 described above further includes a second transmitting unit for transmitting the recovered spectral information to the client side.

[0093] Here, those skilled in the art will understand that the specific functions and operations of each unit and module in the above-described spectral processing device 200 have been referenced above. Figures 1 to 3 The spectral processing methods described are detailed in the text, and therefore, repeated descriptions will be omitted.

[0094] As described above, the spectral processing device 200 according to the embodiments of this application can be implemented in various terminal devices, such as various spectrometers and spectral imaging devices. In one example, the spectral processing device 200 according to the embodiments of this application can be integrated into the terminal device as a software module and / or a hardware module. For example, the spectral processing device 200 can be a software module in the operating system of the terminal device, or it can be an application developed for the terminal device; of course, the spectral processing device 200 can also be one of many hardware modules of the terminal device.

[0095] Alternatively, in another example, the spectral processing device 200 and the terminal device can be separate devices, and the spectral processing device 200 can connect to the verification device via wired and / or wireless networks, transmitting interactive information according to an agreed data format. For example, a Type-C interface can be configured for power supply and data transmission. It can be easily connected to a PC for access, allowing for quick setup of module configuration and viewing of spectral data output. The spectral processing device provided in this application can be used for spectral analysis, color measurement, and color management in various industries such as light sources, displays, cosmetics, fresh food, materials, and plant lighting. It can also provide AI algorithms and modules for visual perception in spectral imaging, medicine, automotive, and consumer electronics photography. The spectral processing device provided in this application can calculate the chromaticity values ​​and color temperature values ​​corresponding to the standard color spaces CIE1931 Yxz or CIE 1976 L*a*b specified by the International Commission on Illumination (CIE) based on spectral data analysis. The module can also be used to measure color charts or other surfaces with reflective colors. Taking the non-destructive testing of apple freshness as an example, the spectral signal variation characteristics of normal and apples with different degrees of decay in the visible and near-infrared bands can be utilized. Sample spectral data can be collected through transmission or diffuse transmission methods, preprocessed, and trained into a model. Then, a prediction model can be used to predict the freshness of unknown samples, thereby detecting the freshness of the sample to be tested. Figure 6 The image shows a comparison of the transmission spectra of normal and apples with different degrees of decay.

[0096] Exemplary spectral processing system

[0097] Figure 7 The illustration shows a schematic diagram of a spectral processing system including a spectral processing apparatus that applies the spectral recovery method according to an embodiment of the present application.

[0098] like Figure 7 As shown, the spectral processing device includes an optical module, a spectral chip, a processor unit, an external interface, and a host computer.

[0099] The spectral processing device can be used for material analysis based on the acquired spectral information, and can also perform spectral image processing based on the acquired spectral information. The spectral processing device can also be connected to a software control module, such as a server's software control module, which can manage one or more of the spectral data processing systems.

[0100] In the spectral processing device, the optical module is used to acquire the object to be photographed and to acquire the incident light of the object entering the optical module (the obtained spectrum may be a projection spectrum or a reflection spectrum), and transmits the acquired incident light to the spectral chip for processing. The spectral chip includes a light modulation layer disposed on the image sensor layer as described above. Each modulation unit in the light modulation layer has a different modulation effect on light of different wavelengths. The modulation methods of the input spectrum by the various modulation units can be the same or different. Different modulation methods may include, but are not limited to, scattering, absorption, transmission, reflection, interference, excitation, resonance enhancement, etc. The final effect of the modulation effect is that the transmission spectrum of light of different wavelengths is different after passing through the modulation unit.

[0101] After light is modulated by the modulation unit, the light intensity information is detected by the sensing unit of the corresponding image sensor layer below the modulation unit. Multiple different modulation units can be distributed at the edges and center of the light modulation layer, with each modulation unit corresponding to one or more sensing units. The modulation units can be located at any edge or center position, and can be continuously or discontinuously distributed; their positions can be arbitrarily chosen. Each modulation unit can be an array of multiple identical modulation sub-units or an array of multiple different modulation sub-units. Each modulation unit and the sensing unit below it constitute a pixel. Algorithms can be used to obtain the intensity distribution of each wavelength at a pixel, thereby acquiring the spectral response of the incident light for the entire image.

[0102] The spectral chip can package and send the acquired spectral images to the processor unit. The processor unit can interconnect with an external interface, connecting to a host computer. For example, the host computer stores various recovery algorithms and includes various data interfaces. Furthermore, as mentioned above, the processor unit can be configured to obtain the required parameter information according to user needs. The processor unit can also be managed by a software control unit, which can simultaneously bind multiple spectral processing devices and manage the light source required for the incident light, including but not limited to adjusting information such as light source type, color temperature, color, or intensity. It can also analyze and process the acquired spectral information for various applications. For example, it can detect the sweetness of fruits, identify the substance of jade, detect the color temperature of the environment, and adjust the entire spectral image and specific information within the image. For example, it can adjust the image color, obtain high-fidelity images, and adjust the image's white balance.

[0103] like Figure 7 As shown, according to their functions, the processor unit may include: an exposure control module, a gain control module, a spectral recovery module, a correction module, an encryption algorithm module, an upgrade control module, a configuration management module, a network server module, a user management module, a log management module, an interface management module, a status detection module, a power management module, a storage management module, a cache module, and a data processing module. The functions of these modules have already been described in the spectral recovery method according to the embodiments of this application, and will not be repeated here.

[0104] In addition, such as Figure 7 As shown, the spectral processing apparatus according to an embodiment of this application may further include a clock reset module for clocking the spectral chip and the processor unit, and a power supply module for supplying power to each module.

[0105] Therefore, the spectral processing apparatus according to the embodiments of this application includes an optical module, a spectral chip, a processor unit, an external interface, and a host computer.

[0106] Furthermore, the processor unit includes an exposure control module, a gain control module, a spectral recovery module, a correction module, an encryption algorithm module, an upgrade control module, a configuration management module, a network server module, a user management module, a log management module, an interface management module, a status detection module, a power management module, a storage management module, a cache module, and a data processing module.

[0107] In addition, the spectral processing device further includes a clock reset module and a power supply module.

[0108] Furthermore, as described above, the spectral processing system according to the embodiments of this application may further include a server and a client, wherein the spectral processing device is connected to the client to send the acquired spectral information to the client, and the spectral processing device is connected to the server to download the firmware required to update the spectral processing device from the server.

[0109] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0110] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0111] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0112] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0113] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A spectral processing method, characterized in that, include: Obtain parameter information of the spectral chip; Configure the parameters corresponding to the captured spectral image based on the parameter information of the spectral chip, and acquire the spectral image; as well as The spectral information of the object under test is recovered based on the spectral image and the corresponding spectral recovery algorithm. The spectral chip includes a filter structure and an image sensor; the filter structure is a broadband filter structure in the frequency domain or wavelength domain. The method further includes: The configuration parameters of the spectral chip are optimized based on the acquired spectral image; and the optimized configuration parameters are stored; or, The recovered spectral information is sent to the server side, which optimizes the configuration parameters based on the difference between the spectral information and the standard data; and the server side receives feedback information from the server side regarding the optimization of the configuration parameters and stores the optimized configuration parameters.

2. The spectral processing method as described in claim 1, wherein, Obtaining parameter information for the spectral chip includes: Pre-set parameter information for different spectral chip types; Determine the type of the current spectral chip; and Find and obtain the parameter information corresponding to the type of the current spectral chip.

3. The spectral processing method as described in claim 2, wherein, Finding and obtaining parameter information corresponding to the type of the current spectral chip includes: Determine whether the current spectral chip type matches the preset type; In response to the current spectral chip type matching a preset type, parameter information of the preset type of spectral chip is obtained; and In response to the mismatch between the current spectral chip type and the preset type, it is determined that the parameter information of the spectral chip cannot be obtained.

4. The spectral processing method as described in claim 1, wherein, Configure the parameters corresponding to the captured spectral image based on the parameter information of the spectral chip, and acquire the spectral image including: Obtain the configuration parameters corresponding to the spectral chip, wherein the configuration parameters correspond to the type of the spectral chip and are calibration parameters, wherein the configuration parameters include at least one or a combination of exposure parameters and gain parameters.

5. The spectral processing method as described in claim 1, wherein, Optimizing the configuration parameters based on the acquired spectral image includes: The spectral images are acquired and cached cyclically according to a set frame rate; and The cached spectral image is subjected to noise reduction processing.

6. The spectral processing method as described in claim 5, wherein, Recovering the spectral information of the object under test based on the spectral image and the corresponding spectral reconstruction algorithm includes: The spectral information of the object under test is recovered based on the denoised spectral image and the corresponding spectral recovery algorithm.

7. The spectral processing method as described in claim 1, wherein, Recovering the spectral information of the object under test based on the spectral image and the corresponding spectral reconstruction algorithm includes: Acquire the spectral information of each modulation unit in the optical modulation layer of the spectral chip corresponding to the pixel point on the image sensor, and the light intensity information of each non-modulation unit in the modulation layer corresponding to the pixel point; Based on the spectral information of the pixel corresponding to each modulation unit, the spectral data of the photographed object is determined; and, The image data of the photographed object is determined based on the light intensity information of the pixel corresponding to each of the non-modulation units.

8. The spectral processing method as described in claim 1, further comprising: The recovered spectral information is sent to the client side.

9. A spectral processing device, characterized in that, Includes an apparatus that applies the spectral recovery method as described in any one of claims 1 to 8.

10. A spectral processing system, characterized in that, The spectral processing device includes a spectral recovery method according to any one of claims 1 to 8, the spectral processing system further includes a server and a client, the spectral processing device is connected to the client to send the acquired spectral information to the client, and the spectral processing device is connected to the server to download and update the firmware required for the spectral processing device from the server.

Images