Spectral image processing method, device, equipment and storage medium

By simultaneously calibrating the spectral device and the compensation device under the preset incident brightness and combining the image brightness with the calibration data, the problem of inaccurate brightness in the prior art is solved, and the accuracy of the spectral image and the reliability of the state information are improved.

CN114494094BActive Publication Date: 2024-11-15GUANGZHOU XAIRCRAFT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210054668.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-18
Publication Date
2024-11-15
Estimated Expiration
2042-01-18

AI Technical Summary

Technical Problem

In the prior art, the compensation device and the spectral device are inaccurate due to independent calibration when fusing brightness, resulting in inaccuracy after fusing brightness.

Method used

By simultaneously calibrating the spectral device and the compensation device under the preset incident brightness, the calibration data of both are acquired and combined, the collected image brightness is fused to determine the status information of the target object.

Benefits of technology

The fusion brightness accuracy of the spectral image is improved, making the state information of the target object more accurate and reliable.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114494094B_ABST
    Figure CN114494094B_ABST
Patent Text Reader

Abstract

The present application provides a spectral image processing method, device, equipment and storage medium, wherein the method comprises: obtaining a first image brightness of a first object spectral image for a target object collected by a spectral device and a second image brightness of a second object spectral image for a target object collected by a compensation device, fusing the first image brightness and the second image brightness according to first calibration data of the spectral device and second calibration data of the compensation device, and obtaining a fused brightness of the first object spectral image and the second object spectral image, wherein the first calibration data is used to characterize the photosensitivity of the spectral device under a preset incident brightness, and the second calibration data is used to characterize the photosensitivity of the compensation device under a preset incident brightness, and determining the state information of the target object according to the fused brightness. The accuracy of the fused brightness is improved, thereby making the state information of the target object more accurate and reliable.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a spectral image processing method, device, equipment and storage medium. Background Art

[0002] In the agricultural field, reflectance is generally used to assess the health of crops, and reflectance is determined by the brightness of the crop spectral image.

[0003] In the related technology, in hyperspectral equipment and multispectral equipment, due to the influence of weather and light, in order to improve the accuracy of spectral image brightness, the images collected by the compensation device and the spectral device are usually brightness fused to obtain the true brightness of the crop spectral image. Therefore, the calibration of the compensation device is particularly critical. At present, the compensation device is usually calibrated independently.

[0004] However, since the compensation device is only calibrated independently, when the compensation device is used in conjunction with the spectral device, the fused brightness is not accurate enough. Summary of the invention

[0005] The purpose of the present application is to provide a spectral image processing method, device, equipment and storage medium to improve the accuracy of the fusion brightness of the spectral image in order to address the deficiencies in the above-mentioned prior art.

[0006] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0007] In a first aspect, an embodiment of the present application provides a spectral image processing method, comprising:

[0008] Acquire a first image brightness of a first object spectral image for a target object collected by a spectral device and a second image brightness of a second object spectral image for the target object collected by a compensation device;

[0009] According to the first calibration data of the spectral device and the second calibration data of the compensation device, the brightness of the first image and the brightness of the second image are fused to obtain the fused brightness of the first object spectral image and the second object spectral image, wherein the first calibration data is used to characterize the photosensitivity performance of the spectral device under a preset incident brightness, and the second calibration data is used to characterize the photosensitivity performance of the compensation device under the preset incident brightness;

[0010] The state information of the target object is determined according to the fused brightness.

[0011] Optionally, the fusing the first image brightness and the second image brightness according to the first calibration data of the spectral device and the second calibration data of the compensation device to obtain the fused brightness of the spectral image of the object includes:

[0012] The first image brightness and the second image brightness are fused according to a ratio of the first calibration data to the second calibration data to obtain the fused brightness.

[0013] Optionally, the first calibration data includes: first shutter time calibration data, first gain calibration data and first incident brightness calibration data, and the second calibration data includes: second shutter time calibration data, second gain calibration data and second incident brightness calibration data;

[0014] The step of fusing the first image brightness and the second image brightness according to the ratio of the first calibration data to the second calibration data to obtain the fused brightness includes:

[0015] Determining a first ratio of the first shutter time calibration data to the second shutter time calibration data;

[0016] determining a second ratio of the first gain calibration data to the second gain calibration data;

[0017] Determining a third ratio of the second incident brightness calibration data to the first incident brightness calibration data;

[0018] The first image brightness and the second image brightness are fused according to the product of the first ratio, the second ratio and the third ratio to obtain the fused brightness.

[0019] Optionally, the fusing the first image brightness and the second image brightness according to the product of the first ratio, the second ratio and the third ratio to obtain the fused brightness includes:

[0020] determining a fourth ratio of the brightness of the second image to the brightness of the first image;

[0021] The product of the first ratio, the second ratio and the third ratio and the product of the fourth ratio are used as the fusion brightness.

[0022] Optionally, before fusing the first image brightness and the second image brightness according to the first calibration data of the spectral device and the second calibration data of the compensation device to obtain the fused brightness of the first object spectral image and the second object spectral image, the method further includes:

[0023] Determining first calibration data of the spectral device according to state change information of the spectral device under preset conditions;

[0024] According to the state change information of the compensation device under the preset condition, second calibration data of the compensation device is determined.

[0025] Optionally, determining first calibration data of the spectral device according to state change information of the spectral device under preset conditions includes:

[0026] Obtaining display brightness of the spectral device corresponding to multiple shutter times under a preset incident brightness;

[0027] generating a first relationship curve between shutter time and display brightness according to the plurality of shutter times and the display brightness corresponding to the plurality of shutter times;

[0028] According to the first relationship curve, first shutter time calibration data in the first calibration data is determined.

[0029] Optionally, generating a first relationship curve between shutter time and display brightness according to the multiple shutter times and the display brightness corresponding to the multiple shutter times includes:

[0030] The first relationship curve is obtained by fitting according to the multiple shutter times and the display brightness corresponding to the multiple shutter times.

[0031] Optionally, determining the first shutter time calibration data in the first calibration data according to the first relationship curve includes:

[0032] The slope of the first relationship curve is determined as the first shutter time calibration data.

[0033] Optionally, determining first calibration data of the spectral device according to state change information of the spectral device under preset conditions includes:

[0034] Obtaining display brightness corresponding to multiple gains of the spectral device under a preset incident brightness;

[0035] generating a second relationship curve between gain and display brightness according to the multiple gains and the display brightness corresponding to the multiple gains;

[0036] According to the second relationship curve, first gain calibration data in the first calibration data is determined.

[0037] Optionally, determining first calibration data of the spectral device according to state change information of the spectral device under preset conditions includes:

[0038] Obtaining display brightness of the spectral device corresponding to multiple incident brightness under a preset shutter time, a preset gain, and a preset aperture;

[0039] generating a third relationship curve between incident brightness and display brightness according to the multiple incident brightnesses and display brightnesses corresponding to the multiple incident brightnesses;

[0040] According to the third relationship curve, first incident brightness calibration data in the first calibration data is determined.

[0041] In a second aspect, another embodiment of the present application provides a spectral image processing device, comprising:

[0042] An acquisition module, configured to acquire a first image brightness of a first object spectral image for a target object acquired by a spectral device and a second image brightness of a second object spectral image for the target object acquired by a compensation device;

[0043] a processing module, configured to fuse the brightness of the first image and the brightness of the second image according to first calibration data of the spectral device and second calibration data of the compensation device, so as to obtain a fused brightness of the first object spectral image and the second object spectral image, wherein the first calibration data is used to characterize the photosensitivity performance of the spectral device under a preset incident brightness, and the second calibration data is used to characterize the photosensitivity performance of the compensation device under the preset incident brightness;

[0044] A determination module is used to determine the state information of the target object according to the fused brightness.

[0045] Optionally, the processing module is specifically used to:

[0046] The first image brightness and the second image brightness are fused according to a ratio of the first calibration data to the second calibration data to obtain the fused brightness.

[0047] Optionally, the first calibration data includes: first shutter time calibration data, first gain calibration data and first incident brightness calibration data, and the second calibration data includes: second shutter time calibration data, second gain calibration data and second incident brightness calibration data;

[0048] The processing module is specifically used for:

[0049] Determining a first ratio of the first shutter time calibration data to the second shutter time calibration data;

[0050] determining a second ratio of the first gain calibration data to the second gain calibration data;

[0051] Determining a third ratio of the second incident brightness calibration data to the first incident brightness calibration data;

[0052] The first image brightness and the second image brightness are fused according to the product of the first ratio, the second ratio and the third ratio to obtain the fused brightness.

[0053] Optionally, the processing module is specifically used to:

[0054] determining a fourth ratio of the brightness of the second image to the brightness of the first image;

[0055] The product of the first ratio, the second ratio and the third ratio and the product of the fourth ratio are used as the fusion brightness.

[0056] Optionally, the determining module is further used to:

[0057] Determining first calibration data of the spectral device according to state change information of the spectral device under preset conditions;

[0058] According to the state change information of the compensation device under the preset condition, second calibration data of the compensation device is determined.

[0059] Optionally, the determining module is specifically configured to:

[0060] Obtaining display brightness of the spectral device corresponding to multiple shutter times under a preset incident brightness;

[0061] generating a first relationship curve between shutter time and display brightness according to the plurality of shutter times and the display brightness corresponding to the plurality of shutter times;

[0062] According to the first relationship curve, first shutter time calibration data in the first calibration data is determined.

[0063] Optionally, the determining module is specifically configured to:

[0064] The first relationship curve is obtained by fitting according to the multiple shutter times and the display brightness corresponding to the multiple shutter times.

[0065] Optionally, the determining module is specifically configured to:

[0066] The slope of the first relationship curve is determined as the first shutter time calibration data.

[0067] Optionally, the determining module is specifically configured to:

[0068] Obtaining display brightness corresponding to multiple gains of the spectral device under a preset incident brightness;

[0069] generating a second relationship curve between gain and display brightness according to the multiple gains and the display brightness corresponding to the multiple gains;

[0070] According to the second relationship curve, first gain calibration data in the first calibration data is determined.

[0071] Optionally, the determining module is specifically configured to:

[0072] Obtaining display brightness of the spectral device corresponding to multiple incident brightness under a preset shutter time, a preset gain, and a preset aperture;

[0073] generating a third relationship curve between incident brightness and display brightness according to the multiple incident brightnesses and display brightnesses corresponding to the multiple incident brightnesses;

[0074] According to the third relationship curve, first incident brightness calibration data in the first calibration data is determined.

[0075] In the third aspect, another embodiment of the present application provides a spectral image processing device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, when the spectral image device is running, the processor and the memory communicate through the bus, and the processor executes the machine-readable instructions to execute any one of the methods described in the first aspect.

[0076] In a fourth aspect, another embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method described in any one of the first aspects is executed.

[0077] The beneficial effects of this application are:

[0078] The present application provides a spectral image processing method, device, equipment and storage medium, wherein the method includes: obtaining the brightness of a first image of a first object spectral image collected by a spectral device for a target object and the brightness of a second image of a second object spectral image collected by a compensation device for the target object, fusing the brightness of the first image and the brightness of the second image according to the first calibration data of the spectral device and the second calibration data of the compensation device to obtain the fused brightness of the first object spectral image and the second object spectral image, wherein the first calibration data is used to characterize the photosensitivity of the spectral device under a preset incident brightness, and the second calibration data is used to characterize the photosensitivity of the compensation device under a preset incident brightness, and determining the state information of the target object according to the fused brightness. By combining the photosensitivity of the spectral device and the compensation device under a preset incident brightness to determine the fused brightness, the accuracy of the fused brightness is improved, thereby making the state information of the target object more accurate and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0080] Figure 1 Schematic diagram of the process of the spectral image processing method provided in the embodiment of the present application Figure 1 ;

[0081] Figure 2 Schematic diagram of the process of the spectral image processing method provided in the embodiment of the present application Figure 2 ;

[0082] Figure 3 Schematic diagram of the process of the spectral image processing method provided in the embodiment of the present application Figure 3 ;

[0083] Figure 4 Schematic diagram of the process of the spectral image processing method provided in the embodiment of the present application Figure 4 ;

[0084] Figure 5 Schematic diagram of the process of the spectral image processing method provided in the embodiment of the present application Figure 5 ;

[0085] Figure 6 Schematic diagram of the process of the spectral image processing method provided in the embodiment of the present application Figure 6 ;

[0086] Figure 7 A schematic diagram of the structure of a spectral image processing device provided in an embodiment of the present application;

[0087] Figure 8 A schematic diagram of the structure of a spectral image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0088] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0089] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0090] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0091] The calibration method of existing compensation equipment is generally to calibrate itself independently. However, when used in conjunction with a spectral device, due to the differences between the devices, such as differences in their own processes and models, the consistency of the devices cannot be ensured, resulting in the fused brightness being not accurate enough when brightness fusion is performed based on the spectral images collected by the compensation device and the spectral device. Based on this, the present application calibrates the spectral device and the compensation device at the same time under a preset incident brightness to maintain the consistency of the spectral device and the compensation device, and uses the first calibration data of the spectral device and the second calibration data of the compensation device to perform brightness fusion, thereby improving the accuracy of the fused brightness.

[0092] The technical solution of the spectral image processing method provided by the present application is described in detail below in conjunction with several specific embodiments.

[0093] Figure 1 Schematic diagram of the process of the spectral image processing method provided in the embodiment of the present application Figure 1 The execution subject of this embodiment may be a spectral image processing device, for example, a laptop computer, a desktop computer, a tablet computer, or other electronic device with data processing capabilities.

[0094] like Figure 1 As shown, the method includes:

[0095] S101 : Acquire a first image brightness of a first object spectral image for a target object collected by a spectral device and a second image brightness of a second object spectral image for the target object collected by a compensation device.

[0096] The compensation device may be a device that performs brightness compensation on the spectral image collected by the spectral device, and the target object may be a monitoring object whose state is to be determined, such as crops whose reflectivity is to be determined.

[0097] The spectral device can collect a first object spectral image for the target object, and the compensation device can collect a second object spectral image for the target object, wherein both the spectral device and the compensation device can have built-in brightness sensors, which are respectively used to detect the first image brightness of the first object spectral image and the second image brightness of the second object spectral image, and upload the first image brightness and the second image brightness to the spectral image processing device, respectively, or store the first image brightness and the second image brightness in the spectral image processing device, so that the spectral image processing device can obtain the first image brightness and the second image brightness.

[0098] S102 . According to first calibration data of the spectral device and second calibration data of the compensation device, fuse the brightness of the first image and the brightness of the second image to obtain a fused brightness of the first object spectral image and the second object spectral image.

[0099] Among them, the first calibration data is used to characterize the photosensitivity performance of the spectral device under a preset incident brightness, and the second calibration data is used to characterize the photosensitivity performance of the compensation device under a preset incident brightness. That is to say, the calibration environment of the first calibration data and the second calibration data is the same, that is, the first calibration data and the second calibration data are obtained by calibrating the spectral device and the compensation device at the same time.

[0100] According to the first calibration data of the spectral device and the second calibration data of the compensation device, the brightness of the first image and the brightness of the second image are fused to obtain the fused brightness of the first object spectral image and the second object spectral image, that is, the first calibration data and the second calibration data for calibrating the spectral device and the compensation device at the same time are taken as factors, and the brightness of the first image and the brightness of the second image are fused to obtain the fused brightness of the first object spectral image and the second object spectral image. For example, the first calibration data can be used as a fusion coefficient of the brightness of the second image to determine the fusion value of the brightness of the second image, and the second calibration data can be used as a fusion coefficient of the brightness of the first image to determine the fusion value of the brightness of the first image, and then the fusion value of the brightness of the second image and the fusion value of the brightness of the first image are weighted averaged to obtain the fusion brightness.

[0101] S103: Determine the state information of the target object according to the fused brightness.

[0102] Among them, the target object can be a monitoring object whose state is to be determined, such as crops whose reflectivity is to be determined, and the state information can be the reflectivity. That is to say, the first object spectral image and the second object spectral image both include the target object, and the fused brightness of the first object spectral image and the second object spectral image can be the fused brightness of the target object in the first object spectral image and the second object spectral image. The reflectivity of the target object can be determined based on the fused brightness, and thus the health of the target object can be evaluated based on the reflectivity.

[0103] It should be noted that the fused brightness of the target object in the first object spectral image and the second object spectral image and the state information of the target object may have a certain mapping relationship. After the fused brightness is determined, the state information of the target object may be determined according to the mapping relationship.

[0104] In the spectral image processing method of this embodiment, the brightness of the first image and the brightness of the second image are fused according to the first calibration data of the spectral device and the second calibration data of the compensation device to obtain the fused brightness of the first object spectral image and the second object spectral image, and the state information of the target object is determined according to the fused brightness. By combining the photosensitivity of the spectral device and the compensation device under the preset incident brightness to determine the fused brightness, the accuracy of the fused brightness is improved, thereby making the state information of the target object more accurate and reliable.

[0105] Figure 2 Schematic diagram of the process of the spectral image processing method provided in the embodiment of the present application Figure 2 ,like Figure 2 As shown, step S102, according to the first calibration data of the spectral device and the second calibration data of the compensation device, the brightness of the first image and the brightness of the second image are fused to obtain the fused brightness of the spectral image of the object, including:

[0106] S201. According to a ratio of first calibration data to second calibration data, fuse the brightness of a first image and the brightness of a second image to obtain a fused brightness.

[0107] Determine the ratio of the first calibration data to the second calibration data, use the ratio as a fusion coefficient, fuse the first image brightness and the second image brightness to obtain a fused brightness. For example, use the ratio of the first calibration data to the second calibration data as a fusion coefficient of the ratio of the second image brightness to the first image brightness, and fuse to obtain the fused brightness.

[0108] Optionally, the first calibration data includes: first shutter time calibration data, first gain calibration data and first incident brightness calibration data, and the second calibration data includes: second shutter time calibration data, second gain calibration data and second incident brightness calibration data.

[0109] Accordingly, step S201 includes:

[0110] A first ratio of the first shutter time calibration data to the second shutter time calibration data is determined.

[0111] A second ratio of the first gain calibration data to the second gain calibration data is determined.

[0112] A third ratio of the second incident brightness calibration data to the first incident brightness calibration data is determined.

[0113] The brightness of the first image and the brightness of the second image are fused according to the product of the first ratio, the second ratio and the third ratio to obtain fused brightness.

[0114] Among them, the first shutter time calibration data is the calibration data of the shutter time of the spectrum device, the first gain calibration data is the calibration data of the gain of the spectrum device, and the first incident brightness calibration data is the calibration data of the incident brightness of the spectrum device. Similarly, the second shutter time calibration data is the calibration data of the shutter time of the compensation device, the second gain calibration data is the calibration data of the gain of the compensation device, and the second incident brightness calibration data is the calibration data of the incident brightness of the compensation device.

[0115] In some embodiments, a first ratio of first shutter time calibration data to second shutter time calibration data is determined, a second ratio of first gain calibration data to second gain calibration data is determined, a third ratio of second incident brightness calibration data to first incident brightness calibration data is determined, and then the product of the first ratio, the second ratio and the third ratio is determined. According to the product of the first ratio, the second ratio and the third ratio, the brightness of the first image and the brightness of the second image are fused to obtain fused brightness. That is, the product of the first ratio, the second ratio and the third ratio is used as a fusion coefficient, and the brightness of the first image and the brightness of the second image are fused to obtain fused brightness.

[0116] Optionally, a fourth ratio of the brightness of the second image to the brightness of the first image may be determined first, and the product of the first ratio, the second ratio and the third ratio and the product of the fourth ratio are used as the fusion brightness.

[0117] Determine a fourth ratio of the brightness of the second image to the brightness of the first image, and use the product of the first ratio, the second ratio and the third ratio as a fusion coefficient of the fourth ratio, that is, use the product of the first ratio, the second ratio and the third ratio and the product of the fourth ratio as the fusion brightness.

[0118] As shown in the following formula:

[0119]

[0120] Among them, x is the brightness of the second image, x1 is the brightness of the first image, is the second incident brightness calibration data, p1 is the first incident brightness calibration data, g1 is the first gain calibration data, g is the second gain calibration data, t1 is the first shutter time calibration data, and t is the second shutter time calibration data.

[0121] In this way, the corresponding relationship between the spectrum device and the compensation device can be established, so as to link the brightness of the first image with the brightness of the second image, which is convenient for brightness fusion.

[0122] It should be noted that the first shutter time calibration data and the second shutter time calibration data can be obtained in the following manner: fix the incident brightness, fix the device gain and the aperture size, adjust the shutter time of the spectral device and the compensation device respectively, to obtain a relationship curve between the shutter time and the device display brightness, and then fit the first shutter time and the second shutter time based on the relationship curve.

[0123] The first gain calibration data and the second gain calibration data can be obtained in the following manner: fixing the incident brightness, fixing the device aperture size and shutter time, adjusting the gains of the spectral device and the compensation device respectively to obtain a relationship curve between the gain and the device display brightness, and then fitting the first gain calibration data and the second gain calibration data based on the relationship curve.

[0124] The first incident brightness calibration data and the second incident brightness calibration data can be obtained in the following manner: fix the device shutter time, gain and aperture size, adjust the incident brightness to obtain a relationship curve between the incident brightness and the device display brightness, and then fit the first incident brightness calibration data and the second incident brightness calibration data based on the relationship curve.

[0125] Figure 3 Schematic diagram of the process of the spectral image processing method provided in the embodiment of the present application Figure 3 ,like Figure 3 As shown, in step S102, before fusing the brightness of the first image and the brightness of the second image according to the first calibration data of the spectral device and the second calibration data of the compensation device to obtain the fused brightness of the first object spectral image and the second object spectral image, the method further includes:

[0126] S301 : Determine first calibration data of a spectral device according to state change information of the spectral device under preset conditions.

[0127] S302: Determine second calibration data of the compensation device according to state change information of the compensation device under preset conditions.

[0128] Among them, the preset condition can be a preset calibration condition, that is, under the same calibration condition, the state change information of the spectral device and the state change information of the compensation device are determined, and then the first calibration data of the spectral device is determined according to the state change information of the spectral device, and the second calibration data of the compensation device is determined according to the state change information of the compensation device, thereby realizing the simultaneous calibration of the spectral device and the compensation device, so that when the brightness fusion is performed subsequently, the accuracy of the fused brightness can be improved.

[0129] Figure 4 Schematic diagram of the process of the spectral image processing method provided in the embodiment of the present application Figure 4 ,like Figure 4 As shown, step S301, determining first calibration data of the spectral device according to state change information of the spectral device under preset conditions, includes:

[0130] S401, obtaining display brightness of a spectral device corresponding to multiple shutter times under a preset incident brightness.

[0131] S402: Generate a first relationship curve between shutter time and display brightness according to a plurality of shutter times and display brightness corresponding to the plurality of shutter times.

[0132] S403: Determine first shutter time calibration data in the first calibration data according to the first relationship curve.

[0133] Wherein, under the preset incident brightness, each shutter time corresponds to a display brightness, and the display brightness is the brightness on the display screen of the spectral device.

[0134] The shutter time is adjusted multiple times to determine the corresponding display brightness, and then a first relationship curve between the shutter time and the display brightness is generated according to the multiple shutter times and the display brightness corresponding to the multiple shutter times. Then, the first shutter time calibration data is determined according to the first relationship curve, wherein the first calibration data includes the first shutter time calibration data, and the first shutter time calibration data is used to calibrate the shutter time of the spectral device.

[0135] It should be noted that, usually, the incident brightness and shutter time are inversely proportional, as shown below:

[0136]

[0137] Among them, l is the incident brightness and t is the shutter time.

[0138] Optionally, step S402, generating a first relationship curve between shutter time and display brightness according to a plurality of shutter times and display brightness corresponding to the plurality of shutter times, includes:

[0139] A first relationship curve is obtained by fitting according to the multiple shutter times and the display brightness corresponding to the multiple shutter times.

[0140] By adjusting the shutter time, the corresponding display brightness is obtained, and then a first relationship curve is fitted according to multiple shutter times and the display brightness corresponding to the multiple shutter times. The abscissa of the first relationship curve can be the shutter time, and the ordinate can be the display brightness.

[0141] Optionally, step S403, determining first shutter time calibration data in the first calibration data according to the first relationship curve, includes:

[0142] The slope of the first relationship curve is determined as the first shutter time calibration data.

[0143] The first calibration data includes first shutter time calibration data. Based on the first relationship curve, the slope of the first relationship curve can be fitted according to the least squares method, and then the slope of the first relationship curve is used as the first shutter time calibration data, wherein the slope of the first relationship curve is used to indicate the relative relationship between the shutter time and the display brightness, that is, the photosensitivity of the spectral device based on the shutter time is taken into consideration to obtain the first shutter time calibration data.

[0144] Correspondingly, step S302 determines the second calibration data of the compensation device based on the state change information of the compensation device under preset conditions, including: obtaining the display brightness of the compensation device corresponding to multiple shutter times under preset incident brightness, generating a relationship curve between shutter time and display brightness based on multiple shutter times and the display brightness corresponding to the multiple shutter times, and then determining the second shutter time calibration data in the second calibration data based on the relationship curve between shutter time and display brightness.

[0145] The above specific implementation process is similar to steps S401-S403 and will not be repeated here.

[0146] In practical applications, the preset incident brightness can be achieved through an integrating sphere, that is, the brightness of the integrating sphere is controlled to maintain a fixed brightness, the output port of the integrating sphere is simultaneously aligned with the display screens of the spectral device and the compensation device, and then the gain and aperture size of the spectral device and the compensation device are fixed, wherein the gain and the aperture size are the same for the spectral device and the compensation device, and the shutter time of the spectral device is adjusted so that the brightness on the display screen of the spectral device changes from dark to bright. When the picture brightness reaches the brightest, a first relationship curve between the shutter time and the display brightness is fitted, and the slope of the first relationship curve is fitted according to the least squares method, and the slope is used as the first shutter time calibration data.

[0147] Similarly, the shutter time of the compensation device is adjusted so that the brightness on the display screen of the compensation device changes from dark to bright. When the screen brightness reaches the brightest, a relationship curve between the shutter time and the display brightness is fitted. The slope of the relationship curve can be fitted according to the least squares method, and the slope is used as the second shutter time calibration data.

[0148] In the spectral image processing method of this embodiment, the display brightness corresponding to multiple shutter times of the spectral device under the preset incident brightness is obtained, and a first relationship curve between the shutter time and the display brightness is generated according to the multiple shutter times and the display brightness corresponding to the multiple shutter times. According to the first relationship curve, the first shutter time calibration data in the first calibration data is determined. In this way, the photosensitivity performance of the spectral device based on the shutter time is taken as a factor to obtain the first incident brightness calibration data.

[0149] Figure 5 Schematic diagram of the process of the spectral image processing method provided in the embodiment of the present application Figure 5 ,like Figure 5 As shown, step S301, determining first calibration data of the spectral device according to state change information of the spectral device under preset conditions, includes:

[0150] S501: Obtain display brightness corresponding to multiple gains of a spectral device under a preset incident brightness.

[0151] S502: Generate a second relationship curve between gain and display brightness according to the multiple gains and the display brightness corresponding to the multiple gains.

[0152] S503: Determine first gain calibration data in the first calibration data according to the second relationship curve.

[0153] Wherein, under a preset incident brightness, each gain corresponds to a display brightness, and the display brightness is the brightness on the display screen of the spectral device.

[0154] The gain is adjusted multiple times to determine the corresponding display brightness, and then a second relationship curve between the gain and the display brightness is generated based on the multiple gains and the display brightness corresponding to the multiple gains. Then, the first gain calibration data is determined based on the second relationship curve, wherein the first calibration data includes first gain calibration data, and the first gain calibration data is used to calibrate the gain of the spectral device.

[0155] It should be noted that, usually, the incident brightness and gain are inversely proportional, as shown below:

[0156]

[0157] Among them, l is the incident brightness and g is the shutter time.

[0158] Optionally, step S502, generating a second relationship curve between gain and display brightness according to the multiple gains and the display brightness corresponding to the multiple gains, includes:

[0159] A second relationship curve is obtained by fitting according to the multiple gains and the display brightness corresponding to the multiple gains.

[0160] The corresponding display brightness is obtained by adjusting the gain, and then a second relationship curve is obtained by fitting according to multiple gains and the display brightness corresponding to the multiple gains. The abscissa of the second relationship curve may be the gain, and the ordinate may be the display brightness.

[0161] Optionally, step S503, determining first gain calibration data in the first calibration data according to the second relationship curve, includes:

[0162] The slope of the second relationship curve is determined as the first gain calibration data.

[0163] The first calibration data includes first gain calibration data. Based on the second relationship curve, the slope of the second relationship curve can be fitted according to the least squares method, and then the slope of the second relationship curve is used as the first gain calibration data, wherein the slope of the second relationship curve is used to indicate the relative relationship between gain and display brightness, that is, the photosensitivity of the spectral device based on gain is taken into consideration to obtain the first gain calibration data.

[0164] Correspondingly, step S302 determines the second calibration data of the compensation device based on the state change information of the compensation device under preset conditions, including: obtaining the display brightness corresponding to multiple gains of the compensation device under a preset incident brightness, generating a relationship curve between gain and display brightness based on the multiple gains and the display brightness corresponding to the multiple gains, and determining the second gain calibration data in the second calibration data based on the relationship curve between gain and display brightness.

[0165] The above specific implementation process is similar to steps S501-S503 and will not be repeated here.

[0166] In practical applications, the preset incident brightness can be achieved through an integrating sphere, that is, the brightness of the integrating sphere is controlled to maintain a fixed brightness, the output port of the integrating sphere is simultaneously aligned with the spectral device and the compensation device, and then the shutter time and aperture size of the spectral device and the compensation device are fixed, wherein the shutter time and the aperture size are the same for the spectral device and the compensation device, and the gain of the spectral device is adjusted to make the brightness on the display screen of the spectral device change from dark to bright. When the screen brightness reaches the brightest, a second relationship curve between the gain and the display brightness is fitted, and the slope of the second relationship curve is fitted according to the least squares method, and the slope is used as the second gain calibration data.

[0167] Similarly, the gain of the compensation device is adjusted so that the brightness on the display screen of the compensation device changes from dark to bright. When the picture reaches the brightest, a relationship curve between the gain and the display brightness is fitted. The slope of the relationship curve can be fitted according to the least squares method, and the slope is used as the second gain calibration data.

[0168] In the spectral image processing method of this embodiment, the display brightness corresponding to multiple gains of the spectral device under a preset incident brightness is obtained, and a second relationship curve between the gain and the display brightness is generated according to the multiple gains and the display brightness corresponding to the multiple gains, and the first gain calibration data in the first calibration data is determined according to the second relationship curve. In this way, the photosensitivity of the spectral device based on the gain is taken into consideration to obtain the first gain calibration data.

[0169] Figure 6 Schematic diagram of the process of the spectral image processing method provided in the embodiment of the present application Figure 6 ,like Figure 6 As shown, step S301, determining first calibration data of the spectral device according to state change information of the spectral device under preset conditions, includes:

[0170] S601, obtaining display brightness of a spectral device corresponding to multiple incident brightness under a preset shutter time, a preset gain, and a preset aperture.

[0171] S602: Generate a third relationship curve between incident brightness and display brightness according to multiple incident brightnesses and display brightnesses corresponding to the multiple incident brightnesses.

[0172] S603: Determine first incident brightness calibration data in the first calibration data according to the third relationship curve.

[0173] Wherein, under a preset shutter time, a preset gain, and a preset aperture, each incident brightness corresponds to a display brightness, and the display brightness is the brightness on the display screen of the spectral device.

[0174] The incident brightness is adjusted multiple times to determine the corresponding display brightness, and then a third relationship curve between the incident brightness and the display brightness is generated based on the multiple incident brightnesses and the display brightnesses corresponding to the multiple incident brightnesses. Then, the first incident brightness calibration data is determined based on the third relationship curve, wherein the first calibration data includes first incident brightness calibration data, and the first incident brightness calibration data is used to calibrate the incident brightness of the spectral device.

[0175] It should be noted that, usually, the incident brightness and display brightness are positively correlated, as shown below:

[0176] Y=p*X

[0177] Wherein, p is the preset proportional coefficient, Y is the display brightness, and X is the incident brightness.

[0178] Optionally, step S602, generating a third relationship curve between incident brightness and display brightness according to multiple incident brightnesses and display brightnesses corresponding to the multiple incident brightnesses, includes:

[0179] A third relationship curve is fitted according to the multiple incident brightnesses and the display brightnesses corresponding to the multiple incident brightnesses.

[0180] The corresponding display brightness is obtained by adjusting the incident brightness, and then a third relationship curve is fitted according to multiple incident brightnesses and the display brightness corresponding to the incident brightness. The abscissa of the third relationship curve can be the incident brightness, and the ordinate can be the display brightness.

[0181] Optionally, step S603, determining first incident brightness calibration data in the first calibration data according to the third relationship curve, includes:

[0182] The slope of the third relationship curve is determined as the first incident brightness calibration data.

[0183] The first calibration data includes first incident brightness calibration data. Based on the third relationship curve, the slope of the third relationship curve can be fitted according to the least squares method, and then the slope of the third relationship curve is used as the first incident brightness calibration data, wherein the slope of the third relationship curve is used to indicate the relative relationship between the incident brightness and the display brightness, that is, the photosensitivity of the spectral device based on the incident brightness is taken into consideration to obtain the first incident brightness calibration data.

[0184] Correspondingly, step S202 determines the second calibration data of the compensation device based on the state change information of the compensation device under preset conditions, including: obtaining the display brightness of the compensation device corresponding to multiple incident brightnesses under a preset shutter time, a preset gain, and a preset aperture, generating a relationship curve between the incident brightness and the display brightness based on the multiple incident brightnesses and the display brightness corresponding to the multiple incident brightnesses, and determining the second incident brightness calibration data in the second calibration data based on the relationship curve between the incident brightness and the display brightness.

[0185] The above specific implementation process is similar to steps S601-S603 and will not be repeated here.

[0186] In practical applications, the incident brightness can be achieved through an integrating sphere. The output port of the integrating sphere is simultaneously aligned with the spectral device and the compensation device, and then the shutter time, gain and aperture size of the spectral device and the compensation device are fixed. Among them, for the spectral device and the compensation device, the shutter time is the same, the gain is the same, and the aperture size is the same. The incident brightness of the integrating sphere is adjusted to make the brightness on the display screen of the spectral device change from dark to bright. When the screen brightness reaches the brightest, a third relationship curve between the incident brightness and the display brightness is fitted, and the slope of the third relationship curve is fitted according to the least squares method, and the slope is used as the second incident brightness calibration data.

[0187] Similarly, when adjusting the incident brightness of the integrating sphere, the display brightness of the compensation device can also be changed from dark to bright. When the picture reaches the brightest, the relationship curve between gain and display brightness is fitted. The slope of the relationship curve can be fitted according to the least squares method, and the slope is used as the second incident brightness calibration data.

[0188] In the spectral image processing method of this embodiment, the display brightness corresponding to multiple incident brightnesses of the spectral device under the preset shutter time, preset gain and preset aperture is obtained, and a third relationship curve between the incident brightness and the display brightness is generated according to the multiple incident brightnesses and the display brightness corresponding to the multiple incident brightnesses, and the first incident brightness calibration data in the first calibration data is determined according to the third relationship curve. In this way, the photosensitivity of the spectral device based on the incident brightness is taken as a factor to obtain the first incident brightness calibration data.

[0189] Figure 7 This is a schematic diagram of the structure of the spectral image processing device provided in the embodiment of the present application. The device can be integrated into a spectral image processing device. Figure 7 As shown, the device comprises:

[0190] An acquisition module 701 is used to acquire a first image brightness of a first object spectrum image for a target object collected by a spectrum device and a second image brightness of a second object spectrum image for the target object collected by a compensation device;

[0191] A processing module 702 is used to fuse the brightness of the first image and the brightness of the second image according to the first calibration data of the spectral device and the second calibration data of the compensation device to obtain a fused brightness of the first object spectral image and the second object spectral image, wherein the first calibration data is used to characterize the photosensitivity performance of the spectral device under a preset incident brightness, and the second calibration data is used to characterize the photosensitivity performance of the compensation device under the preset incident brightness;

[0192] The determination module 703 is used to determine the state information of the target object according to the fusion brightness.

[0193] Optionally, the processing module 702 is specifically configured to:

[0194] The first image brightness and the second image brightness are fused according to a ratio of the first calibration data to the second calibration data to obtain the fused brightness.

[0195] Optionally, the first calibration data includes: first shutter time calibration data, first gain calibration data and first incident brightness calibration data, and the second calibration data includes: second shutter time calibration data, second gain calibration data and second incident brightness calibration data;

[0196] The processing module 702 is specifically used for:

[0197] Determining a first ratio of the first shutter time calibration data to the second shutter time calibration data;

[0198] determining a second ratio of the first gain calibration data to the second gain calibration data;

[0199] Determining a third ratio of the second incident brightness calibration data to the first incident brightness calibration data;

[0200] The first image brightness and the second image brightness are fused according to the product of the first ratio, the second ratio and the third ratio to obtain the fused brightness.

[0201] Optionally, the processing module 702 is specifically configured to:

[0202] determining a fourth ratio of the brightness of the second image to the brightness of the first image;

[0203] The product of the first ratio, the second ratio and the third ratio and the product of the fourth ratio are used as the fusion brightness.

[0204] Optionally, the determining module 703 is further configured to:

[0205] Determining first calibration data of the spectral device according to state change information of the spectral device under preset conditions;

[0206] According to the state change information of the compensation device under the preset condition, second calibration data of the compensation device is determined.

[0207] Optionally, the determining module 703 is specifically configured to:

[0208] Obtaining display brightness of the spectral device corresponding to multiple shutter times under a preset incident brightness;

[0209] generating a first relationship curve between shutter time and display brightness according to the plurality of shutter times and the display brightness corresponding to the plurality of shutter times;

[0210] According to the first relationship curve, first shutter time calibration data in the first calibration data is determined.

[0211] Optionally, the determining module 703 is specifically configured to:

[0212] The first relationship curve is obtained by fitting according to the multiple shutter times and the display brightness corresponding to the multiple shutter times.

[0213] Optionally, the determining module 703 is specifically configured to:

[0214] The slope of the first relationship curve is determined as the first shutter time calibration data.

[0215] Optionally, the determining module 703 is specifically configured to:

[0216] Obtaining display brightness corresponding to multiple gains of the spectral device under a preset incident brightness;

[0217] generating a second relationship curve between gain and display brightness according to the multiple gains and the display brightness corresponding to the multiple gains;

[0218] According to the second relationship curve, first gain calibration data in the first calibration data is determined.

[0219] Optionally, the determining module 703 is specifically configured to:

[0220] Obtaining display brightness of the spectral device corresponding to multiple incident brightness under a preset shutter time, a preset gain, and a preset aperture;

[0221] generating a third relationship curve between incident brightness and display brightness according to the multiple incident brightnesses and display brightnesses corresponding to the multiple incident brightnesses;

[0222] According to the third relationship curve, first incident brightness calibration data in the first calibration data is determined.

[0223] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0224] Figure 8 A schematic diagram of the structure of a spectral image processing device provided in an embodiment of the present application is shown in FIG. Figure 8As shown, the device includes: a processor 801, a memory 802 and a bus 803. The memory 802 stores machine-readable instructions executable by the processor 801. When the spectral image processing device is running, the processor 801 communicates with the memory 802 through the bus 803, and the processor 801 executes the machine-readable instructions to execute the above method embodiment.

[0225] The embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above method embodiment is executed.

[0226] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0227] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or part of the technical solution that contributes to the prior art or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), disk or optical disk and other media that can store program code.

Claims

1. A spectral image processing method, characterized in that: include: Acquire a first image brightness of a first object spectral image for a target object collected by a spectral device and a second image brightness of a second object spectral image for the target object collected by a compensation device; According to the first calibration data of the spectral device and the second calibration data of the compensation device, the brightness of the first image and the brightness of the second image are fused to obtain the fused brightness of the first object spectral image and the second object spectral image, wherein the first calibration data is used to characterize the photosensitivity performance of the spectral device under a preset incident brightness, and the second calibration data is used to characterize the photosensitivity performance of the compensation device under the preset incident brightness; the first calibration data includes: first shutter time calibration data, first gain calibration data and first incident brightness calibration data, and the second calibration data includes: second shutter time calibration data, second gain calibration data and second incident brightness calibration data; The state information of the target object is determined according to the fused brightness.

2. The method according to claim 1, characterized in that: The step of fusing the brightness of the first image and the brightness of the second image according to the first calibration data of the spectral device and the second calibration data of the compensation device to obtain the fused brightness of the spectral image of the object includes: The first image brightness and the second image brightness are fused according to a ratio of the first calibration data to the second calibration data to obtain the fused brightness.

3. The method according to claim 2, characterized in that The step of fusing the first image brightness and the second image brightness according to the ratio of the first calibration data to the second calibration data to obtain the fused brightness includes: Determining a first ratio of the first shutter time calibration data to the second shutter time calibration data; determining a second ratio of the first gain calibration data to the second gain calibration data; Determining a third ratio of the second incident brightness calibration data to the first incident brightness calibration data; The first image brightness and the second image brightness are fused according to the product of the first ratio, the second ratio and the third ratio to obtain the fused brightness.

4. The method according to claim 3, characterized in that: The fusing the first image brightness and the second image brightness according to the product of the first ratio, the second ratio and the third ratio to obtain the fused brightness includes: determining a fourth ratio of the brightness of the second image to the brightness of the first image; The product of the first ratio, the second ratio and the third ratio and the product of the fourth ratio are used as the fusion brightness.

5. The method according to claim 1, characterized in that Before fusing the first image brightness and the second image brightness according to the first calibration data of the spectral device and the second calibration data of the compensation device to obtain the fused brightness of the first object spectral image and the second object spectral image, the method further includes: Determining first calibration data of the spectral device according to state change information of the spectral device under preset conditions; According to the state change information of the compensation device under the preset condition, second calibration data of the compensation device is determined.

6. The method according to claim 5, characterized in that The determining the first calibration data of the spectral device according to the state change information of the spectral device under the preset conditions includes: Obtaining display brightness of the spectral device corresponding to multiple shutter times under a preset incident brightness; generating a first relationship curve between shutter time and display brightness according to the plurality of shutter times and the display brightness corresponding to the plurality of shutter times; According to the first relationship curve, first shutter time calibration data in the first calibration data is determined.

7. The method according to claim 6, characterized in that The step of generating a first relationship curve between shutter time and display brightness according to the plurality of shutter times and the display brightness corresponding to the plurality of shutter times comprises: The first relationship curve is obtained by fitting according to the multiple shutter times and the display brightness corresponding to the multiple shutter times.

8. The method according to claim 6, characterized in that Determining the first shutter time calibration data in the first calibration data according to the first relationship curve includes: The slope of the first relationship curve is determined as the first shutter time calibration data.

9. The method according to claim 5, characterized in that The determining the first calibration data of the spectral device according to the state change information of the spectral device under the preset conditions includes: Obtaining display brightness corresponding to multiple gains of the spectral device under a preset incident brightness; generating a second relationship curve between gain and display brightness according to the multiple gains and the display brightness corresponding to the multiple gains; According to the second relationship curve, first gain calibration data in the first calibration data is determined.

10. The method according to claim 5, characterized in that The determining the first calibration data of the spectral device according to the state change information of the spectral device under the preset conditions includes: Obtaining display brightness of the spectral device corresponding to multiple incident brightness under a preset shutter time, a preset gain, and a preset aperture; generating a third relationship curve between incident brightness and display brightness according to the multiple incident brightnesses and display brightnesses corresponding to the multiple incident brightnesses; According to the third relationship curve, first incident brightness calibration data in the first calibration data is determined.

11. A spectral image processing device, characterized in that: include: An acquisition module, configured to acquire a first image brightness of a first object spectral image for a target object acquired by a spectral device and a second image brightness of a second object spectral image for the target object acquired by a compensation device; a processing module, configured to fuse the brightness of the first image and the brightness of the second image according to the first calibration data of the spectral device and the second calibration data of the compensation device, so as to obtain the fused brightness of the first object spectral image and the second object spectral image, wherein the first calibration data is used to characterize the photosensitivity performance of the spectral device under a preset incident brightness, and the second calibration data is used to characterize the photosensitivity performance of the compensation device under the preset incident brightness; the first calibration data includes: first shutter time calibration data, first gain calibration data and first incident brightness calibration data, and the second calibration data includes: second shutter time calibration data, second gain calibration data and second incident brightness calibration data; A determination module is used to determine the state information of the target object according to the fused brightness.

12. A spectral image processing device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the spectral image processing device is running, the processor and the memory communicate via the bus, and the processor executes the machine-readable instructions to perform the method described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is executed.