Multispectral image correction method, device, equipment and storage medium

Abstract

The application is suitable for the technical field of image processing, and provides a multispectral image correction method, device and equipment and storage medium. The method first extracts spectral data of all channels in a target multispectral image, obtains highlight areas of the spectral data of all channels, then matches reference spectral data of all channels corresponding to the highlight areas of the spectral data of all channels from a spectral database, calculates a correction coefficient set according to the reference spectral data of all channels, and obtains a corrected target multispectral image according to the correction coefficient set and the spectral data of all channels. Thus, more accurate reference spectral data is obtained by matching the spectral data in the spectral database, the original multispectral data is corrected based on the reference spectral data, and the model trained by using the corrected multispectral data has strong robustness to different light sources.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to a multispectral image correction method, apparatus, device and storage medium. Background Technology

[0002] In the process of multispectral image acquisition, active light sources are typically used, such as automatic illumination by the device. In this case, the light source includes both active light sources and ambient light sources. The original multispectral image will change with the change of light source. If the original multispectral image is directly input into a deep learning network for training, the resulting model will have poor robustness to different light sources. Summary of the Invention

[0003] This application provides a multispectral image correction method, apparatus, device, and storage medium, which can solve the problem that the trained model has poor robustness to different light sources when the original multispectral image is directly input into the deep learning network for training.

[0004] A first aspect of this application provides a multispectral image correction method, comprising: extracting spectral data of all channels in a target multispectral image; obtaining the highlight regions of the spectral data of all channels; matching reference spectral data of all channels corresponding to the highlight regions of the spectral data of all channels from a spectral database; calculating a correction coefficient set based on the reference spectral data of all channels; and obtaining the corrected target multispectral image based on the correction coefficient set and the spectral data of all channels.

[0005] In some embodiments, obtaining the highlight regions of the spectral data of all channels includes: selecting the spectral data of the channel with the largest pixel mean from the spectral data of all channels as the spectral data of the sample channel; obtaining the highlight pixel region corresponding to the spectral data of the sample channel, and determining the highlight pixel regions corresponding to the spectral data of other channels in the spectral data of all channels based on the highlight pixel region corresponding to the spectral data of the sample channel. Specifically, obtaining the highlight pixel region corresponding to the spectral data of the sample channel includes: determining the overexposed pixels in the spectral data of the sample channel based on the pixel value of each pixel in the spectral data of the sample channel and a preset overexposure threshold; setting the pixel values ​​of all overexposed pixels in the spectral data of the sample channel to zero to generate the de-overexposure spectral data of the sample channel; and determining the highlight pixel region corresponding to the spectral data of the sample channel based on the pixel value of each pixel in the de-overexposure spectral data of the sample channel and a preset highlight region ratio. Specifically, determining the overexposed pixels in the spectral data of the sample channel based on the pixel values ​​of each pixel in the spectral data of the sample channel and a preset overexposed threshold includes: identifying pixels in the spectral data of the sample channel whose pixel values ​​are greater than the preset overexposed threshold as overexposed pixels in the spectral data of the sample channel.

[0006] In some embodiments, matching the reference spectral data for all channels corresponding to the highlighted regions of the spectral data of all channels from the spectral database includes: obtaining the pixel mean of the highlighted regions of the spectral data of all channels; and finding the spectral data from the spectral database that is closest to the pixel mean of the highlighted regions of the spectral data of all channels as the reference spectral data for all channels based on the pixel mean of each channel. Calculating a set of correction coefficients based on the reference spectral data of all channels, and obtaining the corrected target multispectral image based on the set of correction coefficients and the spectral data of all channels includes: calculating the average reference spectral data of all channels based on the reference spectral data of all channels; determining the ratio of the reference spectral data of each channel to the average reference spectral data of all channels as the set of correction coefficients for each channel; and correcting the spectral data of all channels based on the set of correction coefficients for each channel to obtain the corrected target multispectral image.

[0007] In one embodiment, before extracting spectral data from all channels of the target multispectral data, the method further includes: obtaining an original multispectral image; and extracting a multispectral image within a preset region from the original multispectral image as the target multispectral image.

[0008] A second aspect of this application provides a multispectral data correction apparatus, comprising: a spectral data extraction module for extracting spectral data of all channels in a target multispectral image; a highlight region acquisition module for acquiring highlight regions of the spectral data of all channels; a reference spectral data determination module for matching reference spectral data of all channels corresponding to the highlight regions of the spectral data of all channels from a spectral database; and an image correction module for calculating a correction coefficient set based on the reference spectral data of all channels, and obtaining a corrected target multispectral image based on the correction coefficient set and the spectral data of all channels.

[0009] A third aspect of this application provides a terminal device, including: a multispectral camera for acquiring raw multispectral images; and a processor for receiving the raw multispectral images and then executing the multispectral image correction method of the first aspect described above.

[0010] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the multispectral image correction method of the first aspect described above.

[0011] The fifth aspect of this application provides a computer program product that, when run on a terminal device, causes the terminal device to execute the multispectral image correction method of the first aspect described above.

[0012] The beneficial effects of this application's embodiments compared to the prior art are as follows: This application discloses a multispectral image correction method, apparatus, device, and storage medium. The method first extracts spectral data from all channels of the target multispectral image and obtains the highlight regions of the spectral data for all channels. Then, it matches the reference spectral data for all channels corresponding to the highlight regions of the spectral data from a spectral database. A correction coefficient set is calculated based on the reference spectral data for all channels, and the corrected target multispectral image is obtained based on the correction coefficient set and the spectral data for all channels. Thus, by matching spectral data within the spectral database, more accurate reference spectral data is obtained. The original multispectral data is corrected based on the reference spectral data, and the model trained using the corrected multispectral data exhibits strong robustness to different light sources. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of the structure of a terminal device provided in one embodiment of this application;

[0015] Figure 2 This is a schematic flowchart of a multispectral image correction method provided in an embodiment of this application;

[0016] Figure 3 This is a schematic diagram of a process for obtaining the highlight region of spectral data for all channels, provided in an embodiment of this application;

[0017] Figure 4 This is a schematic diagram of the spectral image of a sample channel in a face scene provided in an embodiment of this application;

[0018] Figure 5 This is a schematic diagram of a highlighted pixel area provided in an embodiment of this application;

[0019] Figure 6 This is a schematic diagram of an image correction process provided in an embodiment of this application;

[0020] Figure 7 This is a schematic diagram of the structure of a multispectral data correction device according to an embodiment of this application. Detailed Implementation

[0021] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0022] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0023] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0024] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0025] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0026] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0027] It should be understood that the sequence number of each step in this embodiment does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0028] In related technologies, the correction method for RGB images is to multiply each channel by a random coefficient. However, when multispectral imaging technology applies this correction method, the random enhancement between channels, with different coefficients for each channel, will change the curve of the spectral data.

[0029] In view of this, embodiments of this application provide a multispectral image correction method, apparatus, device, and storage medium. By matching a spectral database, more accurate reference spectral data is obtained. Based on the reference spectral data, a corresponding correction parameter set is obtained to correct the original multispectral data. The model trained using the corrected multispectral data exhibits strong robustness to different light sources. Furthermore, embodiments of this application can also be applied to hyperspectral images; this embodiment does not impose any limitations on this application.

[0030] The following examples illustrate the application scenarios of the multispectral image correction method provided in the embodiments of this application.

[0031] This application can be applied to liveness detection, such as in payment and access control applications. A light source illuminates various parts of a living body, such as the face and arms, to obtain a raw multispectral image. Taking access control as an example, the background is removed from the raw multispectral image to obtain a face image as the target multispectral image. Then, spectral data from all channels are extracted from the target multispectral image, and the corresponding highlighted areas and reference spectral data from a matching spectral database are obtained. A set of correction coefficients is further calculated, and the corrected target multispectral image is obtained based on the spectral data of all channels in the correction coefficient set. This effectively reflects facial features and improves the security of access control.

[0032] This application can be applied to deep learning scenarios. Original multispectral images change with the light source, and deep learning models trained on uncorrected multispectral images have poor robustness to different light sources. Incorporating the multispectral image correction method, apparatus, device, and storage medium of this application into the data preprocessing stage of deep learning is equivalent to performing spectral dimension data augmentation on the original multispectral data, thereby enhancing the generalization ability of the trained deep learning model.

[0033] To illustrate the technical solution of this application, specific embodiments are described below.

[0034] Reference Figure 1 The diagram shows a structural schematic of a terminal device 1 provided in Embodiment 1 of this application. Figure 1As shown, terminal device 1 may include, but is not limited to, a processor 11, a memory 12, and a multispectral camera 14. Those skilled in the art will understand that... Figure 1 This is merely an example of terminal device 1 and does not constitute a limitation on terminal device 1. Terminal device 1 may include more or fewer components than shown, or combine certain components, or different components. For example, terminal device 1 may not include memory 12, which may be an external storage device, or memory 12 may be cloud storage where computer program 13 is stored. Terminal device 1 may also include input / output interfaces, network access interfaces, buses, etc.

[0035] This embodiment uses Figure 1 Taking the illustrated terminal device 1 as an example, the computer program 13 and the preset spectral database 15 are stored in the memory 12. The processor 11 retrieves the computer program 13 and the preset spectral database 15 from the memory 12 and executes the computer program 13 to implement the multispectral image correction method. The preset spectral database 15 consists of pre-collected spectral data from common light sources and different scenarios. Alternatively, when the processor 11 executes the computer program 13, it implements the functions of each module in the various device embodiments described later.

[0036] Processor 11 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0037] The memory 12 can be an internal storage unit of the terminal device 1, such as a hard disk or RAM of the terminal device 1. The memory 12 can also be an external storage device of the terminal device 1, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the terminal device 1. Furthermore, the memory 12 can include both internal and external storage units of the terminal device 1. The memory 12 is used to store computer programs and other programs and data required by the terminal device 1. The memory 12 can also be used to temporarily store data that has been output or will be output.

[0038] The multispectral camera 14 is mainly used to acquire multispectral images, which may include facial images or images containing other objects (such as vehicles, obstacles, etc.). In some embodiments of this application, the multispectral camera 14 may include a photosensitive chip and a filter array, wherein the filter array may include filters corresponding to multiple spectral bands. After the incident light beam is filtered by the filter array, an image corresponding to the spectral band is obtained. The photosensitive chip is divided into multiple imaging units corresponding to the shape of each filter in the filter array. Each imaging unit has a spectral response channel with a different spectral transmittance, and each spectral response channel corresponds to receiving a target light beam of one spectral band. For example, the multispectral data obtained by a 3×3 filter array contains data from nine different channels. The spectral response value information (pixel value) of each first pixel region is obtained, and the average spectral response value of a certain first pixel region can be further calculated and used as the spectral response value information of that first pixel region, thereby obtaining the spectral response value information (pixel average) of each spectral band in n (n is a positive integer) spectral bands.

[0039] The multispectral camera 14 transmits the acquired multispectral images to the processor. The processor 11 calls the multispectral image correction program stored in the memory 12 and the preset spectral database 15 to perform correction processing on the target multispectral data, obtain the corrected target multispectral data, and send the corrected target multispectral data to the model deep learning network for training, such as inputting it into the learning network for liveness detection.

[0040] Reference Figure 2-6 As shown, the multispectral image correction method may include the following steps:

[0041] Step 101: Extract spectral data from all channels of the target multispectral image.

[0042] Step 102: Obtain the highlight areas of the spectral data for all channels.

[0043] Step 103: Match the reference spectral data of all channels corresponding to the highlighted regions of the spectral data of all channels from the spectral database.

[0044] Step 104: Calculate the set of correction coefficients based on the reference spectral data of all channels, and obtain the corrected target multispectral image based on the set of correction coefficients and the spectral data of all channels.

[0045] For example, the target multispectral image can be represented as rad(112,112,9), where rad represents the multi-channel response data obtained by the multispectral sensor, i.e., the target multispectral image to be corrected, and (112,112,9) indicates that the target multispectral image to be corrected has 112 rows, 112 columns, and 9 channels.

[0046] As one possible implementation, foreground elements are generally more important than background elements in the original multispectral image. Therefore, when correcting the original multispectral image, the spectral characteristics of the foreground elements can be used to improve the accuracy of the correction. Specifically, in one possible implementation of this application, the original multispectral image can be acquired first, and foreground detection can be performed. Multispectral images within a preset region are extracted from the original multispectral data as the target multispectral image. The preset image region can be determined based on the actual application scenario, such as a face region. This is because the original multispectral image often contains interference factors. For example, in a face recognition access control scenario, the original multispectral image obtained when light shines on a face includes not only the face's multispectral image but also some surrounding environmental multispectral images. Since we only need the face's multispectral image data, we only need to determine the target face region. Subsequent processing targeting the target face region better reflects the face information, and the correction process is more accurate. Foreground detection is a process of identifying regions of interest or moving targets (foreground) and static parts (background) in an image.

[0047] After acquiring the target multispectral image, the spectral data of each channel can be extracted. Taking a nine-channel image as an example, the spectral data of channels 1 to 9 can be extracted, represented by rad(112,112,9). In one embodiment, after extracting the spectral data of each channel, dark noise correction can be performed on the spectral data of all the above channels according to a preset dark noise average response value (dark). Dark noise in spectral measurement refers to stray light that affects the original spectral data; it affects optical measurements, so dark noise correction is necessary first. The dark noise average response value (dark) of the multispectral sensor in a dark environment can be measured in advance according to the actual application scenario. The dark noise average response value (dark) is generally a relatively fixed value, for example, dark = 16. After dark noise correction, more accurate grayscale data for each channel can be extracted.

[0048] For example, the 9-channel spectral data msi(112,112,9) = rad(112,112,9) - dark are used to obtain the 9-channel spectral data after removing dark noise, namely: msi1, msi2, msi3, msi4, msi5, msi6, msi7, msi8 and msi9.

[0049] Knowing the characteristics of the filter curves for each channel, i.e., the pixel values ​​corresponding to the image data of each channel, determining the image data of the channel with the highest brightness facilitates subsequent identification of overexposed points and bright pixel areas. The specific steps for obtaining the bright areas of the spectral data for all channels are as follows:

[0050] Step 1021: Select the spectral data of the channel with the largest pixel mean from the spectral data of all channels as the spectral data of the sample channel.

[0051] Step 1022: Determine the overexposed pixels in the spectral data of the sample channel based on the pixel values ​​of each pixel in the spectral data of the sample channel and the preset overexposure threshold.

[0052] Step 1023: Set the pixel values ​​of all overexposed pixels in the spectral data of the sample channel to zero to generate the de-overexposed spectral data of the sample channel.

[0053] Step 1024: Determine the bright pixel region corresponding to the spectral data of the sample channel based on the pixel value of each pixel in the overexposure spectrum data of the sample channel and the preset bright region ratio.

[0054] Step 1025: Determine the bright pixel regions corresponding to the spectral data of other channels in the spectral data of all channels based on the bright pixel regions corresponding to the spectral data of the sample channels.

[0055] Specifically, taking 9-channel spectral data as an example, based on the characteristics of the 9-channel filter curve, the channel with the largest pixel mean among all channels is selected, which is the brightest channel among the 9 channels. If the third channel is the brightest response channel msi3, then msi3 is used as the spectral data of the sample channel.

[0056] Since facial images sometimes have highlights, and these highlights may contain overexposed areas, the highlights, or bright pixel regions, are selected image areas that are the focus of image analysis. Delineating these areas for targeted correction reduces processing time and increases accuracy. In this embodiment, the bright pixel regions corresponding to the spectral data of the sample channels are first located, and then the bright pixel regions corresponding to the spectral data of all channels are determined. Overexposed points are identified based on the pixel values ​​of each pixel in the spectral data of the sample channels. After setting the pixel values ​​of the overexposed points to zero, the bright pixel regions are determined according to a preset highlight region ratio. These bright pixel regions are the highlighted portions of the normally exposed areas, thus obtaining the bright pixel regions corresponding to each channel.

[0057] For example, in a face scene, Figure 4 This is a schematic diagram of the spectral image of the sample channels in a face scene. The black dots on the face represent the highlighted parts in the normally exposed area. Figure 5 For based on Figure 4 A schematic diagram of the identified highlighted pixel area.

[0058] In one embodiment, pixels in the spectral data of a sample channel whose pixel value is greater than a preset overexposure threshold can be identified as overexposed pixels. For example, in order to find overexposed pixels, an overexposure threshold is preset, which can be (255-dark)-deta, where deta can be a number between 1 and 10. deta is used to obtain a value close to the calculated overexposed point, thereby expanding the search range for overexposed points.

[0059] If the spectral data of the sample channel is msi3(112,112), then pixels in the spectral data msi3(112,112) that are greater than the overexposure threshold (255-dark)-deta are identified as overexposed pixels. These overexposed pixels in the spectral data msi3 are then set to zero, resulting in the de-overexposed image data msi3_normal, which is the pixel matrix of normal pixels. The pixels with a preset highlight region ratio in msi3_normal, such as the maximum percentage before the pixel value (which could be 1%-5%, 10%, etc., generally not exceeding 10%), are then identified, and their highlight pixel region (ROI) is obtained.

[0060] Since the pixels in each channel have a corresponding relationship, the bright pixel regions corresponding to the spectral data of other channels can be directly found based on the bright pixel regions corresponding to the spectral data of the sample channel, thereby obtaining the bright pixel regions corresponding to the spectral data of all channels.

[0061] The reference spectral data for all channels is obtained by matching the highlighted regions of the spectral data for all channels from the spectral database. First, the average pixel value of the highlighted pixel region corresponding to the spectral data of all channels needs to be obtained. For example, in the channel corresponding to msi1, the pixels in the highlighted pixel region are determined, and the average pixel value of the pixels in this region is determined to be white1. This process is repeated to determine the average pixel values ​​white2, white3, white4, ... for each channel within the highlighted pixel region, denoted as white = [white1, white2, white3, white4, ...]. Then, white is matched with each spectral data in the spectral database, and the spectral data with the closest match is determined as the reference spectral data. It should be noted that the reference spectral data illu mentioned in this application contains spectral data for multiple channels, denoted as illu = [illu1, illu2, illu3, illu4, ...].

[0062] The spectral database contains multiple sets of standard spectral data, which can be obtained through experiments or pre-tests. For example, spectral data of common light sources and different scenarios can be collected in advance. Among them, for testing light sources in multiple scenarios, the spectral data acquisition method can be to use a multispectral sensor to photograph a standard whiteboard, and the multi-channel response of the standard whiteboard is the spectral data under that light source.

[0063] For example, taking faces as an example, the spectral database can store spectral data under various conditions, including typical face detection distances, common face detection ambient light, and common face detection scenarios. The typical face detection distance can be 40-120cm, the face detection ambient light can be various indoor white light conditions, semi-outdoor conditions, and outdoor conditions, and the face detection scenario can be payment, door locks, access control, etc. In one possible embodiment, when a new device needs to be installed, new spectral data can be collected in the fixed scenario to update the spectral database, further adapting to the spectral data prediction accuracy under different scenarios.

[0064] Furthermore, a set of correction coefficients is calculated based on the reference spectral data of all channels, and the corrected target multispectral image is obtained based on the set of correction coefficients and the spectral data of all channels. The specific steps are as follows:

[0065] Step 1041: Calculate the average reference spectral data for all channels based on the reference spectral data for all channels;

[0066] Step 1042: The ratio of the reference spectral data of each channel in all channels to the average reference spectral data of all channels is determined as the set of correction coefficients corresponding to each channel in all channels;

[0067] Step 1043: Correct the spectral data of all channels based on the set of correction coefficients corresponding to each channel in all channels to obtain the corrected target multispectral image.

[0068] Specifically, the average spectral data illu_ave can be the average of illu1, illu2, illu3, illu4, ..., or the maximum and minimum values ​​of illu can be removed before averaging. The correction coefficient set for the corresponding channel msi1 is gain1 = illu_ave / illu1, the correction coefficient set for the corresponding channel msi2 is gain2 = illu_ave / illu2, the correction coefficient set for the corresponding channel msi3 is gain3 = illu_ave / illu3, ..., and the final corrected dataset is gain = [gain1, gain2, gain3, gain4, gain5, gain6, ...]. Assuming the spectral data of all channels of the target are [msi1, msi2, msi3, msi4, msi5, msi6, ...], the corrected image data for each channel are: msi1*gain1, msi2*gain2, msi3*gain3, msi4*gain4, ...

[0069] The above embodiments of this application disclose a multispectral image correction method. By matching spectral data in a spectral database, more accurate reference spectral data is obtained. The target multispectral image is corrected based on the reference spectral data. The model trained using the corrected target multispectral image has strong robustness to different light sources.

[0070] In one possible implementation, this embodiment can be subsequently applied to deep learning. In this case, the corrected image data for each channel needs to be: msi1*gain1, msi2*gain2, msi3*gain3, msi4*gain4, ..., and reorganized into a three-dimensional reflectance multispectral matrix msi_correct in the dimension of the brightest channel. Normalization is then performed, which can be maximum value normalization, 0-1 normalization, mean normalization, etc. After normalization, the data is input into the deep learning network for training.

[0071] See Figure 7 For ease of explanation, only the parts relevant to the embodiments of this application are shown.

[0072] The multispectral data correction device may specifically include the following modules:

[0073] The spectral data extraction module 701 is used to extract spectral data from all channels in the target multispectral image;

[0074] The highlight area acquisition module 702 is used to acquire the highlight areas of the spectral data of all channels;

[0075] The reference spectral data determination module 703 is used to match the reference spectral data of all channels corresponding to the highlighted areas of the spectral data of all channels from the spectral database;

[0076] The image correction module 704 is used to calculate a set of correction coefficients based on the reference spectral data of all channels, and to obtain the corrected target multispectral image based on the set of correction coefficients and the spectral data of all channels.

[0077] In one embodiment, the highlight region acquisition module 702 may specifically include the following sub-modules: a sample data acquisition sub-module, which selects the spectral data of the channel with the largest pixel mean from the spectral data of all channels as the spectral data of the sample channel; and a highlight region acquisition sub-module, which obtains the highlight pixel region corresponding to the spectral data of the sample channel, and determines the highlight pixel region corresponding to the spectral data of other channels in the spectral data of all channels based on the highlight pixel region corresponding to the spectral data of the sample channel.

[0078] In one embodiment, the highlight area acquisition submodule includes: an overexposure determination unit, configured to determine overexposed pixels in the spectral data of the sample channel based on the pixel values ​​of each pixel in the spectral data of the sample channel and a preset overexposure threshold; an overexposure removal unit, configured to set the pixel values ​​of all overexposed pixels in the spectral data of the sample channel to zero, generating overexposure removal spectral data of the sample channel; and a highlight area acquisition unit, configured to determine the highlight pixel area corresponding to the spectral data of the sample channel based on the pixel values ​​of each pixel in the overexposure removal spectral data of the sample channel and a preset highlight area ratio. In one embodiment, the overexposure determination unit is specifically configured to determine pixels in the spectral data of the sample channel whose pixel values ​​are greater than the preset overexposure threshold as overexposed pixels in the spectral data of the sample channel.

[0079] In one embodiment, the reference spectral data determination module 703 includes: a highlight mean acquisition submodule, used to obtain the pixel mean of the highlight region of the spectral data of all channels; and a reference spectral data determination submodule, used to find, based on the pixel mean of each channel, the spectral data that is closest to the pixel mean of the highlight region of the spectral data of all channels from the spectral database as the reference spectral data of all channels.

[0080] In one embodiment, the image correction module 704 includes: an average reference spectral data acquisition submodule, configured to calculate the average reference spectral data of all channels based on the reference spectral data of all channels; a correction coefficient set acquisition submodule, configured to determine the ratio of the reference spectral data of each channel to the average reference spectral data of all channels as the correction coefficient set corresponding to each channel; and a correction submodule, configured to correct the spectral data of all channels based on the correction coefficient set corresponding to each channel to obtain a corrected target multispectral image.

[0081] In one embodiment, the multispectral data correction device further includes: an original image acquisition module for acquiring an original multispectral image; and a target multispectral image determination module for extracting a multispectral image within a preset region from the original multispectral image as a target multispectral image.

[0082] The multispectral data correction device provided in this application embodiment can be applied in the foregoing method embodiments. For details, please refer to the description of the above method embodiments, which will not be repeated here.

[0083] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0084] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0085] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0086] In the embodiments provided in this application, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0087] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0088] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0089] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0090] The implementation of all or part of the processes in the methods of the above embodiments can also be accomplished by a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the various method embodiments described above.

[0091] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A multispectral image correction method for obtaining training data for a deep learning model, characterized in that, include: Extract spectral data from all channels of the target multispectral image; Obtain the highlight regions of the spectral data of all channels; wherein, obtaining the highlight regions of the spectral data of all channels includes: selecting the spectral data of the channel with the largest pixel mean from the spectral data of all channels as the spectral data of the sample channel; determining the overexposed pixels in the spectral data of the sample channel based on the pixel values ​​of each pixel in the spectral data of the sample channel and a preset overexposure threshold; setting the pixel values ​​of all the overexposed pixels in the spectral data of the sample channel to zero to generate the de-overexposure spectral data of the sample channel; determining the highlight pixel region corresponding to the spectral data of the sample channel based on the pixel values ​​of each pixel in the de-overexposure spectral data of the sample channel and a preset highlight region ratio; and determining the highlight pixel regions corresponding to the spectral data of other channels in the spectral data of all channels based on the highlight pixel regions corresponding to the spectral data of the sample channel. The reference spectral data for all channels corresponding to the highlighted regions of the spectral data for all channels are matched from the spectral database. A set of correction coefficients is calculated based on the reference spectral data of all channels, and a corrected target multispectral image is obtained based on the set of correction coefficients and the spectral data of all channels; wherein, the corrected target multispectral image is used as training data for a deep learning model to achieve data augmentation of the spectral dimension of the original multispectral image.

2. The multispectral image correction method as described in claim 1, characterized in that, The step of determining the overexposed pixels in the spectral data of the sample channel based on the pixel values ​​of each pixel in the spectral data of the sample channel and a preset overexposure threshold includes: Pixels whose pixel values ​​in the spectral data of the sample channel are greater than the preset overexposure threshold are identified as overexposed pixels in the spectral data of the sample channel.

3. The multispectral image correction method as described in claim 1, characterized in that, Reference spectral data for all channels, corresponding to the highlighted regions of the spectral data for all channels, matched from the spectral database, including: Obtain the pixel mean of the highlighted region of the spectral data for all channels; Based on the pixel mean of each channel, the spectral data that is closest to the pixel mean of the highlighted area of ​​all channels is found from the spectral database and used as the reference spectral data for all channels.

4. The multispectral image correction method as described in claim 1, characterized in that, The step of calculating a set of correction coefficients based on reference spectral data from all channels, and obtaining the corrected target multispectral image based on the set of correction coefficients and the spectral data from all channels, includes: The average reference spectral data of all channels is calculated based on the reference spectral data of all channels; The ratio of the reference spectral data of each channel to the average reference spectral data of all channels is used to determine the set of correction coefficients for each channel. The spectral data of all channels are corrected based on the set of correction coefficients corresponding to each channel in all channels to obtain the corrected target multispectral image.

5. The multispectral image correction method according to any one of claims 1-4, characterized in that, Before extracting the spectral data of all channels in the target multispectral image, the process also includes: Obtain the original multispectral image; Extract the multispectral image within a preset region from the original multispectral image, and use it as the target multispectral image.

6. A multispectral data correction device for obtaining training data for a deep learning model, characterized in that, include: The spectral data extraction module is used to extract spectral data from all channels in the target multispectral image; A highlight area acquisition module is used to acquire the highlight areas of the spectral data of all channels. The highlight area acquisition module includes the following sub-modules: a sample data acquisition sub-module, used to select the spectral data of the channel with the largest pixel mean from the spectral data of all channels as the spectral data of the sample channel; a highlight area acquisition sub-module, used to obtain the highlight pixel area corresponding to the spectral data of the sample channel, and determine the highlight pixel areas corresponding to the spectral data of other channels in the spectral data of all channels based on the highlight pixel area corresponding to the spectral data of the sample channel; the highlight area acquisition sub-module includes: an overexposure determination unit, used to determine the overexposed pixels in the spectral data of the sample channel based on the pixel value of each pixel in the spectral data of the sample channel and a preset overexposure threshold; an overexposure removal unit, used to set the pixel value of all overexposed pixels in the spectral data of the sample channel to zero, generating overexposure removal spectral data of the sample channel; and a highlight area acquisition unit, used to determine the highlight pixel area corresponding to the spectral data of the sample channel based on the pixel value of each pixel in the overexposure removal spectral data of the sample channel and a preset highlight area ratio. The reference spectral data determination module is used to match the reference spectral data of all channels corresponding to the highlighted regions of the spectral data of all channels from the spectral database; The image correction module is used to calculate a set of correction coefficients based on the reference spectral data of all channels, and to obtain a corrected target multispectral image based on the set of correction coefficients and the spectral data of all channels; wherein the corrected target multispectral image is used as training data for a deep learning model to achieve data augmentation of the spectral dimension of the original multispectral image.

7. A terminal device, characterized in that, include: A multispectral camera is used to acquire raw multispectral images; A processor is configured to receive the raw multispectral image and then execute the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 5.

Images