Method for calculating correction matrix, non-uniformity correction method and related device

Abstract

The application is suitable for the field of multispectral technology, and provides a correction matrix calculation method, a non-uniformity error correction method and related devices. The calculation method comprises the following steps: acquiring a calibration multispectral image of a whiteboard, and acquiring first data of each channel in the calibration multispectral image; acquiring a reference position of a multispectral image sensor; determining a reference area in the calibration multispectral image according to the reference position, and acquiring second data corresponding to each channel in the reference area; and calculating a correction matrix corresponding to each channel according to the first data and the second data. According to the above scheme, the target correction matrix of each channel is calculated through the data of each channel in the calibration multispectral image and the data corresponding to each channel in the reference area, so that the non-uniformity error of the multispectral camera can be corrected according to the target correction matrix, and the multispectral image obtained by the multispectral camera is more real.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of multispectral technology, and particularly relates to a correction matrix calculation method, a multispectral image non-uniformity error correction method, a correction matrix calculation device, a multispectral image non-uniformity error correction device, a terminal, and a computer readable storage medium. BACKGROUND

[0002] A multispectral image sensor has more channels than an RGB sensor, but due to a more complex manufacturing process, the same channel at different spatial positions of the multispectral image sensor has inconsistent spectral response curves due to manufacturing process (for example, uneven filter coating) and other reasons, which manifests as non-uniformity of the response of the multispectral image sensor in the spatial domain, and further causes the generated multispectral image to have non-uniformity errors. SUMMARY

[0003] The application embodiment provides a correction matrix calculation method, a multispectral image non-uniformity error correction method, a correction matrix calculation device, a multispectral image non-uniformity error correction device, a terminal, and a computer readable storage medium, which can solve the above problems.

[0004] In a first aspect, the application embodiment provides a correction matrix calculation method, the correction matrix being used to correct non-uniformity errors of a multispectral image sensor, comprising: acquiring a calibration multispectral image of a white board, and acquiring first data of each channel in the calibration multispectral image, the calibration multispectral image being acquired by a multispectral image sensor; acquiring a reference position in the multispectral image sensor; wherein the coincidence degree between an actual multi-channel response curve of the multispectral image sensor at the reference position and an ideal multi-channel response curve is greater than a preset threshold; determining a reference region in the calibration multispectral image according to the reference position, and acquiring second data of each channel in the reference region; and calculating a correction matrix corresponding to each channel according to the first data and the second data.

[0005] In some embodiments, acquiring the first data of each channel in the calibration multispectral image comprises: extracting a channel sub-image corresponding to each channel in the calibration multispectral image, and extracting data of each pixel point in each channel sub-image to obtain the first data of each channel; wherein the first data is in the form of a matrix. In some embodiments, the reference region includes one or more reference pixel points, and when the number of reference pixel points is multiple, acquiring the second data of each channel in the reference region comprises: respectively extracting data of each channel in the multiple reference pixel points; and performing averaging on the multiple data of each channel to obtain the second data of each channel in the reference region.

[0006] In some embodiments, the number of the calibration multispectral images is one, and the correction matrix corresponding to each channel is calculated according to the first data and the second data, including: dividing the first data and the second data of each channel to obtain the correction matrix corresponding to each channel. In some embodiments, the number of the calibration multispectral images is multiple, and the multiple calibration multispectral images are respectively multispectral images of the whiteboard captured when the multispectral image sensor is at multiple distances from the whiteboard, and the correction matrix corresponding to each channel is calculated according to the first data and the second data, including: obtaining the first data of each channel in each calibration multispectral image to obtain multiple first data; summing or averaging the multiple first data to obtain third data; obtaining the second data of each calibration multispectral image to obtain multiple second data; summing or averaging the multiple second data to obtain fourth data; and dividing the third data and the fourth data to obtain the correction matrix corresponding to each channel in the multispectral image sensor.

[0007] In a second aspect, the embodiments of the present application provide a non-uniformity error correction method of a multispectral image, including: obtaining an initial multispectral image; performing non-uniformity correction on the initial multispectral image according to a preset correction matrix to obtain a target multispectral image; wherein the correction matrix is calculated by the calculation method of the correction matrix in the first aspect.

[0008] In some embodiments, the correction matrix includes a sub-correction matrix corresponding to each channel, and the non-uniformity correction on the initial multispectral image according to the preset correction matrix to obtain the target multispectral image includes: respectively correcting the data of each channel in the initial multispectral image according to the sub-correction matrix corresponding to each channel to obtain the target multispectral image.

[0009] In a third aspect, the embodiments of the present application provide a calculation device of a correction matrix, including: a first obtaining unit, a second obtaining unit, a processing unit and a calculation unit. The first obtaining unit is configured to obtain a calibration multispectral image of a whiteboard and obtain first data of each channel in the calibration multispectral image, the calibration multispectral image being obtained by a multispectral image sensor; the second obtaining unit is configured to obtain a reference position in the multispectral image sensor; wherein the coincidence degree between the actual multi-channel response curve of the multispectral image sensor at the reference position and the ideal multi-channel response curve is greater than a preset threshold; the processing unit is configured to determine a reference area in the calibration multispectral image according to the reference position and obtain second data of each channel in the reference area; and the calculation unit is configured to calculate a correction matrix corresponding to each channel according to the first data and the second data.

[0010] Fourthly, embodiments of this application provide a non-uniformity error correction device for a multispectral image, including an acquisition unit and a non-uniformity correction unit. The acquisition unit is used to acquire an initial multispectral image; the non-uniformity correction unit is used to perform non-uniformity correction on the initial multispectral image according to a preset correction matrix to obtain a target multispectral image; wherein, the correction matrix is ​​calculated using the correction matrix calculation method described in the first aspect.

[0011] Fifthly, embodiments of this application provide a terminal, including a multispectral image sensor and a processor. The multispectral image sensor is used to acquire multispectral images. In one embodiment, the processor is used to execute the correction matrix calculation method in the first aspect after receiving the multispectral image. In another embodiment, the processor is used to execute the non-uniformity error correction method for the multispectral image in the second aspect after receiving the multispectral image.

[0012] Sixthly, embodiments of this application provide a computer-readable storage medium storing a computer program. In one embodiment, when executed by a processor, the computer program implements the method for calculating the correction matrix as described in the first aspect above. In another embodiment, when executed by a processor, the computer program implements the method for correcting non-uniformity errors in multispectral images as described in the second aspect above.

[0013] In this embodiment, by acquiring the calibration multispectral image of the whiteboard and extracting the data of each channel in the calibration multispectral image and the data corresponding to each channel in the reference area, the correction matrix of each channel is calculated, thereby obtaining the correction matrix of each channel. Then, the non-uniformity error correction can be performed on the multispectral image acquired based on the correction matrix, reducing the error caused by the non-uniformity of the multispectral image sensor, and the obtained target multispectral image is more realistic. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the terminal structure provided in the first embodiment of this application;

[0016] Figure 2 This is a schematic diagram of the 9 channel sub-images extracted from a 9-channel multispectral image;

[0017] Figure 3is a flowchart of a non-uniformity error correction method of a multi-spectrum image sensor according to a second embodiment of the present application;

[0018] Figure 4 is a flowchart of a method for calculating a correction matrix according to a third embodiment of the present application;

[0019] Figure 5 is a schematic diagram of a non-uniformity error correction device for multi-spectrum images according to a fourth embodiment of the present application;

[0020] Figure 6 is a schematic diagram of a device for calculating a correction matrix according to a fifth embodiment of the present application. DETAILED DESCRIPTION

[0021] In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0022] It should be understood that the term "comprises" when used in this specification and the appended claims, specifies the presence of stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0023] It should also be understood that the term "and / or" when used in this specification and the appended claims, means any one or more of the associated listed items can be present, and includes multiples of those items that can be present.

[0024] As used in this specification and the appended claims, the term "if" can be construed to mean "when" or "once" or "in response to a determination" or "in response to the occurrence of" that follows, depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be construed to mean "once it is determined" or "in response to a determination" or "once [the described condition or event] is detected" or "in response to the detection of [the described condition or event]", depending on the context.

[0025] In addition, the terms "first", "second", "third", etc. are used herein only to distinguish one element from another, and do not imply or suggest a relative importance of the elements so designated.

[0026] Reference within the specification of this application to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in additional embodiments," and so on, in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily referring to some, but not all, embodiments, unless otherwise indicated. The terms "including," "comprising," "having," and variations thereof, are meant to encompass the items listed thereafter and equivalents thereof as well as additional items.

[0027] Figure 1 is a schematic diagram of a terminal provided by the first embodiment of the present application. The terminal of this embodiment includes a processor 11, a memory 12, a multi-spectrum camera 13, and a computer program 14 stored in the memory 11 and executable on the processor 11. The processor 11 is connected with the memory 12 and the multi-spectrum camera 13 respectively, the processor 11 can store data such as images to the memory 12, the processor 11 can also call data such as the computer program 14 and images in the memory 12, and the processor 11 can also control the multi-spectrum camera 13 to take images.

[0028] The terminal 1 can include but is not limited to the processor 11, the memory 12, and the multi-spectrum camera 13. Those skilled in the art can understand that the terminal 1 can include more or less components, or combine certain components, or different components, for example, the terminal 1 can also include an input / output interface, a network access interface, a bus, etc.; for another example, the terminal 1 can not include the memory 12, the computer program 14 can be burned in the processor 11, or the memory 12 can be a cloud memory. Figure 1 The terminal 1 shown in the figure is only an example and does not constitute a limitation on the terminal 1, and can include more or less components than those shown in the figure, or combine certain components, or different components, for example, the terminal 1 can also include an input / output interface, a network access interface, a bus, etc.; for another example, the terminal 1 can not include the memory 12, the computer program 14 can be burned in the processor 11, or the memory 12 can be a cloud memory.

[0029] The processor 11 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0030] The memory 12 can be an internal storage unit of the terminal 1, such as a hard disk or a memory of the terminal 1. The memory 12 can also be an external storage device of the terminal 1, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the terminal 1. Further, the terminal 1 can also include both an internal storage unit and an external storage device of the terminal 1. The memory 12 is used to store the computer program 14 and data required for executing the program, such as a correction matrix, a multi-spectral image, etc. The memory 11 can also be used to temporarily store data that has been output or will be output.

[0031] The multi-spectral camera 13 is used to acquire a multi-spectral image of the detected object. The acquired multi-spectral image can be stored in the memory 12 or directly transmitted to the processor 11. The multi-spectral camera 13 can receive light rays of at least 4 different wave bands and then generate a multi-spectral image. For example, the multi-spectral camera 13 can receive light rays of 4, 5, 6, 7, 8, 9, 16 or more different wave bands and then generate an image.

[0032] Figure 2 A schematic diagram of sub-images corresponding to 9 channels extracted from a multi-spectral image is shown. Figure 2 In a corresponding embodiment, the multi-spectral camera 13 can receive light rays of 9 different wave bands and generate a multi-spectral image, which can be referred to as a 9-channel multi-spectral image. The image obtained by extracting the channel data corresponding to each pixel point in the 9-channel multi-spectral image is a one-channel sub-image, and similarly, two-channel sub-images to nine-channel sub-images can be obtained. Figure 2 ch1, ch2, ch3, ch4, ch5, ch6, ch7, ch8 and ch9 in FIG. 1 are respectively a one-channel sub-image, a two-channel sub-image, a three-channel sub-image, a four-channel sub-image, a five-channel sub-image, a six-channel sub-image, a seven-channel sub-image, an eight-channel sub-image and a nine-channel sub-image.

[0033] Specifically, the multispectral camera 13 includes a lens for converging light rays to a multispectral image sensor, and the multispectral image sensor generates a corresponding multispectral image after receiving the light rays. The multispectral image sensor includes a filter array for filtering light rays and a pixel array for receiving filtered light rays and converting them into electrical signals. The filter array includes filters of multiple different center wavelengths, thereby enabling the multispectral image sensor to implement multi-channel imaging. Due to manufacturing processes (e.g., uneven filter coating), the response curves of the same channel of the multispectral image sensor at different spatial positions to light rays of the same waveband are inconsistent, which can be understood as a non-uniformity of the multispectral image sensor in the spatial domain. This non-uniformity can cause the same object to exhibit inconsistent colors or spectral curves at different imaging spaces. For example, in biopsy, the biopsy effect is good at the center position and slightly poor at other positions. Therefore, the non-uniformity error of the multispectral image sensor needs to be corrected.

[0034] The computer program 14 is stored in the memory 12, and the computer program 14 includes a non-uniformity error correction program of a multispectral image and a correction matrix calculation program, etc. The non-uniformity error correction program can be called by the processor 11 to execute instructions for non-uniformity error correction of a multispectral image, such as the non-uniformity error correction method of a multispectral image shown in Figure 3 The correction matrix calculation program can be called by the processor 11 to execute instructions for calculating a correction matrix for correcting the non-uniformity error of a multispectral image sensor, such as the calculation method of a correction matrix shown in Figure 4 In some embodiments, the non-uniformity error correction program and the correction matrix calculation program can also be located in two memories 12, respectively.

[0035] Please refer to Figure 3 , Figure 3 is a flowchart of a non-uniformity error correction method of a multispectral image provided by the second embodiment of the present application. The non-uniformity error correction method of a multispectral image can include:

[0036] S201: Obtain an initial multispectral image.

[0037] The initial multispectral image can be a real-time shot by a multispectral camera, or an image stored in a memory after shooting. The initial multispectral image can include data of at least four different wavebands, i.e., the initial multispectral image is at least a four-channel multispectral image.

[0038] S202: Perform non-uniformity correction on the initial multispectral image according to a preset correction matrix to obtain a target multispectral image; wherein the correction matrix can be calculated by, for example, Figure 4The calculation method of the correction matrix shown is stored in the memory after being calculated.

[0039] The processor can call the preset correction matrix from the memory, the correction matrix including a sub-correction matrix corresponding to each channel, and then perform non-uniformity correction on the data of each channel in each pixel point in the initial multi-spectral image according to the sub-correction matrix, to generate a target multi-spectral image after correction.

[0040] Taking an initial multi-spectral image as a 9-channel multi-spectral image as an example, the multi-spectral data of the entire initial multi-spectral image is MSI_target(m, n, 9), where MSI_target(m, n, 9) is a three-dimensional matrix, m is the number of rows of pixel points in the initial multi-spectral image, n is the number of columns of pixel points in the initial multi-spectral image, and 9 is the data of 9 channels of each pixel point. The correction matrix is Correct(m, n, 9), Correct(m, n, 9) is a three-dimensional matrix, m is the number of rows of pixel points, n is the number of columns of pixel points, and 9 is the correction parameter of 9 channels of each pixel point. In an embodiment, the non-uniformity correction method for the initial multi-spectral image is as follows:

[0041] MSI_target_correct(m, n, 9) = MSI_target(m, n, 9). / Correct(m, n, 9)

[0042] Where. / represents that the data of corresponding points in two matrices are divided, and MSI_target_correct(m, n, 9) is the multi-spectral data of the target multi-spectral image, which is also represented in the form of a three-dimensional matrix.

[0043] In an embodiment, the correction matrix Correct(m, n, 9) can be decomposed into sub-correction matrices Correct1(m, n, 1), Correct2(m, n, 2), Correct3(m, n, 3), Correct4(m, n, 4), Correct5(m, n, 5), Correct6(m, n, 6), Correct7(m, n, 7), Correct8(m, n, 8), and Correct9(m, n, 9) corresponding to the 9 channels, which are all two-dimensional matrices. Then, the data of the 9-channel sub-image are respectively divided by the 9 sub-correction matrices to obtain the corrected data of the 9 sub-channels, and then the target multi-spectral image is obtained.

[0044] In the embodiment, the collected multi-spectral image is corrected by calling the correction matrix, the error caused by non-uniformity is reduced, and the generated target multi-spectral image is more accurate.

[0045] In some embodiments, the correction matrix can be calculated by a method for calculating a correction matrix. As shown in Figure 4 Figure 4 is a flowchart of a method for calculating a correction matrix according to a third embodiment of the present application. The method for calculating a correction matrix can include the following steps:

[0046] S401: Obtain a calibration multispectral image of a whiteboard, and obtain first data of each channel in the calibration multispectral image.

[0047] The whiteboard has good response to light of each waveband. The calibration multispectral image of the whiteboard is obtained, and then the correction matrix is calculated according to the multispectral data of the whiteboard, so that the obtained correction matrix is more accurate. When the correction matrix needs to be calculated, the processor can control the multispectral camera to capture the calibration multispectral image of the whiteboard. When the calibration multispectral image is captured, the whiteboard needs to completely cover the field of view of the multispectral camera. In this way, the entire image of the captured calibration multispectral image is the whiteboard, avoiding the interference of other objects, and being conducive to improving the accuracy of the subsequent correction matrix. In an embodiment, the whiteboard is a diffuse reflection whiteboard. The diffuse reflection whiteboard has a reflectivity of more than 98% to ultraviolet light, visible light, and near-infrared light. The use of the diffuse reflection whiteboard can better reflect the differences between the same channels at different positions, and thus the correction matrix obtained by calibration will be more accurate.

[0048] After the calibration multispectral image is obtained, the first data of each channel in the calibration multispectral image is further obtained. Specifically, a channel sub-image corresponding to each channel in the calibration multispectral image can be extracted, and then the data of each pixel point in each channel sub-image can be extracted, so that the first data of each channel can be obtained, and the first data is represented in the form of a matrix. The first data of each channel includes the data of each pixel point in the calibration multispectral image in the channel, and is represented in the form of an m-row n-column matrix, where m is the number of rows of pixel points in the calibration multispectral image, and n is the number of columns of pixel points in the calibration multispectral image.

[0049] Taking the calibration multispectral image as a 9-channel image as an example, the first data corresponding to channels 1 to 9 obtained is: ch1(m, n) = MSI_1(m, n, 1); ch2(m, n) = MSI_1(m, n, 2); ch3(m, n) = MSI_1(m, n, 3); ch4(m, n) = MSI_1(m, n, 4); ch5(m, n) = MSI_1(m, n, 5); ch6(m, n) = MSI_1(m, n, 6); ch7(m, n) = MSI_1(m, n, 7); ch8(m, n) = MSI_1(m, n, 8); ch9(m, n) = MSI_1(m, n, 9). Wherein, MSI_1(m, n, 1) is the first data of channel 1 in the calibration multispectral image.​

[0050] S402: Obtain a reference position in the multispectral image sensor; wherein the coincidence degree between the actual multi-channel response curve and the ideal multi-channel response curve of the multispectral image sensor at the reference position is greater than a preset threshold.

[0051] The reference position is a position where the coincidence degree between the actual multi-channel response curve and the ideal multi-channel response curve of the multispectral image sensor is greater than a preset threshold, and the preset threshold can be 80%, 90%, 95%, etc., which are not listed here. The ideal multi-channel response curve can be understood as the response curve of each channel to a wide band of spectrum when there is no non-uniformity error (for example, when the filter is uniformly coated), and the actual multi-channel response curve is the response curve of each channel to a wide band of spectrum when actually used. It can be understood that the difference between the actual multi-channel response curve and the ideal multi-channel response curve corresponding to the reference position is within an allowable range, and the data obtained at the reference position is more accurate. Specifically, the reference position can be pre-stored in the memory, and the reference positions of different multispectral image sensors can be the same or different. The reference position is mainly determined by the manufacturing process of the multispectral image sensor, and the reference position can be determined after the multispectral image sensor is produced. The non-uniformity error at the reference position is the smallest, for example, the filter coating in the central region of the multispectral image sensor is more uniform, and the reference position can be the central coordinate point.

[0052] In one embodiment, the reference position is a coordinate point, which can be obtained by calibration. For example, a sample image is collected by a multispectral camera, the channel response curve of each pixel point on the obtained sample image is obtained, and then the ideal pixel point of the most ideal channel response curve is found, and thus the reference position of the multispectral image sensor can be determined. When the ideal pixel point is multiple, the 9-channel response curves of the pixel points in a certain region around the ideal pixel point can be compared, and then the most ideal pixel point is found as the reference position; or, one of the multiple ideal pixel points can be arbitrarily selected as the reference position; or, the multiple ideal pixel points can all be used as the reference position.

[0053] Since the single coordinate point method may be affected by the noise interference of the data of the single coordinate point, affecting the accuracy of the correction matrix, in another embodiment, the reference position can be one or more regions, and the actual multi-channel response curve of the multispectral image sensor in the region is closer to the ideal multi-channel response curve. The specific position of the region can also be determined by calibration and the like.

[0054] S403: Determine a reference region in the calibration multispectral image according to the reference position, and obtain second data of each channel in the reference region.

[0055] Since the calibration multispectral image is generated by the multispectral camera, the reference region in the calibration multispectral image can be determined according to the information of the reference position. It can be understood that the image data corresponding to the reference region is generated by the pixel of the reference position. When the reference region is a pixel point, the data of each channel in the pixel point can be extracted as the second region of each channel of the reference region. When the reference region includes multiple pixel points, the data of each channel in each pixel point needs to be extracted respectively, and multiple data of each channel can be obtained. Then, the average value of the multiple data of each channel is obtained to obtain the second data of each channel of the reference region. It can be understood that the second data is not a matrix but a numerical value. The manner of obtaining the second data can refer to the manner of obtaining the first data in step S401, which will not be described herein.

[0056] S404: Calculate the correction matrix corresponding to each channel according to the first data and the second data.

[0057] In an embodiment, the number of calibration multispectral images is one, which is the multispectral image of the whiteboard collected when the multispectral camera is at a preset distance from the whiteboard. Then, the first data and the second data of each channel are divided to obtain the correction matrix corresponding to each channel. For example, Correct(m, n, k) = ch k (m, n) / ch k (x, y), where Correct(m, n, k) is the correction matrix of the kth channel, ch k (m, n) is the first data of the kth channel, which is represented in the form of a matrix, ch k (x, y) is the second data of the kth channel, which is represented in the form of a numerical value. The correction matrix corresponding to each channel is the sub-correction matrix of each channel described in the above non-uniformity correction method embodiment.

[0058] In another embodiment, in order to more accurately calculate the correction matrix, the multispectral camera collects a calibration multispectral image at each of multiple distances from the whiteboard, and multiple calibration multispectral images can be obtained. Then, the first data of each channel in each calibration multispectral image and the second data of each channel in the reference region of each calibration multispectral image need to be obtained. It can be understood that the number of first data of each channel is multiple, and the number of second data of each channel is multiple. When calculating the correction matrix, the sum or average value of the multiple first data is obtained to obtain the third data corresponding to each channel. The sum or average value of the multiple second data is obtained to obtain the fourth data. Then, the third data and the fourth data are divided to obtain the correction matrix corresponding to each channel of the multispectral image sensor.

[0059] Taking the 9-channel calibrated multi-spectral image as an example, p whiteboard calibrated multi-spectral images corresponding to p distances are collected, each channel has p first data, and the third data corresponding to channels 1 to 9 can be calculated in the following way:

[0060] ch1(m,n)=MSI_1(m,n,1)+MSI_2(m,n,1)+......+MSI_p(m,n,1);

[0061] ch2(m,n)=MSI_1(m,n,2)+MSI_2(m,n,2)+......+MSI_p(m,n,2);

[0062] ch3(m,n)=MSI_1(m,n,3)+MSI_2(m,n,3)+......+MSI_p(m,n,3);

[0063] ch4(m,n)=MSI_1(m,n,4)+MSI_2(m,n,4)+......+MSI_p(m,n,4);

[0064] ch5(m,n)=MSI_1(m,n,5)+MSI_2(m,n,5)+......+MSI_p(m,n,5);

[0065] ch6(m,n)=MSI_1(m,n,6)+MSI_2(m,n,6)+......+MSI_p(m,n,6);

[0066] ch7(m,n)=MSI_1(m,n,7)+MSI_2(m,n,7)+......+MSI_p(m,n,7);

[0067] ch8(m,n)=MSI_1(m,n,8)+MSI_2(m,n,8)+......+MSI_p(m,n,8);

[0068] ch9(m,n)=MSI_1(m,n,9)+MSI_2(m,n,9)+......+MSI_p(m,n,9)。

[0069] Each channel in the reference region corresponds to p second data, and the fourth data corresponding to channels 1 to 9 can be calculated in the following way:

[0070] ch1(x,y)= MSI_1(x,y,1)+MSI_2(x,y,1)+......+MSI_p(x,y,1);

[0071] ch2(x, y) = MSI_1(x, y, 2) + MSI_2(x, y, 2) +... + MSI_p(x, y, 2);

[0072] ch3(x, y) = MSI_1(x, y, 3) + MSI_2(x, y, 3) +... + MSI_p(x, y, 3);

[0073] ch4(x, y) = MSI_1(x, y, 4) + MSI_2(x, y, 4) +... + MSI_p(x, y, 4);

[0074] ch5(x, y) = MSI_1(x, y, 5) + MSI_2(x, y, 5) +... + MSI_p(x, y, 5);

[0075] ch6(x, y) = MSI_1(x, y, 6) + MSI_2(x, y, 6) +... + MSI_p(x, y, 6);

[0076] ch7(x, y) = MSI_1(x, y, 7) + MSI_2(x, y, 7) +... + MSI_p(x, y, 7);

[0077] ch8(x, y) = MSI_1(x, y, 8) + MSI_2(x, y, 8) +... + MSI_p(x, y, 8);

[0078] ch9(x, y) = MSI_1(x, y, 9) + MSI_2(x, y, 9) +... + MSI_p(x, y, 9).

[0079] Wherein, x, y refer to the position coordinates of the reference pixel point when the reference region is the reference pixel point.

[0080] Divide the third data and the fourth data of each channel, the third data of each channel includes m*n data, and the fourth data has only one data, that is, the m*n data is divided by the fourth data, and then the correction matrix corresponding to each channel is obtained. For example, the correction matrix of 9 channels is calculated as follows:

[0081] Correct(m, n, 1) = ch1(m, n) / ch1(x, y);

[0082] Correct(m, n, 2) = ch2(m, n) / ch2(x, y);

[0083] Correct(m, n, 3) = ch3(m, n) / ch3(x, y);

[0084] Correct(m, n, 4) = ch4(m, n) / ch4(x, y);

[0085] Correct(m, n, 5) = ch5(m, n) / ch5(x, y);

[0086] Correct(m, n, 6) = ch6(m, n) / ch6(x, y);

[0087] Correct(m, n, 7) = ch7(m, n) / ch7(x, y);

[0088] Correct(m, n, 8) = ch8(m, n) / ch8(x, y);

[0089] Correct(m, n, 9) = ch9(m, n) / ch9(x, y).

[0090] In addition, the third data and the fourth data are calculated by summation in the above example, which is only one implementation, and in other implementations, the third data and the fourth data can also be calculated by averaging, etc. In the above example, the first data is divided by the second data when calculating the correction matrix, and in other examples, the second data can also be divided by the first data, which is not limited herein.

[0091] The correction matrix of each channel obtained can be stored in the memory, so as to be called by the processor when executing the non-uniformity error correction method. In one embodiment, after obtaining the correction matrix of each channel, the correction matrices of multiple channels can be combined to obtain a multi-channel correction matrix, and the multi-channel correction matrix is a three-dimensional matrix, which can be understood as the sub-correction matrix of each channel described in the above non-uniformity correction method embodiment. In addition, the correction matrix of each channel obtained in the embodiment can be applicable to non-uniformity correction of a multi-spectral image obtained at any shooting distance.

[0092] In the embodiment of the application, by obtaining the calibration multi-spectral image of the whiteboard and extracting the data of each channel in the calibration multi-spectral image and the data corresponding to each channel in the reference region, the correction matrix of each channel is calculated, and then the multi-spectral image collected according to the correction matrix can be subjected to non-uniformity error correction, thereby reducing the error caused by the non-uniformity of the multi-spectral image sensor, and the obtained target multi-spectral image is more real.

[0093] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the application.

[0094] Please refer to Figure 5 , Figure 5 is a schematic diagram of a multi-spectral image non-uniformity error correction device provided in the fourth embodiment of the present application. The units included are configured to perform Figure 3 the steps in the corresponding embodiments. For details, please refer to Figure 3 the related description in the corresponding embodiments. For ease of illustration, only the parts related to the present embodiment are shown. Please refer to Figure 5 , the multi-spectral image non-uniformity error correction device 5 includes an acquisition unit 510 and a non-uniformity correction unit 520.

[0095] Specifically, the acquisition unit 510 is configured to acquire an initial multi-spectral image; the non-uniformity correction unit 520 is configured to perform non-uniformity correction on the initial multi-spectral image according to a preset correction matrix to obtain a target multi-spectral image; wherein the correction matrix can be calculated by the calculation method of the correction matrix as shown in Figure 4 .

[0096] Please refer to Figure 6 , Figure 6 is a schematic diagram of a correction matrix calculation device provided in the fifth embodiment of the present application. The units included are configured to perform Figure 4 the steps in the corresponding embodiments. For details, please refer to Figure 4 the related description in the corresponding embodiments. For ease of illustration, only the parts related to the present embodiment are shown. Please refer to Figure 6 , the correction matrix calculation device 6 includes a first acquisition unit 610, a second acquisition unit 620, a processing unit 630, and a calculation unit 640.

[0097] The first acquisition unit 610 is configured to acquire a calibration multi-spectral image of a whiteboard and acquire first data of each channel in the calibration multi-spectral image, the calibration multi-spectral image being acquired by a multi-spectral image sensor; the second acquisition unit 620 is configured to acquire a reference position in the multi-spectral image sensor; wherein the coincidence degree between the actual multi-channel response curve of the multi-spectral image sensor at the reference position and the ideal multi-channel response curve is greater than a preset threshold; the processing unit 630 is configured to determine a reference area in the calibration multi-spectral image according to the reference position and acquire second data of each channel in the reference area; and the calculation unit 640 is configured to calculate a correction matrix corresponding to each channel according to the first data and the second data.

[0098] It should be noted that the information interaction between the above devices / units, the execution process, and the like, are based on the same concept as the method embodiments of the present application, and for their specific functions and the technical effects brought about, please refer to the method embodiments part, which will not be repeated here.

[0099] The embodiment of the present application further provides a network device, comprising at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps in any of the method embodiments described above when executing the computer program.

[0100] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executable by a processor to implement the steps in any of the method embodiments described above.

[0101] The embodiment of the present application provides a computer program product, which, when executed on a mobile terminal, enables the mobile terminal to implement the steps in any of the method embodiments described above.

[0102] The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the embodiment of the present application can implement all or part of the processes in the above method through a computer program to instruct related hardware to complete, and the computer program can be stored in a computer readable storage medium. The computer program, when executed by a processor, can implement the steps in any of the method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the photographing device / terminal, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0103] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0104] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0105] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other manners. For example, the embodiments of the apparatus / network device described above are merely illustrative. For example, the division of the modules or units is merely logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0106] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0107] The above-described embodiments are merely used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalent replacements; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of calculating a correction matrix for correcting non-uniformity errors of a multi-spectral image sensor, characterized in that, The method comprises: acquiring a calibration multispectral image of a whiteboard, and acquiring first data of each channel in the calibration multispectral image, wherein the calibration multispectral image is acquired by a multispectral image sensor; wherein the first data is a plurality of pixel point data corresponding to each channel in the calibration multispectral image; acquiring a reference position in the multispectral image sensor; wherein the coincidence degree between the actual multi-channel response curve of the multispectral image sensor at the reference position and the ideal multi-channel response curve is greater than a preset threshold; determining a reference region in the calibration multispectral image according to the reference position, and acquiring second data of each channel in the reference region; wherein the second data is the average value of a plurality of pixel point data of each channel in the reference region; calculating a correction matrix corresponding to each channel according to the first data and the second data.

2. The method of claim 1, wherein, The method comprises: extracting a channel sub-image corresponding to each channel in the calibration multispectral image, and extracting data of each pixel point in each channel sub-image to obtain the first data of each channel; wherein the first data is represented in the form of a matrix.

3. The method of claim 1, wherein the correction matrix is calculated by, The reference region comprises one or more reference pixel points, and when the number of reference pixel points is multiple, the method comprises: extracting data of each channel in multiple reference pixel points to obtain multiple data of each channel; averaging the multiple data of each channel to obtain the second data of each channel in the reference region.

4. The method of claim 1, wherein the correction matrix is calculated by, The number of calibration multispectral images is one, and the method comprises: dividing the first data and the second data of each channel to obtain the correction matrix corresponding to each channel.

5. The method of claim 1, wherein the correction matrix is calculated by, The number of calibration multispectral images is multiple, and each of the multiple calibration multispectral images is a multispectral image of the whiteboard acquired when the multispectral image sensor is at multiple distances from the whiteboard, and the method comprises: acquiring the first data of each channel in each of the calibration multispectral images to obtain multiple first data; summing or averaging the multiple first data to obtain third data; acquiring the second data of each of the calibration multispectral images to obtain multiple second data; summing or averaging the multiple second data to obtain fourth data; dividing the third data and the fourth data to obtain the correction matrix corresponding to each channel in the multispectral image sensor.

6. A method of non-uniformity error correction of a multispectral image, characterized in that, The method comprises: acquiring an initial multispectral image; performing non-uniform correction on the initial multispectral image according to a preset correction matrix to obtain a target multispectral image; wherein the correction matrix is calculated by the method for calculating a correction matrix according to any one of claims 1-5.

7. The method of non-uniformity error correction of a multispectral image of claim 6, wherein, The correction matrix includes a sub-correction matrix corresponding to each channel, and the non-uniformity correction of the initial multi-spectral image according to the preset correction matrix obtains a target multi-spectral image, including: According to the sub-correction matrix corresponding to each channel, the data of each channel in the initial multi-spectral image is corrected respectively to obtain the target multi-spectral image.

8. A computing device for correcting a matrix, characterized by Including: The first acquisition unit is configured to acquire a calibration multi-spectral image of a whiteboard and acquire first data of each channel in the calibration multi-spectral image, wherein the calibration multi-spectral image is acquired by the multi-spectral image sensor; and the first data is a plurality of pixel point data corresponding to each channel in the calibration multi-spectral image. The second acquisition unit is configured to acquire a reference position in the multi-spectral image sensor; wherein the coincidence degree between an actual multi-channel response curve and an ideal multi-channel response curve of the multi-spectral image sensor at the reference position is greater than a preset threshold. The processing unit is configured to determine a reference area in the calibration multi-spectral image according to the reference position and acquire second data of each channel in the reference area; wherein the second data is an average value of a plurality of pixel point data of each channel in the reference area. The calculation unit is configured to calculate a correction matrix corresponding to each channel according to the first data and the second data.

9. A device for non-uniformity error correction of a multispectral image, characterized in that, Including: The acquisition unit is configured to acquire an initial multi-spectral image. The non-uniformity correction unit is configured to perform non-uniformity correction on the initial multi-spectral image according to a preset correction matrix to obtain a target multi-spectral image; wherein the correction matrix is calculated by the calculation method of the correction matrix of any one of claims 1-5.

10. A terminal, characterized by comprising: Including: The multi-spectral image sensor is configured to acquire a multi-spectral image. The processor is configured to receive the multi-spectral image and execute the calculation method of the correction matrix of any one of claims 1-5 or the non-uniformity error correction method of the multi-spectral image of claim 6 or 7. 11.A computer readable storage medium, storing a computer program, characterized in that, The computer program is executed by the processor to implement the calculation method of the correction matrix of any one of claims 1-5 or the non-uniformity error correction method of the multi-spectral image of claim 6 or 7.

Images