Multispectral Imaging Method and System Based on Camera Array

Abstract

This disclosure relates to the field of spectral imaging technology, specifically to a multispectral imaging method and system based on a camera array. The multispectral imaging method based on a camera array includes the following steps: correcting the background, radiance, and distance of multiple spectral images obtained simultaneously to obtain multiple corrected spectral images, wherein each of the multiple spectral images represents the spectral image acquired by each lens in the array; correcting the deviation of the multiple corrected spectral images to obtain a corrected spectral image, wherein the deviation represents the difference in image content among different spectral images; and using the corrected spectral image as the target spectral image. This invention, based on a camera array, corrects the background, radiance, and distance of spectral images obtained by the array lenses simultaneously, and then corrects the deviation of the corrected spectral images to obtain the target spectral image, thereby reducing or avoiding the problem of deviation in spectral imaging.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of spectral imaging technology, in particular, the present disclosure relates to a multi-spectral imaging method and system based on camera array. BACKGROUND

[0002] In recent years, with the rapid development of economic society, the application of multi-spectral camera is more and more widely. Multi-spectral camera can obtain the information radiated or reflected by ground objects in multiple spectral bands through spectral technology, and the spectral technology determines the structure and volume of the camera and the image data processing method.

[0003] However, the existing multi-spectral camera often has deviation in the content of the spectral image, and there is no good solution to the deviation of the spectral image. SUMMARY

[0004] The present disclosure provides a multi-spectral imaging method and system based on camera array to solve the problem of deviation in the content of the spectral image.

[0005] In one aspect, the present disclosure provides a multi-spectral imaging method based on camera array, comprising the following steps:

[0006] According to the multiple spectral images at the same time, the background, radiation value and distance of the spectral images are respectively corrected to obtain multiple corrected spectral images, wherein the multiple spectral images represent the spectral images collected by each lens in the array lens;

[0007] The multiple corrected spectral images are corrected for deviation to obtain a corrected spectral image, wherein the deviation represents the difference in the content of the different spectral images;

[0008] The corrected spectral image is taken as a target spectral image.

[0009] Preferably, in one embodiment, the step of correcting the background, radiation value and distance of the spectral images according to the multiple spectral images at the same time to obtain the corrected spectral image further comprises:

[0010] According to the spectral image, the background of the spectral image is corrected; wherein the background correction I' satisfies the following formula:

[0011] I'(x,y,λ)=I(x,y,λ)-B(x,y,λ);

[0012] Wherein I is the spectral image collected by the collecting lens with wavelength λ, B is the radiation spectrum of different positions of the atmosphere, and x and y are pixel coordinate points in the spectral image;

[0013] According to the spectral image corrected by the background, the radiation value of the spectral image is corrected; wherein the radiation value correction I" satisfies the following formula:

[0014]

[0015] Where η is the sensitivity response value of different wavelengths at different pixel positions of the imaging spectrometer, S is the radiation spectrum curve of the illumination source, and k is the correction coefficient constant.

[0016] Based on the spectral image corrected for radiometric values, a distance correction is performed on the spectral image; wherein the distance correction I”’ satisfies the following formula:

[0017]

[0018] Where L is the distance from the measurement point to the radiator, and a is the absorption coefficient curve of the atmosphere at different wavelengths.

[0019] In a preferred embodiment, the step of correcting deviations in multiple corrected spectral images to obtain a corrected spectral image further includes:

[0020] Acquire a first spectral image and a second spectral image, wherein the first spectral image and the second spectral image are spectral images corresponding to two different channels, and the first spectral image is a spectral image contained in the spectral images acquired by any array lens;

[0021] The pixel brightness values ​​of the first and second spectral images are standardized;

[0022] Select a portion of the image from the second spectral image and compare the portion of the image along at least two directions of the first spectral image to obtain the deviation value;

[0023] The deviation value is matched to the first spectral image to form a third spectral image, wherein the third spectral image is the image formed by combining the first spectral image and the deviation value.

[0024] In a preferred embodiment, after the step of correcting multiple spectral image deviations, the method further includes:

[0025] Based on a preset time, obtain spectral images of multiple channels at that time.

[0026] Select regions that each spectral image can contain from multiple spectral images to construct a spectral curve.

[0027] Based on the spectral curve, a color table for the region is obtained. The color table represents the sum of each column of pixels in the region, and the brightness of the column position at the wavelength in the spectral curve represents its relative magnitude.

[0028] In a preferred embodiment, after the step of selecting regions that each of the multiple spectral images can possess to construct a spectral curve, the method further includes:

[0029] Based on the selected region, determine the three-dimensional data, where the three-dimensional data indicates the three-dimensional spectral matrix data of the region;

[0030] Several initial points are randomly selected from the three-dimensional data;

[0031] Select data at a preset first distance from the first initial point to form the first generation unit region;

[0032] Within several first-generation unit regions, a second initial point is selected;

[0033] Data at a preset second distance from the second initial point is selected to form the second-generation unit region;

[0034] The pixel data contained in the second-generation unit area is mapped to the three-color RGB value domain using a pseudo-color algorithm for display.

[0035] In a preferred embodiment, the step of mapping the pixel data contained in the second-generation unit region to the three-color gamut for display using a pseudo-color algorithm further includes:

[0036] Based on the second-generation unit region, a two-dimensional matrix is ​​obtained, and the value is labeled for each pixel data of the two-dimensional matrix pair;

[0037] Obtain the RGB values ​​of the spectral image I(x,y) to increase its contrast, where the spectral image I satisfies the following formula:

[0038]

[0039] Where m is k is the pixel data label value, n is the value range of the label for each pixel, and k∈[0,n].

[0040] Another aspect of the present invention provides a camera array-based multispectral imaging system, and an imaging method applied to the above-mentioned camera array-based multispectral imaging system, comprising:

[0041] Camera module, including fixed-focus array lenses;

[0042] Steering control module, used to drive the camera module to turn;

[0043] Raspberry Pi control components, electrically connected to the camera module;

[0044] A computer system is used to process information collected by the Raspberry Pi control components to obtain images;

[0045] The data transmission module is used to connect the camera module and the computer system respectively for information transmission; among them,

[0046] The array lens includes at least one monitoring lens and multiple acquisition lenses. The multiple acquisition lenses are used to acquire spectral images. Each acquisition lens is equipped with a filter. At least some of the multiple filters allow the transmission of a specified wavelength so that the computer system can acquire spectral images of the corresponding wavelength through the filter lenses.

[0047] In a preferred embodiment, the camera module includes a positioning plate, and an array of lenses is distributed on the positioning plate.

[0048] In a preferred embodiment, the steering control module includes at least two drive motors, one of which drives the camera module to rotate horizontally, and the other drive motor drives the camera module to rotate vertically.

[0049] In one preferred embodiment, the Raspberry Pi control component is connected to the acquisition lenses, and the information acquired by each acquisition lens is stored independently within the Raspberry Pi control component.

[0050] This invention provides a multispectral imaging method based on a camera array. It corrects the background, radiation value, and distance of the spectral images obtained by each lens in the array at the same time. Then, it corrects the deviation of the corrected spectral images to obtain the target spectral image. By correcting the spectral images from multiple angles, it reduces or avoids the problem of deviation in spectral imaging. Attached Figure Description

[0051] Figure 1 This is a schematic flowchart of a multispectral imaging method based on a camera array according to the present invention;

[0052] Figure 2 This is a schematic diagram illustrating the background, radiometric values, and distance correction process of a multispectral imaging method based on a camera array according to the present invention.

[0053] Figure 3 This is a schematic diagram of the array lens deviation spectral image;

[0054] Figure 4 This is a schematic diagram of the spectral image integration process of a multispectral imaging method based on a camera array according to the present invention;

[0055] Figure 5 This is a schematic diagram of edge detection;

[0056] Figure 6 This is a diagram showing the comparison of deviation values;

[0057] Figure 7 This is a schematic diagram of spectral image integration based on deviation values, where a is the template image, b is the original image, and c is the assembled image;

[0058] Figure 8 This is a flowchart of data dimensionality reduction and extraction.

[0059] Figure 9 This is the intended use of color, where the dashed box in a corresponds to the color change in b;

[0060] Figure 10 This is a flowchart of graphic segmentation;

[0061] Figure 11 This is a flowchart of the pseudo-color algorithm;

[0062] Figure 12 The image is obtained using a pseudo-color algorithm;

[0063] Figure 13 This is a schematic diagram of a multispectral imaging system based on a camera array according to the present invention.

[0064] Figure label:

[0065] 1. Camera module; 2. Steering control module; 3. Computer system; 4. Raspberry Pi control components; 5. Data transmission module; 6. Screen; 7. Lens; 8. Filter. Detailed Implementation

[0066] In the description of the embodiments of the present invention, those skilled in the art should understand that the embodiments of the present invention can be implemented as methods, apparatuses, electronic devices, and computer-readable storage media. Therefore, the embodiments of the present invention can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. Furthermore, in some embodiments, the embodiments of the present invention can also be implemented as a computer program product in one or more computer-readable storage media, the computer-readable storage media containing computer program code.

[0067] The aforementioned computer-readable storage medium may be any combination of one or more computer-readable storage media. Computer-readable storage media include: electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any combination thereof. In embodiments of the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0068] The computer program code contained in the aforementioned computer-readable storage medium may be transmitted using any suitable medium, including wireless, wire, optical fiber, radio frequency (RF), or any suitable combination thereof.

[0069] Computer program code for performing the operations of the embodiments of the present invention can be written in assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The computer program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer or an external computer via any type of network, including a local area network (LAN) or a wide area network (WAN).

[0070] The embodiments of the present invention describe the provided methods, apparatus, and electronic devices through flowcharts and / or block diagrams.

[0071] It should be understood that each block of a flowchart and / or block diagram, as well as combinations of blocks in a flowchart and / or block diagram, can be implemented by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine that, when executed by a computer or other programmable data processing apparatus, creates means for implementing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0072] These computer-readable program instructions may also be stored in a computer-readable storage medium that enables a computer or other programmable data processing device to function in a particular manner. In this way, the instructions stored in the computer-readable storage medium produce an instruction apparatus product that includes the functions / operations specified in the blocks of a flowchart and / or block diagram.

[0073] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer-implemented process, such that the instructions that execute on the computer or other programmable data processing apparatus provide a process for implementing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0074] See Figures 1 to 12 This invention provides a multispectral imaging method based on a camera array, comprising the following steps:

[0075] like Figure 1 As shown, S101: Based on multiple spectral images taken at the same time, the background, radiation values ​​and distances are corrected respectively to obtain multiple corrected spectral images.

[0076] Among them, multiple spectral images represent the spectral images acquired by each lens in the array lens.

[0077] Specifically, the array lens comprises multiple independent lenses, each capable of acquiring a spectral image at a specific wavelength.

[0078] Each lens can acquire a spectral image at the same time. When correcting the spectral image, factors such as the background, radiation value, and distance of the spectral image should be taken into account.

[0079] It is understandable that a corrected spectral image is a spectral image obtained by correcting the background, radiance values, and distance of a spectral image.

[0080] Considering factors such as background, radiation levels, and distance, refer to steps S1011-S1013.

[0081] like Figure 2 As shown, S1011: Based on the spectral image, perform background correction on the spectral image; where the background correction I' satisfies the following formula:

[0082] I'(x,y,λ)=I(x,y,λ)-B(x,y,λ);

[0083] Where I is the spectral image with wavelength λ captured by the acquisition lens, B is the radiation spectrum at different locations in the atmosphere, and x and y are the pixel coordinates in the spectral image.

[0084] It should be noted that since most multispectral measurements are conducted during the day, the background atmospheric radiation included in the spectral images needs to be corrected. In the above formula, measuring B is relatively difficult, and can be replaced by the background radiation b from a region close to or around the radiator being measured, such as a region with cloud interference.

[0085] In this step, the background of the spectral image is corrected using a formula, which can avoid or reduce the influence of background radiation on the spectral image and improve the imaging effect.

[0086] S1012: Based on the background-corrected spectral image, perform radiometric numerical correction on the spectral image; wherein, the radiometric numerical correction I” satisfies the following formula:

[0087]

[0088] Where η is the sensitivity response value of different wavelengths at different pixel positions of the imaging spectrometer, S is the radiation spectrum curve of the illumination source, and k is a correction coefficient constant, which is related to the detector calibration conditions.

[0089] It should be noted that the radiation numerical correction is mainly used to correct the response deviation of the array lens. The specific correction formula is as shown above. If the radiator is an actively emitting test object without light source illumination, S does not need to be considered.

[0090] After radiometric correction, the changes in the light source were normalized, and the spectral responses of different imaging positions and wavelengths of the array lens were corrected, thus obtaining the absolute radiometric values ​​of the observed sample.

[0091] In this step, when the array lens acquires spectral images, since multiple lenses are acquiring them, response deviations are inevitable. To avoid or reduce this situation, it is necessary to correct the response deviations, thereby improving the spectral imaging effect.

[0092] S1013: Based on the spectral image corrected for radiometric values, perform distance correction on the spectral image; wherein the distance correction I”' satisfies the following formula:

[0093]

[0094] Where L is the distance from the measurement point to the radiator (the radiator is understood as the test object; for the sake of convenience, the terms radiator and test object may differ in the text, but they represent the same meaning), and a is the absorption coefficient curve of different wavelengths of the atmosphere.

[0095] It should be noted that since the distance between the array lens and the measuring radiator is often far and the distance may not be fixed, it is necessary to perform distance correction on the radiator. The specific correction formula is as shown in the above formula, where k is a correction coefficient constant that is related to the instrument calibration conditions.

[0096] In this step, when the array lens acquires spectral images of the radiator, since the distance between the radiator and the array lens is not fixed, the influence of distance changes on the spectral image is considered during spectral image imaging. The above formula is used for distance correction in order to improve the imaging effect of spectral imaging.

[0097] In this embodiment, the spectral images acquired by the array lens are corrected by adjusting multiple aspects such as background, radiation value, and distance to obtain multiple corrected spectral images, thereby improving the final imaging effect of the spectral images.

[0098] S102: Correct the deviations of multiple corrected spectral images to obtain corrected spectral images.

[0099] Among them, deviation characterizes the differences in image content in different spectral images.

[0100] like Figure 3 As shown, since each lens in the array lens is positioned differently in the coordinate system when acquiring images, there are also differences in the acquired spectral images. In order to avoid or reduce these differences, it is necessary to correct the differences in the image content of the acquired spectral images in order to obtain corrected spectral images.

[0101] Specifically, the difference in lens position causes a shift between spectral images. When the central lens of the array is the source image, and the spectral images obtained by the other lenses around it are template images, the template images will display content that was not displayed in the source image due to the difference in position.

[0102] Therefore, the template image needs to be adjusted to integrate content in the template image that is not displayed in the source image into the source image. Specifically, this includes steps S10211-S10214. The specific details of these steps are as follows:

[0103] like Figure 4 As shown, S10211: Acquire the first spectral image and the second spectral image.

[0104] The first spectral image and the second spectral image are spectral images corresponding to two different channels, and the first spectral image is a spectral image that is included in the spectral images acquired by any array lens.

[0105] Specifically, the array lens includes several lenses. If the number of lenses is 1 to n, it forms n channels. The images acquired by the 1 to n lenses are a first spectral image, a second spectral image, ..., an nth spectral image. The aforementioned first spectral image can be a spectral image acquired by any array lens. Preferably, the first spectral image is the spectral image acquired by the central lens of the array lens, and the first spectral image is used as the source image. The second spectral image is a spectral image acquired by a lens different from the first spectral image, and is used as the template image.

[0106] It is understood that the first spectral image and the second spectral image are spectral images at the same time, and the second spectral image can represent one spectral image or multiple spectral images. This technical solution does not limit them.

[0107] S10212: Standardize the pixel brightness values ​​of the first spectral image and the second spectral image.

[0108] Since the first and second spectral images originate from channels at different locations, the corresponding radiators may not necessarily have the same brightness distribution when acquired. This is because the reflectivity of the same material varies for different wavelengths of light. Therefore, even if the actual translational deviation of the two images is obtained through the positional relationship of the lenses on the array lens, the corresponding maximum co-correlation (Dif) will not be zero. In other words, there may be image patches in the source image that do not match the template image, even though their overall average brightness values ​​are similar, resulting in the lowest possible Dif. Based on the above reasons, the source and template images to be matched are preprocessed, specifically by standardizing the brightness value of each pixel in both images. The standardization satisfies the following formula:

[0109]

[0110] like Figure 5 As shown, the Canny algorithm (edge ​​detection) is used to outline the image edges, similar to calculating the first-order difference of a two-dimensional image, and then using the difference data to replace the original data for template matching. This maximizes the chances of correctly finding the matching position between the two images.

[0111] The algorithm for DIF satisfies the following formula:

[0112] Dif(Δx, Δy)=∑ x y(I1(x,y)-I2(x+Δx,y+Δy)) 2 ,

[0113] like Figure 6 As shown, I1 is the template image, I2 is the original image, and Δx and Δy are the offsets. After the calculation is completed, the relative positions of the two images are modified, and this process is repeated until the minimum Δx and Δy of the Dif image are found.

[0114] S10213: Select a portion of the image from the second spectral image, and compare the portion of the image along at least two directions of the first spectral image to obtain the deviation value.

[0115] like Figure 7 As shown, it's understandable that the purpose of comparing the first and second spectral images is to determine the discrepancies between them, i.e., the deviation values. It's known that because the first and second spectral images were acquired using an array lens with a small distance between the lenses, most of the content in the two images is identical, and the relative brightness distribution at various locations in the two images is not linearly matched. Directly comparing the first and second spectral images would result in a long image processing time and uncertain outcomes.

[0116] For the reasons mentioned above, a portion of the second spectral image, such as one-third of its content, is selected for comparison with the first spectral image. Furthermore, to accelerate the comparison, comparisons can be performed from multiple directions, such as from two opposing directions. This comparison method not only increases processing speed but also reduces the probability of matching errors.

[0117] It should be noted that, to further improve the processing speed, the second spectral image preferably has image blocks with high contrast and significant brightness variations. Of course, these image blocks need to be included in all channels simultaneously, and highly repetitive image blocks should be avoided as much as possible. For example, a portion of the second spectral image can be used as a template to scan the first spectral image, obtaining the corresponding portion in the first spectral image. Finally, the deviation value between the two images is calculated, and this deviation value is used to process the second spectral image.

[0118] S10214: Match the deviation value to the first spectral image to form the third spectral image.

[0119] The third spectral image is formed by combining the first spectral image with the deviation value. It can be understood that the third spectral image proposed here specifically refers to the spectral image obtained by matching the first spectral image and the second spectral image through the deviation value.

[0120] Specifically, after obtaining the deviation value through the previous step, the deviation value is applied to the second spectral image to obtain the third spectral image.

[0121] It should be noted that the third spectral image can be used as a new first spectral image and compared with other spectral images besides the original first and second spectral images. The final spectral image is obtained after multiple comparisons.

[0122] In this embodiment, the deviation value between the first spectral image and the second spectral image is calculated, and the second spectral image is corrected using the deviation value to obtain the third spectral image. This step can then be repeated to obtain the final spectral image. This solves the problem that the different positions of each lens in the array lens cause deviations in the imaging of the corresponding channel, resulting in poor final spectral imaging effect.

[0123] Step S102 can be followed by data dimensionality reduction and extraction steps S10221-S10223.

[0124] like Figure 8 As shown, S10221: Based on the preset time, obtain the spectral images of multiple channels at that time.

[0125] The preset time is a time when the channel acquires the spectral image. This time can be any time from the start time to the end time of acquiring the spectral image. The spectral images of multiple channels corresponding to this preset time are used as the spectral images required for data dimensionality reduction extraction.

[0126] S10222: Select regions that each spectral image can have from multiple spectral images to form a spectral curve.

[0127] like Figure 9 As shown, a spectral curve is generated by selecting a specific point (which can be a region) from multiple channels' spectral images taken at the same time. This spectral curve is constructed from the selected points in each spectral image. If the spectrum of the entire region of the spectral image is used for calculation, the average brightness of the pixels in that region is taken as the brightness value for that wavelength, thus forming the spectral curve.

[0128] S10223: Based on the spectral curve, obtain the color table for the region, wherein the color table represents the sum of each column of pixels in the region, which forms the brightness of the column position at the wavelength, representing its relative size.

[0129] This involves extracting the line region spectral image of a specific area, which is taken from a physical region of the original 3D multispectral image, such as... Figure 9 As shown in the dashed box, the sum of each column of pixels in the dashed box is used as the brightness of that column position at that wavelength, and a corresponding color table is formed. The relative size is represented in the form of the color table.

[0130] It should be noted that a three-dimensional multispectral image is formed by the overlapping directions of multiple spectral images and two directions within the spectral images.

[0131] After extracting the three-dimensional spectral matrix data of a certain region, it can be stored for later use.

[0132] In this embodiment, data dimensionality reduction is performed on a certain region of a three-dimensional multispectral image to form a color table for easy display and analysis of the three-dimensional matrix data.

[0133] like Figure 10 As shown, after obtaining the region in step S10222, the process also includes corrective steps S102221-S102226 for graphical segmentation of the region.

[0134] S102221: Determine the three-dimensional data based on the selected area.

[0135] The three-dimensional data refers to the three-dimensional spectral matrix data of the indicated region, which is formed by extracting a certain region from multiple spectral images. This three-dimensional spectral matrix data is then used as the data required for image segmentation.

[0136] S102222: Several initial points are randomly selected from the three-dimensional data.

[0137] In this process, several initial points are selected from the three-dimensional spectral matrix data. These initial points can be randomly selected.

[0138] S102223: Select data at a preset first distance from the first initial point to form the first generation unit region.

[0139] In this process, after selecting a random first initial point, pixels in the three-dimensional spectral matrix data that fall within the first distance are selected according to a preset first distance to form a first-generation unit region with the first initial point.

[0140] Understandably, by using the above method, pixels that meet a preset first distance and are centered on the first initial point are combined to form the first-generation unit region, thus achieving the first clustering. Clustering can be understood as grouping the pixels contained in the first-generation unit region into one class.

[0141] S102224: Select a second initial point in several first-generation unit regions.

[0142] In this process, based on several first-generation unit regions, a second initial point is selected and used as the center point.

[0143] S102225: Select data at a preset second distance from the second initial point to form the second generation unit region.

[0144] Specifically, the second initial point is used as the center point, and the data of the second distance is selected and the pixels of the second distance are combined to form the second generation unit region.

[0145] It should be noted that the second-generation unit region is constructed based on the first-generation unit region.

[0146] S102226: The pixel data contained in the second-generation unit area is mapped to the three-color RGB value domain for display using a pseudo-color algorithm.

[0147] It can be understood that the second-generation unit region can be understood as the region that passes through the first-generation unit region, or it can be understood as the second-generation unit region formed by repeated iterations, such as passing through the first-generation unit region, the second-generation unit region, the third-generation unit region, and so on to the Nth-generation unit region.

[0148] The pixel data contained in the Nth generation unit region is processed using a pseudo-color algorithm. Preferably, the number of iterations is 10. After 10 iterations, the contained pixel data is obtained and mapped to the RGB value domain for display. For example, when the input data is a 3D data set of (x,y,λ) = 300×400×35, if the first initial point is 12, a 300×400 2D matrix is ​​returned, with values ​​from 0 to 11 as integers, serving as labels for each pixel. Then, a pseudo-color algorithm is used to map these 0-11 values ​​to a 24-bit RGB value domain, displaying the 300×400 image in pseudo-color.

[0149] In this embodiment, by dividing the region, the computational load is reduced through repeated iteration. During the repeated iteration, as the initial point value increases, the indicator of the quality of the iteration result (which represents the standard for terminating the iteration in case of overshoot) will continuously decrease. The initial value at which the rate of decrease of the curve suddenly drops is the value of the initial point that can more realistically reflect the objective category data.

[0150] like Figure 11 As shown, in step S102226, the pseudo-color algorithm is obtained through the following steps S102227 and S102228:

[0151] S102227: Based on the second-generation unit region, obtain a two-dimensional matrix and label the value of each pixel data in the two-dimensional matrix pair.

[0152] The second-generation cell region is understood as the cell region obtained by evaluating the quality of the iterative running results, as mentioned above. It can be understood as the second-generation cell region obtained through 1 to N iterations.

[0153] S102228: Obtain the RGB values ​​of the spectral image I(x,y) to increase the contrast of the spectral image, wherein the spectral image I satisfies the following formula:

[0154]

[0155] Where m is k is the pixel data label value, and n is k∈[0,n].

[0156] like Figure 12 As shown, specifically, the obtained two-dimensional label matrix for each pixel is converted into an RGB image, where the value range of the label for each pixel is n, and k ∈ [0, n]. Let the label value of the pixel at point x, y be k. The RGB values ​​of the final spectral image I(x,y) can then be obtained using the formula above.

[0157] In this embodiment, the pseudo-color algorithm obtains the RGB values ​​through the above formula, which is simple and clear, can fully increase the image contrast, and intuitively show the image segmentation structure.

[0158] S103: Use the corrected spectral image as the target spectral image.

[0159] The target spectral image is the final spectral image obtained during the spectral imaging process.

[0160] In this embodiment, the extracted point spectrum and line region spectrum images are displayed through data dimensionality reduction extraction. The region graphic segmentation simplifies the segmentation algorithm, and the pseudo-color algorithm is used to intuitively display the image segmentation results. Through the above scheme, the calculation speed is improved in the process of correcting the deviation of the spectral image, and the segmentation structure can be intuitively displayed. The problem of matching spectral images under different channels is solved, and the target spectral image is finally obtained.

[0161] like Figure 13 As shown, the present invention also provides a multispectral imaging system based on a camera array, and an imaging method applied to the above-mentioned multispectral imaging system based on a camera array, comprising: a camera module 1, a steering control module 2, a computer system 3, a Raspberry Pi control component 4, and a data transmission module 5. The camera module 1 is driven by the steering control module 2 to enable it to rotate in a predetermined direction. The computer system 3 is connected to the Raspberry Pi control component 5 via the data transmission module 4, and the computer system 3 processes the information acquired by the camera module 1 from the Raspberry Pi control component 5.

[0162] The camera module 1 includes a fixed-focus array lens 7.

[0163] The array lens 7 includes at least one monitoring lens and multiple acquisition lenses. The multiple acquisition lenses are used to acquire spectral images. Each acquisition lens is equipped with a filter, and at least some of the multiple filters allow a specified wavelength to pass through.

[0164] The camera module 1 includes a positioning plate on which array lenses 7 are distributed. The positioning plate not only positions the array lenses 7, but also enables the steering control module 2 to control the rotation of the positioning plate, thereby achieving the steering of the camera module 1.

[0165] Specifically, the array lens 7 can be a fixed-focus lens, for example, an 8mm fixed-focus lens. The acquisition lenses are equipped with filters 8 of corresponding wavelengths. The center wavelength of the filters 8 is shown in Table 1. Table 1 shows the center wavelengths of the filters 8 corresponding to 35 lenses 7.

[0166] Channel Wavelength / nm Channel Wavelength / nm 0 360 18 610 1 380 19 620 2 400 20 630 3 420 21 640 4 440 22 660 5 460 23 680 6 480 24 700 7 500 25 740 8 510 26 760 9 520 27 780 10 530 28 800 11 540 29 810 12 550 30 880 13 560 31 890 14 570 32 900 15 580 33 910 16 590 34 1000 17 600

[0167] Table 1

[0168] The acquisition lens and the monitoring lens are connected using different lines. Specifically, the acquisition lens is connected to the Raspberry Pi control component 4, and the monitoring lens is connected to the monitoring equipment, such as a monitor, to output real-time video monitoring data.

[0169] The acquisition lens and the monitoring lens are both set on the positioning plate, which can be conveniently monitored while the acquisition is being carried out. This allows the operator to observe the acquired information in real time through the screen 6 (the screen 6 is connected to the computer system 3), thus improving convenience.

[0170] The steering control module 2 is used to drive the camera module 1 to turn and adjust the shooting field of view of the camera module 1 to achieve focus on the camera module 1. Specifically, the steering control module 2 includes at least two drive motors, one of which drives the camera module to rotate horizontally by 0-180°; the other drive motor drives the camera module to rotate vertically by 0-90°. These two drive motors form an omnidirectional gimbal, facilitating target tracking and observation.

[0171] Computer system 3 processes the information acquired by Raspberry Pi control component 4 to obtain an image. Specifically, computer system 3 is used to correct the background, radiometric values, and distance of the spectral image, as well as to correct any deviations.

[0172] The Raspberry Pi Controller 4, as a small single-board computer, offers a high degree of freedom, low cost, and is widely used due to its modular and open design.

[0173] The Raspberry Pi control unit 4 and array lens 7 are connected (FFC, Flexible Flat Cable). The FFC cable includes both a MIPICSI2 (MIPI, Mobile Industry Processor Interface; CSI, Camera Serial Interface) interface and an I2C interface (I2C, Internal Integrated Circuit, is an onboard communication protocol). The I2C bus (Inter-Integrated Circuit) is a serial communication bus that uses multi-controller / multi-target, packet switching, single-ended, serial communication.

[0174] It should be noted that CSI is a one-way data transmission interface used for image transmission, while I2C is a two-way command transmission interface used for sending control commands and transmitting commands back.

[0175] After the Raspberry Pi control component 4 is connected to the array lens 7, parameters such as the lens 7's exposure time, analog signal gain, and sensitivity can be adjusted. These parameters can all be modified via commands from the Raspberry Pi control component 4 and transmitted by the data transmission module 5. With a maximum image acquisition frame rate of 30 frames per second for each lens 7, the image acquisition interval frequency can be set via the Raspberry Pi control component 4 to appropriately reduce the number of frames that need to be stored. This avoids over-acquisition and wasting the storage resources of the Raspberry Pi control component 4 in scenarios where high-frequency imaging is not required. This allows the computer system 3 to acquire spectral images of the corresponding wavelengths through the lens 7 equipped with the filter 8.

[0176] Understandably, each camera module 1 is connected to a dedicated Raspberry Pi for driving. Each Raspberry Pi has an RJ45 cable connected to a hub (a multi-port repeater, i.e., a data transmission module), controlled by the backend (referring to computer system 3) via a local area network. Using consecutive IPv4 addresses, an independent program runs on the Raspberry Pi, monitoring the execution of lens shooting and ensuring synchronized operation of all lenses 7. When image capture is required, the backend sends commands to the Raspberry Pi, which then sends the capture signals to the array lenses 7 via the Raspberry Pi control program. The array lenses 7 capture and transmit RAW format images, temporarily stored on an SD card inserted into the Raspberry Pi. The information captured by the array lenses 7 is transmitted back as 35 folders with 35 channels, as shown in Table 1. Each channel represents an image captured by one lens 7, i.e., one wavelength. Each folder stores several frames of RAW format image files. The number of images captured per second depends on the set interval frequency, with a maximum of 30 frames. For example, when the image acquisition interval frame rate is 3, 30 / 3 = 10 frames of images are acquired per second. The images are 8-bit grayscale images. When processing the data in the backend, the pixel brightness value of a certain point in a certain image is extracted as the relative brightness of that point. Other factors, such as analog magnification and CMOS sensitivity, are also considered to calculate the true relative brightness value. Furthermore, the backend control program can convert RAW format images into BMP device-independent bitmaps.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An imaging method of a camera array based multispectral imaging system, characterized in that, The method comprises the following steps: According to a plurality of spectral images at the same time, the background, radiation value and distance of the spectral images are respectively corrected to obtain a plurality of corrected spectral images, wherein the plurality of spectral images represent the spectral images collected by each lens in the array lens; The deviation of the plurality of corrected spectral images is corrected to obtain a corrected spectral image, wherein the deviation represents the difference in image content in different spectral images; The corrected spectral image is taken as a target spectral image; The step of correcting the plurality of spectral images at the same time to obtain a plurality of corrected spectral images further comprises: According to the spectral image, the background of the spectral image is corrected; wherein the background correction I' satisfies the following formula: I'(x, y, λ) = I(x, y, λ) - B(x, y, λ); Wherein I is the spectral image collected by the collection lens, B is the radiation spectrum of different positions of the atmosphere, x and y are pixel coordinate points in the spectral image; According to the background corrected spectral image, the radiation value of the spectral image is corrected; wherein the radiation value correction I" satisfies the following formula: Wherein η is the sensitivity response value of different wavelengths at different pixel positions of the imaging spectrometer, S is the radiation spectrum curve of the illumination light source, and k is a correction coefficient constant; According to the radiation value corrected spectral image, the distance correction of the spectral image is carried out; wherein the distance correction I'" satisfies the following formula: Wherein L is the distance from the measurement point to the radiation body, and a is the different wavelength absorption coefficient curve of the atmosphere.

2. The imaging method of claim 1, wherein, The step of correcting the deviation of the plurality of corrected spectral images to obtain a corrected spectral image further comprises: Obtain a first spectral image and a second spectral image, wherein the first spectral image and the second spectral image are spectral images corresponding to two different channels, and the first spectral image contains spectral images collected by any array lens; The pixel brightness values of the first spectral image and the second spectral image are standardized; Select part of the image from the second spectral image, and respectively compare the part of the image along at least two directions of the first spectral image to obtain a deviation value; Match the deviation value to the first spectral image to form a third spectral image, wherein the third spectral image is formed by combining the first spectral image and the deviation value.

3. The imaging method of claim 1, wherein, After the step of correcting the deviation of the plurality of corrected spectral images to obtain a corrected spectral image, the following steps are further included: According to a predetermined time, a plurality of channels of the spectral image at the time are obtained; Select an area that each of the plurality of spectral images can have from the plurality of spectral images to form a spectral curve graph; Based on the spectral curve graph, a color table of the area is obtained, wherein the color table represents the sum of each column of pixels in the area as the brightness of the column position in the spectral curve to represent the relative size.

4. The imaging method of claim 3, wherein, After the step of selecting an area that each of the plurality of spectral images can have from the plurality of spectral images to form a spectral curve graph, the following steps are further included: According to the selected region, determine three-dimensional data, wherein the three-dimensional data indicates the three-dimensional spectral matrix data of the region; Randomly select a plurality of first initial points in the three-dimensional data; Select data with a preset first distance from the first initial point to form a first generation unit region; In a plurality of first generation unit regions, select a second initial point; Select data with a preset second distance from the second initial point to form a second generation unit region; The pixel data contained in the second generation unit region is mapped to the three-color RGB value domain using a pseudo-color algorithm, and the pixel data is displayed.

5. The imaging method of claim 4, wherein, The step of displaying the pixel data contained in the second generation unit region using a pseudo-color algorithm to map the pixel data to the three-color domain further comprises: According to the second generation unit region, obtain a two-dimensional matrix, and mark the value of each pixel data of the two-dimensional matrix pair; Obtain the RGB value of the spectral image I(x, y) to increase the contrast of the spectral image, wherein the spectral image I satisfies the following formula: wherein m is k is a pixel data tag value, n is a value range of the tag of each pixel, and k ∈ [0, n].

6. A multi-spectral imaging system based on a camera array, characterized in that, The imaging method of the multi-spectral imaging system based on the camera array applied to one of claims 1-5, comprising: Camera module, including fixed focus array lens; Steering control module, used to drive the camera module to steer; Raspberry Pi control component, electrically connected with the camera module; Computer system, used to process the information collected by the Raspberry Pi control component to obtain an image; Data transmission module, used to connect the camera module and the computer system respectively for information transmission; wherein The array lens includes at least one monitoring lens and a plurality of acquisition lenses, and the plurality of acquisition lenses are used to acquire spectral images, each acquisition lens is provided with a filter, and at least part of the plurality of filters allows specified wavelengths to pass through, so that the computer system acquires the spectral images of corresponding wavelengths through the filter lens.

7. The camera array based multispectral imaging system of claim 6, wherein, The camera module includes a positioning plate, and the array lens is distributed on the positioning plate.

8. The camera array based multispectral imaging system of claim 6, wherein, The steering control module includes at least two drive motors, one of which drives the camera module to rotate horizontally, and the other drives the camera module to rotate vertically.

9. The camera array based multispectral imaging system of claim 6, wherein, The Raspberry Pi control component is connected with the acquisition lens respectively, and the information collected by each acquisition lens is stored independently in the Raspberry Pi control component.

Images