Multispectral image generation method, terminal device and computer-readable storage medium

Abstract

The present application is applicable to the field of image processing technology and provides a multispectral image generation method, terminal device, and computer-readable storage medium, including: acquiring a detection image; performing spatial classification based on the spectral similarity of the detection image to determine similar regions; reconstructing a reconstructed multispectral curve for the similar region based on the detection average value of the similar region; and correcting the reconstructed multispectral curve based on the reconstructed multispectral curve of the similar region and the detection image to obtain a target multispectral image of the similar region. By determining similar regions in the detection image and reconstructing the similar regions once to obtain the reconstructed multispectral curve corresponding to the similar regions, the amount of calculation can be effectively reduced. Then, the reconstructed multispectral curve is corrected based on the detection image, effectively improving the accuracy of the generated multispectral image.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular relates to a multispectral image generation method, a terminal device, and a computer-readable storage medium. Background Art

[0002] A multispectral image is an image formed when an object reflects and transmits light across multiple wavelengths. Traditional multispectral imaging systems utilize devices such as gratings to perform spectroscopic scanning to produce multispectral images. However, these systems suffer from complex architectures and high costs. To reduce costs and simplify the system, a multispectral imaging system based on LED lighting has been proposed. This system requires no moving scanning parts, facilitates wavelength switching, and avoids the phenomenon of heterochromatic color shift.

[0003] Current multispectral imaging systems based on LED illumination typically utilize a few common wavelengths of LED illumination. These systems sequentially capture images under different LED illumination conditions and then combine them with spectral reconstruction algorithms to generate reflectance spectra from a wider range of wavelengths. While these systems are simple and easy to implement, the spectral reconstruction process is computationally complex and requires significant computational resources and time, leading to low imaging efficiency and poor real-time performance. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide a multispectral image generation method, a terminal device, and a computer-readable storage medium to solve the problem that the current spectral reconstruction process requires a large amount of computing resources and time, resulting in low imaging efficiency and poor real-time performance.

[0005] In a first aspect, an embodiment of the present application provides a multispectral image generation method, comprising:

[0006] Acquire detection images;

[0007] Perform spatial classification based on the spectral similarity of the detected images to determine similar areas;

[0008] reconstructing a reconstructed multispectral curve of the similar area according to the detection average value of the similar area;

[0009] The reconstructed multispectral curve is corrected according to the reconstructed multispectral curve and the detection image of the similar area to obtain a target multispectral image of the similar area.

[0010] Optionally, performing spatial classification based on spectral similarity of the detected images to determine similar regions includes:

[0011] Setting a reference point and obtaining a reference point detection vector of the reference point;

[0012] Obtaining detection vectors of all pixels in the detection image except the reference point;

[0013] Calculating the cosine similarity between the detection vector of each pixel point and the detection vector of the reference point;

[0014] All pixels of the detection image are traversed, and when the cosine similarity between the detection vector of the pixel point and the detection vector of the reference point is less than a preset similarity threshold, the pixel point is determined to be a pixel point in a similar area to the reference point.

[0015] Optionally, performing spatial classification based on spectral similarity of the detected images to determine similar regions further includes:

[0016] If there are remaining pixels outside the similar area, the reference point is reset from the remaining pixels;

[0017] Obtain a reference point detection vector of the reset reference point;

[0018] Calculating the cosine similarity between the detection vectors of the remaining pixel points and the reference point detection vector of the reset reference point;

[0019] The remaining pixel points are traversed, and when the cosine similarity between the detection vector of the remaining pixel point and the reference point detection vector of the reset reference point is less than a preset similarity threshold, the pixel point is determined to be a pixel point in a similar area to the reset reference point.

[0020] Optionally, reconstructing the reconstructed multispectral curve of the similar area according to the detection average value of the similar area includes:

[0021] Calculate the detection average of similar regions;

[0022] The similar area is reconstructed according to the detection average value to obtain a reconstructed multispectral curve of the similar area.

[0023] Optionally, the correcting the reconstructed multispectral curve according to the reconstructed multispectral curve and the detection image of the similar area to obtain the target multispectral image of the similar area includes:

[0024] Determining the reconstructed spectrum value of each point in the similar area according to the reconstructed multispectral curve of the similar area;

[0025] Determine the corresponding actual detection value according to the detection image;

[0026] The reconstructed spectrum curve is corrected according to the reconstructed spectrum value of each point and the corresponding actual detection value.

[0027] Optionally, the correcting the reconstructed spectrum curve according to the reconstructed spectrum value of each point and the corresponding actual detection value includes:

[0028] Calculate the calculated detection value of each point according to the reconstructed spectrum value of each point;

[0029] The reconstructed spectrum curve is linearly segmented corrected according to the ratio of the calculated detection value to the actual detection value at each point.

[0030] Optionally, if there are multiple similar areas in the detection image, the reconstructed multispectral curve is corrected according to the reconstructed multispectral curve of the similar areas and the detection image to obtain the target multispectral image of the similar areas, including:

[0031] The reconstructed multispectral curve of each similar area is corrected to obtain a target multispectral curve corresponding to the detection image.

[0032] In a second aspect, an embodiment of the present application provides a terminal device, including:

[0033] An acquisition module, used for acquiring a detection image;

[0034] Partitioning module, used to perform spatial classification based on the spectral similarity of the detection image and determine similar areas;

[0035] A reconstruction module, configured to reconstruct a reconstructed multispectral curve of the similar area according to the detection average value of the similar area;

[0036] The correction module is used to correct the reconstructed multispectral curve according to the reconstructed multispectral curve and the detection image of the similar area to obtain a target multispectral image of the similar area.

[0037] In a third aspect, an embodiment of the present application provides a terminal device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method described in the first aspect or any optional method of the first aspect is implemented.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect or any optional method of the first aspect is implemented.

[0039] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the method described in the first aspect or any optional method of the first aspect.

[0040] The multispectral image generation method, terminal device, computer-readable storage medium, and computer program product provided by the embodiments of the present application have the following beneficial effects:

[0041] By determining the similar areas in the detection image and reconstructing the similar areas once to obtain the reconstructed multispectral curve corresponding to the similar areas, the amount of calculation can be effectively reduced. The reconstructed multispectral curve is then corrected based on the detection image to effectively improve the accuracy of the generated multispectral image. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0043] Figure 1 This is a structural diagram of a multispectral imaging system based on LED lighting provided in an embodiment of the present application;

[0044] Figure 2 This is a schematic flow chart of a multispectral image generation method provided in an embodiment of the present application;

[0045] Figure 3 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0046] Figure 4 This is a schematic structural diagram of a terminal device provided in another embodiment of the present application;

[0047] Figure 5 It is a structural diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0049] It should be understood that the term "and / or" used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations. In addition, in the description of this specification and the appended claims, the terms "first," "second," "third," etc. are used only to distinguish descriptions and are not to be understood as indicating or implying relative importance.

[0050] It should also be understood that references to "one embodiment" or "some embodiments" in the present specification mean that one or more embodiments of the present application include a particular feature, structure, or characteristic described in conjunction with that embodiment. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0051] See also Figure 1 , Figure 1 The schematic diagram of the structure of the multi-spectral imaging system based on LED lighting is shown in FIG. Figure 1 As shown, the above-mentioned multispectral imaging system based on LED lighting may include a multi-band LED lighting source and a detector.

[0052] LED lighting sources usually use several LED bands to illuminate the detected object. The detector will collect the detection images reflected / transmitted by different LED bands on the detected object. The multiple detection images collected by the detector (after each LED band is irradiated on the detected object, the detector will collect an image) are spectrally reconstructed to obtain a multispectral image (for example, if the number of LED bands of the LED lighting source is N, the number of bands of the obtained multispectral image is L, where N and L are positive numbers and N is a positive number). <L)。

[0053] In a specific application, the detector may be a video camera or a still camera that can capture spectral images.

[0054] Current spectral reconstruction methods include pseudo-inverse method, Wiener estimation method, principal component analysis, polynomial regression, machine learning, etc. These methods can improve the accuracy of reconstructed multispectral images, but they will make the overall algorithm more complicated and require more computation.

[0055] For example, after the detection light of the LED lighting source is reflected / transmitted by the detected object, the detection image collected by the detector can be expressed as:

[0056]

[0057] Among them, I is the detection value collected by the detector (that is, the value on the detection image), S λ is the reflection spectrum of the detected object (i.e. the multispectral image to be reconstructed), C λ is the spectral response of the detector (which can be measured by standard test instruments), P λ It is the spectral response of the LED light source (which can be measured by standard test instruments).

[0058] Since the spectral reflectance is usually relatively smooth, the principal component analysis method can be used for reconstruction. Therefore, the spectrum to be measured (i.e. the reflectance spectrum of the detected object) can be expressed as:

[0059]

[0060] Among them, σ k is the characteristic coefficient, b k (λ) is the trained feature vector.

[0061] Combining formula (1) and formula (2), we can get:

[0062]

[0063] Since the spectral values ​​are all positive, a constraint is added: Written in matrix form, A σ >0. Considering the smoothness of the spectrum, increase the second derivative of the spectrum The optimal characteristic coefficient σ can be calculated k , the calculation formula is as follows:

[0064] argmin σ [‖F σ -I‖2 2 +α‖P σ ‖ 2 ], A σ >0; (4)

[0065] The reconstructed multispectral image can be obtained by formula (2).

[0066] It should be noted that the above reconstruction process is to reconstruct the spectrum of a pixel point on the image, and the multispectral curve of the pixel point can be obtained. To obtain the entire multispectral image, each pixel point needs to be processed once. If the number of pixels is large, the overall calculation amount will be very large, which will require a lot of computing resources and time, resulting in low imaging efficiency and poor real-time performance.

[0067] In order to solve the above problems, an embodiment of the present application proposes a multispectral image generation method. By determining similar areas in the detection image and reconstructing the similar areas once to obtain a reconstructed multispectral curve corresponding to the similar areas, the amount of calculation can be effectively reduced. The reconstructed multispectral curve is then corrected based on the detection image, effectively improving the accuracy of the generated multispectral image.

[0068] The following is a detailed description of the multispectral image generation method, terminal device, and computer-readable storage medium provided in the embodiments of the present application:

[0069] See also Figure 2 , Figure 2 This is a schematic flow chart of a multispectral image generation method provided in an embodiment of the present application. In this embodiment of the present application, the multispectral image generation method can be executed by a terminal device. Such a terminal device includes, but is not limited to, a mobile phone, tablet computer, desktop computer, server, or other computing device.

[0070] It should be noted that the multispectral image generation method provided in the embodiment of the present application is to generate a multispectral image by Figure 1 The LED illumination light source of the illustrated LED-based multispectral imaging system uses N bands of LED light to illuminate the detected object. A detector collects the detection images formed by the reflection of each band of LED light on the detected object, and reconstructs a multispectral image of L bands based on the detection images. Unlike current multispectral image generation methods, the embodiments of the present application improve the process of reconstructing the L bands of multispectral images based on the detection images, effectively increasing the computational speed of the reconstruction process and improving the efficiency of multispectral image generation.

[0071] Specifically, if Figure 2 As shown, the multispectral image generation method may include S11 to S14, which are described in detail as follows:

[0072] S11: Acquire a detection image.

[0073] In the embodiment of the present application, the detection image refers to an image captured by a detector after an LED illumination light source uses N bands of LED light to illuminate the detected object. The number of the detection images is N, where N is a positive integer. Specifically, N is a positive integer greater than 1.

[0074] In an embodiment of the present application, the detector can transmit the collected detection image to the terminal device in real time, so that the terminal device can obtain the detection image in real time. Of course, after the detector collects the detection image, it can first store the collected detection image in the detector's memory. When it detects an image upload instruction issued by the terminal device, it can then transmit the detection image stored in the detector's memory to the terminal device so that the terminal device can obtain the detection image.

[0075] In each detection image, each pixel has a corresponding detection value (referred to herein as a measured value).

[0076] S12: Perform spatial classification based on the spectral similarity of the detected images to determine similar areas.

[0077] In the embodiment of the present application, a reference point is selected, and N detection values ​​corresponding to the reference point in N detection images form a reference point detection vector I ref , when the detection vector I of any point in the detection image p (that is, the vector composed of the measured values ​​of the pixels in each detection image) and the reference point detection vector I ref When the cosine similarity of the point is less than the preset similarity threshold, the point is determined as a similar point of the reference point and is included in the similar area of ​​the reference point.

[0078] In the embodiment of the present application, the detection vector I is calculated p Detection vector I with reference point ref The calculation formula of cosine similarity is as follows:

[0079]

[0080] Among them, sim refers to the detection vector I p Detection vector I with reference point ref The cosine similarity of I ref is the reference point detection vector, I p is the detection vector of any point in the detection image.

[0081] It should be noted that the above-mentioned preset similarity threshold can be set based on the reconstruction accuracy and computational efficiency.

[0082] In an embodiment of the present application, the above S12 may include the following steps:

[0083] Setting a reference point and obtaining a reference point detection vector of the reference point;

[0084] Obtaining detection vectors of all pixels in the detection image except the reference point;

[0085] Calculating the cosine similarity between the detection vector of each pixel point and the detection vector of the reference point;

[0086] All pixels of the detection image are traversed, and when the cosine similarity between the detection vector of the pixel point and the detection vector of the reference point is less than a preset similarity threshold, the pixel point is determined to be a pixel point in a similar area to the reference point.

[0087] In another embodiment of the present application, the above S12 may further include the following steps:

[0088] If there are remaining pixels outside the similar area, the reference point is reset from the remaining pixels;

[0089] Obtain a reference point detection vector of the reset reference point;

[0090] Calculating the cosine similarity between the detection vectors of the remaining pixel points and the reference point detection vector of the reset reference point;

[0091] The remaining pixel points are traversed, and when the cosine similarity between the detection vector of the remaining pixel point and the reference point detection vector of the reset reference point is less than a preset similarity threshold, the pixel point is determined to be a pixel point in a similar area to the reset reference point.

[0092] In an embodiment of the present application, there may be multiple similar areas in the detection image. After determining a similar area, another reference point can be set in the remaining area, and then the similar areas of this reference point are determined by traversing the points except those determined as in the previous similar area, until the partitioning of the similar areas is completed for each pixel point in the detection image.

[0093] It should be noted that after determining a similar area, the terminal device can first execute subsequent steps S13 to S14 on the similar area, and then determine whether there are other similar areas; or it can first determine all similar areas in the detection image, and then execute steps S13 to S14 for each similar area. This application does not impose any restrictions on this.

[0094] S13: reconstructing a reconstructed multispectral curve of the similar area according to the detection average value of the similar area.

[0095] In an embodiment of the present application, for each similar area, it is necessary to determine its detection average value, which is the average value of the detection value of each point in the similar area. Then, the similar area is reconstructed according to this detection average value, that is, each point in the similar area will use the corresponding multispectral curve after reconstruction, that is, a similar area only needs to be reconstructed once, which effectively reduces the amount of calculation.

[0096] In one embodiment of the present application, the above S13 may include the following steps:

[0097] Calculate the detection average of similar regions;

[0098] The similar area is reconstructed according to the detection average value to obtain a reconstructed multispectral curve of the similar area.

[0099] After the similar area is determined, the detection average value of the similar area can be calculated.

[0100] Specifically, assuming that the number of points in the similar area determined in S12 is num points, the average detection value of the similar area is calculated as: mean =(I ref +...+I num ) / num.

[0101] In this way, the characteristic coefficient σ of the similar area can be determined using the following optimization function opt :

[0102] argmin σ [‖F σ -I mean ‖2 2 +α‖P σ ‖ 2 ].

[0103] The reconstructed multispectral curve of the similar area can be obtained:

[0104]

[0105] Where S′ is the reconstructed spectrum value of the point in the similar area, b k (λ) is the trained feature vector.

[0106] In the embodiment of the present application, if there are multiple similar regions in the detection image, it is necessary to perform the same operation on each similar region to obtain a reconstructed multispectral curve for each similar region.

[0107] S14: Correcting the reconstructed multispectral curve according to the reconstructed multispectral curve and the detected image of the similar area to obtain a target multispectral image of the similar area.

[0108] In an embodiment of the present application, the reconstructed spectral value of each point in the similar area can be obtained based on the reconstructed multispectral curve of the similar area, the actual detection value of the point can be known based on the detection image, and the point is corrected based on the reconstructed spectral value and the actual detection value of the point. This is repeated until every point in the similar area is corrected, and the final target multispectral image can be obtained.

[0109] In specific applications, the calculated detection value of each point in the similar area can be inverted through the reconstructed multispectral curve of the similar area, and then the calculated detection value is compared with the actual detection value of the point, and based on this, the reconstructed spectrum value of the point is linearly corrected in segments.

[0110] Specifically, the corrected target multispectral curve can be expressed as:

[0111]

[0112] scale(t)=I p / I p ' t=1,2,3...,N

[0113] Among them, I p ' is the calculated detection value, I p is the actual detection value, S′(λ) is the reconstructed spectral curve of the points in the similar area, and S(λ) is the corrected target multispectral curve.

[0114] It should be noted that the linear segmentation L1, L2, ..., L N This is related to the spectral response of the N LED light sources. The spectral response of LEDs is usually distributed in different bands. Near the response range, the corresponding scale value is used for correction. For example, assuming that the spectral response range of the first LED is 400-450nm, the position of L1 corresponds to 450nm. Based on this, the corresponding scale value can be determined.

[0115] It should be noted that if there are multiple similar areas in the detection image, then after obtaining the reconstructed multispectral curve of each similar area, the reconstructed multispectral curve needs to be corrected to obtain the target multispectral image of the similar area until the reconstructed multispectral curves of all similar areas are corrected.

[0116] In the embodiment of the present application, after the linear segmented correction is performed on the reconstructed spectral curve of the similar region, it can also be smoothed to avoid mutations.

[0117] It should be noted that the smoothing of the reconstructed spectrum curve after the linear segmentation correction can be achieved using an existing smoothing algorithm, which is not described in detail in this application.

[0118] In one embodiment of the present application, the above S14 may include the following steps:

[0119] Determining the reconstructed spectrum value of each point in the similar area according to the reconstructed multispectral curve of the similar area;

[0120] Determine the corresponding actual detection value according to the detection image;

[0121] The reconstructed spectrum curve is corrected according to the reconstructed spectrum value of each point and the corresponding actual detection value.

[0122] In the embodiment of the present application, after the reconstructed multispectral curve in each similar area is corrected, a reconstructed multispectral image corresponding to the entire detection image can be obtained, and a multispectral image corresponding to the detection image can be generated.

[0123] In the embodiment of the present application, the above-mentioned correction of the reconstructed spectrum curve according to the reconstructed spectrum value of each point and the corresponding actual detection value includes:

[0124] Calculate the calculated detection value of each point according to the reconstructed spectrum value of each point;

[0125] The reconstructed spectrum curve is linearly segmented corrected according to the ratio of the calculated detection value to the actual detection value at each point.

[0126] In order to further illustrate that the multispectral image generation method provided in the embodiment of the present application can ensure the spectral accuracy after imaging on the basis of improving imaging efficiency, the embodiment of the present application also verifies the effect of the multispectral image generation method provided in the embodiment of the present application through simulation experiments. Table 1 shows the correlation coefficient of reconstruction by the existing principal component analysis method, the correlation coefficient of the reconstructed multispectral curve obtained after partition optimization by the method provided in the embodiment of the present application, and the correlation coefficient of the target spectral image obtained after correction of the reconstructed multispectral curve by the method provided in the embodiment of the present application. Table 2 shows the cosine similarity of the image reconstructed by the existing principal component analysis method, the cosine similarity of the reconstructed multispectral curve obtained after partition optimization by the method provided in the embodiment of the present application, and the cosine similarity of the target spectral image obtained after correction of the reconstructed multispectral curve by the method provided in the embodiment of the present application. Among them, the correlation coefficient and cosine similarity are used to represent the difference between the reconstructed spectral curve and the original spectrum. The larger the correlation coefficient, the smaller the difference between the reconstructed spectral curve and the original spectrum, and the smaller the cosine similarity, the smaller the difference between the reconstructed spectral curve and the original spectrum.

[0127] Table 1:

[0128]

[0129] Table 2:

[0130]

[0131] As can be seen from Tables 1 and 2, as the preset similarity threshold increases, the number of similar points increases, the accuracy of the reconstructed spectral curve after partition optimization decreases, while the accuracy of the corrected target multispectral image improves. In addition, when the preset similarity threshold is set to 0.8, the spectral differences obtained by the partition optimization calculation are large, but the corrected spectral curve is significantly improved. Although the accuracy of the reconstructed multispectral curve is also good when the preset similarity threshold is large, as the preset similarity threshold increases, the correlation coefficient gradually decreases, the cosine similarity gradually increases, and the spectral accuracy still has a downward trend. Therefore, it is not recommended to set the preset similarity threshold too large.

[0132] To further illustrate that the multispectral image generation method provided in the embodiment of the present application can effectively improve computational efficiency and the efficiency of multispectral image generation, Table 3 shows the time consumption and image quality of reconstructing each pixel point based on the existing principal component analysis method obtained through simulation experiments, and the comparative data of the time consumption and image quality when reconstructing using the multispectral image generation method provided in the embodiment of the present application.

[0133] In the specific application, the above simulation experiment was programmed using Matlab R2014a and an Intel Core i5-6200U CPU @ 2.3GHz computer to reconstruct a 512*512 image. To measure image quality, the Peak Signal-to-Noise Ratio (PSNR) and spectral cosine similarity of the reconstructed image and the original image were compared. The PSNR compares the difference between each band and then takes the average value of all bands. A higher PSNR value indicates higher reconstructed image quality. The cosine similarity of the spectral curve is calculated by calculating the cosine angle of the spectral curve at the same spatial point and then averaging the different points. A smaller value indicates a closer reconstructed spectral curve to the original spectrum (i.e., higher reconstructed image quality).

[0134] Table 3:

[0135]

[0136] As shown in Table 3, the time taken to reconstruct each pixel based on the existing principal component analysis method is 1162.2s, while when the multispectral image generation method provided in the embodiment of the present application is used for reconstruction, when the preset similarity threshold is set to 0.1, the time taken is only 342s, which is 29.4% of the time taken by the method based on the existing principal component analysis method to reconstruct each pixel; when the preset similarity threshold is 0.3, the time taken is only 74.5s, which is 6.4% of the time taken by the method based on the existing principal component analysis method to reconstruct each pixel; when the preset similarity threshold is 0.5, the time taken is 64s, which is 5.5% of the time taken by the method based on the existing principal component analysis method to reconstruct each pixel. It can be seen that the multispectral image generation method improved by the present application can effectively reduce the calculation time of the multispectral image reconstruction process and improve the reconstruction efficiency. According to the data in Table 3, it can also be seen that the image quality of the multispectral image obtained by the multispectral image generation method provided in the embodiment of the present application is better than the image quality of the multispectral image obtained by reconstructing each pixel based on the existing principal component analysis method. In other words, the multispectral image generation method provided in the embodiment of the present application can not only improve the calculation speed of image reconstruction, improve imaging efficiency and real-time performance, but also improve the image quality of the generated multispectral image and improve image accuracy.

[0137] As can be seen from the above, the multispectral image generation method provided in the embodiment of the present application can effectively reduce the amount of calculation by determining the similar areas in the detection image and reconstructing the similar areas once to obtain the reconstructed multispectral curve corresponding to the similar areas. The reconstructed multispectral curve is then corrected based on the detection image, effectively improving the accuracy of the generated multispectral image.

[0138] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0139] Based on the multispectral image generation method provided in the above embodiment, the embodiment of the present invention further provides an embodiment of a terminal device that implements the above method embodiment.

[0140] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. In the embodiment of the present application, the terminal device includes various units for executing Figure 2 Each step in the corresponding embodiment. Please refer to Figure 2 as well as Figure 2 For the sake of convenience, only the parts related to this embodiment are shown. Figure 3As shown, the terminal device 30 includes: an acquisition module 31 , a partitioning module 32 , a reconstruction module 33 and a correction module 34 .

[0141] The acquisition module 31 is used to acquire the detection image.

[0142] The partitioning module 32 is used to perform spatial classification based on the spectral similarity of the detection images and determine similar areas.

[0143] The reconstruction module 33 is configured to reconstruct a reconstructed multispectral curve of the similar area according to the detection average value of the similar area.

[0144] The correction module 34 is configured to correct the reconstructed multispectral curve according to the reconstructed multispectral curve and the detection image of the similar area to obtain a target multispectral image of the similar area.

[0145] Optionally, the partitioning unit 32 is specifically configured to:

[0146] Setting a reference point and obtaining a reference point detection vector of the reference point;

[0147] Obtaining detection vectors of all pixels in the detection image except the reference point;

[0148] Calculating the cosine similarity between the detection vector of each pixel point and the detection vector of the reference point;

[0149] All pixels of the detection image are traversed, and when the cosine similarity between the detection vector of the pixel point and the detection vector of the reference point is less than a preset similarity threshold, the pixel point is determined to be a pixel point in a similar area to the reference point.

[0150] Optionally, the partitioning unit 32 is further configured to:

[0151] The performing spatial classification based on the spectral similarity of the detected images to determine similar areas further includes:

[0152] If there are remaining pixels outside the similar area, the reference point is reset from the remaining pixels;

[0153] Obtain a reference point detection vector of the reset reference point;

[0154] Calculating the cosine similarity between the detection vectors of the remaining pixel points and the reference point detection vector of the reset reference point;

[0155] The remaining pixel points are traversed, and when the cosine similarity between the detection vector of the remaining pixel point and the reference point detection vector of the reset reference point is less than a preset similarity threshold, the pixel point is determined to be a pixel point in a similar area to the reset reference point.

[0156] Optionally, the reconstruction module 33 is specifically configured to:

[0157] Calculate the detection average of similar regions;

[0158] The similar area is reconstructed according to the detection average value to obtain a reconstructed multispectral curve of the similar area.

[0159] Optionally, the correction module 34 is specifically configured to:

[0160] Determining the reconstructed spectrum value of each point in the similar area according to the reconstructed multispectral curve of the similar area;

[0161] Determine the corresponding actual detection value according to the detection image;

[0162] The reconstructed spectrum curve is corrected according to the reconstructed spectrum value of each point and the corresponding actual detection value.

[0163] It should be noted that the information interaction, execution process, etc. between the above-mentioned modules / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0164] Figure 4 This is a schematic diagram of the structure of a terminal device provided by another embodiment of the present application. Figure 4 As shown, the terminal device 4 provided in this embodiment includes: a processor 40, a memory 41, and a computer program 42 stored in the memory 41 and executable on the processor 40, such as a program for cooperative control of a multi-agent system. When the processor 40 executes the computer program 42, the steps in the above-mentioned embodiments of the multispectral image generation method are implemented, such as Figure 2 Alternatively, when the processor 40 executes the computer program 42, the functions of the modules / units in the above-mentioned terminal device embodiments are realized, for example, Figure 3 The functions of units 31 to 34 are shown.

[0165] For example, the computer program 42 may be divided into one or more modules / units, which are stored in the memory 41 and executed by the processor 40 to complete the present application. The one or more modules / units may be a series of computer program instruction segments that can perform specific functions, and the instruction segments are used to describe the execution process of the computer program 42 in the terminal device 4. For example, the computer program 42 may be divided into various units / modules, and the specific functions of each unit / module may be described in detail in the following sections. Figure 3 The relevant descriptions in the corresponding embodiments are not repeated here.

[0166] The terminal device may include but is not limited to a processor 40 and a memory 41. Those skilled in the art will understand that Figure 4 It is only an example of the terminal device 4 and does not constitute a limitation on the terminal device 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0167] The processor 40 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0168] The memory 41 may be an internal storage unit of the terminal device 4, such as a hard disk or memory of the terminal device 4. The memory 41 may also be an external storage device of the terminal device 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 4. Furthermore, the memory 41 may include both an internal storage unit of the terminal device 4 and an external storage device. The memory 41 is used to store the computer program and other programs and data required by the terminal device. The memory 41 may also be used to temporarily store data that has been output or is about to be output.

[0169] The present application also provides a computer-readable storage medium. Figure 5 , Figure 5 is a schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present application, such as Figure 5 As shown, a computer program 51 is stored in the computer-readable storage medium 5. When the computer program 51 is executed by a processor, the multispectral image generation method described above can be implemented.

[0170] An embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the above-mentioned multispectral image generation method when executing the computer program product.

[0171] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the terminal device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0172] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0173] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0174] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A multispectral image generation method, characterized in that: include: Acquire detection images; The detection image refers to an image captured by a detector when an LED illumination light source uses LED light of N bands to illuminate the detected object, and the number of detection images is N, where N is a positive integer greater than 1; Perform spatial classification based on the spectral similarity of the detection images to determine similar regions; wherein the cosine similarity between the detection vector of a point in the similar region and the detection vector of a reference point in the similar region is less than a preset similarity threshold, and the detection vector is a vector composed of the measured values ​​of the pixel points in each detection image; and the reference point detection vector is composed of N detection values ​​corresponding to the reference point in N detection images; reconstructing a reconstructed multispectral curve of the similar area according to the detection average value of the similar area; Correcting the reconstructed multispectral curve according to the reconstructed multispectral curve and the detection image of the similar area to obtain a target multispectral image of the similar area; The spatial classification based on the spectral similarity of the detected images to determine similar areas includes: Setting a reference point and obtaining a reference point detection vector of the reference point; Obtaining detection vectors of all pixels in the detection image except the reference point; Calculating the cosine similarity between the detection vector of each pixel point and the detection vector of the reference point; All pixels of the detection image are traversed, and when the cosine similarity between the detection vector of the pixel point and the detection vector of the reference point is less than a preset similarity threshold, the pixel point is determined to be a pixel point in a similar area to the reference point.

2. The multispectral image generation method according to claim 1, wherein: The performing spatial classification based on the spectral similarity of the detected images to determine similar areas further includes: If there are remaining pixels outside the similar area, the reference point is reset from the remaining pixels; Obtain a reference point detection vector of the reset reference point; Calculating the cosine similarity between the detection vectors of the remaining pixel points and the reference point detection vector of the reset reference point; The remaining pixel points are traversed, and when the cosine similarity between the detection vector of the remaining pixel point and the reference point detection vector of the reset reference point is less than a preset similarity threshold, the pixel point is determined to be a pixel point in a similar area to the reset reference point.

3. The multispectral image generation method according to claim 1, characterized in that: The reconstructing the multispectral curve of the similar area according to the detection average value of the similar area includes: Calculate the detection average of similar regions; The similar area is reconstructed according to the detection average value to obtain a reconstructed multispectral curve of the similar area.

4. The multispectral image generation method according to claim 1, wherein: The step of correcting the reconstructed multispectral curve according to the reconstructed multispectral curve and the detection image of the similar area to obtain a target multispectral image of the similar area includes: Determining the reconstructed spectrum value of each point in the similar area according to the reconstructed multispectral curve of the similar area; Determine the corresponding actual detection value according to the detection image; The reconstructed spectrum curve is corrected according to the reconstructed spectrum value of each point and the corresponding actual detection value.

5. The multispectral image generation method according to claim 1, characterized in that: The reconstructed spectrum curve is corrected according to the reconstructed spectrum value of each point and the corresponding actual detection value, including: Calculate the calculated detection value of each point according to the reconstructed spectrum value of each point; The reconstructed spectrum curve is linearly segmented corrected according to the ratio of the calculated detection value to the actual detection value at each point.

6. The multispectral image generation method according to any one of claims 1 to 5, characterized in that: If there are multiple similar areas in the detection image, the reconstructed multispectral curve is corrected according to the reconstructed multispectral curve of the similar areas and the detection image to obtain a target multispectral image of the similar areas, including: The reconstructed multispectral curve of each similar area is corrected to obtain a target multispectral curve corresponding to the detection image.

7. A terminal device, characterized in that: include: An acquisition module, used for acquiring a detection image; The detection image refers to an image captured by a detector when an LED illumination light source uses LED light of N bands to illuminate the detected object, and the number of detection images is N, where N is a positive integer greater than 1; A partitioning module is configured to perform spatial classification based on spectral similarity of the detection images and determine similar regions; wherein the cosine similarity between the detection vector of a point in the similar region and the detection vector of a reference point in the similar region is less than a preset similarity threshold, and the detection vector is a vector composed of the measured values ​​of the pixels in each detection image; and the reference point detection vector is composed of the N detection values ​​corresponding to the reference point in the N detection images; A reconstruction module, configured to reconstruct a reconstructed multispectral curve of the similar area according to the detection average value of the similar area; a correction module, configured to correct the reconstructed multispectral curve according to the reconstructed multispectral curve and the detection image of the similar area to obtain a target multispectral image of the similar area; The partitioning module is used to set a reference point and obtain a reference point detection vector of the reference point; Obtaining detection vectors of all pixels in the detection image except the reference point; Calculating the cosine similarity between the detection vector of each pixel point and the detection vector of the reference point; All pixels of the detection image are traversed, and when the cosine similarity between the detection vector of the pixel point and the detection vector of the reference point is less than a preset similarity threshold, the pixel point is determined to be a pixel point in a similar area to the reference point.

8. A terminal device, characterized in that: The terminal device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images