Calibration method and device of spectral camera, equipment and storage medium

Abstract

This application discloses a calibration method applied to an electronic device. The electronic device is equipped with a multispectral camera and sends a module type identifier corresponding to the multispectral camera; receives model data corresponding to the module type identifier; determines a calibration model corresponding to the multispectral camera based on the model data; wherein, the calibration model corresponding to the multispectral camera is used to calibrate the multispectral camera; and obtains the calibrated performance parameters of the multispectral camera based on the first image data acquired by the multispectral camera through the calibration model.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a calibration method, apparatus, device, and storage medium for a spectral camera. Background Technology

[0002] Multispectral photography refers to extending beyond visible light into infrared and ultraviolet light, and by combining various filters or spectrometers with multiple types of photosensitive film, it allows the same target to simultaneously receive information radiated or reflected in different narrow spectral bands, thus obtaining several photographs of the target in different spectral bands.

[0003] Given the significant differences between individual multispectral camera hardware components, calibration is necessary to optimize hardware consistency. However, current common calibration schemes primarily perform differential calibration on hardware characteristics according to uniform requirements and standards, failing to address potential interference factors that may exist in different types of modules. Summary of the Invention

[0004] This application provides a calibration method, apparatus, device, and storage medium for a multispectral camera. It can perform calibration processing on possible interference factors of the module of a certain type by using a calibration model corresponding to the type of multispectral camera. While solving the multivariate fusion problem in the calibration process, it reduces the complexity of the process and improves the accuracy and efficiency of calibration.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a calibration method for a spectral camera, applied to an electronic device, wherein the electronic device is equipped with a multispectral camera, and the method includes:

[0007] Send the module type identifier corresponding to the multispectral camera;

[0008] Receive the model data corresponding to the module type identifier;

[0009] The calibration model corresponding to the multispectral camera is determined based on the model data; wherein, the calibration model corresponding to the multispectral camera is used to calibrate the multispectral camera;

[0010] Based on the first image data acquired by the multispectral camera, the calibration model is used to obtain the calibrated performance parameters of the multispectral camera.

[0011] Secondly, embodiments of this application provide a calibration device for a spectroscopic camera, the calibration device for the spectroscopic camera comprising:

[0012] A transmitting unit is used to transmit the module type identifier corresponding to the multispectral camera;

[0013] The receiving unit is used to receive the model data corresponding to the module type identifier;

[0014] A determining unit is used to determine the calibration model corresponding to the multispectral camera based on the model data; wherein, the calibration model corresponding to the multispectral camera is used to perform calibration processing on the multispectral camera;

[0015] The calibration unit is used to obtain the calibrated performance parameters of the multispectral camera based on the first image data acquired by the multispectral camera using the calibration model.

[0016] Thirdly, embodiments of this application provide an electronic device, which includes a processor and a memory storing processor-executable instructions, wherein when the instructions are executed by the processor, the method as described in the first aspect is implemented.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the method as described in the first aspect.

[0018] This application provides a calibration method, apparatus, device, and storage medium for a multispectral camera, applied to an electronic device. The electronic device is equipped with a multispectral camera and sends a module type identifier corresponding to the multispectral camera; receives model data corresponding to the module type identifier; and determines a calibration model corresponding to the multispectral camera based on the model data. The calibration model corresponding to the multispectral camera is used to perform calibration processing on the multispectral camera. Through the calibration model, based on the first image data acquired by the multispectral camera, the calibrated performance parameters of the multispectral camera are obtained. In other words, in this application, the obtained calibration model corresponding to the module of the multispectral camera can be used to perform the calibration processing of the multispectral camera, and the corresponding calibrated performance parameters can be obtained for the multispectral camera. Therefore, in this application, on the one hand, the calibration model is corresponding to the multispectral camera, which can obtain a better calibration effect for the multispectral camera; on the other hand, by performing multispectral camera calibration processing through the calibration model, the multivariate fusion problem in the calibration process can be solved by computing power, and the calibration of possible interference factors can be achieved. In summary, the embodiments of this application can perform calibration processing on possible interference factors of a module of a certain type by using a calibration model corresponding to the type of multispectral camera. While solving the multivariate fusion problem in the calibration process, it reduces the complexity of the process and improves the accuracy and efficiency of calibration. Attached Figure Description

[0019] Figure 1 This is a calibration diagram for common hardware characteristics;

[0020] Figure 2 This is a schematic diagram illustrating the implementation process of the calibration method for the spectral camera proposed in this application embodiment;

[0021] Figure 3 This is a schematic diagram illustrating the implementation of the calibration method for the spectral camera proposed in this application embodiment;

[0022] Figure 4 This is a schematic diagram illustrating the implementation process of the training method for the calibration model proposed in this application embodiment;

[0023] Figure 5 This is a schematic diagram illustrating the implementation of the training method for the calibration model proposed in this application embodiment;

[0024] Figure 6 This is a schematic diagram illustrating the implementation flow of the data processing method proposed in the embodiments of this application;

[0025] Figure 7 This is a schematic diagram illustrating the implementation of the data processing method proposed in the embodiments of this application;

[0026] Figure 8 This is a schematic diagram of the data acquisition method proposed in the embodiments of this application;

[0027] Figure 9 This is a schematic diagram of the training method for the calibration model proposed in the embodiments of this application;

[0028] Figure 10 This is a schematic diagram of the data processing method proposed in an embodiment of this application;

[0029] Figure 11 This is a schematic diagram of the production line whole machine testing method proposed in the embodiments of this application;

[0030] Figure 12 This is a schematic diagram of the model data synchronization method proposed in the embodiments of this application;

[0031] Figure 13 This is a schematic diagram of the calibration method proposed in the embodiments of this application;

[0032] Figure 14 This is a schematic diagram of the composition of the calibration device for the spectral camera proposed in the embodiments of this application;

[0033] Figure 15 This is a schematic diagram of the composition of the training device for the calibration model proposed in the embodiments of this application;

[0034] Figure 16 This is a schematic diagram of the composition structure of the data processing device proposed in the embodiments of this application;

[0035] Figure 17 This is a schematic diagram of the composition structure of the electronic device proposed in the embodiments of this application. Detailed Implementation

[0036] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining the differences between the applications and are not intended to limit the application. Furthermore, it should be noted that, for ease of description, only the parts that differ from the relevant applications are shown in the accompanying drawings.

[0037] Multispectral cameras were developed based on ordinary aerial cameras. They are able to capture information in a wider spectral range, including bands beyond visible light, such as infrared and ultraviolet light.

[0038] The primary purpose of multispectral cameras is to achieve high-precision spectral reconstruction. Traditional solutions employ prism-based spectral dispersion, which offers good hardware stability. However, in mobile devices, miniaturization requirements often necessitate a sensor-based coating technology approach. Whether using resin-dye, multilayer film, or metasurface solutions, the hardware components vary significantly, necessitating calibration to optimize hardware consistency.

[0039] Currently, common calibration schemes mainly focus on calibrating differences based on hardware characteristics. Figure 1 A calibration diagram for common hardware characteristics, such as Figure 1 As shown, hardware calibration can generally include sensor consistency, spectral coating consistency, optical consistency, module assembly consistency, and overall system consistency.

[0040] However, this method of differentiating based on hardware characteristics cannot take all interference factors into account. If all interference factors were taken into account, the calibration process would be very complex, and the calibration modeling specifications would be huge, making it difficult to meet the needs of large-scale mass production.

[0041] In other words, current common calibration schemes for multispectral cameras cannot handle all interference factors.

[0042] To address the aforementioned issues, this application provides a calibration method, apparatus, device, and storage medium for a multispectral camera, applied to an electronic device. The electronic device is equipped with a multispectral camera and sends a module type identifier corresponding to the multispectral camera; receives model data corresponding to the module type identifier; and determines a calibration model corresponding to the multispectral camera based on the model data. The calibration model corresponding to the multispectral camera is used to perform calibration processing on the multispectral camera. Through the calibration model, based on the first image data acquired by the multispectral camera, the calibrated performance parameters of the multispectral camera are obtained. In other words, in the embodiments of this application, the obtained calibration model corresponding to the module of the multispectral camera can be used to perform the calibration processing of the multispectral camera, and for the multispectral camera, the corresponding calibrated performance parameters can be obtained. Therefore, in this application, on the one hand, the calibration model corresponds to the multispectral camera, enabling better calibration results for the multispectral camera; on the other hand, performing multispectral camera calibration processing through the calibration model can solve the multivariate fusion problem in the calibration process through computing power, achieving calibration against potential interference factors. In summary, the embodiments of this application can perform calibration processing on possible interference factors of a module of a certain type by using a calibration model corresponding to the type of multispectral camera. While solving the multivariate fusion problem in the calibration process, it reduces the complexity of the process and improves the accuracy and efficiency of calibration.

[0043] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0044] One embodiment of this application provides a calibration method for a spectroscopic camera. This calibration method can be applied to a calibration device or electronic device for a spectroscopic camera, and can also be applied to any terminal that includes a calibration device or electronic device for a spectroscopic camera.

[0045] The calibration method for a spectral camera proposed in this application will be described below using an electronic device as an example.

[0046] Furthermore, in the embodiments of this application, Figure 2 This is a schematic diagram illustrating the implementation process of the spectral camera calibration method proposed in this application embodiment, as shown below. Figure 2 As shown, the calibration method for a spectroscopic camera may include the following steps:

[0047] Step 101: Send the module type identifier corresponding to the multispectral camera.

[0048] In embodiments of this application, the electronic device may send a module type identifier corresponding to the multispectral camera.

[0049] Furthermore, in embodiments of this application, the electronic device may be configured with a multispectral camera, wherein the multispectral camera can be used to capture image information in different spectral bands.

[0050] Multispectral cameras can be classified into the following categories based on their structure and spectral dispersion methods:

[0051] Multi-lens multispectral cameras: These have 4-9 lenses, each with its own filter, allowing a narrower spectral band of light to pass through. Multiple lenses simultaneously capture the same scene, recording image information from several different spectral bands on a single film.

[0052] Multi-camera multispectral camera: It consists of several cameras combined together, each with a different filter, receiving information from different spectral bands of the scene, and simultaneously photographing the same scene, each obtaining a set of film with a specific spectral band.

[0053] Beam-splitting multispectral camera: It uses a single lens to photograph a scene, and multiple prism beam splitters to separate the light from the scene into beams of several wavelengths. Multiple sets of film are used to record the light information of each wavelength band separately.

[0054] In addition, depending on the method of beam splitting, multispectral cameras can also be divided into prism beam splitting structure, filter wheel structure, and filter beam splitting structure.

[0055] Furthermore, in embodiments of this application, the electronic device may send a module type identifier corresponding to the multispectral camera to a device or apparatus storing model data.

[0056] For example, in some embodiments, when the electronic device is mounted on a production line calibration fixture, the electronic device can send a module type identifier corresponding to the multispectral camera to the production line machine.

[0057] For example, in some embodiments, the calibration process performed by the electronic device for the multispectral camera may be completed during the overall inspection process. During overall inspection, the electronic device may be mounted in a production line calibration fixture.

[0058] Accordingly, in the embodiments of this application, when it is determined that the device is loaded onto the production line calibration fixture, the electronic device can send the module type identifier corresponding to the multispectral camera to the production line machine, thereby receiving the model data corresponding to the module type identifier corresponding to the multispectral camera sent by the production line machine.

[0059] Production line machines are the equipment on the production line, primarily used to complete processing, assembly, and other tasks. Their working principle relies heavily on automated control technology. Products are transported to the machines via conveyor belts or other conveyor equipment, where they operate according to pre-set programs to complete processing, assembly, and other tasks. These machines are typically equipped with various sensors and monitoring systems that can monitor various parameters during the production process in real time, ensuring stable machine operation and product quality.

[0060] It is understood that, in the embodiments of this application, the module type identifier can be used to determine the type of the module. Specifically, the module type identifier corresponding to the multispectral camera can be used to determine the module type of the module in the multispectral camera.

[0061] Step 102: Receive the model data corresponding to the module type identifier.

[0062] In the embodiments of this application, after sending the module type identifier corresponding to the multispectral camera, the model data corresponding to the module type identifier can be further received.

[0063] It is understood that, in the embodiments of this application, in response to the module type identifier of the multispectral camera sent by the electronic device, model data corresponding to the module type identifier can be further received. The model data corresponding to the module type identifier is used to determine the calibration model corresponding to the module type identifier.

[0064] For example, in some embodiments, when the electronic device is mounted on the production line calibration fixture, after the electronic device sends the module type identifier corresponding to the multispectral camera to the production line machine, the production line machine can select the model data of the calibration model corresponding to the module of the multispectral camera based on the module type identifier, and then send the model data to the electronic device.

[0065] It is understood that, in the embodiments of this application, the production line equipment can store model data for all candidate calibration models. Each type of module can correspond to one candidate calibration model, meaning each type of model corresponds to a set of candidate calibration model data.

[0066] For example, in some embodiments, the model data of candidate calibration models in the electronic instrument may be stored in the form of a mapping relationship between module type identifiers and model data, such as Table 1:

[0067] Table 1

[0068] Module type identifier Model data 1 Data 1 2 Data 2 …… ……

[0069] Step 103: Determine the calibration model corresponding to the multispectral camera based on the model data; wherein, the calibration model corresponding to the multispectral camera is used to calibrate the multispectral camera.

[0070] In the embodiments of this application, after receiving the model data corresponding to the module type identifier, the calibration model corresponding to the multispectral camera can be further determined based on the model data.

[0071] It is understood that, in the embodiments of this application, the calibration model corresponding to the multispectral camera can be used to calibrate the multispectral camera. The calibration model is a candidate calibration model corresponding to the multispectral camera, and the candidate calibration models are obtained by training on a training dataset; the training dataset includes at least one set of training data corresponding to at least one module type identifier.

[0072] In other words, in the embodiments of this application, the calibration model used by the electronic device to perform calibration processing can be a model corresponding to the multispectral camera from the candidate calibration models. That is, when the electronic device calibrates the configured multispectral camera, it uses a calibration model that matches the multispectral camera to perform calibration processing. Here, the calibration model that matches the multispectral camera can be understood as the calibration model that matches the module of the multispectral camera.

[0073] It is understood that, in the embodiments of this application, the purpose of calibrating the multispectral camera is to adjust and optimize its performance parameters, achieving high consistency and accuracy across different hardware components. This is of great significance for ensuring that the multispectral camera can operate stably and reliably in practical applications.

[0074] Furthermore, in embodiments of this application, the candidate calibration model may include at least one calibration model corresponding to at least one module. Different modules may have different calibration models.

[0075] Furthermore, in embodiments of this application, the candidate calibration model can be obtained by a computing server training a training dataset. The candidate calibration model can then be used to perform calibration processing on the module.

[0076] Furthermore, in embodiments of this application, the training dataset may include at least one set of training data corresponding to at least one module type identifier. That is, the computing server can use different training data corresponding to different module type identifiers to train the model separately, thereby obtaining different candidate calibration models corresponding to different module type identifiers.

[0077] In the embodiments of this application, the different training data corresponding to different module type identifiers can be understood as different training data corresponding to different types of modules. That is, the module type identifier can be used to determine the type of the module.

[0078] It is understood that, in the embodiments of this application, each set of training data corresponding to each module type identifier may include the second image data corresponding to the module type identifier and the standard module data corresponding to the module type identifier. The second image data may be obtained through the module corresponding to the module type identifier.

[0079] It is understood that, in the embodiments of this application, the second image data can be understood as the raw image data acquired by the module. Specifically, the second image data can be unprocessed image data directly output by the module. The second image data has higher color depth and richer image details, providing greater flexibility and scope for post-processing.

[0080] For example, in some embodiments, the training dataset used for model training may be obtained during the module calibration process. For instance, when a module manufacturer calibrates and inspects the produced modules, on the one hand, it can collect image data from the produced modules and then inspect the collected image data for compliance. If the image data is qualified, it can be further used as second image data for model training. On the other hand, by calibrating the produced modules, standard module data corresponding to the modules can be determined, including different standard module data corresponding to different types of modules.

[0081] Furthermore, in the embodiments of this application, the model data of the candidate calibration model is verified and transmitted through a data processing device.

[0082] It is understood that, in the embodiments of this application, the data processing device may be a device for data verification and transmission, or it may include a verification device for data verification and a transmission device for data transmission, and may also include other components, such as a local server. This application does not make specific limitations.

[0083] In other words, in the embodiments of this application, the data processing device can be understood as a separate device for data verification and transmission, or as a combination of different devices and / or servers that have data verification and transmission functions.

[0084] Furthermore, in the embodiments of this application, the electronic device can acquire model data corresponding to the multispectral camera; then, it can determine the calibration model corresponding to the multispectral camera based on the model data. The model data corresponding to the multispectral camera is the model data of the target model corresponding to the multispectral camera.

[0085] It is understood that, in the embodiments of this application, after acquiring model data, the electronic device can obtain the calibration model corresponding to the multispectral camera based on the model data, and thus can use the calibration model to calibrate the multispectral camera.

[0086] Furthermore, in the embodiments of this application, when determining the calibration model corresponding to the multispectral camera based on model data, the model data corresponding to the module type identifier can be verified first to obtain the verification result; then, based on the verification result, if the model data meets the data correctness requirements, the calibration model corresponding to the multispectral camera can be determined based on the model data.

[0087] Furthermore, in the embodiments of this application, after receiving the model data corresponding to the module type identifier of the multispectral camera, the model data can be verified for data correctness to determine whether the model data meets the data correctness requirements. If the model data meets the data correctness requirements, then the electronic device can store the model data corresponding to the multispectral camera.

[0088] In the embodiments of this application, the correctness verification of model data may be performed by electronic devices or by other devices or equipment on the production line; this application does not impose any specific limitations.

[0089] It is understood that, in the embodiments of this application, the verification method for the correctness verification of model data is not specifically limited.

[0090] For example, in some embodiments, the correctness of model data can be determined using a verification method such as Message-Digest Algorithm 5 (MD5). MD5 checks the correctness of data by performing a hash operation on the received transmitted data.

[0091] Specifically, MD5 can convert data of any length into a fixed-length 32-bit hexadecimal hash value (i.e., a 128-bit, 16-byte hash value), typically used to ensure the integrity and consistency of transmitted information. In other words, the principle of MD5 verification is to take the input data (such as file content) as input and calculate a unique, fixed-length hash value using the MD5 algorithm. If two pieces of data have the same content, their MD5 values ​​will also be the same; conversely, if the two pieces of data have different content, their MD5 values ​​will be different.

[0092] MD5 can be used to verify several aspects, including:

[0093] Data integrity verification: By comparing the MD5 values ​​of data, the integrity and consistency of the data can be verified to ensure that the data has not been tampered with or damaged during transmission or storage.

[0094] File verification: During file transfer or download, MD5 checksums can be used to ensure file integrity. The sender sends the file's MD5 value to the receiver along with the file. Upon receiving the file, the receiver recalculates the MD5 value and compares it to the one provided by the sender. If the two values ​​match, it means the file has not been corrupted or tampered with during transmission.

[0095] Verification of confidential information: MD5 checksums can also be applied to verify confidential information to ensure the security of data during transmission.

[0096] Step 104: Using the calibration model, based on the first image data acquired by the multispectral camera, obtain the calibrated performance parameters of the multispectral camera.

[0097] In the embodiments of this application, after determining the calibration model corresponding to the multispectral camera based on the model data, the calibration performance parameters of the multispectral camera can be obtained further through the calibration model based on the first image data acquired by the multispectral camera.

[0098] In the embodiments of this application, after calibrating the multispectral camera through the calibration model, the calibration data corresponding to the multispectral camera can be obtained, that is, the calibrated performance parameters of the multispectral camera can be obtained. The calibration data (calibrated performance parameters) can be used to adjust the parameters of the multispectral camera to ensure the accuracy of the multispectral camera, so that the parameters or output of the multispectral camera match the known standard values. In this way, it can be ensured that the multispectral camera provides accurate and reliable data in actual use.

[0099] It is understood that, in the embodiments of this application, the calibration process of the multispectral camera is performed by using a calibration model, which can take into account various interference factors during the calibration process, including but not limited to differences in hardware characteristics, brightness, and sensitivity, thereby further ensuring the accuracy and reliability of the multispectral camera.

[0100] It is understood that, in the embodiments of this application, due to factors such as differences in pixel response, brightness, and sensitivity, the electronic device may not achieve ideal imaging results when acquiring spectral image data through the configured multispectral camera. After calibrating the multispectral camera using a calibration model, the influence of various interference factors can be effectively resolved, thereby improving the imaging effect of the multispectral camera.

[0101] Furthermore, in the embodiments of this application, the calibrated performance parameters include, but are not limited to, one or more of the following: image correction coefficient of at least one pixel, module parameters corresponding to the multispectral camera, and model correction parameters corresponding to the multispectral camera. They may also include any type of parameters corresponding to the multispectral camera, and this application does not impose any specific limitations.

[0102] For example, in some embodiments, the calibrated performance parameters may include an image correction coefficient for at least one pixel. Specifically, after calibrating the multispectral camera based on the calibration model and obtaining the image correction coefficient for at least one pixel, the original spectral image data corresponding to the target scene can be further acquired using the multispectral camera. Then, using the image correction coefficient for at least one pixel, the image data of each pixel in the original spectral image data is corrected to obtain corrected spectral image data. Finally, the spectral information corresponding to the target scene can be determined based on the corrected spectral image data.

[0103] It is understood that, in the embodiments of this application, the target scene can be any scene to be photographed, and this application does not impose any specific limitations. The spectral information corresponding to the target scene may include, but is not limited to, the reflectance or radiance of each pixel in different spectral channels of the target scene.

[0104] It is understood that, in the embodiments of this application, for scenarios where multispectral image data is obtained by conventional calculation methods based on a multispectral camera, the image correction coefficient of at least one pixel is obtained after calibration by the calibration model. The image correction coefficient of the pixel can be used to correct the actual acquired pixel value (image data of the pixel) corresponding to the pixel to obtain the corresponding corrected pixel value (image data of the corrected pixel) to further determine the corrected spectral image data.

[0105] For example, in some embodiments, (x, y) are the coordinates of a pixel, F1(x, y) is the actual gray value acquired by the multispectral camera, i.e., the original spectral image data, and H(x, y) is the corresponding image correction coefficient obtained through the calibration model. Then, after using the image correction coefficient to correct the original spectral image data, the corrected gray value obtained, i.e., the corrected spectral image data, can be F2(x, y) = F1(x, y) × H(x, y).

[0106] For example, in some embodiments, the calibrated performance parameters may include model correction parameters corresponding to the multispectral camera. Specifically, after calibrating the multispectral camera based on the calibration model and obtaining the model correction parameters, the model parameters of the image generation model corresponding to the multispectral camera can be further corrected based on the model correction parameters to obtain the corrected image generation model corresponding to the multispectral camera.

[0107] It is understood that, in the embodiments of this application, for the scenario of obtaining multispectral image data based on a multispectral camera using an AI model, the model calibration obtained after calibration is the model correction parameters corresponding to the multispectral camera. The model correction parameters corresponding to the multispectral camera can be used to correct and optimize the image generation model corresponding to the multispectral camera.

[0108] In other words, in the embodiments of this application, the model correction parameters corresponding to the multispectral camera generated by the calibration model can be used to further adjust the model parameters of the image generation model used to generate spectral image data, thereby improving the performance of the image generation model and obtaining better image effects.

[0109] Accordingly, in the embodiments of this application, after correcting the model parameters of the image generation model corresponding to the multispectral camera based on the correction parameters to obtain the corrected image generation model corresponding to the multispectral camera, the spectral image data corresponding to the target scene is obtained through the multispectral camera. The spectral image data corresponding to the target scene can be further determined based on the spectral image data through the corrected image generation model corresponding to the multispectral camera; and then the spectral information corresponding to the target scene is determined based on the corrected spectral image data.

[0110] It is understood that, in the embodiments of this application, the target scene can be any scene to be photographed, and this application does not impose any specific limitations. The spectral information corresponding to the target scene may include, but is not limited to, the reflectance or radiance of each pixel in different spectral channels of the target scene.

[0111] Furthermore, in the embodiments of this application, after obtaining the spectral information corresponding to the target scene through a multispectral camera, the original image information corresponding to the target scene can be further obtained; then, the original image information is optimized based on the spectral information corresponding to the target scene to obtain the optimized image information corresponding to the target scene.

[0112] It is understood that, in the embodiments of this application, when optimizing the original image information based on the spectral information corresponding to the target scene, the original image information can be corrected based on the correspondence between the spectral information and the original image information to obtain the optimized image information corresponding to the target scene.

[0113] Furthermore, in the embodiments of this application, when the multispectral camera calibration mode is enabled, the electronic device can acquire first image data through the multispectral camera.

[0114] It is understood that, in the embodiments of this application, the electronic device may select to enable the multispectral camera calibration mode in response to the received calibration command.

[0115] It is understood that, in the embodiments of this application, calibration instructions may include, but are not limited to, Man-Machine Interface (MMI) instructions. These calibration instructions can be used to instruct the multispectral camera in an electronic device to undergo calibration processing.

[0116] Furthermore, in the embodiments of this application, when the electronic device acquires the first image data through a multispectral camera, it can detect the brightness information corresponding to the brightness setting parameters when the automatic exposure AE is set according to the brightness setting parameters; when the brightness information meets the preset brightness requirements, the first image data is acquired through the multispectral camera.

[0117] It is understood that, in the embodiments of this application, the first image data can be understood as the raw image data acquired by a multispectral camera. Specifically, the first image data can be unprocessed image data directly output by the multispectral camera. The first image data has higher color depth and richer image details, providing greater flexibility and scope for post-processing.

[0118] Accordingly, in some embodiments, the first image data contains image information in multiple spectral bands, which can be used in post-processing to extract specific spectral features to achieve more accurate target detection and analysis.

[0119] Furthermore, in the embodiments of this application, when acquiring the first image data through a multispectral camera, automatic exposure (AE) can be set first; when the AE meets the brightness requirements, the image is acquired through the multispectral camera to obtain the first image data.

[0120] Automatic exposure (AE) is a photographic and video recording technique used to automatically adjust the camera's exposure settings based on the lighting conditions of the shooting environment to ensure that the image has appropriate brightness and contrast.

[0121] It is understood that, in the embodiments of this application, setting AE may include, but is not limited to, setting shutter speed, aperture size, and ISO (International Organization for Standardization).

[0122] In the embodiments of this application, after the electronic device completes the setting of the image AE (Advanced Image Filter), it can further determine whether the AE meets the brightness requirements. Here, brightness refers to the brightness of the image, which is crucial to the visual effect of the image. Appropriate brightness can make the image clear and rich in detail, while excessive brightness or darkness may lead to a decrease in image quality.

[0123] For example, in some embodiments, when determining whether the AE (exposure image) meets the brightness requirements, when the multispectral camera is in automatic exposure mode, the built-in metering system of the multispectral camera can analyze the lighting conditions of the shooting scene to determine whether the brightness requirements are met. If the brightness requirements are not met, the multispectral camera can automatically adjust parameters such as shutter speed, aperture size, and ISO sensitivity based on the analysis results to ensure the image has appropriate brightness.

[0124] For example, in some embodiments, in low-light environments, the multispectral camera automatically reduces the shutter speed or increases the aperture to improve image brightness; while in high-light environments, it reduces the aperture or increases the shutter speed to reduce image brightness.

[0125] Furthermore, in the embodiments of this application, if it is determined that the AE meets the brightness requirements, the electronic device can further acquire the first image data through a multispectral camera.

[0126] In summary, the calibration method for the spectral camera proposed in this application can use a calibration model obtained through training, which corresponds to the multispectral camera of the electronic device, to calibrate the multispectral camera, thereby obtaining a more ideal calibration result.

[0127] Exemplary, in some embodiments, Figure 3 This is a schematic diagram illustrating the implementation of the spectral camera calibration method proposed in this application embodiment, as shown below. Figure 3As shown, assuming the electronic device is a mobile phone, after the phone is placed into the production line calibration fixture, it can receive MMI (calibration command) instructions. In response to the received calibration command, it activates the multispectral camera calibration mode, then executes AE settings, and further performs AE brightness judgment. If the brightness requirement is not met, the AE settings need to be re-executed until the requirement is met. If the brightness requirement is met, the first image data can be acquired using the multispectral camera. Then, based on the acquired calibration model and the first image data, the corresponding calibration data is obtained and stored.

[0128] This application proposes a calibration method applied to an electronic device. The electronic device is equipped with a multispectral camera and sends a module type identifier corresponding to the multispectral camera; receives model data corresponding to the module type identifier; determines a calibration model corresponding to the multispectral camera based on the model data; wherein, the calibration model corresponding to the multispectral camera is used to perform calibration processing on the multispectral camera; and obtains the calibrated performance parameters of the multispectral camera based on the first image data acquired by the multispectral camera using the calibration model. In other words, in this application's embodiment, the obtained calibration model corresponding to the multispectral camera module can be used to perform multispectral camera calibration processing, and for the multispectral camera, the corresponding calibrated performance parameters can be obtained. It is evident that, in this application, on the one hand, the calibration model corresponds to the multispectral camera, enabling better calibration results for the multispectral camera; on the other hand, performing multispectral camera calibration processing through the calibration model can solve the multivariate fusion problem in the calibration process using computing power, achieving calibration against potential interference factors. In summary, the embodiments of this application can perform calibration processing on possible interference factors of a module of a certain type by using a calibration model corresponding to the type of multispectral camera. While solving the multivariate fusion problem in the calibration process, it reduces the complexity of the process and improves the accuracy and efficiency of calibration.

[0129] Based on the above embodiments, another embodiment of this application proposes a training method for a calibration model. This training method for a calibration model can be applied to a training device or computing server for a calibration model, and can also be applied to any terminal that includes a training device or computing server for a calibration model.

[0130] The following example, using a computing server, illustrates the training method for the calibration model proposed in this application.

[0131] Furthermore, in the embodiments of this application, Figure 4 This is a schematic diagram illustrating the implementation flow of the training method for the calibration model proposed in this application. Figure 4 As shown, the training method for the calibration model may include the following steps:

[0132] Step 201: Obtain the training dataset; wherein the training dataset includes at least one set of training data corresponding to at least one module type identifier.

[0133] In embodiments of this application, the computing server may first acquire a training dataset. This training dataset can be used to train a calibration model.

[0134] Furthermore, in embodiments of this application, the training dataset may include at least one set of training data corresponding to at least one module type identifier. That is, the computing server can use different training data corresponding to different module type identifiers to train the model separately, thereby obtaining different candidate calibration models corresponding to different module type identifiers.

[0135] In the embodiments of this application, the different training data corresponding to different module type identifiers can be understood as different training data corresponding to different types of modules. That is, the module type identifier can be used to determine the type of the module.

[0136] It is understood that, in the embodiments of this application, each set of training data corresponding to each module type identifier may include the second image data corresponding to the module type identifier and the standard module data corresponding to the module type identifier. The second image data may be obtained through the module corresponding to the module type identifier.

[0137] It is understood that, in the embodiments of this application, the second image data can be understood as the raw image data acquired by the module. Specifically, the second image data can be unprocessed image data directly output by the module. The second image data has higher color depth and richer image details, providing greater flexibility and scope for post-processing.

[0138] For example, in some embodiments, the training dataset used for model training may be obtained during the module calibration process. For instance, when a module manufacturer calibrates and inspects the produced modules, on the one hand, it can collect image data from the produced modules and then inspect the collected image data for compliance. If the image data is qualified, it can be further used as second image data for model training. On the other hand, by calibrating the produced modules, standard module data corresponding to the modules can be determined, including different standard module data corresponding to different types of modules.

[0139] Step 202: Train the initial model based on the training dataset to obtain candidate calibration models; wherein, the candidate calibration models include at least one candidate calibration model corresponding to at least one module type identifier.

[0140] In the embodiments of this application, after obtaining the training dataset, the second server can further train the initial model based on the training dataset to obtain a candidate calibration model.

[0141] It is understood that, in the embodiments of this application, for a training dataset that includes at least one set of training data corresponding to at least one module type identifier, the candidate calibration model obtained by training the initial model based on the training dataset may include at least one candidate calibration model corresponding to at least one module type identifier.

[0142] Furthermore, in the embodiments of this application, when training the initial model based on the training dataset to obtain a candidate calibration model, the initial model can be used to obtain the predicted module data corresponding to the module type identifier based on the second image data corresponding to the module type identifier; then the initial model can be corrected based on the standard module data and the predicted module data to obtain the candidate calibration model corresponding to the module type identifier.

[0143] It is understood that, in the embodiments of this application, for a model of a certain type corresponding to at least one module type identifier, the second image data in the corresponding training data can be input into the initial model, and the predicted module data can be output, that is, the corresponding predicted module data can be obtained.

[0144] Accordingly, in the embodiments of this application, after obtaining the prediction module data through the initial model, the loss can be calculated based on the prediction module data and the standard module data in the training data corresponding to the module type identifier, combined with the loss function. Then, the initial model is corrected according to the loss, and finally, the candidate calibration model corresponding to the module type identifier can be trained.

[0145] It is understood that in the embodiments of this application, the goal during model training is to optimize the model parameters by minimizing the loss function. This typically involves using optimization algorithms such as gradient descent to iteratively update the model parameters until the loss function reaches its minimum or converges to a stable solution. The loss function can be used to evaluate the difference or error between the model's predictions and the actual observations. By minimizing this loss function, the model parameters can be optimized, thereby improving the model's prediction accuracy. This application does not specifically limit the type of loss function used during model training. For example, the loss function may include, but is not limited to, Mean Squared Error (MSE), Mean Absolute Error (MAE), Huber loss, etc.

[0146] It is understood that, in the embodiments of this application, the initial model can be used to perform the calibration process of the module, and the candidate calibration model obtained by training can be used to perform the calibration process of the module.

[0147] Step 203: Send the model data of the candidate calibration model to the data processing device so that the model data of the candidate calibration model can be verified and transmitted through the data processing device.

[0148] In the embodiments of this application, after training the initial model based on the training dataset to obtain candidate calibration models, the model data of the candidate calibration models can be further sent to the data processing device, so that the model data of the candidate calibration models can be verified and transmitted through the data processing device.

[0149] In other words, in the embodiments of this application, the model data of the candidate calibration model is verified and transmitted through a data processing device.

[0150] It is understood that, in the embodiments of this application, the data processing device may be a device for data verification and transmission, or it may include a verification device for data verification and a transmission device for data transmission, and may also include other components, such as a local server. This application does not make specific limitations.

[0151] In other words, in the embodiments of this application, the data processing device can be understood as a separate device for data verification and transmission, or as a combination of different devices and / or servers that have data verification and transmission functions.

[0152] In summary, the training method for a calibration model proposed in this application can obtain a training dataset containing training data corresponding to different types of modules, and train different candidate calibration models corresponding to different types of modules through the training dataset, which greatly improves the prediction ability and effect of the calibration model.

[0153] Exemplary, in some embodiments, Figure 5 This is a schematic diagram illustrating the implementation of the training method for the calibration model proposed in this application. Figure 5 As shown, the computing server obtains the training dataset, and then can train the initial model based on the training dataset to obtain the corresponding candidate calibration model; then, the model data of the candidate calibration model obtained by training can be sent to the data processing device for subsequent data verification and data transmission.

[0154] This application provides a method for training a calibration model. A computing server acquires a training dataset, which includes at least one set of training data corresponding to at least one module type identifier. An initial model is trained based on the training dataset to obtain candidate calibration models. Each candidate calibration model includes at least one candidate calibration model corresponding to at least one module type identifier. The model data of the candidate calibration models is sent to a data processing device for verification and transmission. In other words, in this application, calibration models can be trained separately for different modules, thereby achieving optimal calibration results through the corresponding calibration models. This addresses the multivariate fusion problem in the calibration process using computing power, while also enabling efficient data flow and computing power deployment. Therefore, this application demonstrates that the calibration model can handle all interfering factors, solving the multivariate fusion problem in the calibration process while reducing process complexity.

[0155] Based on the above embodiments, another embodiment of this application provides a data processing method that can be applied to a data processing device or data processing equipment, and can also be applied to any terminal that includes a data processing device or data processing equipment.

[0156] The following description uses a data processing device as an example to illustrate the data processing method proposed in the embodiments of this application.

[0157] Furthermore, in the embodiments of this application, Figure 6 This is a schematic diagram illustrating the implementation flow of the data processing method proposed in the embodiments of this application, such as... Figure 6 As shown, the data processing method may include the following steps:

[0158] Step 301: Receive the model data of the candidate calibration model sent by the computing power server; wherein, the candidate calibration model is obtained by the computing power server through training based on the training dataset.

[0159] In embodiments of this application, the data processing device can receive model data of candidate calibration models sent by a second server. These candidate calibration models can be used to perform calibration processing on the module.

[0160] Furthermore, in the embodiments of this application, the candidate calibration model can be obtained by a computing server training based on a training dataset.

[0161] Furthermore, in embodiments of this application, the training dataset may include at least one set of training data corresponding to at least one module type identifier. That is, the computing server can use different training data corresponding to different module type identifiers to train the model separately, thereby obtaining different candidate calibration models corresponding to different module type identifiers.

[0162] In the embodiments of this application, the different training data corresponding to different module type identifiers can be understood as different training data corresponding to different types of modules. That is, the module type identifier can be used to determine the type of the module.

[0163] It is understood that, in the embodiments of this application, each set of training data corresponding to each module type identifier may include the second image data corresponding to the module type identifier and the standard module data corresponding to the module type identifier. The second image data may be obtained through the module corresponding to the module type identifier.

[0164] It is understood that, in the embodiments of this application, the second image data can be understood as the raw image data acquired by the module. Specifically, the second image data can be unprocessed image data directly output by the module. This data records the raw information of the light source signals captured by the module and converted into digital signals. The second image data has higher color depth and richer image details, providing greater flexibility and scope for post-processing.

[0165] For example, in some embodiments, the training dataset used for model training may be obtained during the module calibration process. For instance, when a module manufacturer calibrates and inspects the produced modules, on the one hand, it can collect image data from the produced modules and then inspect the collected image data for compliance. If the image data is qualified, it can be further used as second image data for model training. On the other hand, by calibrating the produced modules, standard module data corresponding to the modules can be determined, including different standard module data corresponding to different types of modules.

[0166] Furthermore, in the embodiments of this application, the data processing device can be a device for performing data verification and transmission, or it can be a verification device for data verification and a transmission device for data transmission, and may also include other components, such as a local server. This application does not specifically limit the scope of the data processing device.

[0167] In other words, in the embodiments of this application, the data processing device can be understood as a separate device for data verification and transmission, or as a combination of different devices and / or servers that have data verification and transmission functions.

[0168] Step 302: Verify the model data of the candidate calibration model. If the model data of the candidate calibration model is verified as qualified, send the model data of the candidate calibration model to the target server so that the model data of the calibration model corresponding to the multispectral camera of the electronic device can be synchronized to the electronic device through the target server.

[0169] In the embodiments of this application, after receiving the model data words of the candidate calibration model sent by the computing power server, the model data of the candidate calibration model can be further verified. If the model data of the candidate calibration model is verified as qualified, the model data of the candidate calibration model can be sent to the target server, so that the model data of the calibration model of the multispectral camera corresponding to the electronic device can be synchronized to the electronic device through the target server.

[0170] It is understood that, in the embodiments of this application, the target server can be a server corresponding to an electronic device. For example, in the process of whole-machine inspection, the electronic device performs calibration processing on a multispectral camera. In this case, the target server can be the server corresponding to the production line performing the whole-machine inspection.

[0171] For example, in some embodiments, if the data processing device verifies the model data of the candidate calibration model and the verification result is qualified, the data processing device can upload the model data of the candidate calibration model to the target server. Correspondingly, the target server can load the model data of the candidate calibration model into the production line machine on the production line that performs whole machine testing, so that the model data of the candidate calibration model corresponding to the calibration model of the multispectral camera of the electronic device can be synchronized to the electronic device through the production line machine.

[0172] It is understood that, in the embodiments of this application, during the process of loading the model data of the candidate calibration model onto the production line machine on the production line performing whole machine testing, the model data of the candidate calibration model can be verified for data correctness to determine whether the model data of the candidate calibration model meets the data correctness requirements. If the model data of the candidate calibration model meets the data correctness requirements, then the production line machine can store the model data of the candidate calibration model.

[0173] In the embodiments of this application, the correctness verification of the model data of the candidate calibration model can be performed by the target server or by other devices or equipment on the production line; this application does not impose any specific limitations.

[0174] It is understood that, in the embodiments of this application, the verification method for the correctness verification of model data is not specifically limited. For example, the correctness of model data can be determined by MD5.

[0175] In summary, the data processing method proposed in this application can verify and transmit candidate model data obtained by training on a computing server, thereby enabling electronic devices to obtain a calibration model corresponding to the multispectral camera of the electronic device. Then, the multispectral camera can be calibrated based on the calibration model, thereby obtaining a more ideal calibration result.

[0176] Exemplary, in some embodiments, Figure 7 This is a schematic diagram illustrating the implementation of the data processing method proposed in the embodiments of this application, as shown below. Figure 7 As shown, after receiving the model data of the candidate calibration model sent by the server, the model data can be verified. If the verification result is qualified, the model data of the candidate calibration model can be sent to the target server. If the verification result is unqualified, the model data can be removed.

[0177] This application provides a data processing method in which a data processing device receives model data of candidate calibration models sent by a computing power server. The candidate calibration models are obtained by the computing power server through training on a training dataset. The model data of the candidate calibration models is validated. If the validation is successful, the model data of the candidate calibration models is sent to a target server, so that the target server can synchronize the model data of the calibration model corresponding to the multispectral camera of the electronic device to the electronic device. In other words, in this application, calibration models can be trained separately for different modules, thereby obtaining the best calibration effect through the corresponding calibration model. This solves the multivariate fusion problem in the calibration process using computing power, and also achieves efficient data flow and computing power deployment. Therefore, in this application, the calibration model can be used to perform calibration processing for all interference factors, solving the multivariate fusion problem in the calibration process while reducing process complexity.

[0178] Based on the above embodiments, another embodiment of this application provides a spectral camera calibration method based on Artificial Intelligence (AI) training, including a spectral camera calibration method, a calibration model training method, and a data processing method. This AI-based spectral camera calibration method can be applied to a calibration system, which can be an electronic device, a computing server, and a data processing device.

[0179] Furthermore, in the embodiments of this application, the computing server in the calibration system is used to acquire a training dataset; wherein the training dataset includes at least one set of training data corresponding to at least one module type identifier; the initial model is trained based on the training dataset to obtain a candidate calibration model; wherein the candidate calibration model includes at least one candidate calibration model corresponding to at least one module type identifier; the model data of the candidate calibration model is sent to a data processing device so that the model data of the candidate calibration model can be verified and transmitted by the data processing device.

[0180] Furthermore, in the embodiments of this application, the data processing device in the calibration system is used to perform data verification on the model data of the candidate calibration model. If the model data of the candidate calibration model is verified as qualified, the model data of the candidate calibration model is sent to the target server so that the model data of the calibration model of the multispectral camera corresponding to the electronic device is synchronized to the electronic device through the target server.

[0181] Furthermore, in the embodiments of this application, the electronic device in the calibration system is used to acquire first image data through a multispectral camera when the multispectral camera calibration mode is enabled; and to obtain calibration data corresponding to the multispectral camera based on the first image data through a calibration model; wherein, the calibration model is a model corresponding to the multispectral camera among the candidate calibration models.

[0182] Furthermore, the AI-trained spectral camera calibration method proposed in this application embodiment may include a data acquisition method. Here, data acquisition can be understood as the acquisition of training data, and the data acquisition may be performed during the module calibration process.

[0183] Exemplary, in some embodiments, Figure 8 This is a schematic diagram of the data acquisition method proposed in the embodiments of this application, as shown below. Figure 8 As shown, when a module manufacturer calibrates and inspects its produced modules, after the calibration process begins, it can acquire Raw images through 32 light source channels. The acquired Raw images are then inspected. If the inspection result is unqualified, the Raw image acquisition is repeated. If the inspection result is qualified, the qualified raw image is sent as the second image data to the computing server to form the training dataset for model training. On the other hand, by calibrating the produced modules, standard module data corresponding to each module can be determined. This includes different standard module data corresponding to different types of modules. The standard module data corresponding to each module is also sent to the computing server to form the training dataset for model training.

[0184] Furthermore, regarding the AI-based spectral camera calibration method proposed in this application embodiment, the computing server can obtain the training dataset during model training to train the constructed initial model, obtain candidate calibration models, and send them to the data processing device.

[0185] Exemplary, in some embodiments, Figure 9 This is a schematic diagram of the training method for the calibration model proposed in the embodiments of this application, as shown below. Figure 9 As shown, the training dataset includes different training data corresponding to different module types (i.e., different module type identifiers). For any given module type identifier, the corresponding training data includes a one-to-one correspondence of second image data and standard module data. The computing server trains the initial model based on the different training data corresponding to different module type identifiers, thereby obtaining different candidate calibration models corresponding to the same module type identifier.

[0186] Furthermore, for the AI-trained spectral camera calibration method proposed in this application embodiment, the data processing device can perform data verification and data transmission on the received model data.

[0187] Exemplary, in some embodiments, Figure 10 This is a schematic diagram of the data processing method proposed in the embodiments of this application, as shown below. Figure 10 As shown, the data processing equipment may include a local server and a test station. The local server stores the received model data and loads it onto the test station, which then verifies the downloaded model data. If the verification result is satisfactory, the model data can be uploaded to the target server, and the module corresponding to the satisfactory model data can be added to the database. If the verification result is unsatisfactory, the module corresponding to the model data can be marked as defective.

[0188] Furthermore, the AI-trained spectral camera calibration method proposed in the embodiments of this application may also include a production line whole-machine inspection method.

[0189] Exemplary, in some embodiments, Figure 11 This is a schematic diagram of the production line whole machine testing method proposed in the embodiments of this application, as shown below. Figure 11As shown, during whole-machine testing, electronic devices, such as mobile phones, can be mounted in production line calibration fixtures. The target server can load the model data of candidate calibration models into the production line performing whole-machine testing. The model data of the candidate calibration models can be verified for correctness, such as through MD5 checksums, to determine whether the model data meets the data correctness requirements. If the model data meets the requirements, the production line machine can store the model data; otherwise, the model data needs to be downloaded again from the target server.

[0190] Exemplary, in some embodiments, Figure 12 This is a schematic diagram of the model data synchronization method proposed in the embodiments of this application, as shown below. Figure 12 As shown, during whole-machine testing, electronic devices, such as mobile phones, can be mounted in production line calibration fixtures. The electronic device can send a module type identifier (e.g., Sensor ID) corresponding to the multispectral camera to the production line machine; then, it can receive the model data corresponding to the module type identifier sent by the production line machine. Next, the model data can be further verified for correctness, such as through MD5 checksum verification, to determine if the model data meets the correctness requirements. If the model data meets the correctness requirements, the electronic device can store the model data, for example, by saving it to the Persist partition; otherwise, it needs to resend the module type identifier corresponding to the multispectral camera to the production line machine to re-acquire the model data.

[0191] Furthermore, regarding the AI-trained spectral camera calibration method proposed in this application embodiment, when the electronic device performs multispectral camera calibration, it can use the acquired model data to obtain a calibration model corresponding to the module of the multispectral camera, and then generate calibration data based on the acquired RAW image through the calibration model.

[0192] Exemplary, in some embodiments, Figure 13 This is a schematic diagram of the calibration method proposed in the embodiments of this application, as shown below. Figure 13 As shown, in response to a received calibration command, the multispectral camera calibration mode can be activated, followed by AE settings and then AE brightness judgment. If the brightness requirement is not met, the AE settings need to be re-executed until the brightness requirement is met; if the brightness requirement is met, the first image data can be acquired through the multispectral camera, and then the corresponding calibration result can be obtained based on the first image data using the acquired calibration model.

[0193] In summary, the AI-trained spectral camera calibration method proposed in this application can solve the multivariate fusion problem in the calibration process through computing power, and can also achieve efficient data flow and computing power deployment in conventional production lines.

[0194] In other words, in the embodiments of this application, AI is used to train each module based on its input. Each module has its own set of data models, and the best calibration effect is obtained by matching the calibrated data models with the original modules.

[0195] Furthermore, the AI-trained spectral camera calibration method proposed in this application, including the AI-based calibration process, server setup, data flow, and process design, serves as a reference for future mass production of high-precision and highly complex devices.

[0196] This application provides an AI-trained method for spectral camera calibration. It allows for the training of calibration models for different modules, enabling the acquisition of optimal calibration results through the corresponding models. This method leverages computing power to address the multivariate fusion problem during calibration, while also achieving efficient data transfer and computing power deployment. In other words, this application's embodiments can use calibration models to handle all interfering factors, resolving the multivariate fusion problem while reducing process complexity.

[0197] Based on the above embodiments, in another embodiment of this application... Figure 14 This is a schematic diagram of the composition and structure of the calibration device for the spectral camera proposed in the embodiments of this application, as shown below. Figure 14 As shown, the calibration device 110 for the spectral camera proposed in this application embodiment may include:

[0198] The transmitting unit 1101 is used to transmit the module type identifier corresponding to the multispectral camera;

[0199] The receiving unit 1102 is used to receive model data corresponding to the module type identifier;

[0200] The determining unit 1103 is used to determine the calibration model corresponding to the multispectral camera based on the model data; wherein, the calibration model corresponding to the multispectral camera is used to perform calibration processing on the multispectral camera;

[0201] The calibration unit 1104 is used to obtain the calibrated performance parameters of the multispectral camera based on the first image data acquired by the multispectral camera through the calibration model.

[0202] In the embodiments of this application, further, Figure 15This is a schematic diagram of the composition of the training device for the calibration model proposed in the embodiments of this application, as shown below. Figure 15 As shown, the training device 120 for the calibration model proposed in this application embodiment may include:

[0203] The acquisition unit 1201 is used to acquire the training dataset; wherein the training dataset includes at least one set of training data corresponding to at least one module type identifier;

[0204] Training unit 1202 is used to train the initial model based on the training dataset to obtain candidate calibration models; wherein, the candidate calibration models include at least one candidate calibration model corresponding to at least one module type identifier;

[0205] The sending unit 1203 is used to send the model data of the candidate calibration model to the data processing device so that the data processing device can verify and transmit the model data of the candidate calibration model.

[0206] In the embodiments of this application, further, Figure 16 This is a schematic diagram of the composition structure of the data processing device proposed in the embodiments of this application, as shown below. Figure 16 As shown, the data processing apparatus 130 proposed in this application embodiment may include:

[0207] The receiving unit 1301 is used to receive model data of candidate calibration models sent by the computing power server; wherein, the candidate calibration models are obtained by the computing power server through training based on the training dataset;

[0208] The verification unit 1302 is used to perform data verification on the model data of the candidate calibration model;

[0209] The sending unit 1303 is used to send the model data of the candidate calibration model to the target server when the model data of the candidate calibration model is verified as qualified, so as to synchronize the model data of the calibration model of the multispectral camera corresponding to the electronic device to the electronic device through the target server.

[0210] In the embodiments of this application, further, Figure 17 This is a schematic diagram of the composition structure of the electronic device proposed in the embodiments of this application, such as... Figure 17 As shown, the electronic device 170 proposed in this application embodiment may include a processor 1701, a memory 1702, a communication interface 1703, and a bus 1704 for connecting the processor 1701, the memory 1702, and the communication interface 1703.

[0211] In the embodiments of this application, the processor 1701 can be at least one of the following: Application-Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field-Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that for different devices, the electronic device used to implement the above-mentioned processor function can also be other types, and this application embodiment does not specifically limit the specific types. The electronic device 170 may also include a memory 1702, which can be connected to the processor 1701. The memory 1702 is used to store executable program code, which includes computer operation instructions. The memory 1702 may include high-speed RAM memory and may also include non-volatile memory, such as at least two disk drives.

[0212] In embodiments of this application, bus 1704 is used to connect communication interface 1703, processor 1701, and memory 1702, as well as the mutual communication between these devices.

[0213] In practical applications, the aforementioned memory 1702 can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 1701.

[0214] Further, in an embodiment of this application, the processor 1701 is configured to send a module type identifier corresponding to the multispectral camera; receive model data corresponding to the module type identifier; determine a calibration model corresponding to the multispectral camera based on the model data; wherein the calibration model corresponding to the multispectral camera is used to perform calibration processing on the multispectral camera; and obtain calibrated performance parameters corresponding to the multispectral camera based on the first image data acquired by the multispectral camera through the calibration model.

[0215] Furthermore, in this embodiment, the functional modules can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional module.

[0216] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method of this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0217] This application provides a computer-readable storage medium storing a program that, when executed by a processor, implements the calibration method for a spectral camera as described above.

[0218] Specifically, the program instructions corresponding to the calibration method of a spectroscopic camera in this embodiment can be stored on storage media such as optical discs, hard disks, and USB flash drives. When the program instructions corresponding to the calibration method of a spectroscopic camera in the storage media are read or executed by an electronic device, the following steps are included:

[0219] Send the module type identifier corresponding to the multispectral camera;

[0220] Receive the model data corresponding to the module type identifier;

[0221] The calibration model corresponding to the multispectral camera is determined based on the model data; wherein, the calibration model corresponding to the multispectral camera is used to calibrate the multispectral camera;

[0222] Based on the first image data acquired by the multispectral camera, the calibration model is used to obtain the calibrated performance parameters of the multispectral camera.

[0223] Specifically, the program instructions corresponding to a training method for a calibration model in this embodiment can be stored on storage media such as optical discs, hard disks, and USB flash drives. When the program instructions corresponding to a training method for a calibration model in the storage media are read or executed by an electronic device, the following steps are included:

[0224] Obtain the training dataset; wherein the training dataset includes at least one set of training data corresponding to at least one module type identifier;

[0225] The initial model is trained based on the training dataset to obtain candidate calibration models; wherein, the candidate calibration models include at least one candidate calibration model corresponding to at least one module type identifier;

[0226] The model data of the candidate calibration model is sent to the data processing device for verification and transmission.

[0227] Specifically, the program instructions corresponding to a data processing method in this embodiment can be stored on storage media such as optical discs, hard disks, and USB flash drives. When the program instructions corresponding to a data processing method in the storage media are read or executed by an electronic device, the following steps are included:

[0228] Receive model data of candidate calibration models sent by the computing power server; wherein, the candidate calibration models are obtained by the computing power server through training based on the training dataset;

[0229] The model data of the candidate calibration model is validated. If the model data of the candidate calibration model is validated and passes the validation, the model data of the candidate calibration model is sent to the target server so that the model data of the calibration model corresponding to the multispectral camera of the electronic device can be synchronized to the electronic device through the target server.

[0230] This application also provides a computer program product.

[0231] In some embodiments, the computer program product may include a computer program or instructions.

[0232] In some embodiments, the computer program product can be applied to the computer device in the embodiments of this application, and the computer program instructions cause the computer to execute the corresponding processes implemented by the computer device in the various methods of the embodiments of this application. For the sake of brevity, they will not be described in detail here.

[0233] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0234] This application is described with reference to schematic and / or block diagrams of implementations of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the schematic and / or block diagrams can be implemented by computer program instructions, and combinations of blocks in the schematic and / or block diagrams can be implemented. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the schematic and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0235] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the implementation flow diagram. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0236] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0237] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A calibration method for a spectroscopic camera, characterized in that, The method is applied to an electronic device equipped with a multispectral camera, and the method includes: Send the module type identifier corresponding to the multispectral camera; Receive the model data corresponding to the module type identifier; The calibration model corresponding to the multispectral camera is determined based on the model data; wherein, the calibration model corresponding to the multispectral camera is used to calibrate the multispectral camera; Based on the first image data acquired by the multispectral camera, the calibration model is used to obtain the calibrated performance parameters of the multispectral camera.

2. The method according to claim 1, characterized in that, The calibrated performance parameters include an image correction coefficient for at least one pixel; the method further includes: The original spectral image data corresponding to the target scene is acquired using the multispectral camera. By using the image correction coefficient of at least one pixel, the image data of each pixel in the original spectral image data is corrected to obtain the corrected spectral image data. The spectral information corresponding to the target scene is determined based on the corrected spectral image data.

3. The method according to claim 1, characterized in that, The calibrated performance parameters include the model correction parameters corresponding to the multispectral camera; the method further includes: Based on the aforementioned correction parameters, the model parameters of the image generation model corresponding to the multispectral camera are corrected to obtain the corrected image generation model corresponding to the multispectral camera.

4. The method according to claim 3, characterized in that, The method further includes: The multispectral camera is used to acquire spectral image data corresponding to the target scene. Based on the spectral image data, the spectral image data corresponding to the target scene is determined using the corrected image generation model corresponding to the multispectral camera. The spectral information corresponding to the target scene is determined based on the corrected spectral image data.

5. The method according to claim 2 or 4, characterized in that, The method further includes: Obtain the original image information corresponding to the target scene; The original image information is optimized based on the spectral information corresponding to the target scene to obtain the optimized image information corresponding to the target scene.

6. The method according to any one of claims 1-4, characterized in that, The calibration model is a candidate calibration model that corresponds to the multispectral camera. The candidate calibration model is obtained by training based on the training dataset. The training dataset includes at least one set of training data corresponding to at least one module type identifier.

7. The method according to claim 1, characterized in that, The step of determining the calibration model corresponding to the multispectral camera based on the model data includes: The model data corresponding to the module type identifier is validated to obtain the validation result; If the model data meets the data correctness requirements based on the verification results, the calibration model corresponding to the multispectral camera is determined based on the model data.

8. The method according to claim 1, characterized in that, The method further includes: When the automatic exposure (AE) is set according to the brightness setting parameters, the brightness information corresponding to the brightness setting parameters is detected; When the brightness information meets the preset brightness requirements, the first image data is acquired by the multispectral camera.

9. A calibration device for a spectroscopic camera, characterized in that, The calibration device for the spectral camera includes: A transmitting unit is used to transmit the module type identifier corresponding to the multispectral camera; The receiving unit is used to receive the model data corresponding to the module type identifier; A determining unit is used to determine the calibration model corresponding to the multispectral camera based on the model data; wherein, the calibration model corresponding to the multispectral camera is used to perform calibration processing on the multispectral camera; The calibration unit is used to obtain the calibrated performance parameters of the multispectral camera based on the first image data acquired by the multispectral camera using the calibration model.

10. An electronic device, characterized in that, The electronic device includes a processor and a memory storing processor-executable instructions, which, when executed by the processor, implement the method as described in any one of claims 1-8.

11. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by a processor, it implements the method as described in any one of claims 1-8.

Images