An image processing method, a method for obtaining data, and a device

Abstract

An image processing method, a method and device for obtaining data. The method can be applied to the field of image processing. The method includes: obtaining first image data of an object to be photographed under the illumination of an illumination light source, where the first image data includes m pixel points; adjusting the pixel values of a first pixel point and a second pixel point in the first image data according to first weight data to obtain second image data. The first weight data includes m weight values corresponding one by one to the m pixel points. The first weight value corresponding to the first pixel point is higher than the second weight value corresponding to the second pixel point, and the saturation of the color corresponding to the first pixel point is lower than the saturation of the color corresponding to the second pixel point; determining the spectral information of the illumination light source according to the second image data. A solution for directly obtaining the spectral information of the light source from the image data is provided, eliminating the need for measurement by high-cost dedicated measuring instruments and saving labor costs.

Description

Technical Field

[0001] This application relates to the field of computer software, and in particular, to an image processing method, a method for obtaining data, and related devices. Background Art

[0002] In the field of digital imaging technology, color imaging devices generally use three spectral channels to collect images of the object to be photographed. For example, red (R), green (G), and blue (B). However, due to the low spectral resolution of traditional RGB cameras, there is still much room for improvement in the color reproduction accuracy of RGB camera imaging technology.

[0003] To improve the imaging drawbacks of RGB cameras, multi-spectral cameras or hyperspectral cameras have emerged. Multi-spectral cameras or hyperspectral cameras can collect image data of the object to be photographed in multiple spectral channels, thereby improving the spectral resolution of the imaging system and enabling higher color reproduction accuracy.

[0004] However, in the process of processing the image data collected by multi-spectral cameras, the spectral information of the illumination source needs to be used, and currently, the spectral information of the illumination source is collected by high-cost dedicated measuring instruments, which is cumbersome to operate and costly. Summary of the Invention

[0005] In view of this, embodiments of this application provide an image processing method, providing a solution for directly obtaining the spectral information of the light source from the image data, no longer requiring measurement by high-cost dedicated measuring instruments, and no longer requiring additional operations, saving labor costs.

[0006] In a first aspect, an embodiment of the present application provides an image processing method. This method is applied in the field of image processing and includes: An electronic device acquires first image data of a photographed object under the illumination of an illumination light source. The first image data includes image data of the photographed object in n spectral channels. The first image data includes m pixel points, where n is an integer greater than 3, and m is an integer greater than or equal to 1. The electronic device acquires first weight data corresponding to the first image data. Among them, the first weight data includes m weight values corresponding one by one to the m pixel points. The first weight value corresponding to the first pixel point among the m pixel points is higher than the second weight value corresponding to the second pixel point among the m pixel points. The saturation of the color corresponding to the first pixel point is lower than the saturation of the color corresponding to the second pixel point. It should be noted that the first pixel point and the second pixel point are any two different pixel points among the m pixel points included in the first image data. The concepts of the first pixel point and the second pixel point are introduced here to express the concept that "the weight value of the pixel point corresponding to the color region with lower saturation is higher, and the weight value of the pixel point corresponding to the color region with higher saturation is lower" by comparing the first pixel point and the second pixel point, rather than representing that the above m pixel points are divided into two categories. The electronic device adjusts the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data to obtain second image data. Based on the second image data, the spectral information of the illumination light source is determined. The spectral information of the illumination light source is used to indicate the spectral power distribution of the illumination light source. The spectral information of the illumination light source may include multiple sets of intensity values corresponding one by one to various different wavelengths of colored light. Each set of intensity values represents the numerical value of the radiant energy of each colored light in the illumination light source.

[0007] In this implementation manner, a solution for directly obtaining the spectral information of a light source from image data is provided. There is no longer a need to measure through high-cost dedicated measuring instruments, nor to perform additional operations, and labor costs are saved. In addition, a higher weight is assigned to the first pixel points corresponding to low-saturation colors, and a lower weight is assigned to the second pixel points corresponding to high-saturation colors. According to the weight values of each pixel point, the pixel values of the first pixel points and the second pixel points in the first image data are adjusted to obtain the second image data. The pixel points corresponding to low-saturation colors in the second image data will be more obvious, that is, the interference brought by the pixel points corresponding to high-saturation colors in the second image data is less. Furthermore, according to the second image data, the spectral information of the illumination light source is generated. Since the high-saturation color regions of the object to be photographed absorb the light in most spectral bands of the illumination light source, it is not easy to extract the spectral information of the illumination light source in all spectral bands from the pixel points corresponding to high-saturation colors. And because the low-saturation color regions of the object to be photographed can reflect the light in each band of the illumination light source more comprehensively, it is not easy to extract the spectral information of the illumination light source in all spectral bands from the pixel points corresponding to low-saturation colors. Therefore, relatively accurate spectral information of the illumination light source in n spectral bands can be obtained from the second image data.

[0008] In a feasible implementation manner of the first aspect, if the first weight data has been normalized, that is, the weight value corresponding to each of the m pixel points included in the first image data is between 0 and 1, the electronic device adjusts the pixel values of the first pixel points and the second pixel points in the first image data according to the first weight data, including: the electronic device reduces the pixel values of the first pixel points and the second pixel points in the first image data, and the reduction ratio of the pixel value of the second pixel point is greater than the reduction ratio of the pixel value of the first pixel point. In the embodiments of the present application, the reduction ratio of the pixel value of the second pixel point is greater than the reduction ratio of the pixel value of the first pixel point, thereby achieving the purpose of making the pixel points corresponding to low-saturation colors in the second image data more obvious, and by using the method of reducing the pixel values of the first pixel points and the second pixel points in the first image data, the occurrence of too high pixel values in the second image data is avoided, which is beneficial to reducing the complexity of the subsequent processing process.

[0009] In an implementable manner of the first aspect, the electronic device adjusts the pixel values of a first pixel point and a second pixel point in first image data according to first weight data, including: the electronic device multiplies the pixel value of the first pixel point in the first image data by a first weight value to obtain the pixel value of the first pixel point in second image data; multiplies the pixel value of the second pixel point in the first image data by a second weight value to obtain the pixel value of the second pixel point in the second image data. In the embodiments of the present application, a specific implementation manner of adjusting the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data is provided, which is implemented by multiplying the pixel point by the corresponding weight value, and the operation is simple and easy to implement.

[0010] In an implementable manner of the first aspect, the electronic device adjusts the pixel values of a first pixel point and a second pixel point in first image data according to first weight data to obtain second image data, including: the electronic device performs a Hadamard product operation on the image data of each spectral channel in the image data of n spectral channels included in the first image data and the first weight data to obtain the second image data.

[0011] In an implementable manner of the first aspect, the image data of each spectral channel may specifically be represented as a p×q matrix, and the first image data is represented as a three-dimensional tensor, which includes n p×q matrices; the first weight data may specifically be represented as a p×q matrix. The electronic device performs a Hadamard product operation on the image data of each spectral channel in the image data of n spectral channels included in the first image data and the first weight data, including: the electronic device sequentially obtains the image data of each spectral channel in the first image data, and performs a Hadamard product operation on the image data of a single spectral channel and the first weight data. Alternatively, the electronic device generates a second tensor according to the first weight data, the second tensor is a three-dimensional tensor, which includes n first weight data, and the electronic device directly performs a Hadamard product operation on the first image data and the second tensor.

[0012] In an implementable manner of the first aspect, the method further includes: the electronic device performs matrix multiplication on the first image data and a first vector to obtain second weight data; wherein, the first image data is an m×n matrix, the first vector includes n elements, the element values of the n elements included in the first vector are obtained based on a pre-training operation and configured in the electronic device, and the second weight data includes m weight values corresponding to m pixel points one by one. The electronic device normalizes the m weight values included in the second weight data to generate the first weight data.

[0013] In this implementation manner, an implementation manner for generating the first weight data is provided. Since the pixel values in different multispectral images or hyperspectral images have different value ranges, the value ranges of the second weight data corresponding to different first image data are different. Normalizing the second weight data is beneficial to reducing the complexity of the subsequent processing, and further improving the accuracy of the spectral information of the finally generated illumination source.

[0014] In an implementable manner of the first aspect, the first image data includes m rows of values, and each row has n values. The electronic device performs matrix multiplication on the first image data and the first vector to obtain the second weight data, including: the electronic device arranges the values in each row of the m rows of values in descending order to obtain the first matrix, multiplies the first matrix by the first vector to obtain the second weight data, and the n values in the first vector are arranged in ascending order.

[0015] In this implementation manner, since the characteristic of the pixel points corresponding to high-saturation colors is that the pixel values of several spectral channels among the n spectral channels are very high, and the pixel values of the remaining several spectral channels are very low, that is, the n pixel values corresponding to the pixel points of high-saturation colors are very unevenly distributed. Based on the foregoing characteristics, the values in each row of the m rows of values are arranged in descending order, and the n values in the first vector are arranged in ascending order, which is convenient for distinguishing the pixel points corresponding to high-saturation colors and the pixel points corresponding to low-saturation colors.

[0016] In an implementable manner of the first aspect, for the process of performing matrix multiplication on the first image data and the first vector. The first image data includes m rows of values, and each row has n values, and the n values in the first vector are arranged in descending order. The electronic device arranges the values in each row of the m rows of values in ascending order to obtain the first matrix, multiplies the first matrix by the first vector to obtain the second weight data.

[0017] In an implementable manner of the first aspect, the method further includes: the electronic device increases the value of the third weight value among the m weight values and decreases the value of the fourth weight value among the m weight values to obtain updated first weight data, where the third weight value is a weight value greater than or equal to a preset threshold, and the fourth weight value is a weight value less than the preset threshold. Specifically, the electronic device may be pre-configured with a piecewise mapping function, and the piecewise mapping function may include a first function and a second function. Each of the m weight values included in the first weight data is subjected to a piecewise mapping once to obtain updated first weight data; that is, when one of the m weight values included in the first weight data is less than the preset threshold, the first function is executed to decrease the value of the fourth weight value among the m weight values; when one of the m weight values included in the first weight data is greater than or equal to the preset threshold, the second function is executed to increase the value of the third weight value among the m weight values. The electronic device adjusts the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data, including: the electronic device adjusts the pixel values of the first pixel point and the second pixel point in the first image data according to the updated first weight data.

[0018] In this implementation manner, the weight value of the pixel point corresponding to the color with high saturation is further amplified, and the weight value of the pixel point corresponding to the color with low saturation is further reduced. Since the low-reflection spectral band of the pixel point corresponding to the color with high saturation is more susceptible to noise interference, after further reducing the weight of the pixel point corresponding to the color with high saturation, it is not only beneficial to further reduce the interference of redundant pixel points (i.e., pixel points corresponding to the color with high saturation) on the spectral information of the generated illumination light source, but also beneficial to reducing the interference of noise on the spectral information of the generated illumination light source.

[0019] In an implementable manner of the first aspect, the first image data is obtained by using an image sensor. The electronic device generates the spectral information of the illumination light source according to the second image data, including: the electronic device performs feature extraction on the second image data to obtain first feature information, and the first feature information may specifically be represented as a vector including n values, and the aforementioned n values are respectively the estimated values of the responses of the image sensor to the illumination light source in n spectral channels. The electronic device generates the spectral information of the illumination light source by solving an equation according to the first feature information and the spectral sensitivities of the image sensor in n spectral bands, and the n spectral bands correspond one-to-one with the n spectral channels.

[0020] In this implementation, a specific implementation scheme for generating the spectral information of an illumination light source is provided. First, the response values of the illumination light source in n spectral channels are generated, and then the spectral information of the illumination light source is solved. Splitting a large step into two small steps is beneficial to improving the accuracy of the generation process of the spectral information of the illumination light source.

[0021] In an implementable manner of the first aspect, the method further includes: The electronic device generates first label information corresponding to m pixel points according to first weight data, where the first label information includes m label values corresponding one-to-one to the m pixel points, and is used to indicate the category of each pixel point among the m pixel points. If the classification result of the third pixel point is the first category, the label value of the third pixel point is 0; if the classification result of the third pixel point is the second category, the label value of the third pixel point is 1. The third pixel point is any one of the m pixel points, and the first category and the second category are different categories. Further, the first label information can be represented as a p×q matrix, which has the same size as each layer in the n layers (i.e., the image data of n spectral channels) included in the first image data. The electronic device sets the pixel values of the pixel points among the m pixel points that belong to the first category to zero, obtaining the updated first image data. The electronic device enhances the pixel value of the first pixel point in the first image data and weakens the pixel value of the second pixel point in the first image data, including: The electronic device enhances the pixel value of the first pixel point in the updated first image data and weakens the pixel value of the second pixel point in the updated first image data.

[0022] In this implementation, according to the first weight data corresponding to the first image data, the m pixel points in the first image data are classified, and then the pixel values of the pixel points belonging to the first category in the first image data are set to zero, so as to set the pixel values of the pixel points corresponding to high-saturation colors in the first image data to zero, that is, to further eliminate the redundant information in the first image data, so that the information of more reliable pixel points can be used to generate the spectral information of the illumination light source, thereby improving the accuracy of the generated spectral information of the illumination light source.

[0023] In an implementable manner of the first aspect, the electronic device sets the pixel values of the pixel points among the m pixel points that belong to the first category to zero according to the first label information, including: The electronic device performs a Hadamard product operation on the image data of each spectral channel in the image data of the n spectral channels included in the first image data and the first label information.

[0024] In this implementation, a specific implementation manner for setting the pixel values of the pixel points among the m pixel points that belong to the first category to zero according to the first label information is provided, which is implemented by performing a Hadamard product operation, and the operation is simple and easy to implement.

[0025] In an implementable manner of the first aspect, the method further includes: The electronic device generates the spectral reflectance ratios of the object to be photographed in n spectral bands according to the first image data and the spectral information of the illumination light source, using a third algorithm. The third algorithm includes, but is not limited to, the least squares algorithm, the particle swarm algorithm, the Wiener estimation algorithm, the pseudo-inverse algorithm, etc. The spectral reflectance ratios of the object to be photographed in n spectral bands are used to perform any one of the following operations: classifying the object to be photographed, identifying the object to be photographed, generating a visual image of the object to be photographed, analyzing the physical characteristics of the object to be photographed, analyzing the chemical characteristics of the object to be photographed, and quantitatively measuring the color of the object to be photographed.

[0026] In this implementation manner, the spectral reflectance ratios of the object to be photographed in n spectral bands are also generated according to the first image data and the spectral information of the illumination light source. Furthermore, various application scenarios of the spectral reflectance ratios of the object to be photographed in n spectral bands are enumerated, expanding the application scenarios of this solution and improving the flexibility of this solution.

[0027] In a second aspect, an embodiment of the present application provides a method for acquiring data. This method is applied to the field of image processing and includes: The training device acquires the training image data of the object to be photographed and the third weight data corresponding to the training image data. Among them, the training image data includes the image data of the object to be photographed in n spectral channels. The first image data includes m pixel points. The size and specific manifestation form of each training image data are similar to those of the first image data. The third weight data includes the annotation weights of each of the m pixel points. The weight value of the first pixel point among the m pixel points is higher than the weight value of the second pixel point among the m pixel points, and the saturation of the color corresponding to the first pixel point is lower than the saturation of the color corresponding to the second pixel point. The training device performs matrix multiplication on the training image data and the first vector to obtain the second weight data, and normalizes the m weight values included in the second weight data to generate the first weight data. Among them, the training image data is an m-by-n matrix, the first vector includes n elements, and the second weight data includes m weight values corresponding one-to-one to the m pixel points. The training device iteratively trains the element values of the n elements in the first vector according to the objective function until the preset conditions are met. Among them, the objective function indicates the similarity between the first weight data and the third weight data. The first objective function can be the angular difference between the first weight data and the third weight data, or the first objective function can also be the L2 norm function, or other types of objective functions. The preset conditions can be that the number of training times reaches the preset number of times, or the value of the first objective function is less than the preset value.

[0028] In an implementable manner of the second aspect, the training image data includes m rows of numerical values, with n numerical values in each row. The training device performs matrix multiplication on the training image data and the first vector to obtain second weight data, including: the training device arranges the numerical values in each of the m rows of numerical values in descending order to obtain a first matrix, multiplies the first matrix by the first vector to obtain the second weight data, and the n numerical values in the first vector are arranged in descending order.

[0029] For the specific implementation steps of the second aspect of the embodiments of the present application and various possible implementation manners of the second aspect, as well as the beneficial effects brought by each possible implementation manner, reference can be made to the descriptions in various possible implementation manners of the first aspect, and details will not be repeated here.

[0030] In a third aspect, an image processing device provided by the embodiments of the present application can be used in the field of image processing. The device includes: an acquisition module, configured to acquire first image data of an object to be photographed under the illumination of an illumination light source, where the first image data includes image data of the object to be photographed in n spectral channels, the first image data includes m pixel points, n is an integer greater than 3, and m is an integer greater than or equal to 1; an adjustment module, configured to adjust the pixel values of a first pixel point and a second pixel point in the first image data according to the first weight data to obtain second image data, where the first weight data includes m weight values corresponding one-to-one to the m pixel points, the first weight value corresponding to the first pixel point is higher than the second weight value corresponding to the second pixel point, and the saturation of the color corresponding to the first pixel point is lower than the saturation of the color corresponding to the second pixel point; a determination module, configured to determine the spectral information of the illumination light source according to the second image data, and the spectral information of the illumination light source indicates the spectral power distribution of the illumination light source.

[0031] The image processing device in the third aspect can also perform other steps performed by the electronic device in the first aspect. For the specific implementation steps of the third aspect of the embodiments of the present application and various possible implementation manners of the third aspect, as well as the beneficial effects brought by each possible implementation manner, reference can be made to the descriptions in various possible implementation manners of the first aspect, and details will not be repeated here.

[0032] Fourth aspect, an embodiment of the present application provides a data acquisition device, which can be used in the field of image processing. The device includes: an acquisition module, configured to acquire training image data of a photographed object and third weight data corresponding to the training image data. The training image data includes image data of the photographed object in n spectral channels, the training image data includes m pixel points, the third weight data includes the annotation weights of each of the m pixel points, and the weight value of a first pixel point among the m pixel points is higher than the weight value of a second pixel point among the m pixel points, and the saturation of the color corresponding to the first pixel point is lower than the saturation of the color corresponding to the second pixel point; a generation module, configured to perform matrix multiplication on the training image data and a first vector to obtain second weight data, and perform normalization processing on the m weight values included in the second weight data to generate first weight data, where the training image data is an m×n matrix, the first vector includes n elements, and the second weight data includes m weight values corresponding one by one to the m pixel points; a training module, configured to perform iterative training on the element values of the n elements in the first vector according to an objective function until a preset condition is met, and the objective function indicates the similarity between the first weight data and the third weight data.

[0033] The data acquisition device in the fourth aspect may also perform other steps executed by the training device in the second aspect. For the specific implementation steps of the fourth aspect and various possible implementation manners of the fourth aspect of the embodiments of the present application, as well as the beneficial effects brought by each possible implementation manner, reference may be made to the descriptions in various possible implementation manners in the second aspect, and details are not described herein again.

[0034] Fifth aspect, an embodiment of the present application provides an electronic device, which may include a processor. The processor is coupled to a memory, and the memory stores program instructions. When the program instructions stored in the memory are executed by the processor, the image processing method in the first aspect is implemented. For the steps of the electronic device in each possible implementation manner of the first aspect executed by the processor, reference may be made to the first aspect specifically, and details are not described herein again.

[0035] Sixth aspect, an embodiment of the present application provides a training device, which may include a processor. The processor is coupled to a memory, and the memory stores program instructions. When the program instructions stored in the memory are executed by the processor, the image processing method in the second aspect is implemented. For the steps of the training device in each possible implementation manner of the second aspect executed by the processor, reference may be made to the second aspect specifically, and details are not described herein again.

[0036] Seventh aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the computer is caused to execute the image processing method in the first aspect, or the computer is caused to execute the data acquisition method in the second aspect.

[0037] In an eighth aspect, an embodiment of the present application provides a circuit system, which includes a processing circuit configured to execute the image processing method of the first aspect above, or the processing circuit is configured to execute the data acquisition method of the second aspect above.

[0038] In a ninth aspect, an embodiment of the present application provides a computer program, which when running on a computer, causes the computer to execute the image processing method of the first aspect above, or causes the computer to execute the data acquisition method of the second aspect above.

[0039] In a tenth aspect, an embodiment of the present application provides a chip system, which includes a processor for implementing the functions involved in the above aspects, for example, sending or processing the data and / or information involved in the above methods. In a possible design, the chip system further includes a memory for storing the necessary program instructions and data of the server or communication device. The chip system can be composed of chips or can include chips and other discrete devices. Description of the Drawings

[0040] Figure 1 It is a schematic structural diagram of an artificial intelligence main framework;

[0041] Figure 2 It is a schematic flowchart of the image processing method provided by the embodiment of the present application;

[0042] Figure 3 It is a schematic diagram of the updated first weight data in the image processing method provided by the embodiment of the present application;

[0043] Figure 4 It is a schematic diagram of the first label information in the image processing method provided by the embodiment of the present application;

[0044] Figure 5 It is a schematic diagram of the first image data and the updated first image data in the image processing method provided by the embodiment of the present application;

[0045] Figure 6 It is two schematic diagrams of the second image data in the image processing method provided by the embodiment of the present application;

[0046] Figure 7 It is a schematic diagram of the updated first image data and the second image data in the image processing method provided by the embodiment of the present application;

[0047] Figure 8 It is a schematic diagram of the spectral information of the illumination light source in the image processing method provided by the embodiment of the present application;

[0048] Figure 9 A schematic diagram of the first feature information in the image processing method provided by the embodiment of the present application;

[0049] Figure 10 A schematic flowchart of the data acquisition method provided by the embodiment of the present application;

[0050] Figure 11 A comparison schematic diagram between the estimated values of the response values of the illumination source in n spectral channels generated by using the image processing method provided by the embodiment of the present application and the true response values of the illumination source in n spectral channels;

[0051] Figure 12 A comparison schematic diagram between the spectral information of the illumination source generated by using the image processing method provided by the embodiment of the present application and the actual spectral information of the illumination source;

[0052] Figure 13a A schematic structural diagram of the image processing device provided by the embodiment of the present application;

[0053] Figure 13b Another schematic structural diagram of the image processing device provided by the embodiment of the present application;

[0054] Figure 14 A schematic structural diagram of the data acquisition device provided by the embodiment of the present application;

[0055] Figure 15 A schematic structural diagram of the electronic device provided by the embodiment of the present application;

[0056] Figure 16 A schematic structural diagram of the training device provided by the embodiment of the present application;

[0057] Figure 17 A schematic structural diagram of the chip provided by the embodiment of the present application. Detailed implementation manners

[0058] Next, the embodiments of the present application will be described with reference to the accompanying drawings. Those skilled in the art can know that with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0059] The terms "first", "second", etc. in the description, claims and above-mentioned drawings of this application are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances, which is only a way of distinguishing objects with the same attributes when describing embodiments of this application. In addition, the terms "comprising", "having" and any variations thereof are intended to cover non-exclusive inclusion, so that a process, method, system, product or device including a series of units does not have to be limited to those units, but may include other units not clearly listed or inherent to these processes, methods, products or devices.

[0060] First, the overall working process of the artificial intelligence system will be described. Please refer to Figure 1 , Figure 1 which shows a schematic structural diagram of the artificial intelligence main framework. The above-mentioned artificial intelligence theme framework will be elaborated from two dimensions: the "intelligent information chain" (horizontal axis) and the "information technology (IT) value chain" (vertical axis). Among them, the "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be the general processes of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, data undergoes the refinement process of "data - information - knowledge - wisdom". The "IT value chain" reflects the value brought by artificial intelligence to the information technology industry from the underlying infrastructure of artificial intelligence, information (providing and processing technology implementation) to the industrial ecological process of the system.

[0061] (1) Infrastructure

[0062] The infrastructure provides computing power support for the artificial intelligence system, enables communication with the external world, and is supported through the basic platform. It communicates with the external world through sensors; the computing power is provided by intelligent chips. As an example, the intelligent chips include hardware acceleration chips such as central processing unit (CPU), neural-network processing unit (NPU), graphics processing unit (GPU), application specific integrated circuit (ASIC), and field programmable gate array (FPGA). The basic platform includes relevant platform guarantees and supports such as distributed computing frameworks and networks, and may include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the external world to obtain data, and these data are provided to the intelligent chips in the distributed computing system provided by the basic platform for computing.

[0063] (2) Data

[0064] The data at the upper layer of the infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voices, texts, and also involves the Internet of Things data of traditional devices, including the business data of existing systems and the sensed data such as force, displacement, liquid level, temperature, and humidity.

[0065] (3) Data Processing

[0066] Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making, etc.

[0067] Among them, machine learning and deep learning can perform symbolic and formal intelligent information modeling, extraction, preprocessing, training, etc. on the data.

[0068] Reasoning refers to the process of simulating the intelligent reasoning method of humans in a computer or intelligent system, and using formal information to perform machine thinking and solve problems according to the reasoning control strategy. The typical function is search and matching.

[0069] Decision-making refers to the process of making decisions after the intelligent information is reasoned, and usually provides functions such as classification, sorting, and prediction.

[0070] (4) General Capabilities

[0071] After the data is processed by the above-mentioned data processing, some general capabilities can be further formed based on the results of the data processing, such as algorithms or a general system. For example, it can be translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0072] (5) Intelligent Products and Industrial Applications

[0073] Intelligent products and industrial applications refer to the products and applications of artificial intelligence systems in various fields, which are the encapsulation of the overall artificial intelligence solution, productize intelligent information decision-making, and realize the landing application. Its application fields mainly include: intelligent terminals, intelligent manufacturing, intelligent transportation, smart homes, intelligent healthcare, intelligent security, autonomous driving, safe cities, etc.

[0074] The embodiments of this application can be applied to application scenarios that require the processing of multi-spectral images or hyperspectral images in various fields. As an example, for instance, the embodiments of this application can also be applied to intelligent monitoring in the field of intelligent security. The intelligent monitoring is a multi-spectral camera or a hyperspectral camera. The first image data of the object to be photographed under the illumination of the illumination source is collected. The first image data is a multi-spectral image or a hyperspectral image. Then, the spectral information of the illumination source in n spectral bands and the first image data can be used to obtain the spectral reflectance of the object to be photographed in n spectral bands. Furthermore, the object to be photographed can be identified according to the spectral reflectance of the photographed object in n spectral bands. n is an integer greater than 3. Since the multi-spectral image or the hyperspectral image carries the image data of the object to be photographed in multiple spectral channels, it is beneficial to improve the accuracy of the image recognition process.

[0075] As another example, a multi-spectral camera or a hyperspectral camera can be configured in an intelligent terminal to collect the first image data of the object to be photographed under the illumination of the illumination source, and use the spectral information of the illumination source in n spectral bands and the first image data to obtain the spectral reflectance of the object to be photographed in n spectral bands. Then, according to the spectral reflectance in n spectral bands, a visual image of the object to be photographed (that is, the image for the user to view in the intelligent terminal gallery) is generated, which is beneficial to improving the color reproduction accuracy of the image, the color resolution and the fidelity of the image.

[0076] As another example, a multispectral camera or a hyperspectral camera may also be configured in the electronic device to collect first image data of the object to be photographed under the illumination of the illumination source, and use the spectral information of the illumination source in n spectral bands and the first image data to obtain the spectral reflectance of the object to be photographed in n spectral bands. Furthermore, based on the spectral reflectance of the object to be photographed in n spectral bands, operations such as classifying or identifying the object to be photographed are performed to improve the accuracy of the image recognition process, and further improve the safety of the electronic device during driving, etc. It should be understood that the application scenarios of the embodiments of the present application are exhausted herein.

[0077] In the above-mentioned various application scenarios, it is necessary to obtain the spectral information of the illumination source. At present, the method of collecting the spectral information of the illumination source through a high-cost dedicated measuring instrument is not only costly but also troublesome to operate. To solve the foregoing problems, an embodiment of the present application provides an image processing method. Please refer to Figure 2 . Figure 2 FIG. is a schematic flowchart of an image processing method provided by an embodiment of the present application. The image processing method provided by an embodiment of the present application may include:

[0078] 201. The electronic device obtains first image data of the object to be photographed under the illumination of the illumination source.

[0079] In an embodiment of the present application, the electronic device obtains first image data of the object to be photographed under the illumination of the illumination source. Among them, the first image data is a multispectral image (MSI) or a hyperspectral image, which includes image data of the object to be photographed in n spectral channels. The first image data includes m pixel points, where n is an integer greater than 3 and m is a positive integer. Further, the electronic device may specifically be a complete device or a chip in a complete device. The image data of the n spectral channels included in the first image data can be regarded as n layers. The sizes of the foregoing n layers are the same, and each image includes m pixel values corresponding to the m pixel points one by one. Then, the first image data includes n pixel values of each of the m pixel points. The first image data may specifically be represented as n matrices, each matrix having m pixel values, and each of the foregoing matrices may be represented as a p-by-q matrix.

[0080] Specifically, in some application scenarios, the electronic device is specifically a device configured with a multispectral camera or a hyperspectral camera. Then, the electronic device can collect first image data of the object to be photographed under the illumination of the illumination source through the configured multispectral camera or hyperspectral camera. In some other application scenarios, the electronic device is specifically a chip, and the chip is integrated with the multispectral camera / hyperspectral camera in a complete device. Then, the chip can collect first image data of the object to be photographed under the illumination of the illumination source through the multispectral camera or hyperspectral camera in the complete device. In some other application scenarios, the electronic device is specifically a device configured with a memory, and the first image data can also be pre-stored in the electronic device. The first image data is collected of the object to be photographed under the illumination of the illumination source. In some other scenarios, the electronic device is specifically a chip, and the electronic device is integrated in a complete device, and the complete device is configured with a memory. Then, the electronic device can obtain the first image data from the memory. In some other application scenarios, the electronic device can also download the first image data through a browser, or the electronic device receives the first image data sent by other electronic devices, etc. The acquisition methods of the first image data are not enumerated here.

[0081] 202. The electronic device obtains first weight data corresponding to the first image data.

[0082] In the embodiments of the present application, after the electronic device obtains the first image data, it is necessary to obtain first weight data corresponding to the first image data. Among them, the first weight data includes m weight values corresponding one-to-one to m pixel points. The first weight value corresponding to the first pixel point among the m pixel points is higher than the second weight value corresponding to the second pixel point among the m pixel points, and the saturation of the color corresponding to the first pixel point is lower than the saturation of the color corresponding to the second pixel point. It should be noted that the first pixel point and the second pixel point are any two different pixel points among the m pixel points included in the first image data. The concepts of the first pixel point and the second pixel point are introduced here to express the concept that "the weight value of the pixel point corresponding to the color region with lower saturation is higher, and the weight value of the pixel point corresponding to the color region with higher saturation is lower" by comparing the first pixel point and the second pixel point, rather than dividing the above m pixel points into two categories.

[0083] Specifically, step 202 may include: The electronic device may convert the first image data from 3D data into 2D data, obtaining an m-by-n matrix, that is, a matrix with m rows and n columns. Each of the n pixel values included in each row of the first image data in the 2D data form represents the pixel values of the same pixel point in n layers. The electronic device performs matrix multiplication on the first image data and the first vector to obtain second weight data. Among them, the first vector includes n elements, and the element values of the n elements included in the first vector are obtained based on a pre-training operation and configured in the electronic device. The training process of the first vector will be described in subsequent embodiments and will not be elaborated here. The second weight data includes m weight values corresponding one by one to m pixel points, and the second weight data may also be represented as a p-by-q matrix. The electronic device normalizes each of the m weight values included in the second weight data to generate the first weight data.

[0084] In the embodiment of the present application, an implementation manner for generating the first weight data is provided. Since the pixel values in different multispectral images or hyperspectral images have different value ranges, the value ranges of the second weight data corresponding to different first image data are different. Normalizing the second weight data is beneficial to reducing the complexity of the subsequent processing process, and thus improving the accuracy of the spectral information of the finally generated illumination source.

[0085] More specifically, regarding the process of performing matrix multiplication on the first image data and the first vector. In one implementation manner, the first image data includes m rows of values, each row having n values, and the n values in the first vector are arranged in ascending order. The electronic device arranges each row of the m rows of values in descending order to obtain a first matrix, and multiplies the first matrix by the first vector to obtain the second weight data.

[0086] In the embodiment of the present application, since the characteristic of the pixel points corresponding to high-saturation colors is that the pixel values of several spectral channels among the n spectral channels are very high, and the pixel values of the remaining several spectral channels are very low, that is, the n pixel values corresponding to the pixel points of high-saturation colors are very unevenly distributed. Based on the foregoing characteristics, each row of the m rows of values is arranged in descending order, and the n values in the first vector are arranged in descending order, which is convenient for distinguishing the pixel points corresponding to high-saturation colors and the pixel points corresponding to low-saturation colors.

[0087] To understand this solution more intuitively, the following describes this implementation manner in the form of formulas. After the electronic device converts the first image data from 3D data into 2D matrix data, it can be represented in the following form:

[0088]

[0089] Among them, a 11 to a 1n represent the n pixel values corresponding to the first pixel point in the n layers (i.e., the image data of n spectral channels) included in the first image data, a m1 to a mn represent the n pixel values corresponding to the m-th pixel point in the n layers (i.e., the image data of n spectral channels) included in the first image data, a 11 to a 1n can also be expressed as r 1 , a 21 to a 2n can also be expressed as r 2 , a m1 to a mn can also be expressed as r m , that is, r 1 to r m can respectively correspond to each row of the m rows of data included in the 2D matrix data.

[0090] The second weight data can be calculated by the following formula:

[0091] w = Sort(M) * C = [Sort(r 1 ); Sort(r 2 );...; Sort(r m )] * C; (2)

[0092] Among them, w represents the second weight data, M represents the first image data in the form of 2D data, Sort(M) represents arranging each row of the m rows of data included in the first image data in descending order, C represents the first vector, and Sort(M) * C represents matrix multiplication of the first image data and the first vector.

[0093] In another implementation, the first image data includes m rows of values, each row has n values, and the n values in the first vector are arranged in descending order. The electronic device arranges each row of the m rows of values in ascending order to obtain the first matrix, and multiplies the first matrix by the first vector to obtain the second weight data.

[0094] The process of normalizing each weight value in the second weight data. In one implementation, the electronic device can obtain the fifth weight value with the largest value among the m weight values included in the second weight data, and divide each weight value among the m weight values included in the second weight data by the fifth weight value, that is, normalize each weight value among the m weight values, thereby obtaining the second weight data. To understand this solution more intuitively, the following shows the formula for normalizing the m weight values included in the second weight data in the form of a formula:

[0095] W N = w / Max(w); (3)

[0096] Wherein, W N represents the first weight data, w represents the second weight data, Max(w) represents the fifth weight value with the largest value among the m weight values, and w / Max(w) represents dividing each weight value among the m weight values included in the second weight data by the fifth weight value. It should be understood that the example in formula (3) is only for facilitating the understanding of this solution and does not limit this solution.

[0097] In another implementation, the electronic device can also obtain the sum of the m weight values included in the second weight data, and divide each weight value among the m weight values included in the second weight data by the sum of the foregoing m weight values, that is, normalize each weight value among the m weight values, thereby obtaining the second weight data, etc. Here, the ways for the electronic device to perform normalization are not exhausted.

[0098] Optionally, after the electronic device obtains the first weight data corresponding to the first image data, it can also increase the value of the third weight value among the m weight values and decrease the value of the fourth weight value among the m weight values to obtain the updated first weight data. Among them, the third weight value is the weight value greater than or equal to the preset threshold among the multiple weight values included in the second weight data, the fourth weight value is the weight value less than the preset threshold among the multiple weight values included in the second weight data, and the value of the preset threshold can be between 0.3 and 0.5. As an example, for example, the value of the preset threshold can be 0.35, 0.4, 0.45 or other values, etc. The specific value of the preset threshold can be set in combination with the actual application scenario and is not limited here.

[0099] Specifically, a piecewise mapping function can be pre-configured in the electronic device. The piecewise mapping function can include a first function and a second function. Each of the m weight values included in the first weight data is subjected to a piecewise mapping to obtain the updated first weight data. That is, when one of the m weight values included in the first weight data is less than the preset threshold, the first function is executed; when one of the m weight values included in the first weight data is greater than or equal to the preset threshold, the second function is executed. To understand this solution more intuitively, the following describes this implementation method in the form of a formula:

[0100] W L =mapping(W N );(4)

[0101] Where, W L represents the updated first weight data, and mapping(W N ) means that each of the m weight values included in the first weight data is subjected to a piecewise mapping. Further, the following formula can be used in the process of piecewise mapping:

[0102]

[0103] Where, u represents any one of the weight values in W N (that is, the first weight data), k represents the preset threshold, exp represents the exponential function with e as the base, and both a and b are two hyperparameters. As an example, for example, the value of a can be 50, and the value of b can be 0.6. Specifically, the values of a and b can be determined in combination with the actual application scenario.

[0104] It should be understood that Equation (5) is only an example for facilitating the understanding of this solution. What can also be pre-stored in the electronic device is other types of piecewise mapping functions. As another example, for example, when one of the m weight values included in the first weight data is less than the preset threshold, the electronic device divides the aforementioned one weight value by a first value, and the first value is greater than 1. As an example, for example, the value of the first value can be 1.1, 1.2, 1.3, 2, 5 or other values, etc., which are not limited here. When one of the m weight values included in the first weight data is greater than or equal to the preset threshold, the electronic device multiplies the aforementioned one weight value by a second value, and the second value is greater than 1. As an example, for example, the value of the second value can be 1.1, 1.2, 1.3, 1.4, 1.5, etc., and no exhaustive listing is made here. Or, when one of the m weight values included in the first weight data is greater than or equal to the preset threshold, the electronic device adds a third value to the aforementioned one weight value, and the third value is a positive number. As an example, for example, the third value can be 0.3, 0.4, 0.5, 0.6, etc., and no exhaustive listing is made here.

[0105] To more intuitively understand this solution, in the embodiments of this application, the updated first weight data is converted into the form of a weight map. Please refer to Figure 3 , Figure 3 which is a schematic diagram of the updated first weight data in the image processing method provided by the embodiments of this application. Figure 3 It includes two sub-schematic diagrams (a) and (b). Figure 3 The sub-schematic diagram (a) of represents the first image data. It should be noted that the first image data is invisible, and it is only shown here after visualizing the first image data for the convenience of understanding this solution. Figure 3 The sub-schematic diagram (b) of represents the updated first weight map of the first image data corresponding to the sub-schematic diagram (a) of Figure 3 A1, A2, A3, and A4 respectively correspond to 4 specular points of the object being photographed. The specular points are white, and the saturation of the color is the lowest, so the assigned weight value is the highest. It can be clearly seen from the sub-schematic diagram (b) of Figure 3 that the color depth of A1, A2, A3, and A4 is the deepest, indicating that the weight values of the pixel points in these 4 regions are the highest. It should be understood that Figure 3 the examples in are only for the convenience of understanding this solution and are not used to limit this solution.

[0106] 203. The electronic device generates m tag values corresponding one-to-one to m pixel points according to the first weight data.

[0107] In some embodiments of this application, the electronic device can also classify the m pixel points according to the first weight data or the updated first weight data to generate first label information corresponding to the m pixel points. Among them, the first label information includes m tag values corresponding one-to-one to the m pixel points, and is used to indicate the category of each pixel point among the m pixel points. If the classification result of the third pixel point is the first category, the tag value of the third pixel point is 0. If the classification result of the third pixel point is the second category, the tag value of the third pixel point is 1. The third pixel point is any one of the m pixel points, and the first category and the second category are different categories. The higher the weight value of the pixel point, the greater the probability of being classified into the first category. Further, the first label information can be represented as a p-by-q matrix, which is the same size as each layer in the n layers (i.e., the image data of n spectral channels) included in the first image data.

[0108] Specifically, the process of classifying m pixel points. The electronic device can use a clustering algorithm to classify the m weight values included in the first weight data (or the updated first weight data), that is, classify the m pixel points corresponding one-to-one to the m weight values, and obtain a first classification result. Among them, the aforementioned clustering algorithm can be a K-means clustering algorithm, support vector machines (SVM), logistic regression algorithm, or other clustering algorithms, etc., which will not be enumerated here. The first classification result can specifically be expressed as a p×q matrix, which includes m classification values corresponding one-to-one to the m pixel points.

[0109] Further, in some implementation manners, the classification value corresponding to a reliable pixel point (that is, the second category) in the first classification result is 1, and the classification value corresponding to an unreliable pixel point (that is, the first category) is 0. Then, the electronic device can directly determine the generated first classification result as the first label information. In other implementation manners, if the values included in the first classification result cannot be directly determined as the first label information, the electronic device can convert the first classification result to obtain the first label information. As an example, for instance, the clustering algorithm used is the K-means clustering algorithm, and the classification value of a reliable pixel point in the obtained first classification result is 1, and the classification value of an unreliable pixel point is 2. Then, the first label information corresponding to the m pixel points can be generated according to the first classification result and by using the binarization method. It should be understood that the example here is only for facilitating the understanding of this solution and is not used to limit this solution.

[0110] To more intuitively understand this solution, in the embodiments of this application, the first label information is converted into a schematic diagram form. Please refer to Figure 4 , Figure 4 which is a schematic diagram of the first label information in the image processing method provided by the embodiments of this application. Figure 4 It includes two sub-schematic diagrams (a) and (b), Figure 4 The sub-schematic diagram (a) of Figure 4 represents the first image data, Figure 4 The sub-schematic diagram (b) of Figure 4 represents the label map (LM) corresponding to each pixel point in the first image data corresponding to the sub-schematic diagram (a) of Figure 4 Since there are only two values, 1 or 0, for the label values in the first label information, there are only black and white in the schematic diagram of the first label information. Figure 4The examples in this are only for facilitating the understanding of this solution and are not used to limit this solution.

[0111] 204. The electronic device sets the pixel values of the pixel points of the first category among the m pixel points to zero to obtain updated first image data.

[0112] In some embodiments of this application, after the electronic device generates m tag values corresponding one-to-one to the m pixel points, it can set the pixel values of the pixel points of the first category among the m pixel points to zero to obtain updated first image data. Specifically, step 204 may include: The electronic device performs a Hadamard product operation on the image data of each spectral channel in the image data of the n spectral channels included in the first image data and the first tag information, so as to set the pixel values of the unreliable pixel points among the m pixel points to zero, and obtain updated first image data.

[0113] More specifically, the image data of each spectral channel can be specifically represented as a p×q matrix, and the first image data is represented as a three-dimensional tensor, which includes n p×q matrices; the first tag information can be specifically represented as a p×q matrix. Then, in one implementation, the electronic device sequentially obtains the image data of each spectral channel in the first image data, and performs a Hadamard product operation on the image data of a single spectral channel and the first tag information. In another implementation, the electronic device generates a first tensor according to the first tag information, and the first tensor is a three-dimensional tensor, which includes n first tag information, and the electronic device directly performs a Hadamard product operation on the first image data and the first tensor. For a more intuitive understanding of this solution, the formula used for the Hadamard product operation is disclosed as follows:

[0114]

[0115] where, MSI d represents the updated first image data, LM 1 represents the first tensor, which is generated based on the first tag information, MSI represents the first image data, represents performing a Hadamard product operation on the first tensor and the first image data. It should be understood that the examples in formula (6) are only for facilitating the understanding of this solution and are not used to limit this solution.

[0116] For a more intuitive understanding of this solution, in the embodiments of this application, the first image data and the updated first image data are visualized. Please refer to Figure 5 , Figure 5 which is a schematic diagram of the first image data and the updated first image data in the image processing method provided by the embodiments of this application. Figure 5 It includes two sub-schematic diagrams (a) and (b), Figure 5 The sub-schematic diagram (a) of which represents the first image data,Figure 5 The (b) sub-schematic diagram of Figure 5 represents the updated first image data. It should be noted that both the first image data and the updated first image data are invisible. Here, for the convenience of understanding this solution, the first image data and the updated first image data are visualized and then displayed. Comparing Figure 5 the (a) sub-schematic diagram of Figure 5 and Figure 5 the (b) sub-schematic diagram of Figure 5 , it can be seen that after setting the pixel values of unreliable pixel points to zero, the redundant information to be processed becomes less, which is more convenient for obtaining the accurate spectral information of the illumination light source. It should be understood that Figure 5 the examples in Figure 5 are only for the convenience of understanding this solution and are not used to limit this solution.

[0117] Optionally, after the electronic device sets the pixel values of the pixel points of the first category among the m pixel points to zero to obtain the updated first image data, it can also perform denoising processing on the updated first image data to generate the denoised first image data. Specifically, the electronic device can input the updated first image data (that is, the n layers after setting the pixel values of the pixel points of the first category to zero) into a low-pass filter respectively, so as to perform denoising operations and remove bad pixel operations through the low-pass filter. Among them, the low-pass filter includes but is not limited to Gaussian filter, mean filter, median filter or other low-pass filters, etc., which are not limited here.

[0118] For a more intuitive understanding of this solution, here, taking the low-pass filter as a Gaussian filter as an example, is disclosed

[0119] MSI e = GF5{MSI d}; (7)

[0120] Among them, MSI e represents the denoised first image data, GF5 represents the Gaussian filter with a window size of 5, and MSI d represents the updated first image data. It should be noted that the example in formula (7) is only for the convenience of understanding this solution. In actual situations, other types of low-pass filters can also be used, and the window size can also be other values, which are not limited here.

[0121] 205. The electronic device adjusts the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data to obtain the second image data.

[0122] In the embodiment of this application, if the first weight data is generated in step 202, the electronic device adjusts the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data to obtain the second image data.

[0123] Specifically, if the first weight data has been normalized, that is, the weight value corresponding to each pixel among the m pixels included in the first image data is between 0 and 1, step 205 may include: The electronic device reduces the pixel values of the first pixel and the second pixel in the first image data according to the first weight data, and the reduction ratio of the pixel value of the second pixel is greater than the reduction ratio of the pixel value of the first pixel. In the embodiment of the present application, the reduction ratio of the pixel value of the second pixel is greater than the reduction ratio of the pixel value of the first pixel, so as to achieve the purpose that the pixels corresponding to the low-saturation colors in the second image data will be more obvious, and by reducing the pixel values of the first pixel and the second pixel in the first image data, the occurrence of too high pixel values in the second image data is avoided.

[0124] More specifically, step 205 may include: The electronic device multiplies the pixel value of the first pixel in the first image data by the first weight value to obtain the pixel value of the first pixel in the second image data; multiplies the pixel value of the second pixel in the first image data by the second weight value to obtain the pixel value of the second pixel in the second image data. In the embodiment of the present application, a specific implementation manner of adjusting the pixel values of the first pixel and the second pixel in the first image data according to the first weight data is provided, which is realized by multiplying the pixel by the corresponding weight value, and the operation is simple and easy to implement.

[0125] Further, in one implementation manner, the electronic device may perform a Hadamard product operation on the image data of each spectral channel in the image data of the n spectral channels included in the first image data and the first weight data to adjust the pixel values of the first pixel and the second pixel in the first image data to obtain the second image data.

[0126] Furthermore, the image data of each spectral channel may specifically be represented as a p×q matrix, and the first image data is represented as a three-dimensional tensor, which includes n p×q matrices; the first weight data may specifically be represented as a p×q matrix. Then in one implementation manner, the electronic device sequentially obtains the image data of each spectral channel in the first image data and performs a Hadamard product operation on the image data of a single spectral channel and the first weight data. In another implementation manner, the electronic device generates a second tensor according to the first weight data, and the second tensor is a three-dimensional tensor, which includes n first weight data, and the electronic device directly performs a Hadamard product operation on the first image data and the second tensor.

[0127] In another implementation, the electronic device can also directly obtain the target weight value corresponding to the target pixel point from the first weight data, and directly multiply the target pixel point by the target weight value. The target pixel point is any one of the m pixel points. The electronic device performs the foregoing operation on each of the m pixel points to adjust the pixel values of the first pixel point and the second pixel point in the first image data.

[0128] If the updated first weight data is generated in step 202, the electronic device adjusts the pixel values of the first pixel point and the second pixel point in the first image data according to the updated first weight data. Specifically, the electronic device reduces the pixel values of the first pixel point and the second pixel point in the first image data according to the updated first weight data, and the reduction ratio of the pixel value of the second pixel point is greater than the reduction ratio of the pixel value of the first pixel point. More specifically, step 205 may include: the electronic device multiplies the pixel value of the first pixel point in the first image data by the updated first weight value to obtain the pixel value of the first pixel point in the second image data; multiplies the pixel value of the second pixel point in the first image data by the updated second weight value to obtain the pixel value of the second pixel point in the second image data.

[0129] Further, the electronic device performs a Hadamard product operation on the image data of each spectral channel in the n spectral channel image data included in the first image data with the updated first weight data, so as to enhance the pixel value of the first pixel point in the first image data and weaken the pixel value of the second pixel point in the first image data, and obtain the second image data.

[0130] In the embodiment of the present application, the weight value corresponding to the pixel point with high saturation is further amplified, and the weight value corresponding to the pixel point with low saturation is reduced. Since the low-reflection spectral band of the pixel point corresponding to the high-saturation color is more susceptible to noise interference, after further reducing the weight of the pixel point corresponding to the high-saturation color, it is not only beneficial to further reduce the interference of redundant pixel points (i.e., pixel points corresponding to high-saturation colors) on the spectral information of the generated illumination light source, but also beneficial to reducing the interference of noise on the spectral information of the generated illumination light source.

[0131] Further, steps 203 and 204 are optional steps. If steps 203 and 204 are executed, step 205 may include: the electronic device adjusts the pixel values of the first pixel point and the second pixel point in the updated first image data (or the denoised first image data) generated in step 204 according to the first weight data (or the updated first weight data) generated in step 202 to obtain the second image data. For the specific implementation manner of the electronic device to adjust the pixel value of the pixel point, reference may be made to the above description, and details are not described herein again.

[0132] Specifically, the electronic device performs a Hadamard product operation on the image data of each spectral channel in the updated first image data (or the denoised first image data) generated in step 204 and the first weight data (or the updated first weight data) generated in step 202, so as to adjust the pixel values of the first pixel and the second pixel in the updated first image data (or the denoised first image data) generated in step 204, and obtain second image data.

[0133] In the embodiment of the present application, according to the first weight data corresponding to the first image data, m pixel points in the first image data are classified, and then the pixel values of the pixel points of the first category in the first image data are set to zero, so as to set the pixel values of the pixel points corresponding to the high-saturation colors in the first image data to zero, that is, to further eliminate the redundant information in the first image data, so that the information of more reliable pixel points can be used to generate the spectral information of the illumination light source, so as to improve the accuracy of the generated spectral information of the illumination light source.

[0134] If steps 203 and 204 are not executed, step 205 may include: the electronic device adjusts the pixel values of the first pixel and the second pixel in the first image data obtained in step 201 according to the first weight data (or the updated first weight data) generated in step 202, and obtains second image data.

[0135] Specifically, the electronic device performs a Hadamard product operation on the image data of each spectral channel in the first image data obtained in step 201 and the first weight data (or the updated first weight data) generated in step 202, so as to adjust the pixel values of the first pixel and the second pixel in the first image data obtained in step 201, and obtain second image data.

[0136] For a more intuitive understanding of this solution, please refer to Figure 6 and Figure 7 , Figure 6 which are two schematic diagrams of the second image data in the image processing method provided by the embodiment of the present application. Figure 6 Taking the non-execution of steps 203 and 204 as an example, Figure 6 it includes two sub-schematic diagrams (a) and (b). Figure 6 The sub-schematic diagram (a) of Figure 6 represents the second image data generated according to the first image data and the first weight data, and is obtained after visualizing the aforementioned second image data. Figure 6The sub-diagram (b) represents the second image data generated based on the first image data and the updated first weight data (i.e., after increasing the value of the third weight value among the m weight values and decreasing the value of the fourth weight value among the m weight values), and is obtained by visualizing the aforementioned second image data. Comparing Figure 6 with the sub-diagram (a) of Figure 6 and the sub-diagram (b) of Figure 6 it can be seen that there is less redundant information in the sub-diagram (b) of

[0137] Continuing to refer to Figure 7 , Figure 7 Taking the execution of steps 203 and 204 as an example, the updated first image data and second image data generated in step 204 are visualized. Please refer to Figure 7 , Figure 7 which is a schematic diagram of the updated first image data and second image data in the image processing method provided by the embodiment of the present application. Figure 7 For the purpose of giving an example in combination with Figure 3 , that is, the first image data corresponding to Figure 7 is the sub-diagram (a) of Figure 3 , Figure 7 which includes two sub-diagrams (a) and (b). Figure 7 The sub-diagram (a) of Figure 7 represents the updated first image data, and the sub-diagram (b) of Figure 7 represents the second image data. It should be noted that both the updated first image data and the second image data are invisible, and here they are only shown after being visualized for the convenience of understanding this solution. Comparing the sub-diagram (a) of Figure 7 and the sub-diagram (b) of Figure 7 it can be seen that the pixel points corresponding to the high-saturation colors in the second image data are further weakened, and the pixel points corresponding to the low-saturation colors in the second image data are further highlighted, so that Figure 7 the effective information in the sub-diagram (b) of

[0138] 206. The electronic device determines the spectral information of the illumination light source according to the second image data.

[0139] In an embodiment of the present application, after obtaining the second image data, the electronic device may determine the spectral information of the illumination light source according to the second image data. The spectral information of the illumination light source is used to indicate the spectral power distribution (SPD) of the illumination light source in multiple spectral bands. The spectral information of the illumination light source may include multiple sets of intensity values corresponding one-to-one to colored lights of multiple different wavelengths, and each set of intensity values represents the value of the radiant energy of each colored light in the illumination light source. To understand this solution more intuitively, please refer to Figure 8 , Figure 8 which is a schematic diagram of the spectral information of the illumination light source in the image processing method provided in the embodiment of the present application. Figure 8 Taking the example of showing part of the spectral information of the illumination light source in the form of a coordinate graph, since generally the illumination light source is a composite light composed of colored lights of different wavelengths, Figure 8 is the spectral power distribution graph of the illumination light source, Figure 8 the abscissa of Figure 8 represents the wavelength value of each colored light in multiple colored lights, and it should be noted that Figure 8 the example in

[0140] is only for facilitating the understanding of this solution, and the spectral information of the illumination light source can also be expressed in other ways, which will not be enumerated here. Specifically, the first image data is obtained by using an image sensor, and the electronic device extracts features from the second image data to obtain the first feature information. The first feature information may specifically be represented as a vector including n values, and the aforementioned n values are respectively the estimated values of the response values of the image sensor for the illumination light source in n spectral channels. The electronic device generates the spectral information of the illumination light source according to the first feature information and the spectral sensitivity of the image sensor in n spectral bands. In the embodiment of the present application, a specific implementation scheme for generating the spectral information of the illumination light source is provided. First, the response values of the illumination light source in n spectral channels are generated, and then the spectral information of the illumination light source is solved. Splitting a large step into two small steps is beneficial to improving the accuracy of the generation process of the spectral information of the illumination light source.

[0141] More specifically, it is a process of extracting features from the second image data by an electronic device. The electronic device can convert the second image data into data in the form of two-dimensional data, that is, convert from a three-dimensional tensor (including n matrices, each matrix being a p×q matrix) to a two-dimensional matrix of m rows and n columns. In one implementation, the electronic device can extract features from the second image data in the form of two-dimensional data to directly obtain the first feature information. As an example, after obtaining a two-dimensional matrix of m rows and n columns, the average value of each column of the aforementioned n column values is obtained, so as to obtain n values included in the first feature information. As another example, the electronic device can also input the second image data into a feature extraction network to output the first feature information through the feature extraction network, etc.

[0142] In another implementation, the electronic device uses a first algorithm to extract features from the second image data in the form of two-dimensional data (which can also be called dimensionality reduction processing) to obtain second feature information. The second feature information is a two-dimensional matrix of k rows and n columns, and then the first feature information is obtained from the second feature information. The first feature information is a vector including n values. Among them, the first algorithm can be a principal components analysis (PCA) algorithm, a singular value decomposition algorithm, or other algorithms, etc. For a more intuitive understanding of this solution, the following takes the example of using the PCA algorithm to extract features from the first image data to disclose the formula used in the process of generating the first feature information:

[0143] Coff = PCA(F 2D ); (8)

[0144] Among them, Coff represents the second feature information, and F 2D represents the second image data in the form of two-dimensional data. PCA(F 2D ) represents using the PCF algorithm to extract features from the second image data in the form of two-dimensional data.

[0145] After the electronic device obtains the second feature information through the PCA algorithm, the first principal component information in the second feature information is used as the first feature information, that is:

[0146]

[0147] Among them, represents the first feature information, that is, the estimated value of the response value of the image sensor to the illumination light source in n spectral channels. Coff(:,1) represents obtaining the first row of data from the k×n column data included in the first feature information. It should be understood that the examples in formulas (8) and (9) are only for facilitating the understanding of this solution and are not limited here.

[0148] To understand this solution more intuitively, please refer to Figure 9 , Figure 9 which is a schematic diagram of the first feature information in the image processing method provided by an embodiment of this application. Figure 9 Taking the form of a coordinate graph to display the normalized first feature information as an example, Figure 9 the abscissa of Figure 9 represents the n spectral channels corresponding to the first image data. Taking n = 8 as an example, Figure 9 B1, B2, B3, B4, B5, B6, B7, and B8 in Figure 9 respectively represent the estimated values of the responses of the image sensor to the illumination light source in the n spectral channels. It should be noted that

[0149] The examples in

[0150] are only for facilitating the understanding of this solution. The spectral information of the illumination light source can also be expressed in other ways, which will not be elaborated here.

[0151]

[0152] Regarding the process of generating the spectral information of the illumination light source. After obtaining the first feature information, the electronic device can generate the spectral information of the illumination light source by solving equations according to the first feature information and the spectral sensitivities of the image sensor in the n spectral bands, and the n spectral bands correspond one-to-one with the n spectral channels. T , that is, shown by the following formula:

[0153] L CH = S·L T ; (11)

[0154] where L CH represents the response value of the image sensor to the illumination light source in the n spectral channels. Equation (11) can be equivalent to the following equation (12):

[0155]

[0156] Among them, Φ represents the regularization constraint matrix, α is a hyperparameter representing the regularization constraint coefficient, I represents the identity matrix, D takes the value of 0, diag() represents generating a diagonal matrix, and the meanings of L CH and S are referred to the descriptions in Equations (10) and (11), which will not be elaborated here. To avoid the problem of overfitting in the solved spectral information of the illumination source, Φ can be set as follows:

[0157]

[0158] Combined with the above principle, it can be known that the process of the electronic device generating the spectral information of the illumination source according to the first feature information and the spectral sensitivities of the image sensor in n spectral bands can be converted into the process of solving the spectral information of the illumination source according to the first feature information and the spectral sensitivities of the image sensor in n spectral bands by using a second algorithm. The second algorithm includes but is not limited to the least squares algorithm, the particle swarm algorithm, the genetic algorithm, or other algorithms.

[0159] To understand this solution more intuitively, taking the solution of the equation by the least squares algorithm as an example below, the spectral information of the illumination source can be generated by solving the following formula:

[0160]

[0161] Among them, represents the generated spectral information of the illumination source. The meanings of each letter in Equation (13) can be referred to the descriptions in the above Equations (10) to (12), which will not be elaborated here. It should be understood that Equation (13) is only an example for facilitating the understanding of this solution and is not limited here.

[0162] Optionally, after the electronic device generates the spectral information of the illumination source, it can also generate the spectral reflectance ratios of the object to be photographed in n spectral bands according to the first image data and the spectral information of the illumination source by using a third algorithm. Among them, the third algorithm includes but is not limited to the least squares algorithm, the particle swarm algorithm, the Wiener estimation algorithm, the pseudo-inverse algorithm, etc., and will not be enumerated here.

[0163] Among them, the spectral reflectance ratios of the object to be photographed in n spectral bands are used to perform any one of the following operations: classifying the object to be photographed, identifying the object to be photographed, generating a visualization image of the object to be photographed, analyzing the physical properties of the object to be photographed, analyzing the chemical properties of the object to be photographed, and quantitatively measuring the color of the object to be photographed.

[0164] Further, since different objects have different spectral reflectances for the n spectral bands, the electronic device may input the spectral reflectances of the object to be photographed in the n spectral bands into a neural network for image classification, so as to classify the object to be photographed through the neural network. Alternatively, the electronic device may also use other algorithms other than the neural network to classify the object to be photographed according to the spectral reflectances of the object to be photographed in the n spectral bands.

[0165] The electronic device may also input the spectral reflectances of the object to be photographed in the n spectral bands into a neural network for image recognition, so as to recognize the object to be photographed through the neural network. Alternatively, the electronic device may also use other algorithms other than the neural network to recognize the object to be photographed according to the spectral reflectances of the object to be photographed in the n spectral bands.

[0166] The electronic device may further input the spectral reflectances of the object to be photographed in the n spectral bands into a neural network for physical property analysis, so as to analyze the physical properties of the object to be photographed through the neural network. Alternatively, the electronic device may also use other algorithms other than the neural network to analyze the physical properties of the object to be photographed according to the spectral reflectances of the object to be photographed in the n spectral bands.

[0167] The electronic device may further input the spectral reflectances of the object to be photographed in the n spectral bands into a neural network for chemical property analysis, so as to analyze the chemical properties of the object to be photographed through the neural network. Alternatively, the electronic device may also use other algorithms other than the neural network to analyze the chemical properties of the object to be photographed according to the spectral reflectances of the object to be photographed in the n spectral bands.

[0168] The electronic device may use other algorithms other than the neural network to perform quantitative measurement of the color of the object to be photographed according to the spectral reflectances of the object to be photographed in the n spectral bands.

[0169] The electronic device may further generate a visualization image of the object to be photographed according to the spectral reflectances of the object to be photographed in the n spectral bands and the spectral information of the illumination light source. In the embodiments of the present application, the spectral reflectances of the object to be photographed in the n spectral bands are also generated according to the first image data and the spectral information of the illumination light source, and then various application scenarios of the spectral reflectances of the object to be photographed in the n spectral bands are enumerated, expanding the application scenarios of the present solution and improving the flexibility of the present solution.

[0170] In an embodiment of the present application, a solution for directly obtaining the spectral information of a light source from image data is provided. There is no longer a need to measure through high-cost dedicated measuring instruments, no additional operations need to be performed, and labor costs are saved. In addition, a higher weight is assigned to the first pixel points corresponding to low-saturation colors, a lower weight is assigned to the second pixel points corresponding to high-saturation colors, and according to the weight values of each pixel point, the pixel values of the first pixel points and the second pixel points in the first image data are adjusted to obtain the second image data. The pixel points corresponding to low-saturation colors in the second image data will be more obvious, that is, the interference brought by the pixel points corresponding to high-saturation colors in the second image data is less. Furthermore, based on the second image data, the spectral information of the illumination light source is generated. Since the high-saturation color regions of the object being photographed absorb light in most spectral bands of the illumination light source, it is not easy to extract the spectral information of the illumination light source in all spectral bands from the pixel points corresponding to high-saturation colors. And because the low-saturation color regions of the object being photographed can reflect the light of the illumination light source in each band more comprehensively, it is not easy to extract the spectral information of the illumination light source in all spectral bands from the pixel points corresponding to low-saturation colors. Therefore, relatively accurate spectral information of the illumination light source in n spectral bands can be obtained from the second image data.

[0171] An embodiment of the present application also provides a method for obtaining data. Please refer to Figure 10 , Figure 10 which is a schematic flowchart of a method for obtaining data provided by an embodiment of the present application. The method for obtaining data provided by an embodiment of the present application may include:

[0172] 1001. The training device obtains the training image data of the object being photographed and the third weight data corresponding to the training image data.

[0173] In an embodiment of the present application, multiple training image data and the third weight data corresponding to each training image data may be pre-configured in the training device. Among them, the training image data includes the image data of the object being photographed in n spectral channels. The first image data includes m pixel points. The size and specific form of each training image data may refer to Figure 2 the description of the first image data in the corresponding embodiment. Please refer to the above description and details will not be elaborated here. The third weight data includes the labeled weights of each of the m pixel points. The weight value of the first pixel point among the m pixel points is higher than the weight value of the second pixel point among the m pixel points. The saturation of the color corresponding to the first pixel point is lower than the saturation of the color corresponding to the second pixel point;

[0174] 1002. The training device performs matrix multiplication on the training image data and the first vector to obtain the second weight data.

[0175] In the embodiments of the present application, the training device may perform matrix multiplication on the training image data and the first vector obtained by initialization, or perform matrix multiplication on the training image data and the first vector generated in the previous training process to obtain the second weight data. The training image data is an m-by-n matrix, the first vector includes n elements, and the second weight data includes m weight values corresponding one-to-one to m pixel points. For the specific implementation of step 1002, reference may be made to Figure 2 the description in step 202 of the corresponding embodiment, which will not be elaborated here.

[0176] 1003. The training device normalizes the m weight values included in the second weight data to generate the first weight data.

[0177] In the embodiments of the present application, for the specific implementation of step 1002, reference may be made to Figure 2 the description in step 202 of the corresponding embodiment, which will not be elaborated here.

[0178] 1004. The training device iteratively trains the element values of the n elements in the first vector according to the first objective function until a preset condition is met. The first objective function indicates the similarity between the first weight data and the third weight data.

[0179] In the embodiments of the present application, after the training device generates the first weight data, it generates the function value of the first objective function according to the first weight data generated in step 1003 and the third weight data obtained in step 1001, and performs gradient derivation according to the function value of the first objective function to reversely update the element values of the n elements in the first vector, so as to complete one training of the element values of the n elements in the first vector. After the training device executes step 1004, it re-enters step 1001 to perform the next training on the first vector. The training device repeatedly executes steps 1001 to 1004 to iteratively train the element values of the n elements in the first vector until a preset condition is met, and obtains the trained first vector. Figure 2 The first vector in the corresponding embodiment is the trained first vector.

[0180] The first objective function indicates the similarity between the first weight data and the third weight data. As an example, for example, the first objective function may be the angular difference between the first weight data and the third weight data, or the first objective function may also be an L2 norm function or other types of objective functions. The preset condition may be that the number of training times reaches a preset number, or the value of the first objective function is less than a preset value.

[0181] It should be noted that Figure 2 the electronic device in the corresponding embodiment and Figure 10 the training device in the corresponding embodiment may be the same device or different devices.

[0182] In the embodiments of the present application, a training step for the first vector is further provided, which improves the integrity of the present solution.

[0183] In order to have a more intuitive understanding of the beneficial effects brought by the embodiments of the present application, the following combines Figure 11 and Figure 12 to further introduce the beneficial effects brought by the embodiments of the present application. Figure 11 and Figure 12 Both take the CIE standard light source as an example for the illumination light source. Figure 11 FIG. is a comparison schematic diagram between the estimated values of the response values of the illumination light source generated by using the image processing method provided in the embodiments of the present application in n spectral channels and the true response values of the illumination light source in n spectral channels. Figure 11 In FIG., it is the value after normalizing the aforementioned response values. Figure 11 The abscissa of FIG. represents 8 spectral channels. Figure 11 The ordinate of FIG. represents the response values in each spectral channel. The broken line pointed by C1 represents the n true response values of the illumination light source in n spectral channels, and the broken line pointed by C2 represents the estimated values of the n response values of the illumination light source in n spectral channels. It can be seen from Figure 11 that the accuracy of the estimated values of the response values of the illumination light source generated by using the image processing method provided in the embodiments of the present application in n spectral channels is relatively high.

[0184] Continuing to refer to Figure 12 , Figure 12 FIG. is a comparison schematic diagram between the spectral information of the illumination light source generated by using the image processing method provided in the embodiments of the present application and the actual spectral information of the illumination light source. Figure 12 The abscissa of FIG. represents the wavelength values of each color light among multiple color lights. Figure 8 The ordinate of FIG. represents the numerical value of the radiant energy of each color light in the illumination light source. The broken line pointed by D1 represents the true spectral information of the illumination light source, and the broken line pointed by D2 represents the spectral information of the illumination light source generated by using the image processing method provided in the embodiments of the present application. It can be seen from Figure 12 that the accuracy of the spectral information of the illumination light source generated by using the image processing method provided in the embodiments of the present application is relatively high.

[0185] Based on the embodiments corresponding to Figures 1 to 10 , in order to better implement the above solution of the embodiments of the present application, relevant devices for implementing the above solution are further provided below. Specifically, refer to Figure 13a , Figure 13aA schematic structural diagram of an image processing apparatus provided by an embodiment of the present application. The image processing apparatus 1300 may include an acquisition module 1301, an adjustment module 1302, and a determination module 1303. Among them, the acquisition module 1301 is configured to acquire first image data of an object to be photographed under the illumination of an illumination light source. The first image data includes image data of the object to be photographed in n spectral channels. The first image data includes m pixel points, where n is an integer greater than 3, and m is an integer greater than or equal to 1. The adjustment module 1302 is configured to adjust the pixel values of a first pixel point and a second pixel point in the first image data according to first weight data to obtain second image data. Among them, the first weight data includes m weight values corresponding one-to-one to the m pixel points. The first weight value corresponding to the first pixel point is higher than the second weight value corresponding to the second pixel point, and the saturation of the color corresponding to the first pixel point is lower than the saturation of the color corresponding to the second pixel point. The determination module 1303 is configured to determine spectral information of the illumination light source according to the second image data, and the spectral information of the illumination light source is used to indicate the spectral power distribution of the illumination light source.

[0186] In a possible design, the adjustment module 1302 is specifically configured to reduce the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data, and the reduction ratio of the pixel value of the second pixel point is greater than the reduction ratio of the pixel value of the first pixel point.

[0187] In a possible design, the adjustment module 1302 is specifically configured to: multiply the pixel value of the first pixel point in the first image data by the first weight value to obtain the pixel value of the first pixel point in the second image data; multiply the pixel value of the second pixel point in the first image data by the second weight value to obtain the pixel value of the second pixel point in the second image data.

[0188] In a possible design, please refer to Figure 13b , Figure 13b A schematic structural diagram of an image processing apparatus provided by an embodiment of the present application. The apparatus 1300 further includes: a generation module 1304, configured to perform matrix multiplication on the first image data and a first vector to obtain second weight data. Among them, the first image data is an m-by-n matrix, the first vector includes n elements, and the second weight data includes m weight values corresponding one-to-one to the m pixel points. The generation module 1304 is further configured to perform normalization processing on the m weight values included in the second weight data to generate first weight data.

[0189] In a possible design, the adjustment module 1302 is further configured to increase the value of the third weight value among the m weight values and decrease the value of the fourth weight value among the m weight values to obtain updated first weight data, where the third weight value is a weight value greater than or equal to a preset threshold, and the fourth weight value is a weight value less than the preset threshold; the adjustment module 1302 is specifically configured to enhance the pixel value of the first pixel point in the first image data and weaken the pixel value of the second pixel point in the first image data according to the updated first weight data.

[0190] In a possible design, the first image data is obtained by using an image sensor. The determination module 1303 is specifically configured to: extract features from the second image data to obtain first feature information, where the first feature information is an estimated value of the response value of the image sensor for the illumination light source in n spectral channels; generate spectral information of the illumination light source according to the first feature information and the spectral sensitivity of the image sensor in n spectral bands, where the n spectral bands correspond one-to-one to the n spectral channels.

[0191] In a possible design, please refer to Figure 13b , the apparatus 1300 further includes: a classification module 1305, configured to classify m pixel points according to the first weight data to obtain m label values corresponding one-to-one to the m pixel points, where the m label values are used to indicate the categories of the m pixel points; an update module 1306, configured to set the pixel values of the pixel points of the first category among the m pixel points to zero to obtain updated first image data; the adjustment module 1302 is specifically configured to adjust the magnitudes of the pixel values of the first pixel point and the second pixel point in the updated first image data.

[0192] In a possible design, please refer to Figure 13b , the apparatus 1300 further includes: a generation module 1304, configured to generate spectral reflectance ratios of the object to be photographed in n spectral bands according to the first image data and the spectral information of the illumination light source, where the spectral reflectance ratios of the object to be photographed in n spectral bands are used to perform any one of the following operations: classifying the object to be photographed, identifying the object to be photographed, generating a visualization image of the object to be photographed, analyzing physical characteristics of the object to be photographed, analyzing chemical characteristics of the object to be photographed, and quantitatively measuring the color of the object to be photographed.

[0193] It should be noted that the information interaction, execution process, etc. among the modules / units in the image processing apparatus 1300 are based on the same concept as the corresponding method embodiments in this application. For specific content, reference can be made to the descriptions in the method embodiments shown above in this application, and details are not repeated here. Figures 2 to 9

[0194] Figure 14 The embodiment of this application further provides a data acquisition apparatus. Please refer to Figure 14 ,Figure 14 A schematic structural diagram of an apparatus for obtaining data provided by an embodiment of the present application. The data acquisition apparatus 1400 may include an acquisition module 1401, a generation module 1402, and a training module 1403. The acquisition module 1401 is configured to acquire training image data of a photographed object and third weight data corresponding to the training image data. The training image data includes image data of the photographed object in n spectral channels. The training image data includes m pixel points. The third weight data includes the annotation weights of each of the m pixel points. The weight value of the first pixel point among the m pixel points is higher than the weight value of the second pixel point among the m pixel points. The saturation of the color corresponding to the first pixel point is lower than the saturation of the color corresponding to the second pixel point. The generation module 1402 is configured to perform matrix multiplication on the training image data and a first vector to obtain second weight data, and perform normalization processing on the m weight values included in the second weight data to generate first weight data. The training image data is an m×n matrix. The first vector includes n elements. The second weight data includes m weight values corresponding one by one to the m pixel points. The training module 1403 is configured to iteratively train the element values of the n elements in the first vector according to an objective function until a preset condition is satisfied. The objective function indicates the similarity between the first weight data and the third weight data.

[0195] In a possible design, the generation module 1402 is specifically configured to arrange each row of the m row values in descending order to obtain a first matrix, multiply the first matrix by the first vector to obtain second weight data, and the n values in the first vector are arranged in descending order.

[0196] It should be noted that the information interaction, execution process, etc. between the modules / units in the data acquisition apparatus 1400 are based on the same concept as the corresponding method embodiments in the present application. For specific content, reference can be made to the descriptions in the method embodiments shown above in the present application, and details are not described herein again. Figure 10 Corresponding to the respective method embodiments based on the same concept, for specific content, reference can be made to the descriptions in the method embodiments shown above in the present application, and details are not described herein again.

[0197] An embodiment of the present application further provides an electronic device. Please refer to Figure 15 , Figure 15 A schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device 1500 is used to implement Figures 2 to 9 the functions of the electronic device in the corresponding embodiment. Specifically, the electronic device 1500 includes: a receiver 1501, a transmitter 1502, a processor 1503, and a memory 1504 (the number of processors 1503 in the electronic device 1500 may be one or more, Figure 15Taking a processor as an example, the processor 1503 may include an application processor 15031 and a communication processor 15032. In some embodiments of the present application, the receiver 1501, the transmitter 1502, the processor 1503, and the memory 1504 may be connected through a bus or other means.

[0198] The memory 1504 may include a read-only memory and a random access memory, and provide instructions and data to the processor 1503. A part of the memory 1504 may further include a non-volatile random access memory (NVRAM). The memory 1504 stores processor and operation instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, where the operation instructions may include various operation instructions for implementing various operations.

[0199] The processor 1503 controls the operation of the electronic device. In a specific application, the various components of the electronic device are coupled together through a bus system, where the bus system may include a power bus, a control bus, a status signal bus, etc. in addition to the data bus. However, for the sake of clarity, all kinds of buses are referred to as the bus system in the figure.

[0200] The method disclosed in the embodiments of the present application can be applied to or implemented by the processor 1503. The processor 1503 can be an integrated circuit chip with signal processing capabilities. During implementation, the steps of the above method can be completed through the integrated logic circuit in the hardware of the processor 1503 or instructions in the form of software. The above-mentioned processor 1503 can be a general-purpose processor, a digital signal processor (DSP), a microprocessor or a microcontroller, and can further include an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The processor 1503 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by a combination of the hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 1504, and the processor 1503 reads the information in the memory 1504 and combines its hardware to complete the steps of the above method.

[0201] The receiver 1501 can be used to receive input digital or character information, and generate signal inputs related to the relevant settings and function controls of the electronic device. The transmitter 1502 can be used to output digital or character information through the first interface; the transmitter 1502 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 1502 can also include a display device such as a display screen.

[0202] It should be noted that for the specific implementation manner and the beneficial effects brought by the application processor 15031 to execute the image processing method, reference can be made to Figures 2 to 9 the descriptions in the corresponding method embodiments, which will not be elaborated here one by one.

[0203] Please refer to Figure 16 , Figure 16 which is a schematic structural diagram of a training device provided by an embodiment of the present application. The training device 1600 is used to implement Figure 10Corresponding to the functions of the training device in the embodiments. Specifically, the training device 1600 is implemented by one or more servers. The training device 1600 may vary significantly due to configuration or performance differences and may include one or more central processing units (CPUs) 1622 (e.g., one or more processors) and a memory 1632, and one or more storage media 1630 (e.g., one or more mass storage devices) for storing application programs 1642 or data 1644. Among them, the memory 1632 and the storage media 1630 may be transient storage or persistent storage. The program stored in the storage media 1630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the training device. Further, the central processing unit 1622 may be configured to communicate with the storage media 1630 and execute a series of instruction operations in the storage media 1630 on the training device 1600.

[0204] The training device 1600 may further include one or more power supplies 1626, one or more wired or wireless network interfaces 1650, one or more input / output interfaces 1658, and / or one or more operating systems 1641, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0205] It should be noted that for the specific implementation manner and the beneficial effects brought by the central processing unit 1622 to execute the data acquisition method, reference can be made to Figure 10 the descriptions in the corresponding method embodiments, which will not be elaborated here one by one.

[0206] In the embodiments of the present application, a computer-readable storage medium is further provided. The computer-readable storage medium stores a program for generating the vehicle driving speed. When it runs on a computer, it causes the computer to execute the steps executed by the electronic device in the method described in the foregoing Figures 2 to 9 illustrated embodiments, or causes the computer to execute the steps executed by the training device in the method described in the foregoing Figure 10 illustrated embodiments.

[0207] In the embodiments of the present application, a computer program product is further provided. When it runs on a computer, it causes the computer to execute the steps executed by the electronic device in the method described in the foregoing Figures 2 to 9 illustrated embodiments, or causes the computer to execute the steps executed by the training device in the method described in the foregoing Figure 10 illustrated embodiments.

[0208] An embodiment of the present application further provides a circuit system, which includes a processing circuit configured to execute the steps performed by an electronic device in the method described in the foregoing Figures 2 to 9 illustrated embodiment, or the processing circuit is configured to execute the steps performed by an electronic device in the method described in the foregoing Figure 10 illustrated embodiment.

[0209] The image processing device or electronic device provided in the embodiment of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit. The processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin, a circuit, etc. The processing unit may execute computer-executable instructions stored in a storage unit to cause the chip to execute the Figures 2 to 9 image processing method described in the illustrated embodiment, or to cause the chip to execute the Figure 10 data acquisition method described in the illustrated embodiment. Optionally, the storage unit is a storage unit inside the chip, such as a register, a cache, etc., and the storage unit may also be a storage unit outside the chip located in the radio access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0210] Specifically, please refer to Figure 17 , Figure 17 which is a schematic structural diagram of a chip provided in the embodiment of the present application. The chip may be embodied as a neural network processor NPU 170, and the NPU 170 is mounted on a main CPU (Host CPU) as a coprocessor, and tasks are assigned by the Host CPU. The core part of the NPU is an arithmetic circuit 1703, and the arithmetic circuit 1703 is controlled by a controller 1704 to extract matrix data from a memory and perform a multiplication operation.

[0211] In some implementations, the arithmetic circuit 1703 includes multiple processing units (Process Engine, PE) inside. In some implementations, the arithmetic circuit 1703 is a two-dimensional systolic array. The arithmetic circuit 1703 may also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1703 is a general matrix processor.

[0212] For example, assume there is an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit fetches the corresponding data of matrix B from the weight memory 1702 and caches it on each PE in the arithmetic circuit. The arithmetic circuit fetches the data of matrix A from the input memory 1701 and performs matrix operations with matrix B, and the partial results or final results of the obtained matrix are stored in the accumulator 1708.

[0213] The unified memory 1706 is used to store input data and output data. The weight data is directly transported through the Direct Memory Access Controller (DMAC) 1705 and is carried to the weight memory 1702. The input data is also carried to the unified memory 1706 through the DMAC.

[0214] The BIU is the Bus Interface Unit, that is, the bus interface unit 1710, which is used for the interaction between the AXI bus, the DMAC, and the Instruction Fetch Buffer (IFB) 1709.

[0215] The bus interface unit 1710 (Bus Interface Unit, abbreviated as BIU) is used for the instruction fetch memory 1709 to obtain instructions from the external memory, and is also used for the storage unit access controller 1705 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0216] The DMAC is mainly used to transport the input data in the external memory DDR to the unified memory 1706, or transport the weight data to the weight memory 1702, or transport the input data to the input memory 1701.

[0217] The vector calculation unit 1707 includes multiple arithmetic processing units. When needed, it further processes the output of the arithmetic circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolution / full connection layer network calculations in neural networks, such as Batch Normalization (batch normalization), pixel-level summation, upsampling of the feature plane, etc.

[0218] In some implementations, the vector computing unit 1707 can store the processed output vectors into the unified memory 1706. For example, the vector computing unit 1707 can apply linear functions and / or non-linear functions to the output of the arithmetic circuit 1703, such as performing linear interpolation on the feature planes extracted by the convolutional layer, or for another example, vectors of accumulated values, to generate activation values. In some implementations, the vector computing unit 1707 generates normalized values, pixel-level summation values, or both. In some implementations, the processed output vectors can be used as activation inputs to the arithmetic circuit 1703, such as for use in subsequent layers in a neural network.

[0219] The instruction fetch buffer 1709 connected to the controller 1704 is used to store the instructions used by the controller 1704;

[0220] The unified memory 1706, the input memory 1701, the weight memory 1702, and the instruction fetch memory 1709 are all On-Chip memories. The external memory is private to this NPU hardware architecture.

[0221] Among them, the operations of each layer in the recurrent neural network can be executed by the arithmetic circuit 1703 or the vector computing unit 1707.

[0222] Among them, the processor mentioned anywhere above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the method in the first aspect above.

[0223] In addition, it should be noted that the device embodiments described above are only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the attached drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines.

[0224] Through the description of the above embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general hardware. Of course, it can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CLUs, dedicated memories, dedicated components, etc. Generally, functions accomplished by computer programs can be easily implemented by corresponding hardware, and the specific hardware structures for implementing the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for this application, software program implementation is a better embodiment in more cases. Based on such understanding, the technical solution of this application, in essence, or the part that makes a contribution to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disc of a computer, and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments of this application.

[0225] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product.

[0226] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are generated in whole or in part. The computer can be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

Claims

1. An image processing method, characterized in that, the method includes: obtaining first image data of an object to be photographed under the illumination of an illumination light source, the first image data including image data of the object to be photographed in n spectral channels, the first image data including m pixel points, n being an integer greater than 3, and m being a positive integer; adjusting the pixel values of a first pixel point and a second pixel point in the first image data according to first weight data to obtain second image data, wherein the first weight data includes m weight values corresponding one-to-one to the m pixel points, the first weight value corresponding to the first pixel point being higher than the second weight value corresponding to the second pixel point, and the saturation of the color corresponding to the first pixel point being lower than the saturation of the color corresponding to the second pixel point; determining spectral information of the illumination light source according to the second image data, the spectral information of the illumination light source being used to indicate the spectral power distribution of the illumination light source, the spectral information of the illumination light source including multiple groups of intensity values corresponding one-to-one to multiple different wavelength color lights in the illumination light source, each group of intensity values in the multiple groups of intensity values representing the numerical value of the radiant energy of each color light in the illumination light source, and the multiple groups of intensity values indicating the spectral power distribution of the illumination light source; wherein the first image data is obtained by using an image sensor, and determining the spectral information of the illumination light source according to the second image data includes: performing feature extraction on the second image data to obtain first feature information, the first feature information being an estimated value of the response value of the image sensor for the illumination light source in the n spectral channels; generating the spectral information of the illumination light source according to the first feature information and the spectral sensitivity of the image sensor in n spectral bands, the n spectral bands corresponding one-to-one to the n spectral channels.

2. The method according to claim 1, characterized in that, adjusting the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data includes: reducing the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data, the reduction ratio of the pixel value of the second pixel point being greater than the reduction ratio of the pixel value of the first pixel point.

3. The method according to claim 1 or 2, characterized in that, adjusting the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data includes: multiplying the pixel value of the first pixel point in the first image data by the first weight value to obtain the pixel value of the first pixel point in the second image data; multiplying the pixel value of the second pixel point in the first image data by the second weight value to obtain the pixel value of the second pixel point in the second image data.

4. The method according to claim 1 or 2, characterized in that, the method further includes: Perform matrix multiplication on the first image data and the first vector to obtain second weight data, where the first image data is an m-by-n matrix, the first vector includes n elements, and the second weight data includes m weight values corresponding one-to-one to the m pixel points; Perform normalization processing on the m weight values included in the second weight data to generate the first weight data.

5. The method according to claim 1 or 2, wherein, the method further includes: Increase the value of a third weight value among the m weight values and decrease the value of a fourth weight value among the m weight values to obtain updated first weight data, where the third weight value is a weight value greater than or equal to a preset threshold, and the fourth weight value is a weight value less than the preset threshold; The adjusting the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data includes: Adjusting the pixel values of the first pixel point and the second pixel point in the first image data according to the updated first weight data.

6. The method according to claim 1 or 2, wherein, the method further includes: Classify the m pixel points according to the first weight data to obtain m label values corresponding one-to-one to the m pixel points, for indicating the categories of the m pixel points; Set the pixel values of the pixel points of the first category among the m pixel points to zero to obtain updated first image data; The adjusting the pixel values of the first pixel point and the second pixel point in the first image data includes: Adjusting the pixel values of the first pixel point and the second pixel point in the updated first image data.

7. The method according to claim 1 or 2, wherein, the method further includes: Generate the spectral reflectance ratios of the object to be photographed in the n spectral bands according to the first image data and the spectral information of the illumination light source, and the spectral reflectance ratios of the object to be photographed in the n spectral bands are used to perform any one of the following operations: classifying the object to be photographed, identifying the object to be photographed, generating a visual image of the object to be photographed, analyzing the physical characteristics of the object to be photographed, analyzing the chemical characteristics of the object to be photographed, and quantitatively measuring the color of the object to be photographed.

8. A method for obtaining data, wherein, the method includes: Obtain training image data of the object to be photographed and third weight data corresponding to the training image data, where the training image data includes image data of the object to be photographed in n spectral channels, the training image data includes m pixel points, the third weight data includes the labeled weights of each of the m pixel points, the weight value of the first pixel point among the m pixel points is higher than the weight value of the second pixel point among the m pixel points, and the saturation of the color corresponding to the first pixel point is lower than the saturation of the color corresponding to the second pixel point; Perform matrix multiplication on the training image data and the first vector to obtain second weight data, and normalize the m weight values included in the second weight data to generate first weight data, where the training image data is an m×n matrix, the first vector includes n elements, and the second weight data includes m weight values corresponding one-to-one to the m pixel points; Iteratively train the element values of the n elements in the first vector according to the objective function until a preset condition is satisfied, where the objective function indicates the similarity between the first weight data and the third weight data.

9. The method according to claim 8, wherein, the training image data includes m rows of values, each row having n values, and the performing matrix multiplication on the training image data and the first vector to obtain second weight data includes: Sort each row of the m rows of values in descending order to obtain a first matrix, multiply the first matrix by the first vector to obtain the second weight data, and the n values in the first vector are arranged in descending order.

10. An image processing apparatus, wherein, the apparatus includes: an acquisition module, configured to acquire first image data of a photographed object under the irradiation of an illumination light source, the first image data including image data of the photographed object in n spectral channels, the first image data including m pixel points, n being an integer greater than 3, and m being a positive integer; an adjustment module, configured to adjust the pixel values of a first pixel point and a second pixel point in the first image data according to first weight data to obtain second image data, where the first weight data includes m weight values corresponding one-to-one to the m pixel points, the first weight value corresponding to the first pixel point is higher than the second weight value corresponding to the second pixel point, and the saturation of the color corresponding to the first pixel point is lower than the saturation of the color corresponding to the second pixel point; a determination module, configured to determine spectral information of the illumination light source according to the second image data, the spectral information of the illumination light source being used to indicate the spectral power distribution of the illumination light source, the spectral information of the illumination light source including multiple groups of intensity values corresponding one-to-one to multiple different wavelength color lights in the illumination light source, each group of intensity values in the multiple groups of intensity values representing the value of the radiant energy of each color light in the illumination light source, and the multiple groups of intensity values indicating the spectral power distribution of the illumination light source; wherein the first image data is obtained by using an image sensor, and the determination module is specifically configured to: Extract features from the second image data to obtain first feature information, where the first feature information is an estimated value of the response value of the image sensor for the illumination light source in the n spectral channels; Generate the spectral information of the illumination light source according to the first feature information and the spectral sensitivity of the image sensor in n spectral bands, where the n spectral bands correspond one-to-one to the n spectral channels.

11. The apparatus according to claim 10, wherein, The adjustment module is specifically configured to reduce the pixel values of the first pixel point and the second pixel point in the first image data according to the first weight data, and the reduction ratio of the pixel value of the second pixel point is greater than the reduction ratio of the pixel value of the first pixel point.

12. The apparatus according to claim 10 or 11, wherein, the adjustment module is specifically configured to: multiply the pixel value of the first pixel point in the first image data by the first weight value to obtain the pixel value of the first pixel point in the second image data; multiply the pixel value of the second pixel point in the first image data by the second weight value to obtain the pixel value of the second pixel point in the second image data.

13. The apparatus according to claim 10 or 11, wherein, the apparatus further comprises: a generation module, configured to perform matrix multiplication on the first image data and a first vector to obtain second weight data, wherein the first image data is a matrix of m by n, the first vector includes n elements, and the second weight data includes m weight values corresponding one-to-one to the m pixel points; the generation module is further configured to perform normalization processing on the m weight values included in the second weight data to generate the first weight data.

14. The apparatus according to claim 10 or 11, wherein, the adjustment module is further configured to increase the value of the third weight value among the m weight values and decrease the value of the fourth weight value among the m weight values to obtain updated first weight data, where the third weight value is a weight value greater than or equal to a preset threshold, and the fourth weight value is a weight value less than the preset threshold; the adjustment module is specifically configured to adjust the pixel values of the first pixel point and the second pixel point in the first image data according to the updated first weight data.

15. The apparatus according to claim 10 or 11, wherein, the apparatus further comprises: a classification module, configured to classify the m pixel points according to the first weight data to obtain m label values corresponding one-to-one to the m pixel points, for indicating the categories of the m pixel points; an update module, configured to set the pixel values of the pixel points of the first category among the m pixel points to zero to obtain updated first image data; the adjustment module is specifically configured to adjust the pixel values of the first pixel point and the second pixel point in the updated first image data.

16. The apparatus according to claim 10 or 11, wherein, the apparatus further comprises: A generation module, configured to generate spectral reflectance ratios of the object to be photographed in the n spectral bands according to the first image data and spectral information of the illumination light source, where the spectral reflectance ratios of the object to be photographed in the n spectral bands are used to perform any one of the following operations: classifying the object to be photographed, identifying the object to be photographed, generating a visualization image of the object to be photographed, analyzing physical characteristics of the object to be photographed, analyzing chemical characteristics of the object to be photographed, and quantitatively measuring the color of the object to be photographed.

17. An apparatus for acquiring data Characterized in that The apparatus includes: An acquisition module, configured to acquire training image data of an object to be photographed and third weight data corresponding to the training image data, where the training image data includes image data of the object to be photographed in n spectral channels, the training image data includes m pixel points, the third weight data includes annotation weights of each of the m pixel points, a weight value of a first pixel point among the m pixel points is higher than a weight value of a second pixel point among the m pixel points, and saturation of a color corresponding to the first pixel point is lower than saturation of a color corresponding to the second pixel point; A generation module, configured to perform matrix multiplication on the training image data and a first vector to obtain second weight data, and perform normalization processing on m weight values included in the second weight data to generate first weight data, where the training image data is an m×n matrix, the first vector includes n elements, and the second weight data includes m weight values corresponding to the m pixel points one by one; A training module, configured to iteratively train element values of the n elements in the first vector according to an objective function until a preset condition is satisfied, where the objective function indicates a similarity between the first weight data and the third weight data.

18. The apparatus according to claim 17 Characterized in that The training image data includes m rows of numerical values, and each row has n numerical values; The generation module is specifically configured to arrange each row of numerical values in the m rows of numerical values in descending order to obtain a first matrix, multiply the first matrix by the first vector to obtain the second weight data, and the n numerical values in the first vector are arranged in descending order.

19. A computer-readable storage medium Characterized in that Includes a program, which when running on a computer, causes the computer to execute the method according to any one of claims 1 to 7, or causes the computer to execute the method according to claim 8 or 9.

20. An electronic device Characterized in that Includes a processor, the processor is coupled to a memory, and the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

21. A training device Characterized in that Comprising a processor, the processor being coupled to a memory that stores program instructions, and when the program instructions stored in the memory are executed by the processor, the method according to claim 8 or 9 is implemented.

22. A circuit system, characterized in that the circuit system includes a processing circuit configured to execute the method according to any one of claims 1 to 7, or the processing circuit is configured to execute the method according to claim 8 or 9.

Images