Industrial camera and image processing method

Abstract

An industrial camera and an image processing method, which relate to the technical field of machine vision. The industrial camera comprises a spectral sensor, an image sensor, and a processor. When the image sensor collects an original image of a photographed object, the spectral sensor collects spectral information of an ambient light source. The processor determines, on the basis of the spectral information, a target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database, and finally performs color processing on the original image on the basis of the target color correction matrix, and obtains a target image. By applying the solution provided in embodiments of the present application, the industrial camera can automatically acquire current spectral information of the ambient light source, and perform color processing on the original image on the basis of the color correction matrix corresponding to the spectral information to obtain the target image, thereby simplifying the adjustment process of the industrial camera.

Description

An industrial camera and image processing method

[0001] This application claims priority to Chinese Patent Application No. 202411480367.8, filed on October 22, 2024, entitled "An Industrial Camera and Image Processing Method", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of machine vision technology, and in particular to an industrial camera and image processing method. Background Technology

[0003] With the continuous improvement of industrialization, machine vision systems are widely used in various industries, such as industrial production, medical devices, and license plate detection. In machine vision systems, the industrial camera is the core component that enables the system to perform its functions. In related technologies, a single-chip color camera is typically used to image the object being photographed.

[0004] A single-color camera mainly consists of a lens, an image sensor, a Bayer filter array, and a processor. Each pixel on the image sensor corresponds to one of the three colors: red, green, and blue. These three pixels are arranged according to a certain pattern. The lens focuses light from the object being photographed onto the image sensor. Through the Bayer filter array, each pixel of the image sensor receives only one of the three colors of light signals: red, green, and blue. The light signal is then converted into an electrical signal, and interpolation processing is performed on each monochrome pixel to estimate the missing color value. Finally, the processed pixel data is reassembled into a complete color image.

[0005] However, the imaging effect of monochrome and color cameras is greatly affected by the color of the light source. In actual shooting, changes in the color of the light source may require industrial cameras to frequently perform color correction and white balance adjustment, making the adjustment process cumbersome. Summary of the Invention

[0006] The purpose of this application is to provide an industrial camera and an image processing method to simplify the adjustment process. The specific technical solution is as follows:

[0007] A first aspect of this application provides an industrial camera, the industrial camera comprising: a spectral sensor, a processor, and an image sensor based on a Bayer filter array;

[0008] The spectral sensor is used to determine the spectral information of the ambient light source based on the received light, and send the spectral information to the processor;

[0009] The image sensor is used to generate an original image of the object being photographed based on the received light, and to send the original image to the processor;

[0010] The processor is configured to determine the target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database based on the spectral information; and to perform color processing on the original image based on the target color correction matrix to obtain the target image.

[0011] In one possible implementation, the processor determines the target color correction matrix corresponding to the spectral information from a pre-calibrated color correction matrix database based on the spectral information, including:

[0012] The actual brightness and actual color temperature of the ambient light source are determined based on the spectral information.

[0013] The target color correction matrix corresponding to the actual brightness and the actual color temperature is determined in the pre-calibrated color correction matrix database.

[0014] In one possible implementation, the industrial camera further includes a lens mount;

[0015] The spectral sensor and image sensor receive light through a lens assembly on a lens mount; or...

[0016] The spectral sensor receives light through a light-transmitting plate on the lens mount; the image sensor receives light through the lens assembly on the lens mount.

[0017] In one possible implementation, the industrial camera further includes a beam splitter and a lens mount, the lens mount being used to mount a lens assembly, the lens assembly and the beam splitter being arranged sequentially along the direction of light incidence;

[0018] The lens assembly is used to transmit the light reflected from the object being photographed to the beam splitter.

[0019] The beam splitter is used to split the light into a first light and a second light and emit them to the image sensor and the spectral sensor, wherein the image sensor is disposed in the emission direction of the first light and the spectral sensor is disposed in the emission direction of the second light.

[0020] In one possible implementation, the beam splitter is a beam splitting prism or beam splitter.

[0021] The image sensor is disposed on the optical axis of the lens assembly; the spectral sensor is disposed on a plane perpendicular to the plane on which the image sensor is located.

[0022] In one possible implementation, the industrial camera further includes a filter, a portion of which is located between the lens assembly and the image sensor, and another portion of which is located between the lens assembly and the spectral sensor, the filter being used to filter out light emitted from non-ambient light sources.

[0023] In one possible implementation, the industrial camera further includes a lens mount, which has a lens mounting hole for mounting a lens and a light-transmitting plate mounting hole for mounting a light-transmitting plate.

[0024] The lens is used to transmit the light reflected from the object being photographed to the image sensor;

[0025] The light-transmitting sheet is used to transmit the light reflected from the object being photographed to the spectral sensor.

[0026] In one possible implementation, the industrial camera further includes an output interface; the industrial camera is connected to a host computer via the output interface.

[0027] The output interface is used to output the target image to the host computer.

[0028] In one possible implementation, the industrial camera further includes one or more printed circuit boards; the processor, the image sensor, and the spectral sensor are fixed on one printed circuit board; or, the image sensor and the spectral sensor are fixed on one printed circuit board, and the processor is fixed on another printed circuit board; or, the processor, the image sensor, and the spectral sensor are respectively fixed on three printed circuit boards.

[0029] In one possible implementation, the industrial camera further includes a housing and a plurality of first connectors; the housing mates with the lens mount to form a receiving cavity;

[0030] One or more of the printed circuit boards are fixed within the receiving cavity by the plurality of first connectors;

[0031] The photosensitive surfaces of the spectral sensor and the image sensor are capable of receiving light reflected from the object being photographed through the lens assembly mounted on the lens mount.

[0032] In one possible implementation, the industrial camera further includes a light source assembly and a second connector, the light source assembly being fixed to the outside of the housing via the second connector, and the illumination area of ​​the light source assembly covering the field of view of the mounted lens assembly.

[0033] A second aspect of this application provides an image processing method applied to the industrial camera described in the first aspect above; the method includes:

[0034] Obtain the original image and spectral information;

[0035] Based on the spectral information, the target color correction matrix corresponding to the spectral information is determined in a pre-calibrated color correction matrix database;

[0036] The original image is processed based on the target color correction matrix to obtain the target image.

[0037] In one possible implementation, determining the target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database based on the spectral information includes:

[0038] The actual brightness and actual color temperature of the ambient light source are determined based on the spectral information.

[0039] The target color correction matrix corresponding to the actual brightness and the actual color temperature is determined in the pre-calibrated color correction matrix database.

[0040] In one possible implementation, the color correction matrix includes: a white balance correction submatrix, a color correction submatrix, a gamma correction submatrix, and a three-dimensional lookup table submatrix;

[0041] The step of color processing the original image based on the target color correction matrix includes: color correction of the original image using each sub-matrix.

[0042] In one possible implementation, the color correction matrix database is pre-calibrated according to the following method:

[0043] Under preset light source conditions of multiple color temperatures and multiple brightness levels, the industrial camera is controlled to capture images of each standard color block in the standard color chart, thereby obtaining color block images with different brightness levels under each preset color temperature.

[0044] Obtain the original color correction matrix, and use the original color correction matrix to correct the color block images of each brightness at each preset color temperature to obtain the corrected color block images;

[0045] Calculate the color error between each corrected color patch image and each standard color patch;

[0046] The original color correction matrix is ​​adjusted according to each of the color errors to obtain the color correction matrix corresponding to different brightness at each preset color temperature;

[0047] The color correction matrix database is obtained by storing the color correction matrices corresponding to different brightness levels under each preset color temperature; wherein, the original color correction matrix includes: the original white balance correction sub-matrix, the original color correction sub-matrix, the original gamma correction sub-matrix, and the original three-dimensional lookup table sub-matrix.

[0048] In one possible implementation, the step of performing color processing on the original image based on the target color correction matrix to obtain the target image includes:

[0049] Perform a first color interpolation on the original image to obtain a first color image;

[0050] The first color image is subjected to first color correction according to the target color correction matrix to obtain the target image.

[0051] In one possible implementation, the step of performing color processing on the original image based on the target color correction matrix to obtain the target image includes:

[0052] The original image is subjected to a second color correction based on the target color correction matrix to obtain the original corrected image;

[0053] The original corrected image is then subjected to a second color interpolation to obtain the target image.

[0054] In one possible implementation, performing a second color interpolation on the original corrected image to obtain the target image includes:

[0055] The original corrected image is subjected to a second color interpolation to obtain a second color image;

[0056] The second color image is subjected to third color correction based on the target color correction matrix to obtain the target image.

[0057] In one possible implementation, the industrial camera further includes an output interface, through which the industrial camera is connected to a host computer, and the method further includes:

[0058] At least one of the target image, the spectral information, the target color correction matrix, the color image, and the original corrected image is output to the host computer through the output interface.

[0059] In one possible implementation, determining the target color correction matrix corresponding to the actual brightness and the actual color temperature in a pre-calibrated color correction matrix database includes:

[0060] The brightness that differs from the actual brightness by less than a preset first threshold and the color temperature that differs from the actual color temperature by less than a preset second threshold are searched in the pre-calibrated color correction matrix database to obtain at least one target brightness and at least one target color temperature.

[0061] The first color correction matrix corresponding to each target brightness and each target color temperature is searched in the color correction matrix database to obtain at least one first color correction matrix;

[0062] The target color correction matrix is ​​obtained based on the statistics of each of the first color correction matrices.

[0063] In one possible implementation, the step of statistically obtaining the target color correction matrix based on each of the first color correction matrices includes:

[0064] Interpolation calculations are performed on each of the first color correction matrices to obtain the target color correction matrix.

[0065] In one possible implementation, determining the actual brightness and actual color temperature of the ambient light source based on the spectral information includes:

[0066] The actual brightness and actual color temperature of the ambient light source are determined based on the spectral information received initially.

[0067] The step of determining the target color correction matrix corresponding to the actual brightness and the actual color temperature in a pre-calibrated color correction matrix database includes:

[0068] Determine the target color correction matrix corresponding to the actual brightness and the actual color temperature in the pre-calibrated color correction matrix database, and record the target color correction matrix;

[0069] The step of performing color processing on the original image based on the target color correction matrix to obtain the target image includes:

[0070] The original image is processed based on the recorded target color correction matrix to obtain the target image.

[0071] In one possible implementation, acquiring the original image and spectral information includes:

[0072] The spectral sensor is controlled to determine the current spectral information of the ambient light source, and the image sensor is simultaneously controlled to generate the original image of the object being photographed.

[0073] The step of determining the target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database based on the spectral information includes:

[0074] Determine whether the difference between the current spectral information and the previously received spectral information is greater than a preset threshold;

[0075] If the difference between the current spectral information and the previously received spectral information is less than or equal to a preset threshold, the previously received spectral information is used as the target spectral information, and the previously recorded target color correction matrix is ​​obtained.

[0076] If the difference between the current spectral information and the previously received spectral information is greater than a preset threshold, the current spectral information is used as the target spectral information, and the actual brightness and actual color temperature of the ambient light source are determined based on the target spectral information; the target color correction matrix corresponding to the actual brightness and the actual color temperature is determined in the pre-calibrated color correction matrix database, and the target color correction matrix is ​​recorded.

[0077] Beneficial effects of the embodiments in this application:

[0078] This application provides an industrial camera and image processing method. The industrial camera includes a spectral sensor, an image sensor, and a processor. When the image sensor acquires the original image of the object being photographed, the spectral sensor acquires the spectral information of the ambient light source. The processor determines the target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database based on the spectral information. Finally, the original image is color-processed based on the target color correction matrix to obtain the target image. Because the industrial camera of this application embodiment can acquire the spectral information of the ambient light source in real time, when the light source changes, the industrial camera can automatically acquire the current spectral information of the ambient light source and perform color processing on the original image based on the color correction matrix corresponding to the spectral information to obtain a target image with high color fidelity, without needing to perform color correction and white balance adjustment on the industrial camera according to changes in the light source, thus simplifying the adjustment process of the industrial camera.

[0079] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0080] The accompanying drawings, which are provided to further illustrate this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application.

[0081] Figure 1 is a schematic diagram of the principle of a single-chip color camera generating color images;

[0082] Figure 2 is a schematic diagram of the first principle structure of the industrial camera provided in the embodiment of this application;

[0083] Figure 3a is a diagram of the first process for image processing of the original image provided in an embodiment of this application;

[0084] Figure 3b is a diagram illustrating a second process for image processing of the original image according to an embodiment of this application.

[0085] Figure 3c is a diagram illustrating a third process for image processing of the original image according to an embodiment of this application.

[0086] Figure 4a is a first structural example of the industrial camera provided in the embodiment shown in Figure 2;

[0087] Figure 4b is an example diagram of the first type of internal structure of the industrial camera shown in Figure 4a;

[0088] Figure 4c is an example of the second type of internal structure of the industrial camera shown in Figure 4a;

[0089] Figure 5a is a second structural example of the industrial camera provided in the embodiment shown in Figure 2;

[0090] Figure 5b is an example diagram of the first type of internal structure of the industrial camera shown in Figure 5a;

[0091] Figure 5c is an example of the second type of internal structure of the industrial camera shown in Figure 5a;

[0092] Figure 6 is a schematic diagram of the second principle structure of the industrial camera provided in the embodiment of this application;

[0093] Figure 7a is a first structural example of the industrial camera provided in the embodiment shown in Figure 6;

[0094] Figure 7b is a second structural example of the industrial camera provided in the embodiment shown in Figure 6;

[0095] Figure 8 is a schematic diagram of the application scenario of the industrial camera provided in the embodiment of this application;

[0096] Figure 9 is a schematic diagram of the third principle structure of the industrial camera provided in the embodiment of this application;

[0097] Figure 10 is a schematic diagram of the fourth principle structure of the industrial camera provided in the embodiment of this application;

[0098] Figure 11a is a schematic diagram of the fifth principle structure of the industrial camera provided in the embodiment of this application;

[0099] Figure 11b is a schematic diagram of the sixth principle structure of the industrial camera provided in the embodiments of this application;

[0100] Figure 12 is a first flowchart of the image processing method provided in an embodiment of this application;

[0101] Figure 13 is a second flowchart of the image processing method provided in an embodiment of this application;

[0102] Figure 14 is a first interactive diagram of the image processing method provided in an embodiment of this application;

[0103] Figure 15 is a third flowchart of the image processing method provided in the embodiments of this application;

[0104] Figure 16 is a second interactive diagram of the image processing method provided in the embodiments of this application;

[0105] Figure 17 is a fourth flowchart of the image processing method provided in the embodiments of this application;

[0106] Figure 18 is a fifth flowchart of the image processing method provided in the embodiments of this application;

[0107] Figure 19 is a sixth flowchart of the image processing method provided in the embodiments of this application;

[0108] Figure 20 is a seventh flowchart of the image processing method provided in the embodiments of this application. Detailed Implementation

[0109] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments in this application are within the scope of protection of this application.

[0110] First, the technical terms used in the embodiments of this application will be explained:

[0111] Tristimulus values: These represent the degree of stimulation of the three primary colors that cause the human retina to perceive a certain color, denoted by X (red primary color stimulation), Y (green primary color stimulation), and Z (blue primary color stimulation).

[0112] Chromaticity coordinates: These represent the position of a color in the chromaticity space. In a chromaticity diagram, colors are represented by two coordinates: x and y.

[0113] With the continuous improvement of industrialization, machine vision systems are widely used in various industries, such as industrial production, medical devices, and license plate detection. In machine vision systems, the industrial camera is the core component that enables the system to function. The industrial camera uses an image sensor capable of receiving spectral response as its photosensitive unit, receiving light reflected or projected from the surface of an object, thereby achieving the imaging of the object being photographed.

[0114] In related technologies, color industrial cameras include two types: single-chip color cameras and true-color industrial cameras. True-color industrial cameras use a beam splitter to refract incident light onto three monochrome sensor targets, where photoelectric conversion is performed to obtain red, green, and blue color values, which are then directly synthesized to obtain a color image. True-color industrial cameras have good color reproduction, but they require high-performance beam splitters and are typically expensive. Therefore, single-chip color cameras are usually used to image the subject.

[0115] A single-chip color camera mainly consists of a lens, an image sensor, a Bayer filter array, and a processor. Each pixel on the image sensor corresponds to one of the three colors: red, green, and blue. These three pixels are arranged according to a certain pattern. Figure 1 shows the principle of a single-chip color camera generating a color image. As shown in Figure 1, in related technologies, the lens focuses light from the object being photographed onto the image sensor. The Bayer filter array generates a Bayer array image 1-1 on the image sensor. The Bayer array image is then separated into red, green, and blue components 2-1, 2-2, and 2-3. Color interpolation is then performed on each component to complete the information, resulting in completed red, green, and blue components 3-1, 3-2, and 3-3. Finally, the completed red, green, and blue components are merged to obtain a three-channel image 4-1, i.e., a color image. It is understandable that in Figure 1, the grayscale value of each pixel in the Bayer array image 1-1 contains only one color information (red, green, or blue). In the red component 2-1, only the pixels marked as red in the Bayer array image have valid values; in the green component 2-2, only the pixels marked as green in the Bayer array image have valid values; and in the blue component 2-3, only the pixels marked as blue in the Bayer array image have valid values. The red component 3-1 is obtained by interpolating and filling the grayscale values ​​of each pixel in the red component 2-1; the green component 3-2 is obtained by interpolating and filling the grayscale values ​​of each pixel in the green component 2-2; and the blue component 3-3 is obtained by interpolating and filling the grayscale values ​​of each pixel in the blue component 2-3. Each pixel in the three-channel image 4-1 includes complete information from the red, green, and blue channels.

[0116] However, the imaging effect of single-chip color cameras is greatly affected by the color of the light source. In actual shooting, changes in the color of the light source may require industrial cameras to frequently perform color correction and white balance adjustment, making the adjustment process cumbersome.

[0117] To simplify the adjustment process of a single-chip color camera and enable it to quickly capture images with high color fidelity under different light source conditions, an industrial camera is provided in a first aspect of this application. Figure 2 is a schematic diagram of the first principle structure of the industrial camera provided in this application. As shown in Figure 2, the industrial camera 1 includes a spectral sensor 12, a processor 13, and an image sensor 14. To achieve light intake, the industrial camera 1 also includes a lens mounting base 11 for mounting a lens assembly 110. The lens assembly 110 is used to transmit the light reflected by the object 3 to the image sensor 14 and the spectral sensor 12. The spectral sensor 12 is used to determine the spectral information of the ambient light source (i.e., the light source assembly 2 in the figure) based on the received light and send the spectral information to the processor 13. The image sensor 14 is used to generate an original image based on the received light and send the original image to the processor 13.

[0118] The processor 13 is used to determine the target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database based on the spectral information; and to perform color processing on the original image based on the target color correction matrix to obtain the target image.

[0119] The industrial camera in this application embodiment can acquire the spectral information of the ambient light source in real time. When the light source changes, the industrial camera can automatically acquire the current spectral information of the ambient light source and perform color processing on the original image based on the color correction matrix corresponding to the spectral information to obtain a target image with high color fidelity. This eliminates the need to perform color correction and white balance adjustment on the industrial camera according to the change of the light source, thus simplifying the adjustment process of the industrial camera.

[0120] The spectral sensor 12 is a device capable of measuring and analyzing spectral characteristics. It is used to determine the spectral information of the ambient light source based on the received light and send the spectral information to the processor 13. The spectral sensor 12 has multiple optical channels, each of which detects the light intensity within a corresponding spectral wavelength range. The multiple optical channels cover the spectral range of the incident light of the spectral sensor 12. That is, the spectral sensor 12 can disperse the incident light according to wavelength and measure the light intensity of each wavelength.

[0121] In one possible embodiment, assuming the spectral sensor 12 has n optical channels, and the light intensity detected by each optical channel is denoted as Fn, where n is the channel number corresponding to the spectral wavelength range, then the spectral wavelength range corresponding to each optical channel is shown in Table 1 below, where λ represents the wavelength. By measuring the intensity of light at different wavelengths, the spectral sensor 12 can acquire spectral information.

[0122] Table 1. Spectral wavelength range corresponding to optical channels

[0123] It is understandable that when machine vision systems are applied to scenarios such as industrial inspection and medical imaging, the light sources in these scenarios are usually fixed, meaning that the spectral information of the ambient light sources in these scenarios is usually unchanged. In order to reduce the computational load on the processor 13, for the same ambient light source, it is only necessary to calculate the spectral information of the ambient light source, further determine the target color correction matrix corresponding to the ambient light source, and then perform color processing on all the original images acquired under the ambient light source based on the target color correction matrix, without having to obtain the spectral information of the ambient light source multiple times.

[0124] Therefore, in one possible implementation, processor 13 is specifically used for:

[0125] The actual brightness and color temperature of the ambient light source are determined based on the spectral information received initially.

[0126] Determine the target color correction matrix corresponding to the actual brightness and actual color temperature in the pre-calibrated color correction matrix database, and record the target color correction matrix;

[0127] The original image is processed based on the recorded target color correction matrix to obtain the target image.

[0128] It is understandable that the spectral sensor 12 is not stable when it is first powered on, and the collected spectral information may be inaccurate. Therefore, in this embodiment, the spectral information received for the first time refers to the first valid spectral information received. In one possible implementation, after receiving the spectral information, the processor 13 determines whether the received spectral information has reached a stable state. If it has, the spectral information that has reached a stable state is taken as valid spectral information. Here, a stable state means that within a short period of time, such as within 1 minute, the difference between several consecutively received spectral information is less than a preset threshold, which indicates that a stable state has been reached.

[0129] After the processor 13 obtains the target color correction matrix based on the first received spectral information, it records the target color correction matrix. After receiving each original image sent by the image sensor 14, the recorded target color correction matrix is ​​used to process the color of each original image.

[0130] In this case, the processor 13 may control the spectral sensor 12 to turn off after receiving the spectral information for the first time, so that the spectral sensor 12 no longer acquires and sends spectral information to the processor 13; or the processor 13 may calculate the actual color temperature and actual brightness of the ambient light source based on the first received spectral information after receiving the spectral information for the first time, and then calculate the target color correction matrix as the target color correction matrix corresponding to the ambient light source. If spectral information sent by the spectral sensor 12 is received again later, the target color correction matrix will not be calculated based on the spectral information.

[0131] It is understood that in this embodiment, the image sensor 14 and the spectral sensor 12 can work simultaneously, or the spectral sensor 12 can first collect the spectral information of the ambient light source, and after the processor 13 determines the target color correction matrix based on the first received spectral information, it controls the image sensor 14 to collect the original image. This application embodiment does not limit this.

[0132] Using the embodiments of this application, the processor only needs to receive the spectral information for the first time to determine the target color correction matrix corresponding to the ambient light source, which reduces the amount of computation of the processor and improves the speed of image color processing.

[0133] However, in some scenarios such as sports and entertainment, and medical testing, the ambient light source may change. Therefore, if the processor 13 performs color processing on each original image acquired by the industrial camera 1 based on the target color correction matrix corresponding to the ambient light source determined by the first received spectral information, there may be a problem of inaccurate color reproduction of some original images.

[0134] Based on this, in one possible implementation, the processor 13 can control the spectral sensor 12 to determine the current spectral information of the ambient light source based on the received light, and simultaneously control the image sensor to generate an original image of the object being photographed based on the received light. That is, the processor 13 controls the spectral sensor 12 to acquire spectral information each time, while simultaneously controlling the image sensor 14 to acquire an original image once.

[0135] For each acquired spectral information and original image, the processor 13 determines the target color correction matrix based on the acquired spectral information, and then performs color processing on the acquired original image based on the target color correction matrix.

[0136] Understandably, machine vision systems typically capture images at high speeds, so industrial cameras usually have high frame rates. For example, if industrial camera 1 has a frame rate of 30 frames per second, it captures one image approximately every 0.033 seconds (1 / 30 of a second). It is evident that the time between two adjacent images captured by the industrial camera is extremely short, while the ambient light source does not change significantly in such a short period. Therefore, within this short time, the target color correction matrix determined by processor 13 based on multiple spectral information collected by spectral sensor 12 is actually the same.

[0137] Based on this, in one possible implementation, the processor 13 is specifically used for:

[0138] The control spectral sensor determines the current spectral information of the ambient light source, stores the current spectral information, and simultaneously controls the image sensor to generate the original image of the object being photographed.

[0139] Determine whether the difference between the current spectral information and the previously received spectral information is greater than a preset threshold;

[0140] If the difference between the current spectral information and the previously received spectral information is less than or equal to a preset threshold, the previously received spectral information is used as the target spectral information, and the previously recorded target color correction matrix is ​​obtained.

[0141] If the difference between the current spectral information and the previously received spectral information is greater than a preset threshold, the current spectral information is used as the target spectral information. The actual brightness and actual color temperature of the ambient light source are determined based on the target spectral information. The target color correction matrix corresponding to the actual brightness and actual color temperature is determined in the pre-calibrated color correction matrix database and recorded.

[0142] In other words, when the processor 13 receives the current spectral information of the ambient light source sent by the spectral sensor, it will store the current spectral information and determine whether the difference between the current spectral information and the previously received spectral information is greater than a preset threshold.

[0143] If the difference between the current spectral information and the previously received spectral information is greater than a preset threshold, the current spectral information is used as the target spectral information. The target color correction matrix is ​​determined in the color correction matrix database based on the target spectral information and recorded. Then, the color of the currently acquired original image is processed based on the recorded target color correction matrix to obtain the target image.

[0144] If the difference between the current spectral information and the previously received spectral information is less than or equal to a preset threshold, the previously recorded target color correction matrix is ​​obtained, and the color of the currently acquired original image is processed based on the target color correction matrix to obtain the target image.

[0145] By employing the embodiments of this application, the current spectral information of the ambient light source is obtained each time an original image is acquired, and the target color correction matrix is ​​determined based on the current spectral information, which can significantly improve the accuracy of color processing of the original image.

[0146] In this embodiment, the image sensor 14 is an image sensor based on a Bayer filter array. A Bayer array is a color filter array in which red, green, and blue filters are arranged in a specific pattern above the photosensitive element of the image sensor. Image sensors based on Bayer filter arrays typically use four pixels as a pixel unit, and each pixel unit contains red, green, and blue pixels, as well as one gray pixel or one green pixel.

[0147] Understandably, due to the color filtering characteristics of the Bayer array, the original image acquired by the image sensor 14 is monochrome, that is, each pixel contains only one of the three colors: red, green, and blue. The original image visually presents a mosaic-like color distribution. Therefore, after acquiring the original image, the image sensor 14 needs to send the original image to the processor 13 for color processing.

[0148] Color processing refers to various processes, such as color interpolation and color correction, applied to unprocessed monochrome or incomplete color image data captured by a Bayer array-based image sensor to restore, enhance, or modify the image's color attributes. After color processing, the image becomes closer to the true colors seen by the human eye.

[0149] It is understandable that industrial cameras are typically used in machine vision systems for applications such as industrial inspection, medical imaging, and sports and entertainment. Therefore, industrial cameras require high-speed image acquisition and processing. Based on this, in one possible implementation, the processor 13 is composed of an FPGA (Field-Programmable Gate Array) or a SoC (System-on-a-Chip). For example, if the user has high requirements for the image acquisition speed of the industrial camera, an FPGA is selected as the processor 13; if the user prioritizes the low power consumption of the industrial camera, a SoC is selected as the processor 13. This application does not limit this aspect.

[0150] When the processor 13 performs color processing on the original image, it may first perform color interpolation processing on the original image and then perform color correction processing; or it may first perform color correction processing on the original image and then perform color interpolation processing; or it may perform color correction processing on the original image, then perform color interpolation processing, and then perform color correction processing on the interpolated image to obtain the target image. This application embodiment does not limit this.

[0151] In this embodiment, color correction by the processor 13 refers to the processor 13 performing color correction on the image based on the target color correction matrix corresponding to the spectral information of the ambient light source. The target color correction matrix is ​​determined by the processor 13 from a pre-calibrated color correction database based on the spectral information of the ambient light source.

[0152] The color correction matrix database may include a mapping relationship between spectral information and the color correction matrix. After acquiring the spectral information of the ambient light source, the processor 13 determines the target color correction matrix based on the mapping relationship between the spectral information and the color correction matrix in the color correction matrix database. Alternatively, it may include a mapping relationship between the color temperature and / or brightness of the light source and the color correction matrix. After acquiring the spectral information of the ambient light source, the processor 13 determines the actual brightness and / or actual color temperature of the ambient light source based on the spectral information, and then determines the target color correction matrix corresponding to the actual brightness and / or actual color temperature based on the mapping relationship between the color temperature and / or brightness and the color correction matrix.

[0153] In one possible implementation, processor 13 is specifically used for:

[0154] The actual brightness and color temperature of the ambient light source are determined based on spectral information; the target color correction matrix corresponding to the actual brightness and color temperature is determined in the pre-calibrated color correction database.

[0155] The process by which processor 13 determines the actual brightness and actual color temperature of the current light source based on spectral information, the process of establishing the color correction matrix database, and the process by which processor 13 determines the target color correction matrix in the preset color correction matrix database based on the actual brightness and actual color temperature of the current light source are described below and will not be repeated here.

[0156] Using the embodiments of this application, the industrial camera can collect the spectral information of the ambient light source in real time and determine the actual brightness and actual color temperature. When the light source changes, it can automatically obtain the current actual brightness and actual color temperature of the ambient light source, and perform color processing on the original image based on the color correction matrix corresponding to the actual brightness and actual color temperature, thereby improving the accuracy of color correction of the original image.

[0157] Depending on the order of color interpolation and color correction, the processor 13 performs color processing on the original image based on the target color correction matrix, resulting in a different process for obtaining the target image.

[0158] In one possible implementation, processor 13 is specifically used for:

[0159] Perform a first color interpolation on the original image to obtain a first color image;

[0160] The first color image is corrected by performing a first color correction based on the target color correction matrix to obtain the target image.

[0161] For example, see Figure 3a. Figure 3a shows a first process diagram for image processing of the original image provided by an embodiment of this application. After determining the actual brightness (i.e., current brightness) and actual color temperature (current color temperature) of the current light source, the processor 13 searches and interpolates in the color correction matrix database to determine the target color correction matrix (i.e., the current color correction matrix).

[0162] The processor 13 first performs color interpolation on the original image to obtain the original color image, and then uses the target color matrix to perform color correction on the original color image to obtain the target image (i.e., the color-corrected image).

[0163] By performing color interpolation before color correction in this embodiment, it can be ensured that the image already has complete color information. The color correction algorithm can then be adjusted based on the complete color data to obtain a more accurate correction result.

[0164] Color distortion can occur during color interpolation due to color deviation. Therefore, color correction can be performed on the original image based on the target color correction matrix before color interpolation to reduce color distortion caused by color deviation. In one possible implementation, processor 13 is specifically used for:

[0165] The original image is then subjected to a second color correction based on the target color correction matrix to obtain the original corrected image.

[0166] The target image is obtained by performing a second color interpolation on the original corrected image.

[0167] For example, see Figure 3b, which shows a second process diagram for image processing of the original image provided in an embodiment of this application. After determining the actual brightness (i.e., current brightness) and actual color temperature (current color temperature) of the current light source, the processor 13 searches and interpolates in the color correction matrix database to determine the target color correction matrix (i.e., the current color correction matrix).

[0168] The processor 13 first performs color correction on the original image based on the target color matrix to obtain the original corrected image (i.e., the Bayer corrected image), and then performs color interpolation on the original corrected image to obtain the target image (i.e., the color corrected image).

[0169] To restore the true colors of objects in the image to the greatest extent possible, the original image can be color-corrected, then color-interpolated, and finally color-corrected again to obtain the target image. Therefore, in one possible implementation, the processor 13 is specifically used for:

[0170] The original corrected image is interpolated using a second color to obtain a second color image;

[0171] The second color image is corrected using the target color correction matrix to obtain the target image.

[0172] For example, see Figure 3c, which shows a third process diagram for image processing of the original image provided in the embodiment of this application. After determining the actual brightness (i.e., current brightness) and actual color temperature (current color temperature) of the current light source, the processor 13 searches and interpolates in the color correction matrix database to determine the target color correction matrix (i.e., the current color correction matrix).

[0173] The processor 13 first performs color correction on the original image based on the target color matrix to obtain the original corrected image (i.e., the Bayer corrected image). Then, it performs color interpolation on the original corrected image to obtain the original color image. Finally, it performs color correction on the original color image using the target color matrix to obtain the target image (i.e., the color corrected image).

[0174] Through the embodiments of this application, the target color correction matrix is ​​used to perform color correction on the original image and the color image after color interpolation, so as to restore the true color information of the object captured by the industrial camera to the greatest extent.

[0175] In order to enable the image sensor 14, processor 13 and spectral sensor 12 to work together and further improve the efficiency and accuracy of image processing in the industrial camera, in one possible implementation, the industrial camera 1 also includes one or more printed circuit boards, through which the image sensor 14 and processor 13 can be connected and communicate.

[0176] The processor 13, image sensor 14, and spectral sensor 12 can be fixed on the same printed circuit board; or the image sensor 14 and spectral sensor 12 can be fixed on one printed circuit board, and the processor 13 can be fixed on another printed circuit board; or the image sensor 14, spectral sensor 12, and processor 13 can be fixed on three different printed circuit boards. All of these are possible, and this application embodiment does not limit them.

[0177] It is understandable that, since the spectral sensor 12, image sensor 14, and processor 13 are the core electronic components of the industrial camera 1, they may be damaged when subjected to external physical impacts, vibrations, or pressure. Therefore, to ensure the stable operation of the core components of the industrial camera 1, in one possible embodiment, the industrial camera 1 further includes a housing and a first connector. The housing cooperates with the lens mounting base 11 to form a receiving cavity. The spectral sensor 12, image sensor 14, and processor 13 are fixed on one or more printed circuit boards, and the one or more printed circuit boards are fixed in the receiving cavity by multiple first connectors. The first connectors can be bolts, studs, or other connectors; this embodiment does not limit the type of the first connector.

[0178] The structure of the industrial camera provided in this application will be described below with reference to the accompanying drawings.

[0179] In one possible embodiment, the structure of the industrial camera is shown in Figure 4a. Figure 4a is a first structural example of the industrial camera provided by the embodiment shown in Figure 2. As shown in Figure 4a, the lens mount 11 has only one lens mounting hole 111. The housing 15 cooperates with the lens mount 11 to form a receiving cavity. The spectral sensor 12 and the image sensor in the receiving cavity achieve light intake through the lens assembly mounted by the lens mounting hole 111 (not shown in Figure 4a). It is understood that the projection of the lens mounting hole 111 on the printed circuit board 16 should completely cover the spectral sensor 12 and the image sensor 14.

[0180] When the spectral sensor 12, processor 13, and image sensor 14 are fixed on the same printed circuit board, the internal structure of the industrial camera 1 is shown in Figure 4b. Figure 4b is an example diagram of the first internal structure of the industrial camera shown in Figure 4a. As shown in Figure 4b, the spectral sensor 12, processor 13, and image sensor 14 are fixed on the same printed circuit board 16. The printed circuit board 16 is provided with four screw holes 160 for fixing to the lens mounting base 11 through the first connector.

[0181] When the spectral sensor 12 and the image sensor 14 are fixed on one printed circuit board, and the processor 13 is fixed on another printed circuit board, the internal structure of the industrial camera 1 is shown in Figure 4c. Figure 4c is an example diagram of the second internal structure of the industrial camera shown in Figure 4a. As shown in Figure 4c, the spectral sensor 12 and the image sensor 14 are fixed on the same printed circuit board (i.e., the first printed circuit board 161), and the processor 13 is fixed on another printed circuit board (i.e., the second printed circuit board 162). The first printed circuit board 161 and the second printed circuit board 162 are each provided with four screw holes 160 for fixing to the lens mounting base 11 through the first connector.

[0182] In the case of multiple printed circuit boards in the industrial camera 1, the connection methods between the printed circuit boards include, but are not limited to, soldering, hollow connecting strips, V-cutting, copper foil bridging, plug-in, etc., as shown in Figure 4c. The first printed circuit board 161 and the second printed circuit board 162 are connected by plug-in 163.

[0183] In another possible embodiment, the structure of the industrial camera is shown in Figure 5a. Figure 5a is a second structural example of the industrial camera provided by the embodiment shown in Figure 2. As shown in Figure 5a, the lens mount 11 is provided with a lens mounting hole 111 and a light-transmitting sheet mounting hole 112 for mounting the lens and the light-transmitting sheet, respectively. The housing 15 cooperates with the lens mount 11 to form a receiving cavity. The light reflected from the object being photographed is transmitted through the lens mounted in the lens mounting hole 111 to the image sensor 14 in the receiving cavity, and the light is transmitted through the light-transmitting sheet mounted in the light-transmitting sheet mounting hole 112 to the spectral sensor 12 (not shown in Figure 4a) in the receiving cavity. It is understood that the projection of the lens mounting hole 111 on the printed circuit board 16 should completely cover the spectral sensor 12 and the image sensor 14.

[0184] The specifications of the lens and light-transmitting sheet in this embodiment can be selected and installed according to actual needs, and this embodiment does not limit them.

[0185] When the spectral sensor 12, processor 13, and image sensor 14 are fixed on the same printed circuit board, the internal structure of the industrial camera 1 is shown in Figure 5b. Figure 5b is an example diagram of the third internal structure of the industrial camera provided in this application. As shown in Figure 5b, the spectral sensor 12, processor 13, and image sensor 14 are fixed on the same printed circuit board 16. The light reflected by the object being photographed is transmitted to the image sensor 14 through the lens mounted in the lens mounting hole 111, and to the spectral sensor 12 through the light-transmitting sheet mounted in the light-transmitting sheet mounting hole 112. The printed circuit board 16 is provided with four screw holes 160 for fixing to the lens mounting base 11 through the first connector.

[0186] When the spectral sensor 12 and the image sensor 14 are fixed on one printed circuit board, and the processor 13 is fixed on another printed circuit board, the internal structure of the industrial camera 1 is shown in Figure 5c. Figure 5c is an example diagram of the fourth internal structure of the industrial camera provided in this application. As shown in Figure 5c, the spectral sensor 12 and the image sensor 14 are fixed on the same printed circuit board (i.e., the first printed circuit board 161), and the processor 13 is fixed on the second printed circuit board 162. The light reflected by the photographed object is transmitted to the image sensor 14 through the lens mounted in the lens mounting hole 111, and to the spectral sensor 12 through the light-transmitting sheet mounted in the light-transmitting sheet mounting hole 112. The first printed circuit board 161 and the second printed circuit board 162 are each provided with four screw holes 160 for fixing to the lens mounting base 11 through the first connector.

[0187] In the above embodiments, when there are multiple printed circuit boards in the industrial camera 1, the connection methods between the printed circuit boards include, but are not limited to, soldering, hollow connecting strips, V-shaped cutting, copper foil bridging, plug-in components, etc., as shown in Figures 4c and 5c. The first printed circuit board 161 and the second printed circuit board 162 are connected by a plug-in component 163.

[0188] Based on the above, in order to achieve consistency in the light collected by the image sensor and the spectral sensor, in the embodiments of this application, the light collected by the spectral sensor and the image sensor can enter through the same light-transmitting hole on the lens mount, or through two light-transmitting holes that are close to each other on the lens mount. Therefore, in one possible implementation, the industrial camera further includes a lens mount, with the spectral sensor and the image sensor entering through a lens assembly on the lens mount; alternatively, the spectral sensor enters through a light-transmitting plate on the lens mount, and the image sensor enters through a lens assembly on the lens mount.

[0189] When both the spectral sensor and the image sensor receive light through a lens assembly on a lens mount, the light intake method can be categorized into two types based on the size of the lens assembly:

[0190] I. Lens assembly covers image sensor and spectral sensor

[0191] As shown in Figures 4a-4c, the industrial camera includes a lens mount 11, a spectral sensor 12, a processor 13, and an image sensor 14. The lens mount 11 has a lens assembly mounting hole for mounting the lens assembly. A filter (not shown) is placed between the lens assembly and the sensors (image sensor 14 and spectral sensor 12) to filter out light emitted from non-ambient light sources. Light reflected from the object being photographed is transmitted sequentially through the lens assembly and the filter to the spectral sensor 12 and the image sensor 14. The spectral sensor 12 and the image sensor 14 directly perform spectral analysis and image acquisition on the incident light entering through the lens assembly mounted on the lens mount 11 and transmitted through the filter.

[0192] II. The lens assembly cannot cover the image sensor and the spectral sensor.

[0193] In this case, in order to ensure that the light entering through the lens assembly can be transmitted to the image sensor, thereby ensuring the consistency of the light collected by the spectral sensor and the image sensor, a beam splitter can be used to split the incident light entering the lens assembly 110 into two parts. One part of the light is projected onto the image sensor 14, and the other part of the light is projected onto the spectral sensor 12. Based on this, referring to Figure 6, which is a schematic diagram of the second principle structure of the industrial camera provided in this application embodiment, the industrial camera 1 also includes a beam splitter 17. When the lens assembly 110 is installed on the lens mounting base 11, the lens assembly 110 and the beam splitter 17 are arranged sequentially along the direction of light incidence. The lens assembly 110 is used to transmit the light reflected by the object being photographed to the beam splitter 17. The beam splitter 17 is used to split the light into a first light and a second light and emit them to the image sensor 14 and the spectral sensor 12. The image sensor 14 is arranged in the direction of the first light emission, and the spectral sensor 12 is arranged in the direction of the second light emission.

[0194] Image sensor 14 generates a raw image based on the received light and sends the raw image to processor 13. Spectral sensor 12 determines the spectral information of the current ambient light source based on the received light and sends the spectral information to processor 13. Processor 13 determines the target color correction matrix corresponding to the actual brightness and actual color temperature of the current light source based on the spectral information, and processes the raw image to obtain the target image.

[0195] To reduce the impact of stray light on spectral analysis and imaging, in one possible implementation, a first filter and a second filter can be provided for the spectral sensor 12 and the image sensor 14, respectively. The first filter is positioned in the direction of the first light emission and in front of the image sensor 14, and the second filter is positioned in the direction of the second light emission and in front of the spectral sensor 12. After the beam splitter 17 splits the light into the first light and the second light, the first light is transmitted to the image sensor 14 via the first filter, and the second light is transmitted to the spectral sensor 12 via the second filter.

[0196] A beam splitter is an optical element that divides incident light into two or more beams according to a specified ratio. A beam splitter can be a beam splitter, a beam splitter prism, or a combination of a reflecting mirror and a transmitting mirror; the embodiments of this application do not limit this.

[0197] In one possible implementation, when the beam splitter 17 is a beam splitter, the structure of the industrial camera 1 is shown in Figure 7a. Figure 7a is a first structural example of the industrial camera provided in the embodiment shown in Figure 6. The beam splitter 171 is inclinedly disposed between the lens mount 11 and the spectral sensor, and the beam splitter 171 has a certain angle with the incident light direction, for example, 45°. The image sensor 14 is disposed on the optical axis of the lens assembly, and the spectral sensor 12 is disposed on a plane perpendicular to the plane where the image sensor 14 is located. Light enters the industrial camera through the lens assembly mounted on the lens mount 11. After entering the internal circuit, the light is first split into a first light beam and a second light beam by the beam splitter 171. The first light beam is emitted onto the image sensor 14 fixed on the first printed circuit board 161, and the second light beam is emitted onto the spectral sensor 12 fixed on the third printed circuit board 164. The first printed circuit board 161 and the second printed circuit board 162 are each provided with four screw holes for fixing to the lens mount 11 through the first connector. Furthermore, the first printed circuit board 161 and the second printed circuit board 162, on which the processor 13 is fixed, are connected by a connector 163.

[0198] To ensure the stability of each printed circuit board, in one possible implementation, each printed circuit board can be fixed to a base. As shown in Figure 7b, which is a second structural example of the industrial camera provided in the embodiment shown in Figure 6, the beam splitter 171 is inclinedly disposed between the lens mount 11 and the spectral sensor, with the beam splitter 171 forming a certain angle with the incident light direction. The image sensor 14 is fixed to the first printed circuit board 161, the processor 13 is fixed to the second printed circuit board 162, and the spectral sensor 12 is fixed to the third printed circuit board 164. The first printed circuit board 161 and the second printed circuit board 162 are connected by a connector 163. The first printed circuit board 161, the second printed circuit board 162, and the third printed circuit board are fixed to the base 18. Each of the first printed circuit board 161 and the second printed circuit board 162 has four screw holes 160 for fixing to the lens mount 11 via a first connector.

[0199] The beam splitter 171 can be fixed by gluing or other methods, and this application embodiment does not limit this.

[0200] By fixing the spectral sensor 12, processor 13, and image sensor 14 to different printed circuit boards, it is easy to locate the problem and carry out rapid processing when any component fails, thereby improving the maintainability of the industrial camera.

[0201] In practical applications, industrial cameras are typically used in machine vision systems. These cameras can be either line scan or area scan cameras. In the case of a line scan camera, the photosensitive area of ​​the image sensor 14 is relatively small, and the spectral sensor 12 and the image sensor 14 can be illuminated through a single lens. See Figures 4a to 4c for details, which will not be repeated here.

[0202] In the case of an area scan camera, the image sensor 14 has a large photosensitive surface area, and all the light entering from a lens needs to illuminate the photosensitive surface of the image sensor 14. Therefore, a lens mounting hole and a light-transmitting plate mounting hole are provided on the lens mount for mounting the lens and light-transmitting plate, respectively. All the light entering from the lens illuminates the photosensitive surface of the image sensor 14, while the light entering from the light-transmitting plate illuminates the photosensitive surface of the spectral sensor 12. See Figures 5a to 5c for details, which will not be repeated here. Of course, in the case of a line scan camera, a lens mounting hole and a light-transmitting plate mounting hole can also be provided on the lens mount.

[0203] Understandably, since industrial cameras are typically used in machine vision systems for applications such as industrial inspection, medical imaging, and sports and entertainment, they need to transmit the acquired images in real time.

[0204] Based on this, in order to achieve image transmission, in one possible implementation, the industrial camera also includes an output interface. As shown in Figures 4c, 5c and 7b, the industrial camera is also provided with an output interface 101. The industrial camera 1 is connected to the host computer through the output interface 101. The processor 13 is also used to output at least one of the target image, spectral information, target color correction matrix, color image and original corrected image to the host computer 4 through the output interface 101.

[0205] As shown in Figure 8, Figure 8 is a schematic diagram of the application scenario of the industrial camera provided in the embodiment of this application. The industrial camera 1 is provided with an output interface 101. When the industrial camera 1 establishes a communication connection with the interface 41 of the host computer 4 through the output interface 101, the output interface 101 is used to output at least one of the target image, spectral information, target color correction matrix, color image and original correction image to the host computer 4.

[0206] It is understood that the content output by output interface 101 is controlled by the user through configuration of output parameters. For example, if the user configures the output target image and spectral information, then after determining that the target image has been obtained, the processor 13 sends the target image and the spectral information of the ambient light source to the host computer 4 through output interface 101; in another example, if the user configures the output target image, target color correction matrix, color image and original corrected image, then after determining that the target image has been obtained, the processor 13 sends the target image, target color correction matrix, color image and original corrected image to the host computer 4 through output interface 101.

[0207] In one specific embodiment, the internal structure of the industrial camera is shown in Figure 7b, and when the industrial camera is connected to the host computer via an output interface, the principle structure of the industrial camera is shown in Figure 9. Figure 9 is a schematic diagram of the third principle structure of the industrial camera provided in this application embodiment. After the light signal enters the interior of the industrial camera 1 through the lens assembly 110 installed on the lens mounting base 11, it is split into a first light ray and a second light ray by the beam splitter 17. The first light ray propagates to the spectral sensor 12 for spectral analysis, and the second light ray propagates to the image sensor 14 for image acquisition. After the spectral sensor 12 performs spectral analysis, it sends the obtained spectral information to the processor 13. The image sensor 14 sends the acquired image to the processor 13. The processor 13 performs color correction on the image according to the spectral information to obtain the target image, and then sends the target image, spectral information, etc. to the host computer 4 via the output interface 101.

[0208] Through the embodiments of this application, the industrial camera can be connected to the host computer through the output interface, and transmit data to the host computer in real time through the output interface, ensuring that the machine vision system can process and analyze image data in real time and accurately.

[0209] As can be seen from the above embodiments, the light source in the above embodiments is mounted outside the industrial camera 1. However, considering that there are some scenarios where it is not possible to mount the light source, in one possible implementation, the industrial camera 1 also includes a light source assembly and a second connector. For example, as shown in FIG10, FIG10 is a schematic diagram of the fourth principle structure of the industrial camera provided in the embodiment of this application. The light source assembly 2 is fixed to the outside of the housing of the industrial camera 1 by the second connector 21.

[0210] Understandably, the light source assembly 2 can be placed at any position on the housing of the industrial camera 1, but the illumination area of ​​the light source assembly 2 needs to cover the field of view of the installed lens assembly 110.

[0211] In another possible implementation, the light source assembly 2 can also be disposed inside the industrial camera 1. In this implementation, an optical window is also provided on the housing. The light source assembly 2 is fixed inside the housing by a third connector, and the light source assembly illuminates the object being photographed through the optical window.

[0212] Understandably, after determining the actual color temperature and brightness of the current ambient light source, the industrial camera needs to look up the corresponding target color correction matrix from a pre-calibrated color correction matrix database. To improve the speed at which the processor 13 determines the target color correction matrix, in one possible implementation, a storage device can be provided in the industrial camera to store the color correction matrix database.

[0213] Referring to Figure 11a, which is a schematic diagram of the fifth principle structure of the industrial camera provided in this application embodiment, the industrial camera also includes a storage device 19. After the light signal enters the image sensor 14 and the spectral sensor 12 via the lens assembly 110 mounted on the lens mount 11 of the industrial camera 1, the image sensor 14 generates a raw image based on the received light and sends the raw image to the processor 13. The spectral sensor 12 determines the spectral information of the current ambient light source based on the received light and sends the spectral information to the processor 13. The processor 13 determines the target color correction matrix corresponding to the actual brightness and actual color temperature of the current light source from the color correction matrix database stored in the storage device 19 based on the spectral information, and processes the raw image to obtain the target image.

[0214] It is understandable that, in order to reduce the data transfer between the storage device 19 and the processor 13, in one possible implementation, as shown in FIG11b, FIG11b is a schematic diagram of the sixth principle structure of the industrial camera provided in the embodiment of this application, the storage device 19 may be integrated into the processor 13.

[0215] The following sections will provide detailed explanations of the processes by which the processor determines the actual brightness and color temperature of the current light source based on spectral information, calibrates the color correction matrix database, and determines the target color correction matrix from the preset color correction matrix database based on the actual brightness and color temperature of the current light source.

[0216] I. The process by which the processor determines the actual brightness and color temperature of the current light source based on spectral information is explained.

[0217] Based on the foregoing embodiments and Table 1, the spectral sensor 12 can determine the spectral information of the light source based on the received light. The spectral information is represented by the light intensity of light in different wavelength ranges. After receiving the spectral information, the processor 13 can calculate the tristimulus values ​​XYZ of the light source based on the spectral information, and then calculate the chromaticity coordinates xy, luminance, and color temperature based on the tristimulus values ​​XYZ.

[0218] In one possible implementation, the processor calculates the actual brightness and actual color temperature of the ambient light source according to the following formulas (1) to (4):

[0219] Wherein, Fn is the light intensity detected by the corresponding channel, a is the correction coefficient of the tristimulus value XYZ of the corresponding spectral wavelength band, the correction coefficient is determined in advance based on actual experience, and this application embodiment does not limit it; Lux is the actual brightness of the light source, and CCT is the actual color temperature of the light source.

[0220] II. Explanation of the process of calibrating and obtaining the color correction matrix database.

[0221] Since the color correction matrix may differ under different light source conditions, in one possible implementation, a standard color chart can be placed under light source conditions with different brightness and color temperature. An industrial camera can be used to capture the original color of each color block in the standard color chart. The standard color value of each color block is used as the reference color. The color error between the original color and the reference color of each color block is used as the objective function. An optimization algorithm, such as the least squares method, is used to solve the color correction matrix under light source conditions with different brightness and color temperature, thereby establishing a color correction matrix database.

[0222] It is understood that in the color correction process, in addition to color correction, other technologies are also involved, such as white balance, gamma correction, and 3D lookup table correction. Therefore, the color correction matrix in the embodiments of this application includes, but is not limited to, white balance correction sub-matrix, color correction sub-matrix, 3D lookup table correction sub-matrix, and gamma correction sub-matrix.

[0223] Taking a light source with a brightness of Lux1 and a color temperature of CCT1 as an example, the color correction matrix under this light source condition is obtained in the following way:

[0224] Assuming a standard color chart contains m color patches, with a brightness of Lux1 and a color temperature of CCT1, an industrial camera is used to capture images of the m color patches to obtain their original colors. Then, the original colors of each color patch are corrected under this light source condition using the original color correction matrix to obtain their corrected colors. If the total difference between the corrected color of each color patch and its reference color is greater than a preset first difference threshold, or if the difference between the corrected color of at least one color patch and its reference color is greater than a preset second difference threshold, the elements in the preset color correction matrix are adjusted, and the above operation is repeated until the total difference between the corrected color of each color patch and its reference color is less than the preset first difference threshold, or the difference between the corrected color of each color patch and its reference color is less than the preset second difference threshold. The resulting color correction matrix is ​​the color correction matrix corresponding to the light source condition with a brightness of Lux1 and a color temperature of CCT1.

[0225] Then by changing the brightness Lux j Color temperature (CCT) k Solving for different brightness levels yields Lux values. j CCTs with different color temperatures k The color correction matrix under the given light source conditions is then used to establish a color correction matrix database.

[0226] It is understandable that the original color correction matrix is ​​the color correction matrix originally set in the industrial camera. Taking the color correction matrix as an example, which includes the white balance correction sub-matrix, the color correction sub-matrix, the 3D lookup table correction sub-matrix, and the gamma correction sub-matrix, the original color correction matrix includes the original white balance correction sub-matrix, the original color correction sub-matrix, the original 3D lookup table correction sub-matrix, and the original gamma correction matrix.

[0227] Based on this, in one possible implementation, the processor 13 is further configured to:

[0228] Under preset light source conditions of multiple color temperatures and multiple brightness levels, an industrial camera is controlled to capture images of each standard color block in the standard color chart, resulting in color block images with different brightness levels at each preset color temperature.

[0229] Obtain the original color correction matrix, and use the original color correction matrix to correct the color block images of each brightness at each preset color temperature to obtain the corrected color block images;

[0230] Calculate the color error between each corrected color patch image and each standard color patch;

[0231] The original color correction matrix is ​​adjusted according to each color error to obtain the color correction matrix corresponding to different brightness at each preset color temperature;

[0232] The color correction matrices corresponding to different brightness levels at each preset color temperature are stored to obtain a color correction matrix database.

[0233] Taking a color correction matrix that includes a white balance correction sub-matrix, a color correction sub-matrix, a three-dimensional lookup table correction sub-matrix, and a gamma correction sub-matrix as an example, in one possible embodiment, the color correction matrix database can be represented by the following Table 2:

[0234] Table 2 Color Correction Matrix Database

[0235] Where WB represents white balance, CCM represents color correction matrix, 3DLUT represents 3D lookup, and Gamma represents gamma correction. The color temperature and brightness values ​​in Table 2 are arranged in ascending or descending order.

[0236] III. Explanation of the process by which the processor determines the target color correction matrix from a preset color correction matrix database based on the actual brightness and color temperature of the current light source.

[0237] As shown in Table 2 above, the color correction matrix database actually includes color correction matrices corresponding to light sources with various preset brightness and color temperatures. Therefore, a lookup table can be performed in the color correction database based on the actual brightness and color temperature of the current light source to obtain the color correction matrix corresponding to the actual brightness and color temperature of the current light source. For example, if the actual brightness of the current light source is Lux1 and the actual color temperature is CCT2, the corresponding color correction matrix obtained after looking up the table is... In another embodiment, if the actual brightness of the current light source is Lux2 and the actual color temperature is CCT2, the corresponding color correction matrix is ​​obtained after looking up the table.

[0238] However, in practical applications, the actual color temperature or brightness of the current light source may not necessarily be found in the color calibration database, making it impossible to find the corresponding target color calibration matrix. For example, the color calibration matrix database may have Lux1 value of 25, Lux2 value of 30, CCT1 value of 2700, and CCT2 value of 2900, while the actual brightness of the current light source is 28 and the actual color temperature is 2800. In this case, the corresponding color calibration matrix cannot be found in the color calibration matrix database.

[0239] To address this problem, in one possible implementation, processor 13 is specifically configured to:

[0240] The system searches the color correction matrix database for brightness values ​​that differ from the actual brightness by less than a preset first threshold and color temperatures that differ from the actual color temperature by less than a preset second threshold, thereby obtaining at least one target brightness and at least one target color temperature.

[0241] Then, the first color correction matrix corresponding to each target brightness and each target color temperature is searched in the correction matrix database to obtain at least one first color correction matrix;

[0242] Finally, the target color correction matrix is ​​obtained based on the statistics of each first color correction matrix.

[0243] The target color correction matrix is ​​obtained based on the statistics of each first color correction matrix. This can be achieved by averaging the first color correction matrices or by interpolating the first color correction matrices, such as nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, least squares interpolation, etc.

[0244] Based on this, in one possible implementation, the processor 13 is specifically used to: perform interpolation calculations on each of the first color correction matrices to obtain the target color correction matrix.

[0245] For example, if a target brightness and a target color temperature are found in the color correction matrix database with a difference from the actual brightness less than a preset first threshold and a difference from the actual color temperature less than a preset second threshold, then the nearest neighbor interpolation method can be used to determine the corresponding color correction matrix. In another example, if a target brightness and a target color temperature are found in the color correction matrix database with a difference from the actual brightness less than a preset first threshold and a difference from the actual color temperature less than a preset second threshold, then the nearest neighbor interpolation method or bilinear interpolation method can be used to determine the corresponding color correction matrix. In yet another example, if a target brightness and a target color temperature are found in the color correction matrix database with a difference from the actual brightness less than a preset first threshold and a difference from the actual color temperature less than a preset second threshold, then the nearest neighbor interpolation method, bilinear interpolation method, or bicubic interpolation method can be used to determine the corresponding color correction matrix.

[0246] Based on this, in the embodiments shown in Figures 3a to 3c, the process by which the processor 13 determines the target color correction matrix in the color correction matrix database based on the current brightness and current color temperature is actually the process of searching and interpolating in the color correction database to obtain the target color correction matrix.

[0247] Through the embodiments of this application, the industrial camera can automatically acquire the actual brightness and color temperature of the ambient light source under different light source conditions, quickly search and interpolate the color correction matrix corresponding to the current ambient light source in the color correction matrix database, thereby quickly restoring the true color information of the photographed object and improving the color restoration speed of the industrial camera.

[0248] Based on the above, taking the color correction matrix in this application embodiment as including a white balance correction sub-matrix, a color correction sub-matrix, a gamma correction sub-matrix, and a three-dimensional lookup table sub-matrix as an example, in one possible implementation, the processor 13 is specifically used to perform color correction on the original image using each sub-matrix.

[0249] In the actual correction process, the original image can be corrected simultaneously using each sub-matrix, or it can be corrected sequentially using each sub-matrix. It is understood that, in the process of sequentially correcting the original image using each sub-matrix, the original image can be corrected first using the white balance correction sub-matrix to obtain a white balance corrected image, then the white balance corrected image can be corrected using the color correction matrix to obtain a color corrected image, then the color corrected image can be corrected using the gamma correction sub-matrix to obtain a gamma corrected image, and finally the gamma corrected image can be corrected using the 3D lookup table sub-matrix to obtain the target image; alternatively, the gamma correction sub-matrix can be used first, followed by the 3D lookup table sub-matrix, then the color correction sub-matrix, and finally the white balance correction sub-matrix. All of these are acceptable, and the embodiments of this application do not limit the order of the sub-matrixes.

[0250] Corresponding to the industrial camera of the first aspect mentioned above, a second aspect of the present application provides an image processing method applied to the industrial camera of the first aspect mentioned above; as shown in FIG12, FIG12 is a first flowchart of the image processing method provided by the embodiment of the present application, the method includes:

[0251] Step S10: Obtain the original image and spectral information;

[0252] Step S20: Determine the target color correction matrix corresponding to the spectral information from the pre-calibrated color correction matrix database based on the spectral information;

[0253] Step S30: Perform color processing on the original image based on the target color correction matrix to obtain the target image.

[0254] The method of this application embodiment allows an industrial camera to acquire the spectral information of ambient light sources in real time. When the light source changes, it can automatically acquire the current spectral information of the ambient light source and perform color processing on the original image based on the color correction matrix corresponding to the spectral information to obtain a target image with high color fidelity. This eliminates the need to perform color correction and white balance adjustment on the industrial camera according to changes in the light source, simplifying the adjustment process of the industrial camera.

[0255] In one possible implementation, the color correction matrix includes a white balance correction sub-matrix, a color correction sub-matrix, a gamma correction sub-matrix, and a three-dimensional lookup table sub-matrix; step S30 specifically includes: color correction of the original image using each sub-matrix.

[0256] As mentioned above, the color correction matrix is ​​pre-calibrated in the following way:

[0257] Under preset light source conditions of multiple color temperatures and multiple brightness levels, an industrial camera is controlled to capture images of each standard color patch in the standard color chart, obtaining color patch images at different brightness levels under each preset color temperature. The original color correction matrix is ​​then obtained, and the original color correction matrix is ​​used to correct the color patch images at each brightness level under each preset color temperature, resulting in corrected color patch images. The color error between each corrected color patch image and each standard color patch is calculated. Based on each color error, the original color correction matrix is ​​adjusted to obtain the color correction matrix corresponding to different brightness levels under each preset color temperature. The color correction matrices corresponding to different brightness levels under each preset color temperature are stored to obtain a color correction matrix database.

[0258] In one possible implementation, as shown in FIG13, FIG13 is a second flowchart of the image processing method provided in the embodiment of the present application, wherein the above step S20 includes:

[0259] Step S201: Determine the actual brightness and actual color temperature of the ambient light source based on the spectral information;

[0260] Step S202: Determine the target color correction matrix corresponding to the actual brightness and actual color temperature in the pre-calibrated color correction matrix database.

[0261] Based on the aforementioned first aspect, the industrial camera 1 can either determine the target color correction matrix solely based on the spectral information received initially, and then perform color processing on all received raw images based on this target correction matrix; or it can acquire spectral information in real time for each raw image acquired, determine the target color correction matrix based on the spectral information acquired during raw image acquisition, and then process the raw image. Therefore, in one possible implementation, the aforementioned step S201 includes:

[0262] The actual brightness and color temperature of the ambient light source are determined based on the spectral information received initially.

[0263] The aforementioned step S202 includes:

[0264] Determine the target color correction matrix corresponding to the actual brightness and actual color temperature in the pre-calibrated color correction matrix database, and record the target color correction matrix;

[0265] The aforementioned step S30 includes:

[0266] The original image is processed based on the recorded target color correction matrix to obtain the target image.

[0267] To describe the image processing method of this application embodiment in more detail, refer to Figure 14. Figure 14 is a first interactive diagram of the image processing method provided in this application embodiment. After the spectral sensor sends spectral information to the processor, the processor determines and records the target color correction matrix 1 based on the spectral information. After receiving the original image 1, original image 2, and original image 3 sent by the image sensor, the processor performs color processing on the original image 1, original image 2, and original image 3 respectively using the recorded target color correction matrix 1 to obtain target image 1, target image 2, and target image 3.

[0268] The processor can determine the target color correction matrix in the following way:

[0269] Method 1: The processor determines the target color correction matrix corresponding to the spectral information from a pre-calibrated color correction matrix database based on the spectral information.

[0270] Method 2: The processor inputs spectral information into a deep neural network to obtain the color correction matrix output by the deep neural network, which serves as the target color correction matrix. This deep neural network is a pre-trained neural network used to predict the color correction matrix.

[0271] Method 3: Input the spectral information into a formula pre-set based on prior knowledge to calculate the color correction matrix, which is then used as the target color correction matrix.

[0272] Furthermore, the aforementioned methods one to three are merely three possible examples. In other possible embodiments, the processor may also determine the target color correction matrix in other ways, and this application does not impose any limitations on this.

[0273] Corresponding to the aforementioned situation where the light source changes, in one possible implementation, as shown in FIG15, FIG15 is a third flowchart of the image processing method provided in the embodiments of this application, the method including:

[0274] Step S01: Control the spectral sensor to determine the current spectral information of the ambient light source, and simultaneously control the image sensor to generate the original image of the object being photographed.

[0275] Step S02: Store the current spectral information;

[0276] Step S03: Determine whether the difference between the current spectral information and the previously received spectral information is greater than a preset threshold.

[0277] If the difference between the current spectral information and the previously received spectral information is less than or equal to a preset threshold, then step S04 is executed; if the difference between the current spectral information and the previously received spectral information is greater than a preset threshold, then steps S05 to S07 are executed.

[0278] Step S04: Use the previously received spectral information as the target spectral information and obtain the previously recorded target color correction matrix;

[0279] Step S05: Use the current spectral information as the target spectral information;

[0280] Step S06: Determine the actual brightness and actual color temperature of the ambient light source based on the target spectral information;

[0281] Step S07: Determine the target color correction matrix corresponding to the first actual brightness and actual color temperature in the pre-calibrated color correction matrix database, and record the target color correction matrix.

[0282] As shown in the second interactive diagram of the image processing method in Figure 16, after the industrial camera in this embodiment starts working, the spectral sensor and the image sensor simultaneously send spectral information 1 and the original image 1 to the processor. The processor saves the spectral information 1 and determines the target color correction matrix 1 based on the spectral information 1. Then, it performs color processing on the original image 1 based on the target color correction matrix 1 to obtain the target image 1. After the spectral sensor sends spectral information 2 to the processor and the image sensor simultaneously sends the original image 2 to the processor, the processor first determines whether the difference between the spectral information 2 and the previously received spectral information 1 is greater than a preset threshold. If so, it redetermines the target color correction matrix 2 based on the spectral information 2 and performs color processing on the original image 2 based on the target color correction matrix 2 to obtain the target image 2. After the spectral sensor sends spectral information 3 to the processor and the image sensor simultaneously sends the original image 3 to the processor, the processor first determines whether the difference between the spectral information 3 and the previously received spectral information 2 is greater than a preset threshold. If not, it directly obtains the latest recorded target color correction matrix 2 and performs color processing on the original image 3 based on the target color correction matrix 2 to obtain the target image 3.

[0283] In one possible implementation, the camera further includes a beam splitter, as shown in FIG17. FIG17 is a fourth flowchart of the image processing method provided in the embodiments of this application. The above step S30 includes:

[0284] Step S31: Perform first color interpolation on the original image to obtain a first color image;

[0285] Step S32: Perform first color correction on the first color image according to the target color correction matrix to obtain the target image.

[0286] In another possible implementation, as shown in FIG18, FIG18 is a fifth flowchart of the image processing method provided in the embodiments of this application, wherein the above step S30 includes:

[0287] Step S33: Perform second color correction on the original image according to the target color correction matrix to obtain the original corrected image;

[0288] Step S34: Perform second color interpolation on the original corrected image to obtain the target image.

[0289] In one possible implementation, as shown in FIG19, FIG19 is a sixth flowchart of the image processing method provided in the embodiments of this application. The above step S34 includes:

[0290] Step S341: Perform second color interpolation on the original corrected image to obtain a second color image;

[0291] Step S342: Perform third color correction on the second color image according to the target color correction matrix to obtain the target image.

[0292] In one possible implementation, the industrial camera further includes an output interface, through which the industrial camera is connected to a host computer, as shown in Figure 20. Figure 20 is a seventh flowchart of the image processing method provided in this application embodiment. The above method further includes:

[0293] Step S40: Output at least one of the target image, spectral information, target color correction matrix, color image and original corrected image to the host computer through the output interface.

[0294] In one possible implementation, determining the target color correction matrix corresponding to the actual brightness and actual color temperature from a pre-calibrated color correction matrix database includes:

[0295] The brightness that differs from the actual brightness by less than a preset first threshold and the color temperature that differs from the actual color temperature by less than a preset second threshold are found in the pre-calibrated color correction matrix database to obtain at least one target brightness and at least one target color temperature.

[0296] Search the color correction matrix database for the first color correction matrix corresponding to each target brightness and each target color temperature to obtain at least one first color correction matrix;

[0297] The target color correction matrix is ​​obtained based on the statistics of each first color correction matrix.

[0298] In one possible implementation, the target color correction matrix is ​​obtained statistically based on each first color correction matrix, including:

[0299] Interpolation calculations are performed on each of the first color correction matrices to obtain the target color correction matrix.

[0300] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described image processing methods.

[0301] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the image processing methods described above.

[0302] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0303] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0304] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the method embodiments are basically similar to the industrial camera embodiments, so the description is relatively simple; relevant parts can be referred to the description of the method embodiments.

[0305] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An industrial camera, characterized in that The industrial camera includes: a spectral sensor, a processor, and an image sensor; The spectral sensor is used to determine the spectral information of the ambient light source based on the received light, and send the spectral information to the processor; The image sensor is used to generate an original image of the object being photographed based on the received light, and to send the original image to the processor; The processor is used to control the spectral sensor and the image sensor, and to determine the target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database based on the spectral information; and to perform color processing on the original image based on the target color correction matrix to obtain the target image.

2. The industrial camera according to claim 1, characterized in that The processor determines the target color correction matrix corresponding to the spectral information from a pre-calibrated color correction matrix database based on the spectral information, including: The actual brightness and actual color temperature of the ambient light source are determined based on the spectral information. The target color correction matrix corresponding to the actual brightness and the actual color temperature is determined in the pre-calibrated color correction matrix database.

3. The industrial camera according to claim 1, characterized in that The industrial camera also includes a lens mount; The spectral sensor and image sensor receive light through a lens assembly on a lens mount; or... The spectral sensor receives light through a light-transmitting plate on the lens mount; the image sensor receives light through the lens assembly on the lens mount.

4. The industrial camera according to claim 3, characterized in that The industrial camera also includes a beam splitter, and the lens assembly and the beam splitter are arranged sequentially along the direction of light incidence. The lens assembly is used to transmit the light reflected from the object being photographed to the beam splitter. The beam splitter is used to split the light into a first light and a second light and emit them to the image sensor and the spectral sensor, wherein the image sensor is disposed in the emission direction of the first light and the spectral sensor is disposed in the emission direction of the second light.

5. The industrial camera according to claim 4, characterized in that The beam-splitting device is a beam-splitting prism or beam-splitting plate; The image sensor is disposed on the optical axis of the lens assembly; the spectral sensor is disposed on a plane perpendicular to the plane on which the image sensor is located.

6. The industrial camera according to claim 3, characterized in that, The industrial camera also includes a filter, a portion of which is located between the lens assembly and the image sensor, and another portion of which is located between the lens assembly and the spectral sensor. The filter is used to filter out light emitted from non-ambient light sources.

7. The industrial camera according to claim 3, characterized in that The lens mounting base is provided with a lens mounting hole for mounting a lens assembly and a light-transmitting sheet mounting hole for mounting a light-transmitting sheet. The lens assembly is used to transmit light reflected from the object being photographed to the image sensor; The light-transmitting sheet is used to transmit the light reflected from the object being photographed to the spectral sensor.

8. The industrial camera according to any of claims 3 to 7, characterized in that The industrial camera also includes one or more printed circuit boards, a housing, and multiple first connectors; the housing mates with the lens mount to form a receiving cavity. One or more of the printed circuit boards are fixed within the receiving cavity by the plurality of first connectors; The photosensitive surfaces of the spectral sensor and the image sensor are capable of receiving light reflected from the object being photographed through the lens assembly mounted on the lens mount.

9. The industrial camera according to claim 8, characterized in that The industrial camera also includes a light source assembly and a second connector. The light source assembly is fixed to the outside of the housing via the second connector, and the illumination area of ​​the light source assembly covers the field of view of the installed lens assembly.

10. An image processing method characterized by, The method, applied to the industrial camera according to any one of claims 1 to 9, comprises: Obtain the original image and spectral information; Based on the spectral information, the target color correction matrix corresponding to the spectral information is determined in a pre-calibrated color correction matrix database; The original image is processed based on the target color correction matrix to obtain the target image.

11. The method of claim 10, wherein, The step of determining the target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database based on the spectral information includes: The actual brightness and actual color temperature of the ambient light source are determined based on the spectral information. The target color correction matrix corresponding to the actual brightness and the actual color temperature is determined in the pre-calibrated color correction matrix database.

12. The method according to claim 10, characterized in that, The color correction matrix includes: a white balance correction submatrix, a color correction submatrix, a gamma correction submatrix, and a three-dimensional lookup table submatrix; The step of color processing the original image based on the target color correction matrix includes: color correction of the original image using each sub-matrix.

13. The method of claim 11, wherein, The color correction matrix database was pre-calibrated according to the following method: Under preset light source conditions of multiple color temperatures and multiple brightness levels, the industrial camera is controlled to capture images of each standard color block in the standard color chart, thereby obtaining color block images with different brightness levels under each preset color temperature. Obtain the original color correction matrix, and use the original color correction matrix to correct the color block images of each brightness at each preset color temperature to obtain the corrected color block images; Calculate the color error between each corrected color patch image and each standard color patch; The original color correction matrix is ​​adjusted according to each of the color errors to obtain the color correction matrix corresponding to different brightness at each preset color temperature; The color correction matrix database is obtained by storing the color correction matrices corresponding to different brightness levels under each preset color temperature; wherein, the original color correction matrix includes: the original white balance correction sub-matrix, the original color correction sub-matrix, the original gamma correction sub-matrix, and the original three-dimensional lookup table sub-matrix.

14. The method according to claim 11, characterized in that, The step of determining the target color correction matrix corresponding to the actual brightness and the actual color temperature in a pre-calibrated color correction matrix database includes: The brightness that differs from the actual brightness by less than a preset first threshold and the color temperature that differs from the actual color temperature by less than a preset second threshold are searched in the pre-calibrated color correction matrix database to obtain at least one target brightness and at least one target color temperature. The first color correction matrix corresponding to each target brightness and each target color temperature is searched in the color correction matrix database to obtain at least one first color correction matrix; The target color correction matrix is ​​obtained based on the statistics of each of the first color correction matrices.

15. The method of claim 11, wherein, Determining the actual brightness and actual color temperature of the ambient light source based on the spectral information includes: The actual brightness and actual color temperature of the ambient light source are determined based on the spectral information received initially. The step of determining the target color correction matrix corresponding to the actual brightness and the actual color temperature in a pre-calibrated color correction matrix database includes: Determine the target color correction matrix corresponding to the actual brightness and the actual color temperature in the pre-calibrated color correction matrix database, and record the target color correction matrix; The step of performing color processing on the original image based on the target color correction matrix to obtain the target image includes: The original image is processed based on the recorded target color correction matrix to obtain the target image.

16. The method of claim 10, wherein, The acquisition of the original image and spectral information includes: The method further includes controlling the spectral sensor to determine the current spectral information of the ambient light source, and simultaneously controlling the image sensor to generate an original image of the object being photographed; the method also includes storing the current spectral information. The step of determining the target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database based on the spectral information includes: Determine whether the difference between the current spectral information and the previously received spectral information is greater than a preset threshold; If the difference between the current spectral information and the previously received spectral information is less than or equal to a preset threshold, the previously received spectral information is used as the target spectral information, and the previously recorded target color correction matrix is ​​obtained. If the difference between the current spectral information and the previously received spectral information is greater than a preset threshold, the current spectral information is used as the target spectral information, and the actual brightness and actual color temperature of the ambient light source are determined based on the target spectral information; the target color correction matrix corresponding to the actual brightness and the actual color temperature is determined in the pre-calibrated color correction matrix database, and the target color correction matrix is ​​recorded.

17. An industrial camera characterized by The industrial camera includes: a spectral sensor, a processor, and an image sensor; The spectral sensor is used to determine the spectral information of the ambient light source based on the received light, and send the spectral information to the processor; The image sensor is used to generate an original image of the object being photographed based on the received light, and to send the original image to the processor; The processor is used to control the spectral sensor and the image sensor, and to determine the target color correction matrix based on the spectral information; and to perform color processing on the original image based on the target color correction matrix to obtain the target image.

18. The industrial camera according to claim 17, characterized in that, The processor determines the target color correction matrix based on the spectral information, including: The actual brightness and actual color temperature of the ambient light source are determined based on the spectral information. The target color correction matrix is ​​determined based on the actual brightness and actual color temperature.

19. The industrial camera of claim 17, wherein, The industrial camera also includes a lens mount; The spectral sensor and image sensor receive light through a lens assembly on a lens mount; or... The spectral sensor receives light through a light-transmitting plate on the lens mount; the image sensor receives light through the lens assembly on the lens mount.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the image processing method according to any one of claims 10-16.

Images