Industrial camera and image processing method

Abstract

The embodiment of the invention provides an industrial camera and an image processing method, and relates to the technical field of machine vision, the industrial camera comprises a spectrum sensor, an image sensor and a processor, when the image sensor collects an original image of a shot object, the spectrum sensor collects spectral information of an ambient light source, and the processor processes the spectral information; and the processor determines a target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database according to the spectral information, and finally performs color processing on the original image based on the target color correction matrix to obtain a target image. By applying the scheme provided by the embodiment of the invention, the industrial camera can automatically acquire the current spectral information of the ambient light source, and perform color processing on the original image based on the color correction matrix corresponding to the spectral information to obtain the target image, so that the adjustment process of the industrial camera is simplified.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to an industrial camera and an image processing method. Background Art

[0002] As the level of industrialization continues to improve, machine vision system products are widely used in various industries, such as industrial production, medical equipment, license plate detection, etc. In machine vision systems, industrial cameras are the core components of machine vision systems to achieve their functions. In related technologies, single-chip color cameras are usually used to achieve imaging of the objects being photographed.

[0003] A single-chip color camera is mainly composed of a lens, an image sensor, a Bayer filter array, and a processor. Each pixel on the image sensor corresponds to one of the three colors of red, green, and blue, and the three pixels of red, green, and blue are arranged in a certain pattern. The lens focuses the light from the object being photographed onto the image sensor. Through the Bayer filter array, each pixel of the image sensor only receives one light signal from the three colors of red, green, and blue, and then converts the light signal into an electrical signal, and interpolates each monochrome pixel to estimate the missing color value, and finally recombines the processed pixel data into a complete color image.

[0004] However, the imaging effect of monochrome color cameras is greatly affected by the color of the light source. During the actual shooting process, changes in the color of the light source may require industrial cameras to frequently perform color correction and white balance adjustments, and the adjustment process is cumbersome. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide an industrial camera and an image processing method to simplify the adjustment process. The specific technical solution is as follows:

[0006] In a first aspect of an embodiment of the present application, an industrial camera is provided, the industrial camera comprising: a spectral sensor, a processor, and an image sensor based on a Bayer filter array;

[0007] The spectral sensor is used to determine spectral information of the ambient light source based on the received light, and send the spectral information to the processor;

[0008] The image sensor is used to generate an original image of the photographed object according to the received light, and send the original image to the processor;

[0009] The processor is used to determine the target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database according to the spectral information; and perform color processing on the original image based on the target color correction matrix to obtain a target image.

[0010] In a possible implementation manner, the processor determines, according to the spectral information, a target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database, including:

[0011] Determine the actual brightness and actual color temperature of the ambient light source according to the spectral information;

[0012] A target color correction matrix corresponding to the actual brightness and the actual color temperature is determined in a pre-calibrated color correction matrix database.

[0013] In a possible implementation, the industrial camera further includes a spectroscopic device and a lens mounting seat, wherein the lens mounting seat is used to mount a lens assembly, and the lens assembly and the spectroscopic device are sequentially arranged along the incident direction of the light;

[0014] The lens assembly is used to transmit the light reflected by the photographed object to the light splitting device;

[0015] The spectroscopic device is used to split the light into a first light and a second light and emit the light to the image sensor and the spectral sensor, wherein the image sensor is arranged in the emitting direction of the first light, and the spectral sensor is arranged in the emitting direction of the second light.

[0016] In a possible implementation manner, the light splitting device is a light splitting prism or a light splitting plate;

[0017] The image sensor is arranged on the optical axis of the lens assembly; and the spectrum sensor is arranged on a plane perpendicular to the plane where the image sensor is located.

[0018] In a possible implementation, the industrial camera further includes a lens mounting seat, and the lens mounting seat is provided with a lens mounting hole for mounting a lens and a light-transmitting sheet mounting hole for mounting a light-transmitting sheet;

[0019] The lens is used to transmit the light reflected by the photographed object to the image sensor;

[0020] The light-transmitting sheet is used to transmit the light reflected by the photographed object to the spectrum sensor.

[0021] In a possible implementation manner, the industrial camera further includes: an output interface; the industrial camera is connected to a host computer via the output interface;

[0022] The output interface is used to output the target image to the host computer.

[0023] In a possible implementation, the industrial camera further includes one or more printed circuit boards; the processor, the image sensor, and the spectral sensor are fixed on one printed circuit board; or, the image sensor and the spectral sensor are fixed on one printed circuit board, and the processor is fixed on another printed circuit board; or, the processor, the image sensor, and the spectral sensor are respectively fixed on three printed circuit boards.

[0024] In a possible implementation, the industrial camera further includes a housing and a plurality of first connecting members; the housing cooperates with the lens mounting seat to form a receiving cavity;

[0025] One or more of the printed circuit boards are fixed in the accommodating cavity through the plurality of first connectors;

[0026] The photosensitive surface of the spectrum sensor and the photosensitive surface of the image sensor can receive light reflected by the photographed object through the lens assembly installed on the lens mounting seat.

[0027] In a possible implementation, the industrial camera further includes a light source assembly and a second connecting member, the light source assembly is fixed to the outside of the housing via the second connecting member, and an illumination area of ​​the light source assembly covers a shooting field of view of the installed lens assembly.

[0028] A second aspect of the embodiments of the present application provides an image processing method, which is applied to the industrial camera described in the first aspect; the method comprises:

[0029] Get original image and spectral information;

[0030] Determine a target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database according to the spectral information;

[0031] The original image is subjected to color processing based on the target color correction matrix to obtain a target image.

[0032] In a possible implementation manner, determining a target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database according to the spectral information includes:

[0033] Determine the actual brightness and actual color temperature of the ambient light source according to the spectral information;

[0034] A target color correction matrix corresponding to the actual brightness and the actual color temperature is determined in a pre-calibrated color correction matrix database.

[0035] In a possible implementation, the color correction matrix includes: a white balance correction submatrix, a color correction submatrix, a gamma correction submatrix, and a three-dimensional lookup table submatrix;

[0036] The color processing of the original image based on the target color correction matrix includes: performing color correction on the original image using each sub-matrix.

[0037] In a possible implementation manner, the color correction matrix database is pre-calibrated according to the following method:

[0038] Under the light source conditions of preset multiple color temperatures and multiple brightnesses, the industrial camera is controlled to shoot each standard color block in the standard color card to obtain color block pictures of different brightnesses under each preset color temperature;

[0039] Obtaining an original color correction matrix, and using the original color correction matrix to correct the color block images of each brightness at each preset color temperature to obtain each corrected color block image;

[0040] Calculating the color errors between the corrected color block images and the standard color blocks;

[0041] The original color correction matrix is ​​adjusted according to each color error to obtain a color correction matrix corresponding to different brightness under each preset color temperature;

[0042] The color correction matrices corresponding to different brightness under each preset color temperature are stored to obtain the color correction matrix database; wherein the original color correction matrix includes: an original white balance correction sub-matrix, an original color correction sub-matrix, an original gamma correction sub-matrix and an original three-dimensional lookup table sub-matrix.

[0043] In a possible implementation manner, performing color processing on the original image based on the target color correction matrix to obtain the target image includes:

[0044] Performing a first color interpolation on the original image to obtain a first color image;

[0045] Performing a first color correction on the first color image according to the target color correction matrix to obtain a target image.

[0046] In a possible implementation manner, performing color processing on the original image based on the target color correction matrix to obtain the target image includes:

[0047] Performing a second color correction on the original image according to the target color correction matrix to obtain an original corrected image;

[0048] Perform a second color interpolation on the original corrected image to obtain a target image.

[0049] In a possible implementation manner, performing a second color interpolation on the original corrected image to obtain a target image includes:

[0050] Performing a second color interpolation on the original corrected image to obtain a second color image;

[0051] Perform a third color correction on the second color image according to the target color correction matrix to obtain a target image.

[0052] In a possible implementation, the industrial camera further includes an output interface, and the industrial camera is connected to a host computer via the output interface. The method further includes:

[0053] At least one of the target image, the spectral information, the target color correction matrix, the color image and the original corrected image is output to the host computer through the output interface.

[0054] In a possible implementation manner, determining a target color correction matrix corresponding to the actual brightness and the actual color temperature in a pre-calibrated color correction matrix database includes:

[0055] Searching for a brightness whose difference from the actual brightness is less than a preset first threshold value, and a color temperature whose difference from the actual color temperature is less than a preset second threshold value in a pre-calibrated color correction matrix database, to obtain at least one target brightness and at least one target color temperature;

[0056] Searching the color correction matrix database for the first color correction matrix corresponding to each of the target brightnesses and each of the target color temperatures to obtain at least one first color correction matrix;

[0057] A target color correction matrix is ​​obtained based on the statistics of each of the first color correction matrices.

[0058] In a possible implementation manner, obtaining a target color correction matrix based on statistics of each of the first color correction matrices includes:

[0059] Interpolation calculation is performed on each of the first color correction matrices to obtain a target color correction matrix.

[0060] In a possible implementation, determining the actual brightness and the actual color temperature of the ambient light source according to the spectral information includes:

[0061] Determining the actual brightness and actual color temperature of the ambient light source according to the spectral information received for the first time;

[0062] The step of determining a target color correction matrix corresponding to the actual brightness and the actual color temperature in a pre-calibrated color correction matrix database includes:

[0063] Determine a target color correction matrix corresponding to the actual brightness and the actual color temperature in a pre-calibrated color correction matrix database, and record the target color correction matrix;

[0064] The color processing of the original image based on the target color correction matrix to obtain the target image includes:

[0065] The original image is subjected to color processing based on the recorded target color correction matrix to obtain a target image.

[0066] In a possible implementation manner, the obtaining of the original image and spectral information includes:

[0067] Controlling the spectrum sensor to determine current spectrum information of the ambient light source, and simultaneously controlling the image sensor to generate an original image of the photographed object;

[0068] The step of determining a target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database according to the spectral information includes:

[0069] Determine whether the difference between the current spectrum information and the spectrum information received last time is greater than a preset threshold;

[0070] When the difference between the current spectral information and the spectral information received last time is less than or equal to a preset threshold, the spectral information received last time is used as the target spectral information, and the target color correction matrix recorded last time is obtained;

[0071] When the difference between the current spectral information and the spectral information received last time is greater than a preset threshold, the current spectral information is used as the target spectral information, and the actual brightness and the actual color temperature of the ambient light source are determined according to the target spectral information; the target color correction matrix corresponding to the actual brightness and the actual color temperature is determined in a pre-calibrated color correction matrix database, and the target color correction matrix is ​​recorded.

[0072] Beneficial effects of the embodiments of the present application:

[0073] An industrial camera and image processing method provided by an embodiment of the present application, the industrial camera includes a spectral sensor, an image sensor and a processor. When the image sensor collects the original image of the photographed object, the spectral sensor collects the spectral information of the ambient light source. The processor determines the target color correction matrix corresponding to the spectral information in a color correction matrix database obtained by pre-calibration based on the spectral information, and finally performs color processing on the original image based on the target color correction matrix to obtain the target image. Since the industrial camera of the embodiment of the present application can collect the spectral information of the ambient light source in real time, when the light source changes, the industrial camera can automatically obtain the current spectral information of the ambient light source, and perform color processing on the original image based on the color correction matrix corresponding to the spectral information to obtain a target image with high color reproduction, without the need to perform color correction and white balance adjustment on the industrial camera according to the change of the light source, thereby simplifying the adjustment process of the industrial camera.

[0074] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0076] Figure 1 Schematic diagram for generating color images for a single-chip color camera;

[0077] Figure 2 A schematic diagram of the first principle structure of an industrial camera provided in an embodiment of the present application;

[0078] Figure 3a A first process diagram of image processing of an original image provided in an embodiment of the present application;

[0079] Figure 3b A second process diagram of image processing of an original image provided in an embodiment of the present application;

[0080] Figure 3c A third process diagram for performing image processing on an original image provided in an embodiment of the present application;

[0081] Figure 4a for Figure 2 A first structural example diagram of an industrial camera provided by the illustrated embodiment;

[0082] Figure 4b for Figure 4a The first internal structure example diagram of an industrial camera is shown;

[0083] Figure 4c for Figure 4a The second example diagram of the internal structure of the industrial camera shown;

[0084] Figure 5a for Figure 2 A second structural example diagram of an industrial camera provided by the illustrated embodiment;

[0085] Figure 5b for Figure 5a The first internal structure example diagram of an industrial camera is shown;

[0086] Figure 5c for Figure 5a The second example diagram of the internal structure of the industrial camera shown;

[0087] Figure 6 A schematic diagram of the second principle structure of an industrial camera provided in an embodiment of the present application;

[0088] Figure 7a for Figure 6 A first structural example diagram of an industrial camera provided by the illustrated embodiment;

[0089] Figure 7b for Figure 6 A second structural example diagram of an industrial camera provided by the illustrated embodiment;

[0090] Figure 8 A schematic diagram of an application scenario of an industrial camera provided in an embodiment of the present application;

[0091] Fig. 9 A schematic diagram of the third principle structure of an industrial camera provided in an embodiment of the present application;

[0092] Fig.10 A fourth principle structural diagram of an industrial camera provided in an embodiment of the present application;

[0093] Fig.11a A fifth schematic diagram of the principle structure of an industrial camera provided in an embodiment of the present application;

[0094] Fig.11b A sixth schematic diagram of the principle structure of an industrial camera provided in an embodiment of the present application;

[0095] Fig.12 A first flow chart of the image processing method provided in the embodiment of the present application;

[0096] Fig.13 A second flow chart of the image processing method provided in the embodiment of the present application;

[0097] Fig.14A first interactive diagram of the image processing method provided in an embodiment of the present application;

[0098] Fig.15 A third flow chart of the image processing method provided in the embodiment of the present application;

[0099] Fig.16 A second interactive diagram of the image processing method provided in an embodiment of the present application;

[0100] Fig.17 A fourth flow chart of the image processing method provided in an embodiment of the present application;

[0101] Fig.18 A fifth flow chart of the image processing method provided in the embodiment of the present application;

[0102] Fig.19 A sixth flow chart of the image processing method provided in the embodiment of the present application;

[0103] Fig. 20 This is the seventh flow chart of the image processing method provided in the embodiment of the present application. DETAILED DESCRIPTION

[0104] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0105] First, the professional terms in the embodiments of the present application are explained:

[0106] Three stimulus values: It is the expression of the degree of stimulation of the three primary colors that cause the human retina to perceive a certain color, expressed as X (red primary color stimulation amount), Y (green primary color stimulation amount), and Z (blue primary color stimulation amount).

[0107] Chromaticity coordinates: represent the position of color in the chromaticity space. In the chromaticity diagram, the color is represented by two coordinates, x and y.

[0108] With the continuous improvement of the level of industrialization, machine vision system products are widely used in various industries, such as industrial production, medical equipment, license plate detection, etc. In the machine vision system, the industrial camera is the core component of the machine vision system to realize its function. The industrial camera uses an image sensor that can receive spectral response as a photosensitive unit to receive light reflected or projected from the surface of the object, thereby realizing imaging of the photographed object.

[0109] In the related art, color industrial cameras include single-chip color cameras and true color industrial cameras. True color industrial cameras use a beam splitter prism to refract the incident light onto three monochrome sensor target surfaces, perform photoelectric conversion to obtain the values ​​of red, green, and blue, and then directly synthesize them to obtain a color image. True color industrial cameras have good color reproduction, but they have high requirements for the beam splitting performance of the beam splitter prism and are usually more expensive. Therefore, single-chip color cameras are usually used to achieve imaging of the object being photographed.

[0110] A single-chip color camera is mainly composed of a lens, an image sensor, a Bayer filter array, and a processor. Each pixel on the image sensor corresponds to one of the three colors: red, green, and blue. The three pixels are arranged according to a certain pattern. Figure 1 Schematic diagram for generating color images for a single-chip color camera, such as Figure 1 As shown, in the related art, the lens focuses light from the photographed object onto the image sensor, generates a Bayer array image on the image sensor through the Bayer filter array, performs color gamut separation on the Bayer array image to obtain red component, green component and blue component, then performs color interpolation completion information on each component to obtain the completed red component, green component and blue component, then performs color gamut merging on the completed red component, green component and blue component to obtain a three-channel image, i.e., a color image.

[0111] However, the imaging effect of a single-chip color camera is greatly affected by the color of the light source. During the actual shooting process, changes in the color of the light source may require industrial cameras to frequently perform color correction and white balance adjustments, and the adjustment process is cumbersome.

[0112] In order to simplify the adjustment process of a single-chip color camera so that the single-chip color camera can quickly capture images with high color reproduction under different light source conditions, in a first aspect of an embodiment of the present application, an industrial camera is provided. Figure 2 The first principle structural diagram of the industrial camera provided in the embodiment of the present application is as follows: Figure 2 As shown, the industrial camera 1 includes a spectral sensor 12, a processor 13 and an image sensor 14. In order to achieve light intake, the industrial camera 1 also includes a lens mounting seat 11 for mounting a lens assembly 110. The lens assembly 110 is used to transmit the light reflected by the photographed object 3 to the image sensor 14 and the spectral sensor 12; the spectral sensor 12 is used to determine the spectral information of the ambient light source (i.e., the light source assembly 2 in the figure) based on the received light, and send the spectral information to the processor 13; the image sensor 14 is used to generate an original image according to the received light, and send the original image to the processor 13;

[0113] The processor 13 is used to determine the target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database according to the spectral information; and perform color processing on the original image based on the target color correction matrix to obtain the target image.

[0114] Through the industrial camera of the embodiment of the present application, since the industrial camera can collect spectral information of the ambient light source in real time, when the light source changes, the industrial camera can automatically obtain the current spectral information of the ambient light source, and perform color processing on the original image based on the color correction matrix corresponding to the spectral information to obtain a target image with high color reproduction. There is no need to perform color correction and white balance adjustment on the industrial camera according to changes in the light source, thereby simplifying the adjustment process of the industrial camera.

[0115] The spectrum sensor 12 is a device capable of measuring and analyzing spectral characteristics, and is used to determine the spectrum information of the ambient light source based on the received light, and send the spectrum information to the processor 13. The spectrum sensor 12 has multiple optical channels, each of which detects the light intensity of a corresponding spectral wavelength range, and the multiple optical channels cover the spectral range of the incident light of the spectrum sensor 12, that is, the spectrum sensor 12 can disperse the incident light according to the wavelength and measure the light intensity of each wavelength.

[0116] In a possible embodiment, assuming that the spectral sensor 12 has n optical channels, the light intensity detected by each optical channel is recorded as Fn, n is the serial number corresponding to the channel of the spectral wavelength range, and the spectral wavelength range corresponding to each optical channel is shown in the following Table 1, where λ represents the wavelength. By measuring the intensity of light of different wavelengths, the spectral sensor 12 can obtain spectral information.

[0117] Table 1 Optical channel corresponding spectral wavelength range

[0118]

[0119] It is understandable that when the machine vision system is applied to scenes such as industrial inspection and medical imaging, the light source in the scene is usually fixed, that is, the spectral information of the ambient light source in the scene is usually unchanged. In order to reduce the computational complexity of the processor 13, for the same ambient light source, it is only necessary to calculate the spectral information of the ambient light source, further determine the target color correction matrix corresponding to the ambient light source, and then perform color processing on all original images collected under the ambient light source based on the target color correction matrix, without having to obtain the spectral information of the ambient light source multiple times.

[0120] Therefore, in a possible implementation manner, the processor 13 is specifically configured to:

[0121] Determine the actual brightness and actual color temperature of the ambient light source according to the spectrum information received for the first time;

[0122] Determine a target color correction matrix corresponding to actual brightness and actual color temperature in a pre-calibrated color correction matrix database, and record the target color correction matrix;

[0123] The original image is color processed based on the recorded target color correction matrix to obtain a target image.

[0124] It is understandable that the working state of the spectral sensor 12 is not stable when it is just powered on, and the collected spectral information may be inaccurate. Therefore, the spectral information received for the first time in the embodiment of the present application refers to the valid spectral information received for the first time. In one possible implementation, after receiving the spectral information, the processor 13 will determine whether the received spectral information has reached a stable state. If it has, the spectral information that has reached a stable state will be used as valid spectral information. Among them, the stable state means that within a short period of time, for example: within 1 minute, the difference between several consecutive spectral information received is less than a preset threshold, then it can be determined that the stable state has been reached.

[0125] After the processor 13 obtains the target color correction matrix according to the spectral information received for the first time, the target color correction matrix is ​​recorded, and after receiving each original image sent by the image sensor 14, the recorded target color correction matrix is ​​used to perform color processing on each original image.

[0126] In this case, the processor 13 may control the spectral sensor 12 to turn off after first receiving the spectral information, so that the spectral sensor 12 no longer acquires and sends the spectral information to the processor 13; or the processor 13 may calculate the actual color temperature and actual brightness of the ambient light source based on the spectral information received for the first time after first receiving the spectral information, and then calculate the target color correction matrix as the target color correction matrix corresponding to the ambient light source. If the processor 13 receives the spectral information sent by the spectral sensor 12 again, the target color correction matrix will no longer be calculated based on the spectral information.

[0127] It can be understood that in this embodiment, the image sensor 14 and the spectral sensor 12 can work simultaneously, or the spectral sensor 12 can first collect the spectral information of the ambient light source, and the processor 13 controls the image sensor 14 to collect the original image after determining the target color correction matrix based on the spectral information received for the first time. The embodiment of the present application is not limited to this.

[0128] By using the embodiment of the present application, the processor can determine the target color correction matrix corresponding to the ambient light source only through the spectral information received for the first time, thereby reducing the amount of calculation of the processor and improving the speed of image color processing.

[0129] However, in some scenarios such as sports entertainment, medical testing, etc., the ambient light source may change. Therefore, if the original images captured by the industrial camera 1 are color processed based on the target color correction matrix corresponding to the ambient light source determined by the processor 13 based on the spectral information received for the first time, there may be a problem of inaccurate color restoration of some original images.

[0130] Based on this, in a possible implementation, the processor 13 can control the spectrum sensor 12 to determine the current spectrum information of the ambient light source based on the received light, and simultaneously control the image sensor to generate an original image of the photographed object based on the received light. That is, the processor 13 controls the spectrum sensor 12 to collect spectrum information once, and simultaneously controls the image sensor 14 to collect an original image once.

[0131] For each acquired spectral information and original image, the processor 13 determines a target color correction matrix based on the acquired spectral information, and then performs color processing on the acquired original image based on the target color correction matrix.

[0132] It is understandable that machine vision systems usually shoot at high speed, so the frame rate of industrial cameras is usually high. For example, the frame rate of industrial camera 1 is 30 frames per second, and the camera captures a picture approximately every 0.033 seconds (1 / 30 seconds). It can be seen that the time between two adjacent images captured by the industrial camera is extremely short, and the ambient light source does not change significantly within a short period of time. Therefore, in a short period of time, the target color correction matrix determined by the processor 13 based on the multiple spectral information collected by the spectral sensor 12 is actually the same.

[0133] Based on this, in a possible implementation manner, the processor 13 is specifically configured to:

[0134] Controlling the spectrum sensor to determine the current spectrum information of the ambient light source, storing the current spectrum information, and simultaneously controlling the image sensor to generate an original image of the photographed object;

[0135] Determine whether the difference between the current spectrum information and the spectrum information received last time is greater than a preset threshold;

[0136] When the difference between the current spectral information and the spectral information received last time is less than or equal to a preset threshold, the spectral information received last time is used as the target spectral information, and the target color correction matrix recorded last time is obtained;

[0137] When the difference between the current spectral information and the spectral information received last time is greater than a preset threshold, the current spectral information is used as the target spectral information, and the actual brightness and actual color temperature of the ambient light source are determined based on the target spectral information; the target color correction matrix corresponding to the actual brightness and actual color temperature is determined in the pre-calibrated color correction matrix database, and the target color correction matrix is ​​recorded.

[0138] That is, when the processor 13 receives the current spectrum information of the ambient light source sent by the spectrum sensor, it stores the current spectrum information and determines whether the difference between the current spectrum information and the spectrum information received last time is greater than a preset threshold;

[0139] If the difference between the current spectral information and the spectral information received last time is greater than a preset threshold, the current spectral information is used as the target spectral information, and the target color correction matrix is ​​determined in the color correction matrix database according to the target spectral information, and the target color correction matrix is ​​recorded, and then the color processing of the currently acquired original image is performed based on the recorded target color correction matrix to obtain the target image;

[0140] If the difference between the current spectral information and the last received spectral information is less than or equal to a preset threshold, the last recorded target color correction matrix is ​​obtained, and the currently acquired original image is color processed based on the target color correction matrix to obtain the target image.

[0141] By adopting the embodiment of the present application, the current spectral information of the ambient light source is obtained each time the original image is captured, and the target color correction matrix is ​​determined according to the current spectral information, which can significantly improve the accuracy of color processing of the original image.

[0142] In the embodiment of the present application, the image sensor 14 is an image sensor based on a Bayer filter array. The Bayer array is a color filter array, which is composed of red, green, and blue filters arranged in a specific pattern above the photosensitive element of the image sensor. An image sensor based on a Bayer filter array usually uses four pixels as a pixel unit, and each pixel unit contains red, green, and blue pixels, a gray pixel, or a green pixel.

[0143] It is understandable that due to the color filtering characteristics of the Bayer array, the original image captured by the image sensor 14 is monochrome, that is, each pixel contains only one of the three colors of red, green and blue, and the original image visually presents a mosaic-like color distribution. Therefore, after the image sensor 14 captures the original image, it is necessary to send the original image to the processor 13 for color processing.

[0144] Color processing refers to performing various processing such as color interpolation and color correction on the unprocessed monochrome or incomplete color image data captured by the image sensor based on the Bayer array to restore, enhance or modify the color attributes of the image. After color processing of the original image, the image is closer to the real color seen by the human eye.

[0145] It is understandable that since industrial cameras are usually used in machine vision systems, and thus in industrial inspection, medical imaging, sports entertainment and other scenarios, industrial cameras need to achieve high-speed image acquisition and processing. Based on this, in one possible implementation, the processor 13 is composed of an FPGA (Field-Programmable Gate) or a SoC (System on Chip). For example, if the user has high requirements for the image acquisition speed of the industrial camera, FPGA is selected as the processor 13; if the user focuses on the low power consumption of the industrial camera, SoC is selected as the processor 13, and the embodiments of the present application are not limited to this.

[0146] When the processor 13 performs color processing on the original image, it may be to first perform color interpolation processing on the original image and then perform color correction processing; it may be to first perform color correction processing on the original image and then perform color interpolation processing; it may be to perform color correction processing on the original image and then perform color interpolation processing, and then perform color correction processing on the color interpolated image to obtain the target image, and the embodiments of the present application are not limited to this.

[0147] In the embodiment of the present application, the processor 13 performs color correction, which means that the processor 13 performs color correction on the image according to the target color correction matrix corresponding to the spectral information of the ambient light source. The target color correction matrix is ​​determined by the processor 13 in a pre-calibrated color correction database according to the spectral information of the ambient light source.

[0148] Among them, the color correction matrix database may include a mapping relationship between spectral information and the color correction matrix. After the processor 13 obtains the spectral information of the ambient light source, it determines the target color correction matrix in the color correction matrix database according to the mapping relationship between the spectral information and the color correction matrix; it may also include a mapping relationship between the color temperature and / or brightness of the light source and the color correction matrix. After the processor 13 obtains the spectral information of the ambient light source, it determines the actual brightness and / or actual color temperature of the ambient light source according to the spectral information, and then determines the target color correction matrix corresponding to the actual brightness and / or actual color temperature according to the mapping relationship between the color temperature and / or brightness and the color correction matrix.

[0149] In a possible implementation manner, the processor 13 is specifically configured to:

[0150] The actual brightness and actual color temperature of the ambient light source are determined according to the spectral information; and the target color correction matrix corresponding to the actual brightness and actual color temperature is determined in a pre-calibrated color correction database.

[0151] Among them, the process of the processor 13 determining the actual brightness and actual color temperature of the current light source based on the spectral information, the process of establishing the color correction matrix database, and the process of the processor 13 determining the target color correction matrix in the preset color correction matrix database based on the actual brightness and actual color temperature of the current light source are described below and will not be repeated here.

[0152] By adopting the embodiments of the present application, the industrial camera can collect the spectral information of the ambient light source in real time and determine the actual brightness and actual color temperature. When the light source changes, the current actual brightness and actual color temperature of the ambient light source can be automatically obtained, and the original image can be color processed based on the color correction matrix corresponding to the actual brightness and actual color temperature, thereby improving the accuracy of color correction of the original image.

[0153] Due to the different orders of color interpolation and color correction, the processor 13 performs color processing on the original image based on the target color correction matrix, and the process of obtaining the target image is also different.

[0154] In a possible implementation manner, the processor 13 is specifically configured to:

[0155] Performing a first color interpolation on the original image to obtain a first color image;

[0156] A first color correction is performed on the first color image according to the target color correction matrix to obtain a target image.

[0157] For example, see Figure 3a , Figure 3a The first process diagram of image processing for an original image provided by an embodiment of the present application is shown. After determining the actual brightness (i.e., current brightness) and actual color temperature (i.e., current color temperature) of the current light source, the processor 13 determines the target color correction matrix (i.e., current color correction matrix) in the color correction matrix database;

[0158] The processor 13 first performs color interpolation on the original image to obtain an original color image, and then performs color correction on the original color image using a target color matrix to obtain a target image (ie, a color-corrected image).

[0159] Through the embodiments of the present application, color interpolation is performed before color correction, which can ensure that the image has complete color information, and the color correction algorithm can be adjusted based on the complete color data to obtain a more accurate correction result.

[0160] In the color interpolation process, color distortion may occur due to color deviation. Therefore, the original image may be color corrected based on the target color correction matrix first, and then color interpolation may be performed to reduce the color distortion caused by color deviation. In a possible implementation, the processor 13 is specifically configured to:

[0161] Performing a second color correction on the original image according to the target color correction matrix to obtain an original corrected image;

[0162] Perform a second color interpolation on the original corrected image to obtain a target image.

[0163] For example, see Figure 3b , Figure 3b The second process diagram of image processing of the original image provided by the embodiment of the present application is shown. After determining the actual brightness (i.e., current brightness) and actual color temperature (current color temperature) of the current light source, the processor 13 determines the target color correction matrix (i.e., current color correction matrix) in the color correction matrix database;

[0164] The processor 13 first performs color correction on the original image based on the target color matrix to obtain an original corrected image (ie, a Bayer corrected image), and then performs color interpolation on the original corrected image to obtain a target image (ie, a color corrected image).

[0165] In order to restore the true color of the object in the image to the greatest extent, the original image may be color corrected, then color interpolated, and then the color interpolated image may be color corrected to obtain the target image. Therefore, in a possible implementation, the processor 13 is specifically used to:

[0166] Performing a second color interpolation on the original corrected image to obtain a second color image;

[0167] A third color correction is performed on the second color image according to the target color correction matrix to obtain a target image.

[0168] For example, see Figure 3c , Figure 3c The third process diagram of image processing for the original image provided by the embodiment of the present application is shown. After determining the actual brightness (i.e., current brightness) and actual color temperature (current color temperature) of the current light source, the processor 13 determines the target color correction matrix (i.e., current color correction matrix) in the color correction matrix database;

[0169] The processor 13 first performs color correction on the original image based on the target color matrix to obtain the original corrected image (i.e., the Bayer corrected image), then performs color interpolation on the original corrected image to obtain the original color image, and then performs color correction on the original color image using the target color matrix to obtain the target image (i.e., the color corrected image).

[0170] Through the embodiments of the present application, the target color correction matrix is ​​used to perform color correction on the original image and the color image after color interpolation, so as to restore the true color information of the object photographed by the industrial camera to the greatest extent.

[0171] In order to enable the image sensor 14, the processor 13 and the spectral sensor 12 to work together and further improve the efficiency and accuracy of industrial camera image processing, in a possible implementation, the industrial camera 1 also includes one or more printed circuit boards, and the image sensor 14, the image sensor 14 and the processor 13 can be connected and communicated through the printed circuit boards.

[0172] The processor 13, image sensor 14 and spectral sensor 12 can be fixed on the same printed circuit board; the image sensor 14 and spectral sensor 12 can be fixed on one printed circuit board, and the processor 13 can be fixed on another printed circuit board; the image sensor 14, spectral sensor 12 and processor 13 can be fixed on three printed circuit boards respectively. All of these are possible and are not limited to this in the embodiments of the present application.

[0173] It is understandable that, since the spectral sensor 12, the image sensor 14 and the processor 13 are the core electronic components of the industrial camera 1, they may be damaged when subjected to external physical shocks, vibrations, extrusions, etc. Therefore, in order to ensure the stable operation of the core components of the industrial camera 1, in a possible implementation, the industrial camera 1 also includes a housing and a first connector, the housing cooperates with the lens mount 11 to form a receiving cavity, the spectral sensor 12, the image sensor 14 and the processor 13 are fixed on one or more printed circuit boards, and the one or more printed circuit boards are fixed in the receiving cavity through a plurality of first connectors. Among them, the first connector can be a connector such as a bolt or a stud, and the embodiment of the present application does not limit the type of the first connector.

[0174] The structure of the industrial camera provided in this application is described below with reference to the accompanying drawings.

[0175] In a possible embodiment, the structure of the industrial camera is as follows Figure 4a As shown, Figure 4a for Figure 2 The first structural example diagram of the industrial camera provided by the embodiment shown is as follows: Figure 4aAs shown, there is only one lens mounting hole 111 on the lens mounting seat 11, and the housing 15 cooperates with the lens mounting seat 11 to form a receiving cavity, and the spectral sensor 12 and the image sensor in the receiving cavity are lighted through the lens assembly installed in the lens mounting hole 111 ( Figure 4a It is understood that the projection of the lens mounting hole 111 on the printed circuit board 16 should completely cover the spectral sensor 12 and the image sensor 14.

[0176] When the spectral sensor 12, the processor 13 and the image sensor 14 are fixed on the same printed circuit board, the internal structure of the industrial camera 1 is as follows: Figure 4b As shown, Figure 4b for Figure 4a The first internal structure example of an industrial camera is shown in FIG. Figure 4b As shown, the spectral sensor 12, the processor 13 and the image sensor 14 are fixed on the same printed circuit board 16. Four screw holes 160 are provided on the printed circuit board 16 for fixing to the lens mounting seat 11 through a first connecting member.

[0177] When the spectral sensor 12 and the image sensor 14 are fixed on one printed circuit board, and the processor 13 is fixed on another printed circuit board, the internal structure of the industrial camera 1 is as follows: Figure 4c As shown, Figure 4c for Figure 4a The second internal structure example of an industrial camera is shown in FIG. Figure 4c As shown, the spectral sensor 12 and the image sensor 14 are fixed on the same printed circuit board (i.e., the first printed circuit board 161), and the processor 13 is fixed on another printed circuit board (i.e., the second printed circuit board 162). The first printed circuit board 161 and the second printed circuit board 162 are each provided with four screw holes 160 for fixing to the lens mounting base 11 through the first connecting member.

[0178] In the case where there are multiple printed circuit boards in the industrial camera 1, the connection methods between the printed circuit boards include but are not limited to welding, hollow connecting strips, V-cutting, copper foil bridge connection, connectors, etc. Figure 4c As shown, the first printed circuit board 161 and the second printed circuit board 162 are connected via a plug-in connector 163 .

[0179] In another possible embodiment, the structure of the industrial camera is as follows: Figure 5a As shown, Figure 5a for Figure 2 The second structural example diagram of the industrial camera provided by the embodiment shown is as follows: Figure 5aAs shown, the lens mounting seat 11 is provided with a lens mounting hole 111 and a light-transmitting sheet mounting hole 112, which are used to mount the lens and the light-transmitting sheet respectively. The housing 15 cooperates with the lens mounting seat 11 to form a receiving cavity. The light reflected by the photographed object is transmitted from the lens mounted in the lens mounting hole 111 to the image sensor 14 in the receiving cavity. The light is transmitted from the light-transmitting sheet mounted in the light-transmitting sheet mounting hole 112 to the spectral sensor 12 in the receiving cavity ( Figure 4a It is understood that the projection of the lens mounting hole 111 on the printed circuit board 16 should completely cover the spectral sensor 12 and the image sensor 14.

[0180] The specifications of the lens and the light-transmitting sheet in the embodiment of the present application can be selected and installed according to actual needs, and the embodiment of the present application is not limited to this.

[0181] When the spectral sensor 12, the processor 13 and the image sensor 14 are fixed on the same printed circuit board, the internal structure of the industrial camera 1 is as follows: Figure 5b As shown, Figure 5b This is a third internal structure example diagram of an industrial camera provided in this application. Figure 5b As shown, the spectral sensor 12, the processor 13 and the image sensor 14 are fixed on the same printed circuit board 16. The light reflected by the photographed object is transmitted to the image sensor 14 by the lens installed in the lens mounting hole 111, and is transmitted to the spectral sensor 12 by the light-transmitting sheet installed in the light-transmitting sheet mounting hole 112. Four screw holes 160 are provided on the printed circuit board 16 for fixing to the lens mounting seat 11 through a first connecting member.

[0182] When the spectral sensor 12 and the image sensor 14 are fixed on one printed circuit board, and the processor 13 is fixed on another printed circuit board, the internal structure of the industrial camera 1 is as follows: Figure 5c As shown, Figure 5c This is a fourth example diagram of the internal structure of an industrial camera provided in this application. Figure 5c As shown, the spectral sensor 12 and the image sensor 14 are fixed on the same printed circuit board (i.e., the first printed circuit board 161), the processor 13 is fixed on the second printed circuit board 162, the light reflected by the photographed object is transmitted to the image sensor 14 by the lens installed in the lens mounting hole 111, and is transmitted to the spectral sensor 12 by the light-transmitting sheet installed in the light-transmitting sheet mounting hole 112, and four screw holes 160 are respectively provided on the first printed circuit board 161 and the second printed circuit board 162 for fixing to the lens mounting seat 11 through the first connecting member.

[0183] In the above embodiment, when there are multiple printed circuit boards in the industrial camera 1, the connection methods between the printed circuit boards include but are not limited to welding, hollow connecting strips, V-shaped cutting, copper foil bridge connection, plug-in connectors, etc. Figure 4cand Figure 5c As shown, the first printed circuit board 161 and the second printed circuit board 162 are connected via a plug-in connector 163 .

[0184] exist Figure 4a to Figure 4c In the illustrated embodiment, both the spectral sensor 12 and the image sensor 14 directly perform spectral analysis and image acquisition on the incident light entering through the lens assembly installed on the lens mounting seat 11. In this case, in order to take into account both image acquisition and spectral information acquisition, the lens assembly may introduce more aberrations and distortions, thereby affecting the quality of the image and spectrum. For example, in order to increase the coverage of the spectrum, the lens assembly may need to use a wider spectrum lens, but the spatial resolution of the image may be reduced; for another example, in order to improve the resolution of the image, the lens assembly may use a high-resolution lens, but the spectral information received by the spectral sensor may not be fine enough, affecting the accuracy of the spectral analysis.

[0185] In order to simultaneously enhance the image resolution and improve the accuracy of spectral analysis, in a possible implementation, a spectroscopic device may be used to split the incident light entering the lens assembly 110 into two parts, one part of the light being projected onto the image sensor 14, and the other part of the light being projected onto the spectral sensor 12. Based on this, see Figure 6 , Figure 6 The second principle structural diagram of the industrial camera provided in the embodiment of the present application, the industrial camera 1 also includes a spectrometer 17. When the lens assembly 110 is installed on the lens mounting seat 11, the lens assembly 110 and the spectrometer 17 are arranged in sequence along the direction of incident light; the lens assembly 110 is used to transmit the light reflected by the photographed object to the spectrometer 17; the spectrometer 17 is used to split the light into a first light and a second light and emit them to the image sensor 14 and the spectral sensor 12, the image sensor 14 is arranged in the direction of emission of the first light, and the spectral sensor 12 is arranged in the direction of emission of the second light.

[0186] The image sensor 14 generates an original image based on the received light and sends the original image to the processor 13. The spectrum sensor 12 determines the spectrum information of the current ambient light source based on the received light and sends the spectrum information to the processor 13. The processor 13 determines the target color correction matrix corresponding to the actual brightness and actual color temperature of the current light source based on the spectrum information, and processes the original image to obtain the target image.

[0187] The optical splitter is an optical element that splits incident light into two or more light beams at a specified ratio. The optical splitter can be a beam splitter, a beam splitter prism, or a combination of a reflector and a transmissive mirror, which is not limited in the present application.

[0188] In a possible implementation manner, when the light splitting device 17 is a light splitter, the structure of the industrial camera 1 is as follows: Figure 7a As shown, Figure 7a for Figure 6 In the first structural example diagram of the industrial camera provided by the illustrated embodiment, the beam splitter 171 is obliquely arranged between the lens mounting seat 11 and the spectrum sensor, and the beam splitter 171 has a certain angle with the incident direction of the light, for example, 45°. The image sensor 14 is arranged on the optical axis of the lens assembly, and the spectrum sensor 12 is arranged on a plane perpendicular to the plane where the image sensor 14 is located. After the light enters the interior of the industrial camera 1 through the lens assembly mounted on the lens mounting seat 11, the beam splitter 171 first splits the light into a first light and a second light, and emits the first light to the image sensor 14 fixed to the first printed circuit board 161, and emits the second light to the spectrum sensor 12 fixed to the third printed circuit board 164. The first printed circuit board 161 and the second printed circuit board 162 are respectively provided with four screw holes for fixing to the lens mounting seat 11 through the first connecting member, and the first printed circuit board 161 and the second printed circuit board 162 fixed with the processor 13 are connected through the connector 163.

[0189] In order to ensure the stability of each printed circuit board, in a possible implementation, each printed circuit board can be fixed on a base. Figure 7b As shown, Figure 7b Shown Figure 6 In the second structural example diagram of the industrial camera provided by the illustrated embodiment, the beam splitter 171 is obliquely arranged between the lens mounting seat 11 and the spectrum sensor, and the beam splitter 171 has a certain angle with the incident direction of the light, the image sensor 14 is fixed on the first printed circuit board 161, the processor 13 is fixed on the second printed circuit board 162, and the spectrum sensor 12 is fixed on the third printed circuit board 164. The first printed circuit board 161 and the second printed circuit board 162 are connected by a connector 163, and the first printed circuit board 161, the second printed circuit board 162 and the third printed circuit board are fixed on the base 18. The first printed circuit board 161 and the second printed circuit board 162 are respectively provided with four screw holes 160 for fixing with the lens mounting seat 11 through the first connecting member.

[0190] The beam splitter 171 may be fixed by gluing or other methods, which is not limited in the embodiment of the present application.

[0191] The spectral sensor 12, the processor 13 and the image sensor 14 are fixed to different printed circuit boards respectively, so that when any component fails, it is easy to locate the problem and then quickly deal with it, thereby improving the maintainability of the industrial camera.

[0192] In practical applications, industrial cameras are usually used in machine vision systems. In machine vision systems, industrial cameras can be either linear array cameras or area array cameras. In the case of linear array cameras, the photosensitive surface area of ​​the image sensor 14 is relatively small, and the spectral sensor 12 and the image sensor 14 can be illuminated through one lens. For details, see Figure 4a to Figure 4c , which will not be repeated here.

[0193] In the case of an area array camera, the photosensitive surface area of ​​the image sensor 14 is relatively large, and the light entering a lens needs to be completely irradiated onto the photosensitive surface of the image sensor 14. Therefore, a lens mounting hole and a light-transmitting sheet mounting hole are provided on the lens mounting seat, which are used to install the lens and the light-transmitting sheet respectively. The light entering the lens is completely irradiated onto the photosensitive surface of the image sensor 14, and the light entering the light-transmitting sheet is irradiated onto the photosensitive surface of the spectrum sensor 12. For details, see Figure 5a to Figure 5c Of course, in the case of a linear array camera, a lens mounting hole and a light-transmitting sheet mounting hole may also be provided on the lens mounting seat.

[0194] It is understandable that since industrial cameras are usually used in machine vision systems and thus in scenarios such as industrial inspection, medical imaging, sports and entertainment, industrial cameras need to transmit the captured images in real time.

[0195] Based on this, in order to realize image transmission, in a possible implementation, the industrial camera also includes an output interface. Figure 4c , Figure 5c as well as Figure 7b As shown, the industrial camera is also provided with an output interface 101, and the industrial camera 1 is connected to the host computer via the output interface 101, and the processor 13 is also used to output at least one of the target image, spectral information, target color correction matrix, color image and original corrected image to the host computer 4 via the output interface 101.

[0196] like Figure 8 As shown, Figure 8 A schematic diagram of an application scenario of an industrial camera provided in an embodiment of the present application, wherein an output interface 101 is provided on the industrial camera 1. When the industrial camera 1 establishes a communication connection with the interface 41 of the host computer 4 through the output interface 101, the output interface 101 is used to output at least one of the target image, spectral information, target color correction matrix, color image and original corrected image to the host computer 4.

[0197] It is understandable that the content output by the output interface 101 is controlled by the user by configuring the output parameters. Exemplarily, the user configures the output target image and spectral information, and the processor 13 sends the target image and the spectral information of the ambient light source to the host computer 4 through the output interface 101 after determining that the target image is obtained; in another example, the user configures the output target image, the target color correction matrix, the color image and the original corrected image, and the processor 13 sends the target image, the target color correction matrix, the color image and the original corrected image to the host computer 4 through the output interface 101 after determining that the target image is obtained.

[0198] In a specific embodiment, the internal structure of the industrial camera is as follows Figure 7b As shown, when the industrial camera is connected to the host computer through the output interface, the principle structure of the industrial camera is as follows Fig. 9 As shown, Fig. 9 This is a schematic diagram of the third principle structure of the industrial camera provided in the embodiment of the present application. After the optical signal enters the interior of the industrial camera 1 through the lens assembly 110 installed on the lens mounting seat 11, it is split into a first light ray and a second light ray by the spectrometer 17. The first light ray is transmitted to the spectral sensor 12 for spectral analysis, and the second light ray is transmitted to the image sensor 14 for image acquisition. After the spectral analysis, the spectral sensor 12 sends the obtained spectral information to the processor 13, and the image sensor 14 sends the acquired image to the processor 13. The processor 13 performs color correction on the image according to the spectral information to obtain the target image, and then sends the target image, spectral information, etc. to the host computer 4 via the output interface 101.

[0199] Through the embodiments of the present application, the industrial camera can be connected to the host computer through the output interface, and transmit data to the host computer in real time through the output interface, ensuring that the machine vision system can process and analyze image data in real time and accurately.

[0200] Based on the above embodiments, it can be known that the light source in the above embodiments is installed outside the industrial camera 1. However, considering that the light source cannot be installed in some scenes, in a possible implementation, the industrial camera 1 further includes a light source assembly and a second connecting member. For example, Fig.10 As shown, Fig.10 This is a fourth principle structural diagram of an industrial camera provided in an embodiment of the present application, in which the light source assembly 2 is fixed to the outside of the housing of the industrial camera 1 through a second connecting member 21 .

[0201] It is understandable that the light source assembly 2 can be set at any position of the housing of the industrial camera 1, but the illumination area of ​​the light source assembly 2 needs to cover the shooting field of view of the installed lens assembly 110.

[0202] In another possible implementation, the light source assembly 2 may also be disposed in the industrial camera 1. In this implementation, an optical window is also left on the shell, and the light source assembly 2 is fixed in the shell through a third connecting member. The light source assembly illuminates the object being photographed through the optical window.

[0203] It is understandable that after the industrial camera determines the actual color temperature and actual brightness of the current ambient light source, it needs to search for the corresponding target color correction matrix from the pre-calibrated color correction matrix database. In order to increase the speed of the processor 13 in determining the target color correction matrix, in a possible implementation, a storage device may be provided in the industrial camera, and the color correction matrix database may be stored in the storage device.

[0204] See also Fig.11a , Fig.11a The fifth principle structure diagram of the industrial camera provided in the embodiment of the present application, the industrial camera also includes a storage device 19. After the light signal enters the image sensor 14 and the spectral sensor 12 through the lens assembly 110 installed on the lens mount 11 of the industrial camera 1, the image sensor 14 generates an original image based on the received light and sends the original image to the processor 13, and the spectral sensor 12 determines the spectral information of the current ambient light source based on the received light and sends the spectral information to the processor 13. The processor 13 determines the target color correction matrix corresponding to the actual brightness and actual color temperature of the current light source from the color correction matrix database stored in the storage device 19 based on the spectral information, and processes the original image to obtain the target image.

[0205] It is understandable that in order to reduce the data transfer between the storage device 19 and the processor 13, in a possible implementation, as shown in FIG. Fig.11b As shown, Fig.11b The sixth principle structural diagram of the industrial camera provided in the embodiment of the present application, the storage device 19 may be integrated in the processor 13.

[0206] The following will respectively describe in detail the process of the processor determining the actual brightness and actual color temperature of the current light source according to the spectrum information, the process of calibrating the color correction matrix database, and the process of the processor determining the target color correction matrix in the preset color correction matrix database according to the actual brightness and actual color temperature of the current light source:

[0207] 1. Explain the process of the processor determining the actual brightness and actual color temperature of the current light source based on the spectrum information.

[0208] Based on the above embodiments and Table 1, it can be known that the spectrum sensor 12 can determine the spectrum information of the light source based on the received light, and the spectrum information is represented by the light intensity of light in different wavelength ranges. After receiving the spectrum information, the processor 13 can calculate the tristimulus values ​​XYZ of the light source according to the spectrum information, and then calculate the chromaticity coordinates xy, brightness and color temperature according to the tristimulus values ​​XYZ.

[0209] In a possible implementation, the processor calculates the actual brightness and the actual color temperature of the ambient light source according to the following formulas (1) to (4):

[0210]

[0211] Lux=Y (3)

[0212]

[0213] Among them, Fn is the light intensity detected by the corresponding channel, a is the correction coefficient of the calibrated tristimulus value XYZ of the corresponding spectral wavelength band, the correction coefficient is determined in advance based on actual experience, and the embodiment of the present application is not limited to this; Lux is the actual brightness of the light source, and CCT is the actual color temperature of the light source.

[0214] 2. Explain the process of calibrating the color correction matrix database

[0215] Since the color correction matrix under different light source conditions may be different, in one possible implementation, the standard color card can be placed under light source conditions of different brightness and color temperature, and an industrial camera can be used to capture the original color of each color block in the standard color card. The standard color value of each color block is used as a reference color, and the color error between the original color and the reference color of each color block is used as the objective function. An optimization algorithm such as the least squares method is used to solve the color correction matrix under light source conditions of different brightness and color temperature, thereby establishing a color correction matrix database.

[0216] It can be understood that in the color correction process, in addition to color correction, many other technologies are also involved, such as white balance, gamma correction, three-dimensional lookup, etc. Therefore, the color correction matrix in the embodiment of the present application includes but is not limited to a white balance correction sub-matrix, a color correction sub-matrix, a three-dimensional lookup table correction sub-matrix, and a gamma correction sub-matrix, etc.

[0217] Taking the brightness of the light source as Lux1 and the color temperature as CCT1 as an example, the color correction matrix under this light source condition is established in the following way:

[0218] Assume that the standard color card includes m color blocks. When the brightness is Lux1 and the color temperature is CCT1, the industrial camera is controlled to shoot the m color blocks respectively to obtain the original colors of the m color blocks. Then, the original colors of the color blocks under the light source condition are corrected according to the original color correction matrix to obtain the corrected colors of the color blocks. If the total difference between the corrected colors of the color blocks and the reference colors of the color blocks is greater than the preset first difference threshold or there is a difference between the corrected colors of at least one color block and the reference color is greater than the preset second difference threshold, then the elements in the preset color correction matrix are adjusted and the above operation is performed again until the total difference between the corrected colors of the color blocks and the reference colors of the color blocks is less than the preset first difference threshold or the difference between the corrected colors of each color block and the reference color is less than the preset second difference threshold. At this time, the color correction matrix obtained is the color correction matrix corresponding to the light source condition with a brightness of Lux1 and a color temperature of CCT1.

[0219] Then by changing the brightness Lux j 、Color Temperature CCT k , solve for different brightness Lux j , different color temperatures CCT k The color correction matrix under different light source conditions is obtained, and then a color correction matrix database is established.

[0220] It can be understood that the original color correction matrix is ​​the color correction matrix originally set in the industrial camera. Taking the color correction matrix including the white balance correction submatrix, the color correction submatrix, the three-dimensional lookup table correction submatrix and the gamma correction submatrix as an example, the original color correction matrix includes the original white balance correction submatrix, the original color correction submatrix, the original three-dimensional lookup table correction submatrix and the original gamma correction submatrix.

[0221] Based on this, in a possible implementation manner, the processor 13 is further configured to:

[0222] Under the light source conditions of multiple preset color temperatures and multiple brightnesses, the industrial camera is controlled to shoot each standard color block in the standard color card to obtain color block images of different brightnesses under each preset color temperature;

[0223] Obtain an original color correction matrix, and use the original color correction matrix to correct the color block images of each brightness at each preset color temperature to obtain each corrected color block image;

[0224] Calculate the color error between each corrected color block image and each standard color block;

[0225] The original color correction matrix is ​​adjusted according to each color error to obtain a color correction matrix corresponding to different brightness under each preset color temperature;

[0226] The color correction matrices corresponding to different brightness under each preset color temperature are stored to obtain a color correction matrix database.

[0227] Taking the color correction matrix including the white balance correction submatrix, the color correction submatrix, the three-dimensional lookup table correction submatrix and the gamma correction submatrix as an example, in a possible embodiment, the color correction matrix database can be represented by the following Table 2:

[0228] Table 2 Color correction matrix database

[0229]

[0230]

[0231] in, WB Indicates white balance. CCM represents the color correction matrix, 3DLUT represents a three-dimensional search, Gamma Indicates gamma correction. In Table 2, the color temperature values ​​and brightness values ​​are arranged from small to large or from large to small.

[0232] 3. Explain the process of the processor determining the target color correction matrix in the preset color correction matrix database according to the actual brightness and actual color temperature of the current light source

[0233] As can be seen from Table 2 above, the color correction matrix database actually includes the color correction matrices corresponding to the light source conditions of each preset brightness and each preset color temperature. Therefore, the color correction matrix corresponding to the actual brightness and actual color temperature of the current light source can be obtained by looking up the table in the color correction database. For example, if the actual brightness of the current light source is Lux1 and the actual color temperature is CCT2, the corresponding color correction matrix obtained after looking up the table is In another embodiment, if the actual brightness of the current light source is Lux2 and the actual color temperature is CCT2, the corresponding color correction matrix is ​​obtained after looking up the table:

[0234] However, in actual applications, the actual color temperature or actual brightness of the current light source may not necessarily be found in the color correction database to be exactly the same value, and thus it is impossible to find the target color correction matrix corresponding to the color temperature and brightness. For example, in the color correction matrix database, the value of Lux1 is 25, the value of Lux2 is 30, the value of CCT1 is 2700, and the value of CCT2 is 2900, while the actual brightness of the current light source is 28 and the actual color temperature is 2800, then the corresponding color correction matrix cannot be found in the color correction matrix database.

[0235] To solve this problem, in a possible implementation manner, the processor 13 is specifically configured to:

[0236] Searching in a color correction matrix database for a brightness whose difference from the actual brightness is less than a preset first threshold, and a color temperature whose difference from the actual color temperature is less than a preset second threshold, to obtain at least one target brightness and at least one target color temperature;

[0237] Then, searching the correction matrix database for the first color correction matrix corresponding to each target brightness and each target color temperature to obtain at least one first color correction matrix;

[0238] Finally, a target color correction matrix is ​​obtained based on the statistics of each first color correction matrix.

[0239] Among them, the target color correction matrix is ​​obtained based on the statistics of each first color correction matrix. The corresponding color correction matrix can be determined by averaging the first color correction matrices, or the corresponding color correction matrix can be determined by interpolation calculation on each first color correction matrix, such as nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, least squares interpolation and other interpolation calculations.

[0240] Based on this, in a possible implementation manner, the processor 13 is specifically configured to: perform interpolation calculation on each first color correction matrix to obtain a target color correction matrix.

[0241] Exemplarily, if a brightness whose difference with the actual brightness is less than a preset first threshold, and a color temperature whose difference with the actual color temperature is less than a preset second threshold are searched in the color correction matrix database, and a target brightness and a target color temperature are obtained, the nearest neighbor difference method can be used to determine the corresponding color correction matrix; in another example, if a brightness whose difference with the actual brightness is less than a preset first threshold, and a color temperature whose difference with the actual color temperature is less than a preset second threshold are searched in the color correction matrix database, and two target brightnesses and two target color temperatures are obtained, the corresponding color correction matrix can be determined by nearest interpolation, or the corresponding color correction matrix can be determined by bilinear interpolation; in another example, if a brightness whose difference with the actual brightness is less than a preset first threshold, and a color temperature whose difference with the actual color temperature is less than a preset second threshold are searched in the color correction matrix database, and four target brightnesses and four target color temperatures are obtained, the corresponding color correction matrix can be determined by nearest interpolation, or the corresponding color correction matrix can be determined by bilinear interpolation, or the corresponding color correction matrix can be obtained by bicubic interpolation.

[0242] Based on this, Figure 3a to Figure 3c In the illustrated embodiment, the process in which the processor 13 determines the target color correction matrix in the color correction matrix database according to the current brightness and the current color temperature is actually a process of searching and interpolating in the color correction database to obtain the target color correction matrix.

[0243] Through the embodiments of the present application, the industrial camera can automatically obtain the current actual brightness and actual color temperature of the ambient light source under different light source conditions, and quickly search and interpolate the color correction matrix corresponding to the current ambient light source in the color correction matrix database, thereby quickly restoring the true color information of the photographed object and improving the speed of color restoration of the industrial camera.

[0244] Based on the above, taking the color correction matrix in the embodiment of the present application including a white balance correction sub-matrix, a color correction sub-matrix, a gamma correction sub-matrix and a three-dimensional lookup table sub-matrix as an example, in a possible implementation, the processor 13 is specifically used to perform color correction on the original image using each sub-matrix.

[0245] In the actual correction process, the original image can be corrected by using each sub-matrix at the same time, or the original image can be corrected by using each sub-matrix in sequence. It can be understood that in the process of correcting the original image with each sub-matrix in sequence, the original image can be corrected with the white balance correction sub-matrix first to obtain a white balance correction image, and then the white balance correction image can be corrected with the color correction matrix to obtain a color correction image, and then the color correction image can be corrected with the gamma correction sub-matrix to obtain a gamma correction image, and finally the gamma correction image can be corrected with the three-dimensional lookup table sub-matrix to obtain the target image; it can also be corrected with the gamma correction sub-matrix first, and then with the three-dimensional lookup table sub-matrix, and then with the color correction sub-matrix, and finally with the white balance correction sub-matrix, which are all possible, and the embodiment of the present application does not limit the order of the sub-matrices.

[0246] Corresponding to the industrial camera of the first aspect, the second aspect of the embodiment of the present application provides an image processing method, which is applied to the industrial camera of the first aspect; Fig.12 As shown, Fig.12 A first flow chart of an image processing method provided in an embodiment of the present application, the method comprising:

[0247] Step S10, obtaining the original image and spectral information;

[0248] Step S20, determining a target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database according to the spectral information;

[0249] Step S30, performing color processing on the original image based on the target color correction matrix to obtain a target image.

[0250] Through the method of the embodiment of the present application, the industrial camera can collect the spectral information of the ambient light source in real time. When the light source changes, the current spectral information of the ambient light source can be automatically acquired, and the original image can be color processed based on the color correction matrix corresponding to the spectral information to obtain a target image with high color restoration. There is no need to perform color correction and white balance adjustment on the industrial camera according to the change of the light source, thereby simplifying the adjustment process of the industrial camera.

[0251] In a possible implementation, the color correction matrix includes a white balance correction submatrix, a color correction submatrix, a gamma correction submatrix and a three-dimensional lookup table submatrix; the above step S30 specifically includes: performing color correction on the original image using each submatrix.

[0252] Based on the above, it can be known that the color correction matrix is ​​pre-calibrated according to the following method:

[0253] Under the preset light source conditions of multiple color temperatures and multiple brightnesses, the industrial camera is controlled to shoot each standard color block in the standard color card to obtain color block pictures of different brightnesses at each preset color temperature; the original color correction matrix is ​​obtained, and the color block pictures of each brightness at each preset color temperature are corrected using the original color correction matrix to obtain each corrected color block picture; the color error between each corrected color block picture and each standard color block is calculated; the original color correction matrix is ​​adjusted according to each color error to obtain the color correction matrix corresponding to different brightnesses at each preset color temperature; the color correction matrix corresponding to different brightnesses at each preset color temperature is stored to obtain a color correction matrix database.

[0254] In a possible implementation, Fig.13 As shown, Fig.13 This is a second flow chart of the image processing method provided in the embodiment of the present application, wherein the above step S20 includes:

[0255] Step S201, determining the actual brightness and actual color temperature of the ambient light source according to the spectrum information;

[0256] Step S202: determining a target color correction matrix corresponding to actual brightness and actual color temperature in a pre-calibrated color correction matrix database.

[0257] Based on the first aspect, the industrial camera 1 can determine the target color correction matrix based on the spectral information received for the first time, and perform color processing on all received original images based on the target correction matrix; or it can acquire spectral information in real time every time an original image is acquired, determine the target color correction matrix based on the spectral information acquired when acquiring the original image, and then process the original image. Based on this, in a possible implementation, the aforementioned step S201 includes:

[0258] Determine the actual brightness and actual color temperature of the ambient light source according to the spectrum information received for the first time;

[0259] The aforementioned step S202 includes:

[0260] Determine a target color correction matrix corresponding to actual brightness and actual color temperature in a pre-calibrated color correction matrix database, and record the target color correction matrix;

[0261] The aforementioned step S30 includes:

[0262] The original image is color processed based on the recorded target color correction matrix to obtain a target image.

[0263] For a more detailed description of the image processing method of the present application, see Fig.14 , Fig.14 This is the first interactive diagram of the image processing method provided in the embodiment of the present application. After the spectral sensor sends spectral information to the processor, the processor determines and records the target color correction matrix 1 according to the spectral information. After receiving the original image 1, original image 2, and original image 3 sent by the image sensor, the processor uses the recorded target color correction matrix 1 to perform color processing on the original image 1, original image 2, and original image 3, respectively, to obtain the target image 1, target image 2, and target image 3.

[0264] Corresponding to the above-mentioned situation where the light source changes, in a possible implementation manner, as Fig.15 As shown, Fig.15 A third flow chart of the image processing method provided in the embodiment of the present application, the method comprising:

[0265] Step S01, controlling the spectrum sensor to determine the current spectrum information of the ambient light source, and simultaneously controlling the image sensor to generate an original image of the photographed object;

[0266] Step S02, storing current spectrum information;

[0267] Step S03, determining whether the difference between the current spectrum information and the spectrum information received last time is greater than a preset threshold;

[0268] If the difference between the current spectrum information and the spectrum information received last time is less than or equal to the preset threshold, step S04 is executed; if the difference between the current spectrum information and the spectrum information received last time is greater than the preset threshold, steps S05 to S07 are executed;

[0269] Step S04, using the spectrum information received last time as the target spectrum information, and obtaining the target color correction matrix recorded last time;

[0270] Step S05, taking the current spectrum information as the target spectrum information;

[0271] Step S06, determining the actual brightness and actual color temperature of the ambient light source according to the target spectrum information;

[0272] Step S07 , determining a target color correction matrix corresponding to the first actual brightness and the actual color temperature in a pre-calibrated color correction matrix database, and recording the target color correction matrix.

[0273] Combination Fig.16 As can be seen from the second interaction diagram of the image processing method shown, after the industrial camera of the embodiment of the present application starts working, the spectral sensor and the image sensor first send spectral information 1 and original image 1 to the processor at the same time, the processor saves the spectral information 1 and determines the target color correction matrix 1 according to the spectral information 1, and performs color processing on the original image 1 according to the target color correction matrix 1 to obtain the target image 1; after the spectral sensor sends spectral information 2 to the processor and the image sensor sends the original image 2 to the processor at the same time, the processor first determines whether the difference between the spectral information 2 and the spectral information 1 received last time is greater than a preset threshold value, if so, the target color correction matrix 2 is re-determined according to the spectral information 2, and the original image 2 is color processed according to the target color correction matrix 2 to obtain the target image 2; after the spectral sensor sends spectral information 3 to the processor and the image sensor sends the original image 3 to the processor at the same time, the processor first determines whether the difference between the spectral information 3 and the spectral information 2 received last time is greater than the preset threshold value, if not, the latest recorded target color correction matrix 2 is directly obtained, and the original image 3 is color processed according to the target color correction matrix 2 to obtain the target image 3.

[0274] In a possible implementation, the camera further includes a light splitting device, such as Fig.17 As shown, Fig.17 This is a fourth flow chart of the image processing method provided in the embodiment of the present application. The above step S30 includes:

[0275] Step S31, performing a first color interpolation on the original image to obtain a first color image;

[0276] Step S32, performing a first color correction on the first color image according to a target color correction matrix to obtain a target image.

[0277] In another possible implementation, Fig.18 As shown, Fig.18 The fifth flow chart of the image processing method provided in the embodiment of the present application, the above step S30 includes:

[0278] Step S33, performing a second color correction on the original image according to the target color correction matrix to obtain an original corrected image;

[0279] Step S34, performing a second color interpolation on the original corrected image to obtain a target image.

[0280] In a possible implementation, Fig.19 As shown, Fig.19 The sixth flow chart of the image processing method provided in the embodiment of the present application, the above step S34 includes:

[0281] Step S341, performing a second color interpolation on the original corrected image to obtain a second color image;

[0282] Step S342: Perform a third color correction on the second color image according to the target color correction matrix to obtain a target image.

[0283] In a possible implementation, the industrial camera further includes an output interface, and the industrial camera is connected to the host computer via the output interface, such as Fig. 20 As shown, Fig. 20 This is a seventh flow chart of the image processing method provided in the embodiment of the present application, wherein the method further includes:

[0284] Step S40: output at least one of the target image, the spectrum information, the target color correction matrix, the color image and the original corrected image to a host computer through an output interface.

[0285] In a possible implementation manner, determining a target color correction matrix corresponding to actual brightness and actual color temperature in a pre-calibrated color correction matrix database includes:

[0286] Searching for a brightness whose difference from the actual brightness is less than a preset first threshold value, and a color temperature whose difference from the actual color temperature is less than a preset second threshold value in a pre-calibrated color correction matrix database, to obtain at least one target brightness and at least one target color temperature;

[0287] Searching a color correction matrix database for first color correction matrices corresponding to each target brightness and each target color temperature to obtain at least one first color correction matrix;

[0288] A target color correction matrix is ​​obtained based on statistics of each first color correction matrix.

[0289] In a possible implementation manner, obtaining a target color correction matrix based on statistics of each first color correction matrix includes:

[0290] An interpolation calculation is performed on each first color correction matrix to obtain a target color correction matrix.

[0291] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned image processing methods are implemented.

[0292] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any image processing method in the above embodiments.

[0293] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0294] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0295] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the method embodiment, since it is basically similar to the industrial camera embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0296] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. An industrial camera, characterized in that: The industrial camera comprises: a spectral sensor, a processor and an image sensor; The spectral sensor is used to determine spectral information of the ambient light source based on the received light, and send the spectral information to the processor; The image sensor is used to generate an original image of the photographed object according to the received light, and send the original image to the processor; The processor is used to control the spectral sensor and the image sensor, and determine the target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database according to the spectral information; and perform color processing on the original image based on the target color correction matrix to obtain a target image.

2. The industrial camera according to claim 1, characterized in that: The processor determines, according to the spectral information, a target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database, including: Determine the actual brightness and actual color temperature of the ambient light source according to the spectral information; A target color correction matrix corresponding to the actual brightness and the actual color temperature is determined in a pre-calibrated color correction matrix database.

3. The industrial camera according to claim 1, characterized in that: The industrial camera further comprises a spectroscopic device and a lens mounting seat, wherein the lens mounting seat is used to mount a lens assembly; the lens assembly and the spectroscopic device are sequentially arranged along the incident direction of the light; The lens assembly is used to transmit the light reflected by the photographed object to the light splitting device; The spectroscopic device is used to split the light into a first light and a second light and emit the light to the image sensor and the spectral sensor, wherein the image sensor is arranged in the emitting direction of the first light, and the spectral sensor is arranged in the emitting direction of the second light.

4. The industrial camera according to claim 3, characterized in that: The light splitting device is a light splitting prism or a light splitting plate; The image sensor is arranged on the optical axis of the lens assembly; and the spectrum sensor is arranged on a plane perpendicular to the plane where the image sensor is located.

5. The industrial camera according to claim 1, characterized in that: The industrial camera also includes a lens mounting seat, on which a lens mounting hole for mounting a lens and a light-transmitting sheet mounting hole for mounting a light-transmitting sheet are provided; The lens is used to transmit the light reflected by the photographed object to the image sensor; The light-transmitting sheet is used to transmit the light reflected by the photographed object to the spectrum sensor.

6. The industrial camera according to any one of claims 3 to 5, characterized in that: The industrial camera also includes one or more printed circuit boards; the processor, the image sensor and the spectral sensor are fixed on one printed circuit board; or, the image sensor and the spectral sensor are fixed on one printed circuit board, and the processor is fixed on another printed circuit board; or, the processor, the image sensor and the spectral sensor are respectively fixed on three printed circuit boards.

7. The industrial camera according to claim 6, characterized in that: The industrial camera further comprises a housing and a plurality of first connecting members; the housing cooperates with the lens mounting seat to form a receiving cavity; One or more of the printed circuit boards are fixed in the accommodating cavity through the plurality of first connectors; The photosensitive surface of the spectrum sensor and the photosensitive surface of the image sensor can receive light reflected by the photographed object through the lens assembly installed on the lens mounting seat.

8. The industrial camera according to claim 7, characterized in that: The industrial camera also includes a light source assembly and a second connecting member. The light source assembly is fixed to the outside of the housing via the second connecting member. The illumination area of ​​the light source assembly covers the shooting field of view of the installed lens assembly.

9. The industrial camera according to claim 1, characterized in that: The industrial camera also includes: an output interface; the industrial camera is connected to the host computer through the output interface; The output interface is used to output the target image to the host computer.

10. An image processing method, characterized in that: Applied to the industrial camera according to any one of claims 1 to 9, the method comprises: Get original image and spectral information; Determine a target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database according to the spectral information; The original image is subjected to color processing based on the target color correction matrix to obtain a target image.

11. The method according to claim 10, characterized in that The step of determining a target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database according to the spectral information includes: Determine the actual brightness and actual color temperature of the ambient light source according to the spectral information; A target color correction matrix corresponding to the actual brightness and the actual color temperature is determined in a pre-calibrated color correction matrix database.

12. The method according to claim 10, characterized in that The color correction matrix includes: a white balance correction sub-matrix, a color correction sub-matrix, a gamma correction sub-matrix and a three-dimensional lookup table sub-matrix; The color processing of the original image based on the target color correction matrix includes: performing color correction on the original image using each sub-matrix.

13. The method according to claim 12, characterized in that The color correction matrix database is pre-calibrated according to the following method: Under the light source conditions of preset multiple color temperatures and multiple brightnesses, the industrial camera is controlled to shoot each standard color block in the standard color card to obtain color block pictures of different brightnesses under each preset color temperature; Obtaining an original color correction matrix, and using the original color correction matrix to correct the color block images of each brightness at each preset color temperature to obtain each corrected color block image; Calculating the color errors between the corrected color block images and the standard color blocks; The original color correction matrix is ​​adjusted according to each color error to obtain a color correction matrix corresponding to different brightness under each preset color temperature; The color correction matrices corresponding to different brightness under each preset color temperature are stored to obtain the color correction matrix database; wherein the original color correction matrix includes: an original white balance correction sub-matrix, an original color correction sub-matrix, an original gamma correction sub-matrix and an original three-dimensional lookup table sub-matrix.

14. The method according to claim 10, characterized in that The color processing of the original image based on the target color correction matrix to obtain the target image includes: Performing a first color interpolation on the original image to obtain a first color image; Performing a first color correction on the first color image according to the target color correction matrix to obtain a target image.

15. The method according to claim 10, characterized in that The color processing of the original image based on the target color correction matrix to obtain the target image includes: Performing a second color correction on the original image according to the target color correction matrix to obtain an original corrected image; Perform a second color interpolation on the original corrected image to obtain a target image.

16. The method according to claim 15, characterized in that The performing a second color interpolation on the original corrected image to obtain a target image comprises: Performing a second color interpolation on the original corrected image to obtain a second color image; Perform a third color correction on the second color image according to the target color correction matrix to obtain a target image.

17. The method according to claim 16, characterized in that The industrial camera also includes an output interface, and the industrial camera is connected to a host computer via the output interface. The method also includes: At least one of the target image, the spectral information, the target color correction matrix, the color image and the original corrected image is output to the host computer through the output interface.

18. The method according to claim 11, characterized in that The step of determining a target color correction matrix corresponding to the actual brightness and the actual color temperature in a pre-calibrated color correction matrix database includes: Searching for a brightness whose difference from the actual brightness is less than a preset first threshold value, and a color temperature whose difference from the actual color temperature is less than a preset second threshold value in a pre-calibrated color correction matrix database, to obtain at least one target brightness and at least one target color temperature; Searching the color correction matrix database for the first color correction matrix corresponding to each of the target brightnesses and each of the target color temperatures to obtain at least one first color correction matrix; A target color correction matrix is ​​obtained based on the statistics of each of the first color correction matrices.

19. The method according to claim 18, characterized in that The step of obtaining a target color correction matrix based on the statistics of each of the first color correction matrices includes: Interpolation calculation is performed on each of the first color correction matrices to obtain a target color correction matrix.

20. The method according to claim 11, characterized in that The determining the actual brightness and the actual color temperature of the ambient light source according to the spectral information includes: Determining the actual brightness and actual color temperature of the ambient light source according to the spectral information received for the first time; The step of determining a target color correction matrix corresponding to the actual brightness and the actual color temperature in a pre-calibrated color correction matrix database includes: Determine a target color correction matrix corresponding to the actual brightness and the actual color temperature in a pre-calibrated color correction matrix database, and record the target color correction matrix; The color processing of the original image based on the target color correction matrix to obtain the target image includes: The original image is subjected to color processing based on the recorded target color correction matrix to obtain a target image.

21. The method according to claim 10, characterized in that The obtaining of the original image and spectral information includes: Controlling the spectral sensor to determine the current spectral information of the ambient light source, and simultaneously controlling the image sensor to generate an original image of the photographed object; the method further comprises: storing the current spectral information; The step of determining a target color correction matrix corresponding to the spectral information in a pre-calibrated color correction matrix database according to the spectral information includes: Determine whether the difference between the current spectrum information and the spectrum information received last time is greater than a preset threshold; When the difference between the current spectral information and the spectral information received last time is less than or equal to a preset threshold, the spectral information received last time is used as the target spectral information, and the target color correction matrix recorded last time is obtained; When the difference between the current spectral information and the spectral information received last time is greater than a preset threshold, the current spectral information is used as the target spectral information, and the actual brightness and the actual color temperature of the ambient light source are determined according to the target spectral information; the target color correction matrix corresponding to the actual brightness and the actual color temperature is determined in a pre-calibrated color correction matrix database, and the target color correction matrix is ​​recorded.

Images