METHOD AND DEVICE FOR PROCESSING IMAGES
Patent Information
- Application Number
- DE602020053421
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-11-29
- Filing Date
- 2020-11-24
- Publication Date
- 2025-06-25
- Estimated Expiration
- 2040-11-24
Description
Technical Field
[0001] The invention relates to the field of image processing and more specifically, the processing of images obtained by infrared camera in order to improve the visibility of identifiable objects in the images. Prior art
[0002] Many applications require determining which objects are present in a scene, or at a specific location. For example, driving an autonomous vehicle requires reliably identifying, in real time, which objects are around the vehicle, including those that may be on the roadway in front of the vehicle.
[0003] In a well-known manner, cameras are used, coupled with object identification methods, to identify identifiable objects in the images acquired by the cameras. These cameras operate in different spectra (wavelength ranges), such as the visible spectrum or the infrared spectrum.
[0004] The difficulty of this operation is increased in the presence of fog or precipitation. Indeed, in the presence of fog and more generally poor weather conditions, the visibility of objects observed with a thermal infrared camera (LWIR) is degraded.
[0005] Document CN1946143 discloses a method for removing band noise (dark and light lines).
[0006] Documents US2015371373 and CN105512623 disclose methods for removing fog from images acquired by a vehicle-mounted camera. Statement of the invention
[0007] A first objective of the invention is to propose a method making it possible, from images acquired in foggy weather or during precipitation, to obtain improved images in which the visibility of the objects present in the image is increased, and thus the detection of objects in these images is facilitated.
[0008] This objective is achieved through an image processing method comprising the following steps: a) an initial image A consisting of pixels is acquired; c1) using at least one computer, for each value of an index j in a range of lines from N1 to N2 of the initial image A, a factor F1(j) is determined such that: E moyen G 1 j ; I A , i , j * F 1 j ; I A , i , j + s ≤ E moyen G 1 j ; I A , i , j ; I A , i , j + s where G1(j) is a first group of columns of the row j considered; IA,i,j is an intensity of a pixel of the initial image A located on column i and row j; E mean (G1(j); x(i,j); y(i,j)) is the average deviation in absolute value between values x(i,j) and y(i,j), for the values of i of the group of columns G1(j); s=+1 or -1; and d1) a first improved image (B1) is calculated in which each row Lj of said range of rows going from N1 to N2, is replaced by a row L' j obtained by multiplying each pixel of the row Lj by a multiplying coefficient (M1(j)) equal to the product of the factors F1(k), for k going from N1 to j.
[0009] By this operation, for each line Lj of the range of lines P, a factor F1(j) is determined such that, if each pixel of the line Lj were multiplied by this factor F1(j), a line L'j would be obtained such that the average difference between the intensities of the pixels of the line L'j located in a first group of columns and the intensities of corresponding pixels of the line Lj+s is less (in absolute value) than the average difference between the intensities of the pixels of the line Lj located in the first group of columns and the intensities of the corresponding pixels of the line Lj+s (i.e. the line immediately above or below line j).
[0010] In the above definition of the method, a pixel corresponding to a previously cited pixel is a pixel located in the same column as the previously cited pixel.
[0011] In this method, preferably the initial image is an infrared image, in particular an image in the long-wave infrared (LWIR) region. The initial image is in this case an image acquired using a long-wave infrared camera.
[0012] Indeed, it has appeared that by taking into account a physical model of the degradation of images, in particular of images acquired in the LWIR band, it is possible in a simple manner, in accordance with the method defined above, to improve the quality of the images by simple computer processing and thus to increase the visibility of the objects present in the images.
[0013] The process defined above is based on the following assumptions.
[0014] It is assumed that the average intensity of the radiation emitted by a point in the scene is substantially constant regardless of the position of this point in the scene, whether it is close or far from the camera.
[0015] Therefore, it is assumed that the presence of fog causes a degradation of the images which is proportional to the distance between the camera and the observed point.
[0016] (In the following text, for simplicity, only "fog" will be referred to, but this term should be interpreted broadly to cover any atmospheric phenomenon likely to disrupt the acquisition of images of a scene by disturbing the atmosphere between the camera and the scene, such as fog, rain, drizzle, snow, hail, dust, etc.).
[0017] The principle of the invention therefore consists of detecting a loss of intensity of the pixels of the image attributable to fog, and compensating for this loss of intensity in such a way as to compensate for the degradation caused by the fog.
[0018] For this, the invention is also based on the assumption that at the time of image acquisition, the camera is oriented in such a way that the transverse axis of the camera (parallel to the lines of the acquired image) is oriented in a direction substantially parallel to the ground, - generally horizontal -.
[0019] Under these conditions, on the same line of the image, the points of the scene represented by the different pixels are substantially at the same distance from the camera.
[0020] The geometry of the scene therefore generally leads points located at the same distance from the camera to be found substantially in the same line of the image. It follows that a loss of light intensity between two consecutive lines of the image can be identified as a loss of intensity of the pixels of the image attributable to fog.
[0021] Advantageously, the method proposed above makes it possible to simply quantify this loss of intensity and to compensate for it, thus improving the quality of the image and the visibility of the objects present in the image.
[0022] This process can be integrated into the camera or applied in post-processing to the images captured by the camera. The proposed process works day and night. It can be used to improve the visualization of any scene captured in infrared, for example the scene in front of a vehicle, or a scene filmed by a remote surveillance system.
[0023] In the method defined above, the calculation of the coefficient F1 is generally done by taking into account, for a given line Lj, all the pixels of the line. The 'first group of columns' (denoted according to the context G1 or G1(j)) then contains for the line Lj considered all the columns of the image (the first group of columns G1(j) is determined, or at least can be determined separately, for each line Lj of the image).
[0024] In the above definition, deviations are calculated in absolute value.
[0025] The 'Average_deviation' function is a function which, when applied to two quantities x(i,j) and y(i,j) defined for each value (i,j), i successively taking all the values of the columns of the column group G1(j), provides an average of the deviation in absolute value between the quantities x(i,j) and y(i,j). Any average function can be used to calculate the average deviation, for example an arithmetic, quadratic, geometric, etc.
[0026] For example, the average difference between the pixel intensities of two successive rows in a group of columns can be calculated as an average of the differences (in absolute value) of the intensity values of the two pixels in this column belonging respectively to one row and the following row, using the following formula: E moyen G 1 j ; I A , i , j ; I A , i , j + 1 = Moyenne G 1 j I A , i , j − I A , i , j + 1
[0027] where Mean G1(j) (x(i,j)) denotes a mean function of the function x(i,j), for the set of values of column i of the group G1(j).
[0028] Therefore, the processing carried out by multiplying each of the pixels of the line Lj by the factor F makes it possible to reduce the intensity difference between the lines Lj and Lj+s. Thanks to this, the effect of fog at the level of the treated pixels is reduced, and the visibility of the objects represented by these pixels is increased.
[0029] The F1 factor can be determined in different ways.
[0030] The process is preferably applied either to an entire lower part of the image (in particular a part located below the horizon), or to all the lines of the image.
[0031] In a preferred embodiment, in step c1), for a line j, the factor F1(j) is determined as being the value of the parameter F for which the value of the following functional K is minimal: K = ∑ G 1 j g I A , i , j + s − F ∗ I A , i , j
[0032] where g is a strictly increasing function, and the sum is calculated for all columns i of the group G1(j). The function g can in particular be a norm. The value F1 thus obtained minimizes the overall difference (it depends on the function g chosen) between the pixels of the line L j+s and those of the line L'j. The sum over i in the expression above applies to all columns i of the image.
[0033] For example, in step c1), for a line j, the factor F1(j) can be determined as the value of the parameter F for which the value of the following functional K is minimal: K = ∑ G 1 j I A , i , j + s − F ∗ I A , i , j p where p is a positive number.
[0034] In one embodiment, in step c1), for a line j, the factor F1(j) is determined as being equal to the ratio between the average intensity of the pixels of the line Lj and the average intensity of the pixels of the line Lj+s.
[0035] In determining this ratio, the average intensity is a function of average; the average intensity of the pixels in a line is therefore an average of the intensity of the pixels in the line. Any average function can be used to perform this calculation: an arithmetic mean, a quadratic mean, etc.; a median function, etc.
[0036] In this embodiment, the average of the intensities of the line L'j obtained by multiplying each pixel of the line j by the factor F1(j) is therefore substantially equal to that of the line L j+s: Moyenne G 1 j I A , i , j * F 1 j = Moyenne I A , i , j + s
[0037] The process thus makes it possible to homogenize the average intensities between different lines of the image, and possibly across all the lines of the image.
[0038] The accuracy of the process can be increased by additionally incorporating the following steps: c2) for each e value of the index j in the range of lines from N1 to N2 of the initial image A, using at least one computer: we determine a factor F2(j) such that: E moyen G 2 j ; I A , i , j * F 2 j ; I A , i , j − s ≤ E moyen G 2 j ; I A , i , j ; I A , i , j − s where G2(j) is a second group of columns of the row j considered; and d2) a second enhanced image is calculated in which each row Lj of said range of rows going from N1 to N2 is replaced by a row L"j obtained by multiplying each pixel of the row Lj by a multiplier coefficient (M2(j)) equal to the product of the factors F2(k), for k going from j to N2; and e) an average enhanced image is calculated equal to an average of the first and second enhanced images.
[0039] In practice, the calculation of the coefficients F1 and / or F2 to be applied to the pixels of a line is generally carried out, for each of the lines Lj, by taking into account all the columns of the image. In this case, the first group of columns contains the indices of all the columns of the initial image A.
[0040] However, it may be preferable to calculate the coefficients F1 and / or F2 by taking into account not all the pixels of a given line Lj, but by taking into account only a part of them. The first or second group of columns then defines a restricted number of pixels which must be taken into account for the calculation of the coefficients F1 and / or F2.
[0041] It is appropriate to select the pixels to be taken into account for the calculation of the coefficient F1 in this way, particularly when, for certain pixels considered in a line Lj, the corresponding pixels in the reference line Lj+s or Lj-s from which the factor F1 or F2 is calculated do not represent at all the same object or the same zone as the pixels considered in the line Lj. These pixels can be called 'edge pixels', because in this case they represent the edge or the limit of an object or a zone. In this case, it is preferable not to take the edge pixels into account for the calculation of the coefficients F1 and / or F2. The first group of columns is then determined in such a way that it does not contain the columns in which the edge pixels are located.
[0042] Edge pixels can be detected by noting that the intensity difference between a pixel on line Lj and the corresponding pixel on line Lj+s or Lj-s, as the case may be, is relatively large, and in any case greater than the difference it would have if this difference were only due to the presence of fog. When this situation is detected, the pixel in question can be identified as an edge pixel, which should not be taken into account when calculating the factor F1 or F2.
[0043] To implement this principle, in one embodiment, the method further comprises the following step, for at least one line Lj: b1) for each of the pixels of said at least one line L j , a difference (EA,h,j ) is evaluated between an intensity IA,h,j of a pixel of a column h considered on the line L j and an intensity IA,h,j+s of a corresponding pixel in the column h on the line L j+s: E A , h , j = I A , h , j − I A , h , j + s ; and in step d1, the first group of columns to be taken into account for the row Lj is determined as being the set of column indices h for which the deviation EA,h,j is less than a predetermined value.
[0044] Finally, at the end of the various steps presented above, in one embodiment the method further comprises the following step: f) a final enhanced image is calculated by applying dynamic compression processing to one of the first or second enhanced images, or to the average enhanced image. This operation is called 'tone-mapping' in English.
[0045] Furthermore, after the acquisition of the initial image in step a) and before steps b1 and / or b2, a denoising process of the initial image can be provided.
[0046] In a particular embodiment, the different steps of the image processing method are determined by computer program instructions.
[0047] Consequently, the invention also relates to a computer program on an information medium, this program being capable of being implemented in a computer, this program comprising portions / means / instructions of program code for the execution of the steps of the image processing method defined previously when said program is executed on a computer.
[0048] This program may use any programming language, and be in the form of source code, object code, or intermediate code between source code and object code, such as in a partially compiled form, or in any other desirable form. The program according to the invention may in particular be downloaded from a network such as the Internet.
[0049] The invention also relates to an information medium readable by a computer on which a computer program as defined above is recorded.
[0050] The information carrier may be any entity or device capable of storing the program. For example, the carrier may include a storage medium, such as a ROM, for example a CD ROM or a microelectronic circuit ROM, or a magnetic recording medium, for example a floppy disc or a hard disk. Alternatively, the information carrier may be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of the image processing method.
[0051] A second objective of the invention is to propose a device making it possible, from images acquired in foggy weather or during precipitation, to obtain improved images in which the visibility of the objects present in the image is increased, and thus the detection of objects in these images is facilitated.
[0052] This objective is achieved by means of an image processing device, the device comprising a) an initial image acquisition module, configured to acquire an initial image A consisting of pixels; c1) a corrective factor determination module F1, configured, for each value of an index j in a range of lines from N1 to N2 of the initial image A, to determine a factor F1(j) such that: E moyen G 1 j ; I A , i , j * F 1 j ; I A , i , j + s ≤ E moyen G 1 j ; I A , i , j ; I A , i , j + s where G1(j) is a first group of columns of the row j considered; IA,i,j is an intensity of a pixel of the initial image A located on column i and row j; E mean (G1(j); x(i,j); y(i,j)) is the average deviation in absolute value between the values x(i,j) and y(i,j), for the values of i of the group of columns G1(j); s=+1 or -1; and d1) a first enhanced image calculation module (B1) configured to calculate a first enhanced image, in which each row Lj of said index range j varying from N1 to N2 is replaced by a row L'j obtained by multiplying each pixel of the row Lj by a multiplier coefficient equal to the product of the factors F1(k), for k ranging from N1 to j.
[0053] This device is thus configured for the implementation of the process defined previously, as well as its various improvements.
[0054] Thus in one embodiment, the corrective factor determination module F1 is configured to obtain the factor F1 as being the value of the parameter F for which the value of the following functional K is minimal: K = ∑ G 1 j g I A , i , j + s − F ∗ I A , i , j where g is a strictly increasing function, notably a norm, and the sum is calculated for all columns i of the group G1(j).
[0055] In particular, in one embodiment, the corrective factor determination module F1 is configured to obtain the factor F1 as being the value of the parameter F for which the value of the following functional K is minimal: K = ∑ G 1 j I A , i , j + s − F ∗ I A , i , j p where p is a positive number.
[0056] In one embodiment, the correction factor determination module F1 is configured to determine the factor F1 as being equal to the ratio between the average intensity of the pixels of the line Lj and the average intensity of the pixels of the line Lj+s (the average intensity of the pixels of a line being an average of the intensity of the pixels of the line).
[0057] The average of the intensities of the line L'j obtained by multiplying each pixel of the line j by the factor F1(j) is equal to that of the line L j+s: Moyenne G 1 j I A , i , j ∗ F 1 j = Moyenne I A , i , j + s
[0058] By extension, the present disclosure also relates to an image acquisition system comprising at least one infrared camera, in particular an infrared camera configured to acquire images in the LWIR frequency band, and an image processing device as defined previously. The present disclosure finally relates to a robot or vehicle comprising an image acquisition system as defined previously. Brief description of the drawings
[0059] The invention will be better understood and its advantages will appear better on reading the detailed description which follows, of embodiments shown as non-limiting examples. The description refers to the appended drawings, in which: [ Fig. 1 ] There Figure 1 is a schematic view of an image acquisition device according to the present disclosure; [ Fig. 2 ] There Figure 2is a schematic representation of an image processed by an image processing method according to the present disclosure; and [ Fig. 3 ] There Figure 3 is a block diagram representing an image processing method according to the present disclosure. Description of the embodiments
[0060] Referring to the Figure 1 , a vehicle 100 will now be described. The vehicle 100 (in this case, a private car) is equipped with an image acquisition system 10 in accordance with the present disclosure.
[0061] The image acquisition system 10 comprises an infrared camera 20, mounted at the front of the vehicle 100, and a computer 30. The computer 30 constitutes an image processing device within the meaning of the present disclosure.
[0062] The calculator 30 has the hardware architecture of a computer, as schematically illustrated in the Figure 1 .
[0063] It includes in particular a processor 31, a read-only memory 32, a random access memory 33 and communication means 34 for communicating with the rest of the vehicle 100 and / or with communication networks external to the vehicle.
[0064] The read-only memory 32 of the computer 30 constitutes a recording medium in accordance with the disclosure, readable by the processor 31 and on which is recorded a computer program in accordance with the invention, comprising instructions for executing the steps of the image processing method according to the present disclosure which will be described later in relation to the figures 2 And 3 This computer program defines, in an equivalent manner, functional modules of the image processing device 10, namely an initial image acquisition module, a corrective factor determination module F1, and an improved first image calculation module.
[0065] The camera 20 is directed horizontally along an X axis located in the median plane of the vehicle and makes it possible to acquire images representing the part of the roadway C located in front of the vehicle 100.
[0066] In each acquired image, the lower lines of the image represent the parts of the scene that are closest to the vehicle 100. In fact, the closer a line is to the lower line of the image, the closer it represents points in the scene to the vehicle. Thus, the line Lj+1 corresponds to points closer to the camera 10 than the points Lj.
[0067] It follows that in foggy weather, since the loss of visibility caused by fog is generally proportional to an increasing function of the distance between the camera and the observed point, objects on the image lines are therefore all the more visible the lower they are in the image.
[0068] The implementation of the method according to the present disclosure advantageously makes it possible to take this property into account and to compensate for the loss of visibility in the image which results from the presence of fog.
[0069] An example of implementation of an image processing method in accordance with the present disclosure and implemented by the computer 30 of the image acquisition system 10 will now be described with reference to figures 2 And 3 . a1) Acquisition of the initial image
[0070] We begin by acquiring an initial image A. In the example presented, this acquisition is usually done by acquiring the image provided as output by the camera 20. However, more generally, within the meaning of the present disclosure, the acquisition can be done by reading the image in a computer file, by receiving the image via a remote connection, by calculating the image from other data, or any other means.
[0071] In this case, image A is acquired using an infrared camera operating in the 'long wave' frequency range of the infrared band (LWIR). This image has n rows j and m columns i.
[0072] An image denoising process is applied to the initial image to reduce noise in the image.
[0073] A series of operations are then carried out in parallel.
[0074] In this implementation example, we consider the case where s=+1, N1=1 and N2=n. This disclosure also naturally includes the cases where s=-1, N1>1 and / or N2 <n. b1) Identification of the first groups G1(j) of columns to be taken into account for the calculation of the factors F1
[0075] For image A, for each of the lines Lj, and for each column i, we carry out the following operations (unless these are not defined) to determine the first group of columns G1(j): a) we evaluate the difference (EA,i,j) between the intensity IA,i,j-1 of the pixel (i,j+s) (pixel of column i and row j+s) and the intensity IA,i,j of the corresponding pixel (i,j) on row Lj: E A , i , j = I A , i , j + s − I A , i , j . b) we integrate the column i considered into the first group of columns G1(j) only if the difference EA,i,j is less than a threshold value Emax. b2) Identification of the second groups G2(j) of columns to be taken into account for the calculation of the factors F2
[0076] For image A, for each of the lines Lj, and for each column i, we carry out the following operations (unless these are not defined) to determine the second group of columns G2(j): a) we evaluate the difference (EA,i,j) between the intensity IA,i,j-1 of the pixel (i,js) (pixel of column i and row js) and the intensity IA,i,j of the corresponding pixel (i,j) on row Lj: E A , i , j = I A , i , j − s − I A , i , j . b) we integrate the column i considered into the second group of columns G2(j) only if the difference EA,i,j is less than the threshold value Emax.
[0077] Depending on the value of s, one of the groups G1(1) or G2(1), and one of the groups G1(n) or G2(n), are not defined by the previous calculations.
[0078] We therefore further fix that these groups not defined by the previous calculations are equal respectively to the same group of columns, but respectively of the second line (G1(2) or G2(2) depending on the case) and of the penultimate line (G1(n-1) or G2(n-1) depending on the case). c1) and c2) Determination of the corrective factors F1 and F2
[0079] We then calculate the correction factors F1(j) and F2(j), j=1...n, for each of the lines of the initial image A, in the following manner: The correction factors F1(n) and F2(1) respectively for the last line and for the first line are each equal to 1.
[0080] For each of the lines Lj for which F1 or F2 has not yet been determined, the corrective factors F1(j) and F2(j) are calculated with the formulas: F 1 j = ∑ i I A , i , j ∗ I A , i , j + s ∑ i I A , i , j ∗ I A , i , j et F 2 j = ∑ i I A , i , j ∗ I A , i , j − s ∑ i I A , i , j ∗ I A , i , j
[0081] For each line Lj, now consider the line L'j obtained by multiplying each pixel of the line Lj by the factor F1(j).
[0082] The F1 coefficients are calculated so that for each line Lj, the average deviation between the intensities of the pixels of line L'j located in the first group of columns and the intensities of the corresponding pixels of line Lj+s is less than the average deviation between the intensities of the pixels of line Lj located in the first group of columns and the intensities of the corresponding pixels of line Lj+s.
[0083] The same property is obtained for the F2 coefficients (by replacing +s with -s).
[0084] In the expressions of F1(j) and F2(j), the sums ('Sigma' on i) are calculated only for the indices i of the pixels of the first or respectively of the second group of columns (G1(j),G2(j)), that is to say only for the pixels for which the difference EA,i,j is less than the threshold value Emax. This method therefore leads to not taking into consideration in the calculation of the correction factors F1 and F2 the columns in which there is a significant discontinuity of value between line j and line j+s (or respectively between lines j and js). This method thus advantageously makes it possible not to take into consideration certain irrelevant information, such as the differences in intensity between an object and another object distinct from the observed scene, which have nothing to do with a loss of image quality due to fog.
[0085] Remarks : steps b1) and b2) are optional. If they are not carried out, groups G1 and G2 contain by default all the columns of image A. The calculation of the correction factors F1 and F2 is then carried out by calculating the sums ('Sigma' over i) in the numerator and denominator of the equation above for all the values of i, from the first to the last column. In this example of implementation, the factors F1 and F2 are calculated by the formula above. Naturally, any other of the functions previously proposed to implement the method according to the present disclosure can also be used. d1) and d2) Calculation of the improved images B1 and B2
[0086] Then, during a step d1) for the initial image A, the following operations are carried out to calculate the improved image B1: Calculation of the coefficients M1 and M2 and the pixel values of the improved images, for a first line:
[0087] We initialize the multiplier coefficients M1 and M2 for a first line, respectively: M1, for line 1: M1(1) = F1(1); and M2, for line n: M2(n) = F2(n).
[0088] We then calculate the value of each of the pixels in line 1 (j=1) of the first enhanced image, and of each of the pixels in the last line (j=n) of the second enhanced image, in the following manner: For each value of i ranging from 1 to m: I B 1 , i , 1 = M 1 1 ∗ I A , i , 1 et I B 2 , i , n = M 2 n ∗ I A , i , n .
[0089] Calculation of the coefficients M1,M2 and the pixel values of the enhanced images, for the other lines: First enhanced image: for each value of j ranging from 2 to n: We calculate the multiplier coefficients M1 for line j: M 1 j = M 1 j − 1 ∗ F 1 j We then calculate the value of each of the pixels in line j of the first enhanced image. For each value of i ranging from 1 to m: I B 1 , i , j = M 1 j ∗ I A , i , j
[0090] Second improved image: For each value of j ranging from n-1 to 1: We calculate the multiplier coefficients M2 for line j: M 2 j = M 2 j + 1 ∗ F 2 j We then calculate the value of each of the pixels in line j of the second enhanced image. For each value of i ranging from 1 to m: I B 2 , i , j = M 2 j ∗ I A , i , j
[0091] Thanks to these operations, improved images B1 and B2 are obtained, the lines L"j of which have an average difference between the intensities of the pixels of the line L"j and the intensities of the pixels of the line Lj+s and respectively Lj-s which is less than the average difference between the intensities of the pixels of the line Lj and the intensities of the pixels of the line Lj+s and respectively Lj-s. Thanks to this, the visibility of the objects in the images B1 and B2 is improved compared to their visibility in the images A and Ainv.
[0092] In the above implementation, the entire set of lines in the original image is modified by the specified transformation. However, in other implementations only a portion of the image lines, for example a range of lines in a given interval, may be modified. e) Calculation of the average improved image C
[0093] We then calculate an average improved image C by taking the average of the first and second improved images B1 and B2: each pixel (i,j) of the average improved image takes the value: I C i , j = 1 2 I B 1 , i , j + I B 2 , i , j f) Calculation of the final improved image D
[0094] The final enhanced image D is then calculated by applying a dynamic image compression process to the average enhanced image C.
[0095] Although the present invention has been described with reference to specific exemplary embodiments or implementations, it is obvious that various modifications and changes may be made to these examples without departing from the general scope of the invention as defined by the claims. Furthermore, individual features of the various embodiments discussed may be combined in additional embodiments. For example, while in the example presented, the image processing device (the computer 30) is integrated with the camera 20, in particular so as to be able to produce results in real time. Conversely, in another embodiment the image processing device may be independent of the camera.It may be a computer, for example a computer connected to the internet, and capable of implementing a program in accordance with the present disclosure to process images and provide output images in which the visibility of objects is enhanced. Therefore, the description and drawings should be considered in an illustrative rather than restrictive sense.
Claims
1. An image processing method comprising the following steps: a) an initial image (A) consisting of pixels is acquired; c1) using at least one computer, for each value of an index j in a range of lines from N1 to N2 of the initial image A, a factor F1 (j) is determined as the value of the parameter F for which the following functional K is minimum: K = ∑ G 1 j g I A , i , j + s − F ∗ I A , i , j where G1 (j) is a first group of columns of the line j considered; g is a strictly increasing function, in particular a norm, and the sum is calculated for all columns i of group G1 (j); IA,i,j is an intensity of one pixel of the initial image (A) located on column i and line j; s=+1 or -1; and d1) a first enhanced image (B1) is calculated, wherein each line Lj of said range of lines from N1 to N2 is replaced by a line L'j obtained by multiplying each pixel of line Lj by a multiplier coefficient (M1(j)) equal to the product of the factors F1(k), for k ranging from N1 to j.
2. The method according to claim 1, wherein at step c1), for a line j, the factor F1 (j) is determined as the value of the parameter F for which the value of the following functional K is a minimum: K = ∑ G 1 j I A , i , j + s − F ∗ I A , i , j p where p is a positive number.
3. The method according to claim 1, wherein at step c1), for a line j, the factor F1 (j) is determined to be equal to a ratio between the average intensity of the pixels of the line Lj and the average intensity of the pixels of the line Lj+s, the average intensity of the pixels of a line being an average of the intensity of the pixels of the line.
4. The method according to any of claims 1 to 3, further comprising the following steps: c2) for each value of the index j in the range of lines from N1 to N2 of the initial image (A), a factor F2(j) is determined by means of at least one computer as the value of the parameter F for which the value of the following functional K is a minimum: K = ∑ G 2 j g I A , i , j + s − F ∗ I A , i , j where G2(j) is a second group of columns of the line j considered; g is a strictly increasing function, in particular a norm, and the sum is calculated for all columns i of group G2(j); and d2) a second enhanced image (B2) is calculated, wherein each line Lj of said range of lines from N1 to N2 is replaced by a line L"j obtained by multiplying each pixel of line Lj by a multiplier coefficient (M2(j)) equal to the product of the factors F2(k), for k ranging from j to N2; and e) an average enhanced image (C) equal to an average of the first and second enhanced images (B1, B2) is calculated.
5. The method according to any of claims 1 to 4, wherein the initial image (A) is an infrared image, in particular a long-wave infrared (LWIR) image.
6. The method according to any of claims 1 to 5, further including the following step, for at least one line Lj: b1) for each of the pixels of said at least one line Lj, a deviation (EA,h,j) between an intensity IA,h,j of a pixel of a column h considered on the line Lj and an intensity IA,h,j+s of a corresponding pixel in the column h and on the line Lj+s is evaluated: E A , h , j = I A , h , j − I A , h , j + s ; and at step d1), the first group of columns (G1(j)) to be taken into account for the line Lj is determined as being the set of column indices h for which the deviation EA,h,j is less than a predetermined value.
7. A computer program on an information carrier, said program being executable in a computer, the program including program code portions / means / instructions for carrying out the steps of the image processing method according to any of claims 1 to 6 when said program is run on a computer.
8. A computer-readable information carrier having stored thereon, a computer program according to claim 7.
9. An image processing device (30), the device including a) an initial image acquisition module, configured to acquire an initial image (A) consisting of pixels; c1) a factor-determining module F1(j) configured, for each value of an index j in a range of lines from N1 to N2 of the initial image A, to determine a factor F1(j) as the value of the parameter F for which the value of the following functional K is a minimum: K = ∑ G 1 j g I A , i , j + s − F ∗ I A , i , j where G1(j) is a first group of columns of the line j considered; g is a strictly increasing function, in particular a norm, and the sum is calculated for all columns i of group G1 (j); IA,i,j is an intensity of one pixel of the initial image (A) located on column i and line j; s=+1 or -1; and d1) a first enhanced image calculation module, configured to calculate a first enhanced image wherein, each line Lj of said range of lines from N1 to N2 is replaced by a line L'j obtained by multiplying each pixel of line Lj by a multiplier coefficient (M1(j)) equal to the product of the factors F1(k), for k ranging from N1 to j.
10. The image processing device (30) according to claim 9, wherein the factor F1(j) determining module is configured to obtain the factor F1 (j) as the value of the parameter F for which the value of the following functional K is a minimum: K = ∑ G 1 j I A , i , j + s − F ∗ I A , i , j p where p is a positive number.
11. The image processing device (30) according to claim 9, wherein the factor F1(j) determining module is configured to determine the factor F1 (j) as being equal to a ratio between the average pixel intensity of the line Lj and the average pixel intensity of the line Lj +s, the average pixel intensity of a line being an average of the pixel intensity of the line.
12. An image acquisition system (10) comprising at least one infrared camera (20), in particular an infrared camera (20) configured to acquire images in the LWIR frequency band, and an image processing device (30) according to any of claims 9 to 11.
13. A robot or vehicle (100) comprising an image acquisition system (10) according to claim 12.