IMAGE PROCESSING DEVICE

Abstract

An image processing device according to the technology of the present disclosure comprises: a coherence calculation processing unit (2) for calculating a coherence map; a threshold calculation processing unit (110) for setting a threshold value which is used to extract a change area; a threshold processing unit (200) for performing threshold processing using the threshold value; and a change detection processing unit (3) for identifying a type of change when it is determined that a change in pixels of interest occurs by threshold processing based on the coherence maps for at least three images.

Description

TECHNICAL FIELD

[0001] The technology according to the present disclosure relates to an image processing apparatus. BACKGROUND TO THE STATE OF THE ART

[0002] In general, image processing technologies for radar images are known. More specifically, an image processing device is known that processes a radar image of a ground surface obtained by observing a synthetic aperture radar that performs periodic observation.

[0003] For example, Patent Literature 1 discloses an image processing device that calculates a plurality of complex correlation distributions (coherence maps) from a plurality of radar images and extracts a physical change of the ground surface therefrom. REFERENCE LISTPATENT LITERATURE

[0004] Patent Literature 1: JP 2009-289111 A SUMMARY OF THE INVENTIONTECHNICAL PROBLEM

[0005] The conventional image processing device disclosed in Patent Literature 1 uses only the correlations of two adjacent radar images among the radar images arranged in time series, and therefore the obtained information is limited. Therefore, the technology according to the present disclosure utilizes information obtained from the correlations of three radar images to obtain more information. An object of the technology according to the present disclosure is to achieve the extraction of a physical change on a ground surface included in a radar image with higher accuracy than before by improving a conventional image processing device as disclosed in Patent Literature 1. SOLUTION TO THE TASK

[0006] An image processing apparatus according to the technology of the present disclosure includes: a coherence calculation processing unit for calculating a coherence map; a threshold calculation processing unit for setting a threshold used to extract a change area; a threshold processing unit for performing threshold processing using the threshold; and a change detection processing unit for identifying a type of change when it is determined that a change occurs in pixels of interest by the threshold processing based on the coherence maps for at least three images. ADVANTAGEOUS EFFECTS OF THE INVENTION

[0007] The image processing apparatus according to the technology of the present disclosure has the above-mentioned configuration and can therefore extract a physical change on the ground surface included in three radar images with higher accuracy than before by using information obtained from the coherences of the radar images. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram showing a functional configuration of an image processing apparatus according to a first embodiment. Fig. 2 is a block diagram showing a functional configuration of a threshold calculation processing unit 110a included in the image processing apparatus according to the first embodiment. Fig.3 is a block diagram showing a functional configuration of an image processing apparatus according to a second embodiment. Fig. 4 is a block diagram showing a functional configuration of a threshold calculation processing unit 110b included in the image processing apparatus according to the second embodiment. Fig. 5 is a block diagram showing a functional configuration of an image processing apparatus according to a third embodiment. Fig. 6 is a block diagram showing a functional configuration of an image processing apparatus according to a fourth embodiment. Fig. 7 is a block diagram showing a functional configuration of a threshold calculation processing unit 110d included in the image processing apparatus according to the fourth embodiment. DESCRIPTION OF THE EMBODIMENTSFirst embodiment

[0008] Fig. 1 is a block diagram showing a functional configuration of an image processing apparatus according to a first embodiment. As shown in Fig. 1, the image processing apparatus according to the first embodiment includes a data storage unit 1, a coherence calculation processing unit 2, a threshold calculation processing unit 110a, a threshold processing unit 200a, a change detection processing unit 3a, and an output data storage unit 4.

[0009] Elements with "a" at their reference numerals, such as the threshold calculation processing unit 110a, the threshold processing unit 200a, and the change detection processing unit 3a, indicate modes in the first embodiment, whose changes will be described in the embodiments described later. Elements without a letter after the reference numeral are commonly used in the technology according to the present disclosure, regardless of the embodiments. <<Datenspeichereinheit 1> >

[0010] The data storage unit 1 is a component that stores images to be processed by the image processing device. Specifically, the data storage unit 1 stores data (hereinafter referred to as "image data") related to N images obtained through N observations and captured at different times. The image processed by the image processing device according to the technology of the present disclosure is, for example, a radar image.

[0011] The image data stored in the data storage unit 1 can be read by another functional block of the image processing device, e.g., the coherence calculation processing unit 2. <<Kohärenzberechnung-Verarbeitungseinheit 2> >

[0012] The coherence calculation processing unit 2 is a component that calculates a coherence distribution between images.

[0013] The coherence calculation processing unit 2 calculates, in a first step, a complex radar image for each of the N read radar images. In a second step, the coherence calculation processing unit 2 generates coherence distributions (hereinafter referred to as "coherence maps") between complex radar images for all combinations ( N C2 combinations) of two complex radar images extracted from the calculated N complex radar images. The coherence map can also be referred to as a complex correlation distribution. The coherence calculation processing unit 2 creates the coherence map based on the individual pixel information of the complex radar image. <<Schwellenwertberechnung-Verarbeitungseinheit 110> >

[0014] The threshold calculation processing unit 110 is a component that sets a threshold used for extracting a "change range," which will be described later. The details of the threshold calculation processing unit 110 will be understood from the description of Fig. 2, which will be described later. <<Schwellenwertverarbeitungseinheit 200> >

[0015] The threshold processing unit 200 is a component that performs threshold processing using the threshold value set by the threshold calculation processing unit 110. The purpose of the threshold processing performed by the threshold processing unit 200 is to determine whether or not a change has occurred in a pixel of interest. <<Change Detection Processing Unit 3>>

[0016] The change detection processing unit 3 is a component that, when it detects that a change has occurred in the pixel in question, identifies the type of change. The type of change identified by the change detection processing unit 3 is sent to the output data storage unit 4. The type of change detected by the change detection processing unit 3 can be programmed to be displayed on a display unit (not shown) of the image processing device. <<Ausgabedatenspeichereinheit 4> >

[0017] The output data storage unit 4 is a component that stores the type of change identified by the change detection processing unit 3. <<Detail der von der Bildverarbeitungsvorrichtung durchgeführten Verarbeitung> >

[0018] The coherence calculation processing unit 2 included in the image processing device calculates a complex radar image for each of the N radar images read as described above. The coherence calculation processing unit 2 then creates coherence maps between complex radar images for all combinations ( N C2 combinations) of two complex radar images extracted from the calculated N complex radar images.

[0019] In the present application, n and m represent sampling numbers related to images. That is, both n and m are natural numbers from 1 to N. The sampling numbers (n and m) related to images can be reformulated as image numbers related to images. Note that the radar image on which the technology according to the present disclosure is based consists of a plurality of radar images obtained by observing the same area at different times. That is, the technology according to the present disclosure is based on a radar image related to a synthetic aperture radar device mounted on a flying object (artificial satellite or aircraft) that performs periodic observation multiple times. From this point of view, the sampling period is on a weekly or monthly basis.Therefore, a moving picture created by combining pictures is played at a much higher speed than the actual speed, although the sampling numbers (n and m) in terms of the pictures can be regarded as the frame numbers in terms of the pictures.

[0020] The sampling period of the radar image processed by the image processing device may be short, for example, on a daily or hourly basis. In such a case, even when comparing two images with adjacent sampling numbers, such as an nth radar image and an (n + 1)th radar image, hardly any changes are detected. In this case, when conventional image processing and calculating the coherence of the (n + 1)th radar image viewed from the nth radar image, only the coherence close to 1 is uniformly calculated for each pixel.

[0021] The technology according to the present disclosure is effective in such a case when the sampling period is short as described above. The image processing apparatus according to the technology of the present disclosure calculates the coherences (C n ) not only for two images with adjacent sample numbers, but also for two images with non-consecutive sample numbers.

[0022] From the radar image (hereinafter simply referred to as “n-th radar image”) in the n-th sample, the coherence (C n ) with the radar image (hereinafter simply referred to as "m-th radar image") in the m-th sample is given by the following expression. {for n=1 to N−1 andfor m=n+1 to N Cn(m, y, x)=∑i=1LZn(y,x)Z'm(y,x)∑i=1L|Zn(y,x)|2∑i=1L|Zm(y,x)|2

[0023] In expression (1), Z n(y, x) represents a data value (referred to as a “data value”) at a position (y, x) in the n-th radar image. Similarly, Z m (y, x) in Expression (1) represents a data value at a position (y, x) in the m-th radar image. The parameter L in Expression (1) represents the number of pixels used to calculate the coherence of the pixels of interest at the position (y, x). In addition, the superscript (*) in Expression (1) indicates a complex conjugate. In Expression (1), m is a natural number greater than n. When m = n + 1, Expression (1) is the same as the expression for coherence according to the related art disclosed in Patent Literature 1.

[0024] The coherence (C n ) is based on the assumption that the nth radar image and the mth radar image were acquired in the same satellite orbit. The coherence (C n) has the property that it takes a value closer to "1" when a change in the soil surface is smaller, and conversely takes a smaller value when a change in the soil surface is larger.

[0025] In the expression for coherence (C n ) given by expression (1), an operation for obtaining a sum for L pixels is used as the denominator and numerator. The technology according to the present disclosure may use a result of spatial filtering of the L pixels instead of the sum of the L pixels.

[0026] Furthermore, in the technology according to the present disclosure, a data value (Z n (y, x)) at position (y, x) in the n-th radar image as “the result of the spatial Fourier transform (e.g. in y- or x-direction) of L pixels around position (y, x)” during the calculation of coherence (C n). In this case, the calculation of the sum in expression (1) is not performed. In view of this, it can be further understood that Z n (y, x) is a complex number. Furthermore, the coherence (C n ) a complex number on the unit circumference in a complex plane. The details of a procedure for calculating the coherence (C n ) will be apparent from the description of the sixth embodiment.

[0027] As described above, even when comparing two images with adjacent sampling numbers, such as an nth radar image and an (n+1)th radar image in a case where the sampling period of the radar image processed by the image processing device is short, little change is detected. An effective way to extract changes is to compare the first image and the last image among the chronologically arranged images. Therefore, for coherence (Cn ) calculated based on Expression (1), the coherence (C1(N, y, x)) for the last frame from the first frame is important. Furthermore, a method of tracking the coherence (C1) of the remaining N - 1 frames using the image on the first frame as a reference image is also effective. This method can detect a frame in which a change has occurred, that is, a period in which a change has occurred.

[0028] The threshold calculation processing unit 110 included in the image processing device sets a threshold value used to extract a "change area" as described above. The threshold values ​​set by the threshold calculation processing unit 110 are referred to as P in this specification. thd and P thu The P thd and P thuThe indices used are derived from th for “threshold”, d for “down” and u for “up”.

[0029] The threshold calculation processing unit 110 may P thd and P thu based on the coherences of the neighboring N - 1 images, ie the coherences when m = n + 1, among the coherences calculated by the coherence calculation processing unit 2 with respect to the pixel of interest (position (y, x)). Table 1 Threshold processing Conditional expression Determination result C n (m, y, x) ≤ P thd Pixel of interest is change area Pthd < C n (m, y, x) ≤ P thu Pixel of interest is non-change area P thu ≤ C n (m, y, x) Pixel of interest is change area

[0030] The threshold processing unit 200 included in the image processing device performs threshold processing using the threshold values ​​(P thd and P thu ) as described above. Specifically, the threshold processing performed by the threshold processing unit 200 is based on the following conditional expression.

[0031] The term “pixel of interest” described in Table 1 refers here to the pixel of interest located at (y, x) of the coherence map of the n-th radar image and the m-th radar image.

[0032] As described above, the coherence (C n ) is originally a complex number, but as can be seen in the prior art disclosed in Patent Literature 1, there is a case where the discussion focuses only on the real part of the complex number. In the technology according to the present disclosure, the conditional expression shown in Table 1 means that the real part of the complex number is checked for coherence and compared with a threshold value that is a real number.

[0033] The threshold processing unit 200 performs the threshold processing shown in Table 1 for the coherence maps of all combinations ( N C2 combinations).

[0034] As described above, the coherence (C n ) is a complex number on a unit circumference in a complex plane. In addition, the coherence (C n ) has a value closer to "1" because the change at the ground surface is smaller. In other words, the real part of the coherence (C n ) 1, because the surface of the soil changes less.

[0035] In the conditional expression shown in the upper part of Table 1, it is not surprising that if the real part of the coherence (C n ) equal to or less than the lower threshold (P thd ), it is determined that “the pixel of interest is a region of change”.

[0036] The conditional expression shown in the lower part of Table 1 takes into account regions such as forests and water bodies. In regions such as forests and water bodies, the coherence (C n) is usually low, as, for example, branches and leaves are shaken by the wind or the wind creates waves. The region of forests transitions from a state of low coherence to a state of high coherence when forests disappear due to deforestation or similar. The region of water bodies transitions from a state of low coherence to a state of high coherence when the water bodies become land due to global warming or similar. With regard to regions of low coherence, such as forests and water bodies, it is concluded that “the regions are areas of change” when a transition from the state of low coherence (C n ) to the state with high coherence (C n), that is, when a disappearance occurs. The conditional expression shown in the lower part of Table 1 seems strange at first glance, but it is not strange when regions with low coherence, such as forests and water bodies, are taken into account. The upper threshold (P thu ), which appears in the conditional expression in the lower part of Table 1, is also determined taking into account regions of low coherence such as forests and water areas.

[0037] Fig. Fig. 2 is a block diagram showing a functional configuration of the threshold calculation processing unit 110a included in the image processing apparatus according to the first embodiment. As shown in Fig.2, the threshold calculation processing unit 110a includes a pixel of interest setting unit 120a, an average calculation processing unit 130a, a threshold storage unit 140a, and a threshold setting unit 150a.

[0038] The pixel-of-interest setting unit 120a in the threshold calculation processing unit 110a processes the selection of a pixel to be used for setting the threshold. The pixel-of-interest setting unit 120a detects the coherence (C n ), given by expression (1), in chronological order.

[0039] The average calculation processing unit 130a in the threshold calculation processing unit 110a performs the calculation of the coherence of each pixel set by the pixel of interest setting unit 120a and also calculates an average value.

[0040] The threshold storage unit 140a in the threshold calculation processing unit 110a stores in advance the parameters used to calculate the threshold. The parameters used to calculate the threshold are, for example, the offset values ​​(β d and β u ).

[0041] The threshold setting unit 150a in the threshold calculation processing unit 110a determines the threshold values ​​(P thd , P thu ) based on the average value calculated by the average calculation processing unit 130a and the parameters stored in the threshold storage unit 140a.

[0042] The average value (M c ) is given by the following expression. Mc(y,x)=∑n=1N−1Cn(n+1,y,x)N−1

[0043] In particular, expression (2) gives an average value of the neighboring N - 1 coherences, that is, an average value of the coherences when m = n + 1, among the coherences related to the pixels of interest (position (y, x)).

[0044] The threshold values ​​(P thd and P thu ) determined by the threshold setting unit 150a are given, for example, by the following expression. {Pthd=Mc(y,x)−βdPthu=Mc(y,x)+βu

[0045] As shown in expressions (2) and (3), the image processing apparatus according to the first embodiment sets a range having a width of offset values ​​(β d and β u ) defined in advance around the average value of the measured N - 1 neighboring coherences as a “no change area”.

[0046] The purpose of using the average value in setting the threshold is disclosed, for example, in Patent Literature 1. As disclosed in Patent Literature 1, regions such as forests and water bodies originally have low coherence. To extract a change in an area that originally had low coherence, the image processing device sets a threshold based on an average of the coherence values.

[0047] Furthermore, it is assumed that the forest region transitions from a state of low coherence to a state of high coherence when forests disappear due to deforestation or similar. Therefore, not only the lower threshold (P thd ), but also the upper threshold (P thu ) is specified.

[0048] As described above, the change detection processing unit 3 identifies the type of change when it detects that a change has occurred in the pixel in question. The processing performed by the change detection processing unit 3 is referred to as "change detection" in this specification. The change detection performed by the change detection processing unit 3 can be performed based on coherences calculated from three freely selected images, that is, a change rate of coherences.

[0049] When the sampling times are set to n, m1 and m2, there are nine possible cases, shown in the following table, in which the coherence differs from the coherence (C n (m1)) of the m1-th image viewed from the n-th image and the coherence (C m1 (m2)) of the m2th image viewed from the m1th image changes. Table 2 Nine cases C m1 (m2) ≤ P thd Pthd < C m1 (m2) < P thu P thu ≤ C m1 (m2) C n (m1) ≤ Pthd Case 5 Case 4 Case 6 P thd < C n (m1) < P thu Case 2 Case 1 Case 3 P thu ≤ C n ( m 1) Case 8 Case 7 Case 9

[0050] It should be noted that in Table 2 y and x in the arguments of the function (C n , C m1 ), which represents coherence, can be omitted. Similar to m in Expression (1), m1 and m2 in Table 2 represent sampling numbers with respect to images. The size relationship between n, m1, and m2 is 1≤ n < m1< m2 ≤ N.

[0051] In Case 1 shown in Table 2, the change detection processing unit 3a according to the first embodiment compares the coherence (C n (m2)) of the m2-th image viewed from the n-th image with the threshold. Table 3 Divided by cases in Case 1 Conditional expression Determination result C n (m2, y, x) ≤ P thd Case 1B Gradual change throughout the period P thd < C n (m2, y, x) < P thu Case 1A No change P thu ≤ C n (m2, y, x) Case 1C Slight change in the m1-th image

[0052] Case 1, presented in Table 2, shows a situation where there is no change from the nth image to the m1th image and from the m1th image to the m2th image. Table 3 shows a result when considering a change from the nth image directly to the m2th image.

[0053] The top row of Table 3 shows that there is a change when focusing on a transition from the nth image directly to the m2th image. Therefore, the evaluation result in this case is that "a gradual change occurs throughout the entire period."

[0054] The middle row of Table 3 shows that there is no change when focusing on a transition from the nth image directly to the m2th image. Therefore, the result of the evaluation in this case is "no change."

[0055] The bottom row of Table 3 shows that the coherence is higher when switching from the nth image directly to the m2th image than for the other combinations. This means that in this case, there is almost no change between the nth and m2th images. This phenomenon can occur when the m1th image changes slightly. Furthermore, this phenomenon shows that a slight change occurring in the m1th image can be replicated in the m2th image. Therefore, the determination result in this case is: "There is a slight change in the m1th image."

[0056] With respect to the pixel of interest, the change detection processing unit 3a performs change detection for a total of eleven types of situations (Case 1A, Case 1B, Case 1C, Case 2, Case 3, Case 4, Case 5, Case 6, Case 7, Case 8, and Case 9), as shown in Tables 2 and 3.

[0057] As described above, the image processing apparatus according to the technology of the present disclosure can extract a change even in a case where both C n (m1) as well as C m1 (m2) change areas.

[0058] The change detection processing unit 3 performs change detection for each pixel of interest. The result of the processing performed by the change detection processing unit 3 is stored in the output data storage unit 4. The data stored in the output data storage unit 4 may be a structure. The structure stored in the output data storage unit 4 may include a field for specifying the pixel of interest and a field for specifying the change detection obtained by the processing performed by the change detection processing unit 3. In particular, the field for specifying the pixel of interest may be the position of the pixel of interest (y, x).The field indicating the change detection obtained by the processing of the change detection processing unit 3a may be any field as long as it can identify one of eleven types of situations (Case 1A, Case 1B, Case 1C, Case 2, Case 3, Case 4, Case 5, Case 6, Case 7, Case 8 and Case 9).

[0059] As described above, the image processing apparatus according to the first embodiment has the configuration described above, and therefore, even in a case where only about three to several dozen radar images can be obtained, it is possible to extract a physical change on the ground surface included in the radar images with higher accuracy than before. Second embodiment

[0060] An image processing apparatus according to a second embodiment is a modification of the image processing apparatus according to the technology of the present disclosure. Unless otherwise indicated, the same reference numerals as in the first embodiment are used in the second embodiment. In the second embodiment, the description overlapping with that of the first embodiment is omitted where appropriate.

[0061] Fig. 3 is a block diagram showing a functional configuration of the image processing apparatus according to the second embodiment. As shown in Fig.As shown in Figure 3, the image processing apparatus according to the second embodiment includes a data storage unit 1, a coherence calculation processing unit 2, a threshold calculation processing unit 110b, a threshold processing unit 200a, a change detection processing unit 3a, and an output data storage unit 4. As described above, the configuration of the image processing apparatus according to the second embodiment is substantially the same as that of the first embodiment. A component with "b" after a reference numeral, as in the threshold calculation processing unit 110b, indicates a mode in the second embodiment.

[0062] Fig. Fig. 4 is a block diagram showing a functional configuration of the threshold calculation processing unit 110b included in the image processing apparatus according to the second embodiment. As shown in Fig.4, the threshold calculation processing unit 110b includes a pixel of interest setting unit 120b, an average calculation processing unit 130b, a threshold storage unit 140b, and a threshold setting unit 150b. <<Durchschnittsberechnung-Verarbeitungseinheit 130b nach dem zweiten Ausführungsbeispiel> >

[0063] The average calculation processing unit 130a according to the first embodiment obtains a value given by Expression (2), that is, an average value using only adjacent images, but the technology according to the present disclosure is not limited to this.

[0064] In the image processing apparatus according to the technology of the present disclosure, the average calculation processing unit 130b may use a value given by the following expression. Mc_2(y,x)=∑n−1N−1{∑m=n+1NCn(m,y,x)}NC2

[0065] Here, N C2 in the denominator on the right side of Expression (4) is the number of combinations of two images extracted from N images. That is, the average calculation processing unit 130b according to the second embodiment calculates an average value of the coherences for all combinations. <<Schwellenwerteinstelleinheit 150b gemäß dem zweiten Ausführungsbeispiel> >

[0066] Threshold values ​​(P thd_2 , P thu_2 ) determined by the threshold setting unit 150b are given, for example, by the following expression. {Pthd_2=Mc_2(y,x)−βdPthu_2=Mc_2(y,x)−βu

[0067] As shown in expressions (4) and (5), the image processing apparatus according to the second embodiment sets a range having a width of offset values ​​(β d and β u) defined in advance around the average value of the coherences calculated for all combinations, as the ‘non-change area’.

[0068] As described above, the image processing apparatus according to the second embodiment has the above-described configuration, and thus the same effects as those of the image processing apparatus according to the first embodiment are achieved. Third embodiment

[0069] An image processing apparatus according to a third embodiment is a modification of the image processing apparatus according to the technology of the present disclosure. Unless otherwise indicated, the same reference numerals as those used in the above-described embodiments are used in the third embodiment. In the third embodiment, descriptions overlapping with those used in the above-described embodiments are omitted where appropriate.

[0070] Fig. 5 is a block diagram showing a functional configuration of the image processing apparatus according to the third embodiment. As shown in Fig.As shown in Figure 5, the image processing apparatus according to the third embodiment includes a data storage unit 1, a coherence calculation processing unit 2, a threshold calculation processing unit 110a, a threshold processing unit 200a, a change detection processing unit 3c, and an output data storage unit 4. As described above, the configuration of the image processing apparatus according to the third embodiment is substantially the same as that of the first embodiment. A component with "c" after a reference numeral as in the change detection processing unit 3c indicates a mode in the third embodiment. <<Change detection processing unit 3c according to the third embodiment>>

[0071] The change detection processing unit 3a according to the first embodiment first detects nine cases (Case 1, Case 2, Case 3, Case 4, Case 5, Case 6, Case 7, Case 8 and Case 9) shown in Table 2 from the coherence (C n (m1)) of the m1-th image, seen from the n-th image, and the coherence (C m1 (m2)) of the m2-th image viewed from the m1-th image. The change detection processing unit 3a according to the first embodiment further defines three types of Case 1 (Case 1A, Case 1B, and Case 1C) from the coherence (C n (m2)) of the m2-th image viewed from the n-th image.

[0072] The change detection processing unit 3c according to the third embodiment compares the coherence (C n(m2)) of the m2-th image viewed from the n-th image with a threshold for all nine cases (Case 1, Case 2, Case 3, Case 4, Case 5, Case 6, Case 7, Case 8 and Case 9) shown in Table 2, and further defined three types of each of the nine cases, that is, a total of 9 × 3 = 27 cases.

[0073] As described above, the change detection processing unit 3 performs change detection for each pixel of interest. The result of the processing performed by the change detection processing unit 3 is stored in the output data storage unit 4. The data stored in the output data storage unit 4 may be a structure. The structure stored in the output data storage unit 4 may include a field for indicating the pixel of interest and a field for indicating the change detection obtained by the processing performed by the change detection processing unit 3. Specifically, the field for indicating the pixel of interest may be the position of the pixel of interest (y, x).

[0074] The field indicating the change detection obtained by the processing of the change detection processing unit 3c may be any field as long as it can identify one of 27 kinds of situations.

[0075] As described above, the image processing apparatus according to the third embodiment has the above-described configuration, and thus the same effects as those of the image processing apparatuses according to the above-described embodiments are achieved. Fourth embodiment

[0076] An image processing apparatus according to a fourth embodiment is a modification of the image processing apparatus according to the technology of the present disclosure. Unless otherwise indicated, the same reference numerals as in the above-described embodiments are used in the fourth embodiment. In the fourth embodiment, descriptions overlapping with those of the above-described embodiments are omitted where appropriate.

[0077] Fig. 6 is a block diagram showing a functional configuration of the image processing apparatus according to the fourth embodiment. As shown in Fig.As shown in FIG. 6, the image processing apparatus according to the fourth embodiment includes a data storage unit 1, a coherence calculation processing unit 2, a threshold calculation processing unit 110d, a threshold processing unit 200d, a change detection processing unit 3d, and an output data storage unit 4. As described above, the configuration of the image processing apparatus according to the fourth embodiment is substantially the same as that of the first embodiment. A component with "d" after a reference numeral as in the change detection processing unit 3d indicates a mode in the fourth embodiment.

[0078] Fig. Fig. 7 is a block diagram showing a functional configuration of the threshold calculation processing unit 110d included in the image processing apparatus according to the fourth embodiment. As shown in Fig.7, the threshold calculation processing unit 110d includes a pixel of interest setting unit 120d, an average calculation processing unit 130d, a threshold storage unit 140d, and a threshold setting unit 150d.

[0079] The image processing apparatus according to the technology of the present disclosure can simultaneously include the mode described in the second embodiment and the mode described in the third embodiment. In the image processing apparatus according to the fourth embodiment, the average calculation processing unit 130d performs the processing given by Expression (4) and calculates the average value of the coherences for all combinations. In addition, the change detection processing unit 3d of the image processing apparatus according to the fourth embodiment compares the coherence (C n(m2)) of the m2-th image viewed from the n-th image with a threshold for all nine cases (Case 1, Case 2, Case 3, Case 4, Case 5, Case 6, Case 7, Case 8 and Case 9) shown in Table 2, and further defined three types of each of the nine cases, that is, a total of 9 × 3 = 27 cases.

[0080] As described above, the image processing apparatus according to the technology of the present disclosure is not limited to the aspects described in the respective embodiments, and it is possible to combine the respective embodiments, modify each component of the respective embodiments, or omit each component in the respective embodiments.

[0081] As described above, the image processing apparatus according to the fourth embodiment has the above-described configuration, and thus the same effects as those of the image processing apparatuses according to the above-described embodiments are achieved. Fifth embodiment

[0082] An image processing apparatus according to a fifth embodiment is a modification of the image processing apparatus according to the technology of the present disclosure. Unless otherwise indicated, the same reference numerals as in the above-described embodiments are used in the fifth embodiment. In the fifth embodiment, descriptions overlapping with those of the above-described embodiments are omitted where appropriate.

[0083] The image processing apparatus described in each of the first to fourth embodiments calculates an average value of coherences by the average calculation processing unit 130 and sets a range having a width of offset values ​​(β d and β u ) set in advance around the calculated average value by the threshold setting unit 150 as a “no change range”.

[0084] The image processing apparatus according to the technology of the present disclosure is not limited to the use of preset offset values ​​(β d and β u ) is limited.

[0085] The image processing device according to the fifth embodiment may rely on a theoretical dispersion value or a false alarm probability instead of the preset offset values ​​(βd and ß u ) relate.

[0086] As described above, the image processing apparatus according to the fifth embodiment has the above-described configuration and thus offers an effect of being able to increase a degree of freedom with respect to setting the “range other than the change range” in addition to the effects similar to those of the image processing apparatuses according to the above-described embodiments. Sixth embodiment

[0087] An image processing apparatus according to a sixth embodiment is a modification of the image processing apparatus according to the technology of the present disclosure. Unless otherwise indicated, the sixth embodiment uses the same reference numerals as the above-described embodiments. In the sixth embodiment, descriptions overlapping with those of the above-described embodiments are omitted where appropriate.

[0088] In the image processing device according to the technology of the present disclosure, a radar image to be processed may be preprocessed or unprocessed. The image processing device according to the sixth embodiment performs preprocessing of a radar image.

[0089] As described above, in the technology according to the present disclosure, the data value (Z.(y, x)) at the position (y, x) in the n-th radar image may be considered as “the result of the spatial Fourier transform (e.g., in the y or x direction) of L pixels around the position (y, x)” during the calculation of the coherence (C n ) can be defined. Z n For example, (y, x) can be a value given by the following expression. Zn(y,x)=α+jβ in the middle α=2π∑k=y−L22y+L22b(k,x)cos(k−yL2+1π) β=2π∑k=y−L22y+L22b(k,x)sin(k−yL2+1π)

[0090] Here, L2 in expression (6) corresponds to the value L in expression (1). Expression (6) is the result of the Fourier transform of L2 + 1 pixels around the position (y, x) in the y direction, or more precisely, the result of the transformation by a description function. In expression (6), b(y, x) represents the luminance of the pixel at the position (y, x). In expression (6), j represents an imaginary number. Although expression (6) represents the Fourier transform in the y direction, the Fourier transform in the x direction can also be used. When a certain radar image is focused, a dark part and a bright part of pixels may periodically appear in a certain region, such as forests, when viewed spatially. For example, in a forest region in a radar image, it is assumed that the luminance of pixels at a distance of five pixels is high.In such a case, when the conversion of Expression (6) is performed on the radar image with L = 4, the absolute value of Z,(y, x) (the distance from the origin on the complex plane) is large in the area where the luminance is high at a distance of five pixels. In this way, the technology according to the present disclosure can extract a region of interest (hereinafter referred to as the "region of interest").

[0091] Expression (6) is an example of preprocessing the radar image. When L2 = 0 in expression (6), Z n (y, x) equals b(y, x) multiplied by π / 2, which corresponds to the situation where preprocessing is not performed.

[0092] If Z n (y, x) is given by expression (6), the coherence (C n ) as follows. {for n=1 to N−1 andfor m=n+1 to N Cn(m, y, x)=Zn(y,x)Z'm(y,x)|Zn(y,x)||Zm(y,x)|

[0093] The coherence (C n ) between images that do not change can be derived from expressions (6) and (7). The coherence (C n ) between images that do not change can be calculated as follows: Coherence (C n ) of the nth radar image as seen from the nth radar image. Cn(n,y,x)=Zn(y,x)Z'n(y,x)|Zn(y,x)||Zn(y,x)|=(α+βj)(α−βj)α2+β2α2+β2=α2+β2α2+β2=1

[0094] In a case where the coherence (C n ) of the m1-th radar image seen from the n-th radar image and the coherence (C n ) of the m2-th radar image as seen from the m1-th radar image, the coherence (C n ) of the m2th radar image as seen from the nth radar image is given as follows. Cn(m2,y,x)=Zn(y,x)Z*m2(y,x)|Zn(y,x)||Zm2(y,x)|=Zn(y,x)Z*m2(y,x)|Zn(y,x)||Zm2(y,x)|{Zm1(y,x)Z*m1(y,x)|Zm1(y,x) ||Zm1(y,x)|}︷=1=Zn(y,x)Z*m1(y,x)|Zn(y,x)||Zm1(y,x)|Zm1(y,x)Z*m2(y,x)|Zm1(y,x)||Zm2(y,x)|=Cn(m1,y,x)Cm1(m2,y,x)

[0095] As an example of preprocessing, Expression (6) is given with respect to the spatial Fourier transform, but the technology according to the present disclosure is not limited thereto. The image processing device according to the technology of the present disclosure may perform a temporal Fourier transform instead of Expression (6). More specifically, the image processing device according to the technology of the present disclosure may, for example, divide radar images for each season to be observed, perform a temporal Fourier transform of the radar images for each season, and perform a comparison between the seasons.

[0096] The image processing apparatus according to the technology of the present disclosure can be applied to a processed image on which semantic segmentation, ie, a deep learning algorithm, has been performed. Image processing according to the technology of the present disclosure can also be used in conjunction with semantic segmentation.

[0097] The image processing apparatus according to the technology of the present disclosure is not limited to semantic segmentation and can also be applied to an intermediate product of a learning model from the field of artificial intelligence, such as a neural network, such as a feature value map. Image processing according to the technology of the present disclosure can also be used in conjunction with artificial intelligence.

[0098] As described above, the image processing apparatus according to the sixth embodiment has the above configuration and thus offers the same effects as the image processing apparatuses according to the above-described embodiments. INDUSTRIAL APPLICABILITY

[0099] The technology of the present disclosure can be applied to an image processing apparatus that processes a radar image of a ground surface obtained by observation with a synthetic aperture radar that performs periodic observation, and is industrially applicable. LIST OF REFERENCE SYMBOLS

[0100] 1: Data storage unit, 2: Coherence calculation processing unit, 3: Change detection processing unit, 4: Output data storage unit, 110: Threshold calculation processing unit, 120: Pixel of interest setting unit, 130: Average calculation processing unit, 140: Threshold storage unit, 150: Threshold setting unit, 200: Threshold processing unit QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2009-289111 A

[0004]

Claims

[1] Image processing device comprising: a coherence calculation processing unit for calculating a coherence map; a threshold calculation processing unit for setting a threshold used to extract a change range; a threshold processing unit for performing threshold processing on the coherence map using the threshold; and a change detection processing unit for, when it is determined by the threshold processing based on the coherence maps for at least three images that a change occurs in pixels of interest, to identify the type of change. [2] The image processing apparatus according to claim 1, wherein the threshold calculation processing unit comprises: a pixel-of-interest setting unit for setting the pixels of interest; an averaging processing unit for calculating the coherence of each of the pixels of interest and calculating an average value; and a threshold setting unit for setting the threshold based on the average value calculated by the average calculation processing unit. [3] The image processing apparatus according to claim 2, wherein the average value calculated by the average calculation processing unit is for either the coherence between images having adjacent sampling numbers or the coherence between images of all combinations of two extracted images. [4] An image processing apparatus according to any one of claims 1 to 3, wherein the pixels of interest relate to a radar image or a radar image obtained by pre-processing the radar image.

Images