Image processing device and image processing program

Abstract

To provide an image processing device with which it is possible to detect occurrence of image abnormality.SOLUTION: An image processing device 10 comprises: a plurality of memories 12 provided in correspondence to a plurality of image processing, in which data is sequentially stored, in order from a first memory 12A to an N-th memory 12N; a first calculation unit 14 for calculating the number of the data stored in any one of the plurality of memories 12, the difference in peripheral pixel value of which is greater than or equal to a first threshold; a second calculation unit 16 for calculating the number of the data stored in the other of the plurality of memories 12, the difference in peripheral pixel value of which is greater than or equal to the first threshold; and a detection unit 18 for detecting abnormality of the plurality of memories 12 on the basis of calculation results from the first calculation unit 14 and the second calculation unit 16.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing device and an image processing program. [Background technology]

[0002] Patent Document 1 proposes the following processes for scanned image data: using a pixel threshold calculated from at least one of the original image data and the scanned image data, extracting pixels that exceed the pixel threshold, and setting an area where the extracted pixels are adjacent to each other and made up of multiple consecutive pixels as an abnormal candidate area; calculating the total number of pixels that make up each abnormal candidate area and an accumulated pixel value that is the sum of the differences between the pixel threshold and each pixel that makes up the abnormal candidate area; setting an erroneous judgment exclusion standard according to the characteristics of the original image data or the scanned image data; and determining whether the total number of pixels and the accumulated pixel value of each abnormal candidate area meet the erroneous judgment exclusion standard; and if they do, determining the abnormal candidate area as normal, and if they do not, determining it as abnormal.

[0003] Patent Document 2 proposes that whether a pixel to be detected as a defective pixel is a defective pixel or not is determined based on the absolute difference between the average signal value of a block of a predetermined size including the pixel to be detected as a defective pixel and the signal value of the pixel to be detected as a defective pixel, using a first threshold value and a second threshold value that is greater than the first threshold value; if the absolute difference value is greater than the first threshold value and less than the second threshold value, the defective pixel address is registered in a defective pixel address memory as a pixel to be subjected to static correction; and if the absolute difference value is greater than the second threshold value, the pixel is determined to be a pixel to be subjected to dynamic correction.

[0004] Patent Document 3 proposes a frame memory unit in which image data supplied frame by frame from an infrared imaging unit is sequentially stored, and a judgment circuit which treats each of the pixels constituting the infrared imaging unit as a target pixel, calculates the difference between the luminance value of the current frame supplied from the infrared imaging unit and the luminance value of each of a plurality of delayed frames that are temporally consecutive to the current frame and are stored in the frame memory unit, and checks the relationship between this difference and a threshold value; if the relationship satisfies a judgment condition indicating a normal output, the judgment circuit outputs the luminance value of the current frame as is as the output of the target pixel, and if the relationship satisfies a judgment condition indicating an abnormal output, it outputs a luminance value of the luminance values ​​of the plurality of delayed frames that is considered to be a normal output as the output of the target pixel.

[0005] Patent Document 4 proposes dividing an image in a frame memory into partial regions, providing two types of luminance distribution memories, namely, a reference image luminance distribution memory and a comparison image luminance distribution memory, calculating the difference corresponding to each partial region, and detecting an abnormality when the absolute value of the difference exceeds a preset threshold value. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2021-140468 [Patent Document 2] Japanese Patent Application Laid-Open No. 2012-70319 [Patent Document 3] Japanese Patent Application Laid-Open No. 2008-311734 [Patent Document 4] Japanese Patent Application Publication No. 8-7096 Summary of the Invention [Problem to be solved by the invention]

[0007] When displaying video, if an abnormality occurs in the memory used for video processing (for example, frame memory, line memory, etc.) or the control circuit that controls the memory, the video displayed on the display will be distorted.

[0008] SUMMARY OF THE INVENTION The present invention has been made in consideration of the above circumstances, and has as its object to provide an image processing device capable of detecting the occurrence of an abnormality in an image. [Means for solving the problem]

[0009] The image processing device according to the first aspect includes a first calculation unit that calculates the number of cases where the difference in values ​​of surrounding pixels is equal to or greater than a predetermined first threshold value in data stored in a first memory, which is one of a plurality of memories in which data is stored sequentially; a second calculation unit that calculates the number of cases where the difference in values ​​of surrounding pixels is equal to or greater than the first threshold value in data stored in a second memory other than the first memory, which is one of the plurality of memories; and a detection unit that detects abnormalities based on the calculation results of the first calculation unit and the second calculation unit.

[0010] An image processing program according to a second aspect causes a computer to calculate the number of cases where the difference in values ​​of surrounding pixels is equal to or greater than a predetermined first threshold in data stored in a first memory that is one of a plurality of memories in which data is sequentially stored, calculate the number of cases where the difference in values ​​of surrounding pixels is equal to or greater than the first threshold in data stored in a second memory that is another of the plurality of memories other than the first memory, and execute a process of detecting an abnormality based on each calculation result. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a schematic configuration of an image processing apparatus according to an embodiment of the present invention; [Figure 2] FIG. 1 is a block diagram showing a specific configuration example of an image processing apparatus according to an embodiment of the present invention. [Figure 3] 10 is a flowchart showing an example of the flow of processing performed by a first calculation unit and a second calculation unit. [Figure 4]10 is a flowchart illustrating an example of a flow of processing performed by a detection unit. [Figure 5] 10A and 10B are diagrams showing an example of an input video and a video when a memory failure occurs; DETAILED DESCRIPTION OF THE INVENTION

[0012] An example of an embodiment of the present invention will be described in detail below with reference to the drawings. Fig. 1 is a block diagram showing a schematic configuration of an image processing apparatus according to this embodiment.

[0013] As shown in FIG. 1, the image processing device 10 according to this embodiment includes a plurality of memories 12, a first memory 12A to an N-th memory 12N, a first calculation unit 14, a second calculation unit 16, and a detection unit 18.

[0014] Data is sequentially stored in the multiple memories 12 in the order of the first memory 12A to the Nth memory 12N. For example, multiple memories 12 are provided corresponding to multiple image processes, and each time an image process is performed, data is sequentially stored in the memory corresponding to the next image process.

[0015] The first calculation unit 14 calculates, in the data stored in one of the memories 12, the number of pixels whose difference in value between neighboring pixels is equal to or greater than a predetermined first threshold value.

[0016] The second calculation unit 16 calculates the number of neighboring pixel values ​​whose difference is equal to or greater than the first threshold value in the data stored in another memory among the plurality of memories 12.

[0017] The detection unit 18 detects abnormalities in the memories 12 based on the calculation results of the first calculation unit 14 and the second calculation unit 16.

[0018] FIG. 2 is a block diagram showing a specific example of the configuration of the image processing device 10 according to this embodiment.

[0019] Specifically, the image processing device 10 according to this embodiment includes an input control unit 20, a first memory 12A to an Nth memory 12N, a first image processing unit 22A to an Nth image processing unit 22N, a first calculation unit 14, a second calculation unit 16, and a detection unit 18.

[0020] The input control unit 20 receives data representing an image to be displayed on an electronic mirror, a CID (Center Information Display), or the like from a camera or the like, and sequentially outputs the data to the first memory 12A.

[0021] The first memory 12A to the Nth memory 12N are provided corresponding to the first video processing section 22A to the Nth video processing section 22N, respectively.

[0022] The first to Nth video processing units 22A to 22N each perform a different image processing operation. That is, the first to Nth video processing units 22A to 22N perform a multi-step image processing operation on the input data and output the processed data.

[0023] The first calculation unit 14 is provided corresponding to the first memory 12A, and the second calculation unit 16 is provided corresponding to the Nth memory 12N.

[0024] Each of the first calculation unit 14 and the second calculation unit 16 includes a first difference circuit (upper and lower) 24, a first comparison circuit (upper and lower) 26, a second difference circuit (left and right) 28, a second comparison circuit (left and right) 30, and a counter 32.

[0025] The first difference circuit (up and down) 24 of the first calculation unit 14 calculates the difference between the value of m pixels on n lines of the data stored in the first memory 12A and the value of m pixels on n-1 lines. That is, it calculates the difference between the values ​​of adjacent pixels above and below, which are peripheral pixels of the data stored in the first memory 12A.

[0026] The first comparison circuit (upper and lower) 26 of the first calculation unit 14 compares the calculation result of the first difference circuit (upper and lower) 24 with a predetermined threshold value A, and outputs the comparison result to the counter 32.

[0027] The second difference circuit (left and right) 28 of the first calculation unit 14 calculates the difference between the value of the m pixel on the nth line of the data stored in the first memory 12A and the value of the m-1 pixel on the nth line. That is, it calculates the difference between the values ​​of the adjacent pixels on the left and right, which are peripheral pixels of the data stored in the first memory 12A.

[0028] The second comparison circuit (left and right) 30 of the first calculation unit 14 compares the calculation result of the second difference circuit (left and right) 28 with a predetermined threshold A, and outputs the comparison result to the counter 32.

[0029] The counter 32 of the first calculation unit 14 counts the number of comparison results of the first comparison circuit (upper and lower) 26 and the second comparison circuit (left and right) 30 that are equal to or greater than the threshold value A.

[0030] Meanwhile, the first difference circuit (up and down) 24 of the second calculation unit 16 calculates the difference between the value of m pixels on n lines of the data stored in the Nth memory 12N and the value of m pixels on the (n-1)th line. That is, it calculates the difference between the values ​​of adjacent pixels above and below that are peripheral pixels of the data stored in the Nth memory 12N.

[0031] The first comparison circuit (upper and lower) 26 of the second calculation unit 16 compares the calculation result of the first difference circuit (upper and lower) 24 with a predetermined threshold value A, and outputs the comparison result to the counter 32.

[0032] The second difference circuit (left and right) 28 of the second calculation unit 16 calculates the difference between the value of the m pixel on the nth line of the data stored in the Nth memory 12N and the value of the m-1 pixel on the nth line. That is, it calculates the difference between the values ​​of the adjacent pixels on the left and right, which are peripheral pixels of the data stored in the Nth memory 12N.

[0033] The second comparison circuit (left and right) 30 of the second calculation unit 16 compares the calculation result of the second difference circuit (left and right) 28 with a predetermined threshold A, and outputs the comparison result to the counter 32.

[0034] The counter 32 of the second calculation unit 16 counts the number of comparison results of the first comparison circuit (upper and lower) 26 and the second comparison circuit (left and right) 30 that are equal to or greater than the threshold value A.

[0035] The detection unit 18 includes a third comparison circuit 34 and a fourth comparison circuit 36, and detects abnormalities in the plurality of memories 12, the first memory 12A to the Nth memory 12N.

[0036] The third comparison circuit 34 compares the count values ​​of the counters 32 of the first calculation unit 14 and the second calculation unit 16 and outputs the comparison result to the fourth comparison circuit 36 ​​.

[0037] The fourth comparison circuit 36 ​​compares the comparison result of the third comparison circuit 34 with a predetermined threshold B and outputs a signal indicating normality or abnormality. Specifically, the fourth comparison circuit 36 ​​outputs a signal indicating normality when the comparison result of the third comparison circuit 34 is less than threshold B, and outputs a signal indicating abnormality when the comparison result is equal to or greater than threshold B.

[0038] Next, specific processing performed by each unit of the image processing device 10 according to this embodiment configured as described above will be described.

[0039] First, specific processing performed by the first calculation unit 14 and the second calculation unit 16 will be described. Fig. 3 is a flowchart showing an example of the flow of processing performed by the first calculation unit 14 and the second calculation unit 16. Note that the first calculation unit 14 and the second calculation unit 16 differ only in whether the data source is the first memory 12A or the Nth memory 12N, and therefore the processing of the first calculation unit 14 will be described below as a representative. In addition, in the processing of Fig. 3, for example, the processing of the first calculation unit 14 is started when data is stored in the first memory 12A, and the processing of the second calculation unit 16 is started when data is stored in the Nth memory 12N.

[0040] In step 100, the counter 32 is reset and the process proceeds to step 102.

[0041] In step 102, the first difference circuit (upper and lower) 24 and the second difference circuit (left and right) 28 acquire the data stored in the first memory 12A, and the process proceeds to step 104.

[0042] In step 104, the first difference circuit (upper and lower) 24 and the second difference circuit (left and right) 28 focus on one pixel, and the process proceeds to step 106.

[0043] In step 106, the first difference circuit (upper and lower) 24 calculates the difference between the upper and lower pixels, and the process proceeds to step 108. That is, the difference between the value of m pixels on the nth line and the value of m pixels on the n-1th line is calculated.

[0044] In step 108, the first comparison circuit (upper and lower) 26 determines whether the calculated difference is equal to or greater than a predetermined threshold value A. If the determination is affirmative, the process proceeds to step 110;

[0045] In step 110 , the counter 32 increments the count value by 1 (+1) and then the process proceeds to step 112 .

[0046] In step 112, the second difference circuit (left and right) 28 calculates the difference between the left and right pixels, and the process proceeds to step 114. That is, the difference between the value of the m pixel on the nth line and the value of the m-1 pixel on the nth line is calculated.

[0047] In step 114, the second comparison circuit (left and right) 30 determines whether the calculated difference is equal to or greater than a predetermined threshold value A. If the determination is affirmative, the process proceeds to step 116, and if negative, the process proceeds to step 118.

[0048] In step 116, the counter 32 increments the count value by one (+1) and then the process proceeds to step 118.

[0049] In step 118, the first difference circuit (upper and lower) 24 and the second difference circuit (left and right) 28 determine whether or not the above processing has been performed for all pixels. If the determination is negative, the process proceeds to step 120, and if the determination is positive, the series of processing by the first calculation unit 14 ends.

[0050] In step 120, the first difference circuit (top and bottom) 24 and the second difference circuit (left and right) 28 focus on another pixel, and the process returns to step 106 to repeat the above process.

[0051] Next, a description will be given of specific processing performed by the detection unit 18. Fig. 4 is a flowchart showing an example of the flow of processing performed by the detection unit 18. Note that the processing in Fig. 4 starts, for example, when the processing by the first calculation unit 14 and the second calculation unit 16 is completed.

[0052] In step 200, the third comparison circuit 34 compares the count results of the first calculation unit 14 and the second calculation unit 16, and then proceeds to step 202. That is, the value of the counter 32 of the first calculation unit 14 is compared with the value of the counter 32 of the second calculation unit 16.

[0053] In step 202, the fourth comparison circuit 36 ​​determines whether the comparison result in step 200 is equal to or greater than a predetermined threshold value B. If the determination is negative, the process proceeds to step 204, and if the determination is affirmative, the process proceeds to step 206.

[0054] In step 204, the fourth comparison circuit 36 ​​outputs an abnormality signal indicating a memory failure, and the series of processes of the detection unit 18 ends.

[0055] On the other hand, in step 206, the fourth comparison circuit 36 ​​outputs a normal signal indicating that the memory is normal, and the series of processes ends.

[0056] For example, Figure 5 shows an example of an input video and a video when a memory failure occurs. As shown in Figure 5, when the input video is "A," a memory failure can cause video with different lines to be output, as shown by the dotted line in Video 1 when a memory failure occurs, or the video can be output with a horizontal shift, as shown by the dotted line in Video 2 when a memory failure occurs.

[0057] In this embodiment, the number of neighboring pixel differences equal to or greater than a predetermined threshold A in the data stored in the first memory 12A and the number of neighboring pixel differences equal to or greater than a predetermined threshold A in the data stored in the Nth memory 12N are calculated, and an abnormality is detected based on the respective calculation results. Although the data stored in the Nth memory 12N has undergone multiple steps of image processing on the data stored in the first memory 12A, the image is the same, and therefore the differences between neighboring pixels (above, below, and to the left and right) do not change drastically. Therefore, the number of neighboring pixel differences equal to or greater than the threshold A is calculated for the data stored in the first memory 12A and the Nth memory 12N, and compared, and it is determined whether the comparison result is equal to or greater than a predetermined threshold B, thereby making it possible to detect an abnormality such as the example image at the time of memory failure shown in FIG. 5.

[0058] In the above embodiment, the first calculation unit 14 calculates the number of neighboring pixels whose difference is equal to or greater than the threshold A for the data stored in the first memory 12A, and the second calculation unit calculates the number of neighboring pixels whose difference is equal to or greater than the threshold A for the data stored in the Nth memory 12N, but this is not limiting. For example, the first calculation unit 14 may calculate the number of neighboring pixels whose difference is equal to or greater than the threshold A for the data stored in another memory such as the second memory, and the second calculation unit 16 may calculate the number of neighboring pixels whose difference is equal to or greater than the threshold A for the data stored in another memory such as the N-1th memory.

[0059] In the above embodiment, the difference between the values ​​of the neighboring pixels above and below and to the left and right is calculated, and the number of pixels equal to or greater than the threshold A is counted, but this is not limiting. For example, the difference between at least one of the pixels above and below and to the left and right may be calculated, and the number of pixels equal to or greater than the threshold A may be counted. That is, the embodiment may include at least one of the first difference circuit (up and down) 24 and the first comparison circuit (up and down) 26, and the second difference circuit (left and right) 28 and the second comparison circuit (left and right) 30.

[0060] Furthermore, the processing performed by each unit in each of the above embodiments may be software processing performed by executing a program, or may be hardware processing. For example, the processing may be performed by hardware such as a GPU (Graphics Processing Unit), ASIC (Application Specific Integrated Circuit), or FPGA (Field-Programmable Gate Array). Alternatively, the processing may be a combination of both software and hardware. Furthermore, if the processing is software, the program may be stored in various storage media and distributed.

[0061] Furthermore, the present invention is not limited to the above, and it goes without saying that various modifications can be made without departing from the spirit of the present invention.

[0062] The following additional notes are provided regarding the above-described embodiments.

[0063] (Appendix 1) a first calculation unit that calculates the number of pixels whose difference in value of a neighboring pixel is equal to or greater than a predetermined first threshold value in the data stored in a first memory that is one of a plurality of memories in which data is sequentially stored; a second calculation unit that calculates the number of neighboring pixel values ​​whose difference is equal to or greater than the first threshold value in data stored in the second memory other than the first memory among the plurality of memories; a detection unit that detects an abnormality based on the calculation results of the first calculation unit and the second calculation unit; An image processing device comprising:

[0064] (Appendix 2) The image processing device according to claim 1, wherein the first memory is the first memory among the plurality of memories in which data is stored, and the second memory is the last memory among the plurality of memories in which data is stored.

[0065] (Appendix 3) The image processing device according to claim 1 or 2, wherein the surrounding pixels are adjacent pixels on the top, bottom, left and right sides.

[0066] (Appendix 4) 4. The image processing device according to claim 1, wherein the detection unit detects, as an abnormality, a case where a difference between the calculation results of the first calculation unit and the second calculation unit is equal to or greater than a predetermined second threshold value.

[0067] (Appendix 5) 5. The image processing device according to any one of claims 1 to 4, wherein the plurality of memories are provided corresponding to a plurality of image processes, respectively.

[0068] (Appendix 6) On the computer, Calculating the number of pixels whose difference in value between neighboring pixels is equal to or greater than a predetermined first threshold value in the data stored in a first memory that is one of a plurality of memories in which data is sequentially stored; Calculating the number of neighboring pixel values ​​whose difference is equal to or greater than the first threshold value in the data stored in the second memory other than the first memory among the plurality of memories An image processing program that executes the process of detecting abnormalities based on each calculation result. [Explanation of symbols]

[0069] 10 Image processing device 12 Multiple Memories 12A First Memory 12N Nth memory 14 First calculation section 16 Second calculation unit 18 Detector 22A 1st video processing section 22N Nth Video Processing Unit 24 1st differential circuit (upper and lower) 26 1st comparison circuit (upper and lower) 28 2nd differential circuit (left and right) 30 Second comparison circuit (left and right) 32 counters 34 Third comparison circuit 36 4th comparison circuit

Claims

1. a first calculation unit that calculates the number of neighboring pixel values ​​whose difference is equal to or greater than a predetermined first threshold value in the data stored in a first memory that is one of a plurality of memories in which data is sequentially stored; a second calculation unit that calculates the number of pixels whose difference in value of neighboring pixels is equal to or greater than the first threshold value in data stored in a second memory other than the first memory among the plurality of memories; a detection unit that detects an abnormality based on calculation results of the first calculation unit and the second calculation unit; An image processing device comprising:

2. 2. The image processing device according to claim 1, wherein the first memory is the first memory among the plurality of memories to store data, and the second memory is the last memory among the plurality of memories to store data.

3. The image processing device according to claim 1 , wherein the peripheral pixels are pixels adjacent to each other on the top, bottom, left and right sides.

4. The image processing device according to claim 1 , wherein the detection unit detects an abnormality when a difference between the calculation results of the first calculation unit and the second calculation unit is equal to or greater than a predetermined second threshold value.

5. 2. The image processing apparatus according to claim 1, wherein each of the plurality of memories is provided corresponding to each of a plurality of image processes.

6. On the computer, Calculating the number of pixels whose difference in pixel values ​​between neighboring pixels is equal to or greater than a predetermined first threshold value in the data stored in a first memory that is one of a plurality of memories in which data is sequentially stored; Calculating the number of neighboring pixel values ​​whose difference is equal to or greater than the first threshold value in data stored in a second memory other than the first memory among the plurality of memories; An image processing program that executes the process of detecting abnormalities based on each calculation result.

Images