Circuit device, electronic device, and error detection method

Abstract

The present application provides a circuit device, an electronic device, and an error detection method, which can perform error detection of an icon or the like even if a display position of the icon or the like is unknown. A circuit device (100) includes an image acquisition circuit (130), an error detection circuit (150), and a search circuit (10). The error detection circuit (150) obtains an index indicating a degree of dissimilarity between a foreground image and a background image in a display image (GIN) based on a result of a correlation operation, and performs error detection of the display image (GIN) based on the index. The search circuit (10) searches for a display position of a foreground image in the display image (GIN) based on a reference image (GRF) that is a reference for the foreground image. The error detection circuit (150) performs error detection based on the display position searched by the search circuit (10).

Description

TECHNICAL FIELD

[0001] The present application relates to a circuit device, an electronic apparatus, and an error detection method. BACKGROUND

[0002] In display control of a display device, a processing device such as a CPU transmits image data and a control signal to a display controller, the display controller performs image processing and generation of a timing signal, and a display driver drives a display panel using the image data after the image processing and the timing signal. In such communication, the image data received by the display controller sometimes has data errors due to communication errors and the like. For example, in Patent Documents 1 to 3, a technology is disclosed in which the display controller performs error detection on the image data received from the processing device by CRC. CRC is an abbreviation of Cyclic Redundancy Check. In addition, a method of searching for an object region from an image is known. For example, in Patent Document 4, such a method is disclosed.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2012-35677

[0004] Patent Document 2: Japanese Patent Application Publication No. 2007-101691

[0005] Patent Document 3: Japanese Patent Application Publication No. 2007-72394

[0006] Patent Document 4: Japanese Patent Application Publication No. 2008-15848

[0007] In a case where an icon or the like is displayed on a display image and display errors of the icon or the like are detected, if configured to perform error detection on a display region specified in advance, in order to perform display error detection of the icon or the like, the display position of the icon or the like is limited to the display region specified in advance, and thus there is a problem that the degree of freedom of display design is reduced. The technology disclosed in Patent Document 4 is a technology related only to how to efficiently search for an object region, and Patent Document 4 does not disclose any series of technologies related to detection of an object region and error detection of an image. SUMMARY

[0008] One embodiment of the present application relates to a circuit device including: an image acquisition circuit that acquires a display image; an error detection circuit that calculates a histogram of pixel values of the display image, performs a correlation operation using the histogram, calculates an index indicating a degree of dissimilarity between a foreground image and a background image from a result of the correlation operation, and performs error detection of the display image based on the index, the foreground image being an image of a given region in the display image, the background image corresponding to a background of the foreground image in the display image; and a search circuit that searches for a display position of the foreground image in the display image based on a reference image that is a reference of the foreground image, the error detection circuit performing the error detection based on the display position searched for by the search circuit.

[0009] Further, another embodiment of the present application relates to a circuit device including: an image acquisition circuit that acquires a display image; an error detection circuit that calculates an index indicating a degree of coincidence between a foreground image and a reference image that is a reference of the foreground image, from pixel values of the display image and pixel values of the reference image or from pixel values of an edge image of the display image and pixel values of an edge image of the reference image, and performs error detection of the display image based on the index, the foreground image being an image of a given region in the display image; and a search circuit that searches for a display position of the foreground image in the display image based on the reference image, the error detection circuit performing the error detection based on the display position searched for by the search circuit.

[0010] Further, another embodiment of the present application relates to an electronic device including the circuit device described in any of the above.

[0011] Further, another embodiment of the present application relates to an error detection method in which a display position of a foreground image in a display image is searched for based on a reference image that is a reference of the foreground image, the foreground image being an image of a given region in the display image, a histogram of pixel values of the display image is calculated based on the searched-for display position, a correlation operation using the histogram is performed, an index indicating a degree of dissimilarity between the foreground image and a background image corresponding to a background of the foreground image is calculated from a result of the correlation operation, and error detection of the display image is performed based on the index.

[0012] Further, another aspect of the present application relates to an error detection method in which a display position of a foreground image in a display image is searched for from a reference image that is a reference of the foreground image, the foreground image being an image of a given region in the display image, an index indicating a degree of coincidence of the foreground image with the reference image is calculated from the searched display position, a pixel value of the display image, and a pixel value of the reference image, or from the searched display position, a pixel value of an edge image of the display image, and a pixel value of an edge image of the reference image, and error detection of the display image is performed based on the index. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is a configuration example of a circuit device.

[0014] Figure 2 is a detailed configuration example of a search circuit.

[0015] Figure 3 is an example of a display image.

[0016] Figure 4 is an example of a reference image.

[0017] Figure 5 is a flow of processing performed by an FFT processing section.

[0018] Figure 6 is an example of cross-correlation of a display image and a reference image.

[0019] Figure 7 is a diagram explaining an operation of an error detection circuit.

[0020] Figure 8 is a detailed configuration example of a circuit device.

[0021] Figure 9 is a flow chart showing a processing flow of error detection processing.

[0022] Figure 10 is a histogram of each channel of YCbCr in an error detection region.

[0023] Figure 11 is an autocorrelation value obtained by performing autocorrelation operation on a histogram.

[0024] Figure 12 is a first example of a display image.

[0025] Figure 13 is an example of a histogram in a case where a foreground has multiple tones.

[0026] Figure 14 is an example of a cross-correlation value of a histogram in a case where a foreground has multiple tones.

[0027] Figure 15 is an example of a reference image.

[0028] Figure 16 is an averaged image of a reference image.

[0029] Figure 17 is a second example of a display image.

[0030] Figure 18 is a third example of a display image.

[0031] Figure 19 is a fourth example of a display image.

[0032] Figure 20 is a first example of a reference image, a display image, and a mask image.

[0033] Figure 21 is an example of an edge value calculated from a reference image and a display image according to the first example.

[0034] Figure 22 is a second example of a reference image, a display image, and a mask image.

[0035] Figure 23 is an example of an edge value calculated from a reference image and a display image according to the second example.

[0036] Figure 24 is a structural example of an electronic device.

[0037] Label Explanation

[0038] 10: search circuit; 11: first edge detection section; 12: division section; 13: input line buffer; 14: second edge detection section; 15: FFT processing section; 16: output line buffer; 17: position detection section; 100: circuit device; 110: interface; 120: preprocessing circuit; 130: image acquisition circuit; 140: interface; 150: error detection circuit; 160: CRC circuit; 170: register circuit; 180: icon processing circuit; 190: interface; 195: memory; 200: processing device; 300: electronic device; 310: processing device; 320: circuit device; 330: display driver; 340: display panel; 350: storage device; 360: operation device; 370: communication device; Al: error detection region; A2: icon; CHK: separate region; ECIB: edge image; ERQ: error detection result; fcor: cross correlation; GIN: display image; GRF: reference image; GZN: foreground image; HPS: display position; MSB: mask image; RIA: reference image; SAR: error detection region; SEQ: search result; SRef: averaged image; TH: detection threshold. DETAILED DESCRIPTION

[0039] Hereinafter, a preferred embodiment of the present application will be described in detail. In addition, the present embodiment described below does not unduly limit the content described in the claims, and all the structures described in the present embodiment are not necessarily essential structural elements.

[0040] 1. Circuit device

[0041] Figure 1 is a structural example of the circuit device 100 of the present embodiment. The circuit device 100 includes the search circuit 10 and the error detection circuit 150. The circuit device 100 is realized, for example, by an integrated circuit device in which circuit elements are integrated on a semiconductor substrate.

[0042] The image acquisition circuit 130 acquires the display image GIN, and outputs the display image GIN to the search circuit 10 and the error detection circuit 150. The display image GIN is an image in which a foreground image is synthesized with a background image. In addition, the image acquisition circuit 130 acquires the reference image GRF which is a reference of the foreground image, and outputs the reference image GRF to the search circuit 10 and the error detection circuit 150. For example, in the display image GIN shown in Figure 3 In the display image GIN shown in FIG. 1, the meter image is a background image, and the icon image is superimposed on the meter image as a foreground image GZN. The foreground image GZN is an image in which the icon portion in the reference image GRF shown in FIG. 2 is extracted. Figure 4 In addition, the entire reference image GRF can be superimposed on the meter image as the foreground image GZN.

[0043] The image acquisition circuit 130 can be, for example, an interface that receives the display image GIN and the reference image GRF from the outside of the circuit device 100, or can be a memory access circuit that reads out the display image GIN and the reference image GRF stored in a memory, or can be a superimposition circuit that superimposes the foreground image on the background image. In addition, in the detailed configuration example described later in Figure 8 The image acquisition circuit 130 is a superimposition circuit, but this is only an example. In Figure 1 The image acquisition circuit 130 acquires the display image GIN and the reference image GRF, but is not limited thereto, and the reference image GRF can be input to the search circuit 10 and the error detection circuit 150 from a memory or the like without passing through the image acquisition circuit 130. Further, in Figure 1 The display image GIN and the reference image GRF are input to the error detection circuit 150, but in the case where the error detection circuit 150 does not use the reference image GRF, the reference image GRF can not be input to the error detection circuit 150. For example, in the case where the error detection circuit 150 calculates the visibility index described later, the reference image GRF is not used at times.

[0044] The search circuit 10 searches for the display position of the foreground image in the display image GIN on the basis of the reference image GRF. Specifically, the search circuit 10 calculates the product of the display image GIN and the reference image GRF in the frequency domain using a 2-dimensional FFT, and performs IFFT on the result of the product, thereby calculating the cross-correlation of the display image GIN and the reference image GRF. FFT is an abbreviation of Fast Fourier Transform, and IFFT is an abbreviation of Inverse Fast Fourier Transform. The search circuit 10 compares the cross-correlation with a detection threshold, and detects the position where the correlation value is equal to or higher than the detection threshold as the display position of the foreground image when such a position exists. When no position where the correlation value is equal to or higher than the detection threshold exists, the search circuit 10 determines that the foreground image is not displayed in the display image GIN. The detection threshold is a threshold value for determining whether or not the foreground image is displayed in the display image GIN. The detection threshold can be stored in advance in a memory or a register or the like, or can be set in a memory or a register or the like from the outside of the circuit device 100. The search circuit 10 outputs the search result SEQ of the display position to the error detection circuit 150.

[0045] The error detection circuit 150 performs error detection of the display image GIN based on the display position searched by the search circuit 10, and outputs an error detection result ERQ. Specifically, the error detection circuit 150 sets an error detection region that contains the foreground image and is narrower than the display image GIN based on the display position searched by the search circuit 10, performs image analysis of the error detection region, and thereby performs error detection. The error detection circuit 150 calculates an index indicating whether the foreground image is properly displayed in the error detection region, and performs error detection based on the index. The index is a visibility index for evaluating the visibility of the foreground image, or a shape index for evaluating the similarity of the shape of the foreground image to the reference image. These indexes will be described later. As the error detection result ERQ, a result of comparing the index to a threshold value, or the index itself, or the like is output.

[0046] In the above embodiment, the circuit device 100 includes the image acquisition circuit 130 that acquires the display image GIN, the error detection circuit 150 that performs error detection of the display image GIN, and the search circuit 10. The search circuit 10 searches for the display position of the foreground image in the display image GIN based on the reference image GRF that is a reference for the foreground image. The error detection circuit 150 performs error detection based on the display position searched by the search circuit 10.

[0047] According to the present embodiment, the error detection circuit 150 is able to perform error detection based on the display position searched by the search circuit 10, and thus is able to perform error detection even when the display position of the foreground image is not decided in advance. Thus, it is possible to display an icon or the like at a free position, and it is possible to improve the degree of freedom of display design. The series of processes for performing such error detection are not disclosed at all in the above-mentioned Patent Literature 1 and the like.

[0048] Further, in the present embodiment, the error detection circuit 150 performs error detection when the search circuit 10 detects the display position in the above-mentioned search, and does not perform error detection when the search circuit 10 does not detect the display position in the above-mentioned search. That is, the error detection circuit 150 performs error detection when the search circuit 10 detects a position where the correlation value is equal to or higher than the detection threshold value, and does not perform error detection when the search circuit 10 does not detect a position where the correlation value is equal to or higher than the detection threshold value.

[0049] According to the present embodiment, error detection is performed when the foreground image is included in the display image GIN, and error detection is not performed when the foreground image is not included in the display image GIN. For example, in a case where a frame image of a moving image is input as the display image GIN, it is assumed that there are a time when an icon or the like is displayed in the display image GIN and a time when the icon or the like is not displayed. According to the present embodiment, when the icon or the like is displayed in such a moving image, it is possible to check whether the icon or the like is properly displayed. In addition, a technique in which such image search and error detection are performed as a series of processes is not disclosed at all in the above-described Patent Literature 1 and the like.

[0050] 2. Search circuit and error detection circuit

[0051] Figure 2 is a detailed configuration example of the search circuit 10. The search circuit 10 includes a first edge detection section 11, a division section 12, an input line buffer 13, a second edge detection section 14, an FFT processing section 15, an output line buffer 16, and a position detection section 17. In addition, the following description is an example in which the cross-correlation between a display image and a reference image is indirectly calculated by calculating the cross-correlation between an edge image of the display image and an edge image of the reference image, but is not limited thereto, and the cross-correlation between the display image and the reference image can be directly calculated.

[0052] The first edge detection section 11 extracts an edge image of the display image GIN by performing filter processing on the display image GIN. The filter processing is, for example, Soble filter processing, but can be any filter processing that enables edge extraction. Figure 3 An example of the display image GIN is shown. In this example, an icon as a foreground image GZN is superimposed on an instrument image as a background image. An edge image extracted from the display image GIN includes an edge component of the background image and an edge component of the foreground image GZN.

[0053] The division section 12 divides the edge image of the display image GIN into a plurality of individual regions CHK. In Figure 3 an example in which the individual regions CHK divided into 3 x 8 are shown. In addition, in Figure 3 , the individual regions CHK are shown with respect to the display image GIN, but here, the edge image is divided. Specifically, the division section 12 divides the edge image into a plurality of belt-shaped regions, and divides each of the belt-shaped regions into individual regions. In Figure 3 , by dividing 3 rows of belt-shaped regions into 8 columns, the individual regions CHK divided into 3 x 8 are obtained.

[0054] The second edge detection section 14 extracts an edge image of the reference image GRF by performing a filter process on the reference image GRF. The filter process is, for example, a Sobel filter process, but can be any filter process that enables edge extraction. Figure 4 An example of the reference image GRF is shown. The edge image extracted from the reference image GRF contains an edge image of the icon portion. If the foreground image GZN in the display image GIN is displayed correctly, the edge image of the foreground image GZN is similar to the edge image of the reference image GRF.

[0055] The input line buffer 13 stores the edge image of the individual region CHK and the edge image of the reference image GRF. The FFT processing section 15 calculates the cross-correlation of the edge image of the individual region CHK and the edge image of the reference image GRF using FFT processing and IFFT processing. The output line buffer 16 stores the cross-correlation calculated for each individual region CHK. Furthermore, the input line buffer 13 and the output line buffer 16 temporarily store intermediate generation data of the FFT processing section 15.

[0056] Figure 5 A flow of the process performed by the FFT processing section 15 is shown. The FFT processing section 15 performs the following line FFT on each of the edge image of the individual region CHK and the edge image of the reference image GRF. The FFT processing section 15 reads out the edge image from the input line buffer 13 line by line, performs FFT processing on each line, and writes the result line by line to the output line buffer 16. The FFT result of each line is composed of data of the same number of points, and the result of the line FFT becomes data of points in a matrix shape as a whole. In addition, the line FFT outputs a plurality of spatial frequency components corresponding to a plurality of spatial frequencies as a one-dimensional FFT, but each of the spatial frequency components is referred to as a point here.

[0057] Next, the FFT processing section 15 performs the following column FFT on the result of the line FFT of the edge image of the individual region CHK and the result of the line FFT of the edge image of the reference image GRF. The FFT processing section 15 reads out the result of the line FFT column by column from the output line buffer 16, performs FFT processing on each column, and writes the result column by column into the input line buffer 13. The FFT result of each column is constituted by data of the same number of points, and the result of the 2-dimensional FFT becomes data of points in a matrix shape as a whole. The above line FFT and column FFT are combined as a 2-dimensional FFT. The 2-dimensional FFT result of the edge image of the individual region CHK and the 2-dimensional FFT result of the edge image of the reference image GRF are data of the same number of lines and columns. For example, if the individual region CHK is L x L pixels, the reference image GRF is M x M pixels, and the 2-dimensional FFT result is nc lines nb columns, nc, nb are set in a manner that nc > L + M - 1, nb > L + M - 1. Usually, nc = nb, but nc and nb can also be different. L, M, nc, nb are integers of 2 or more.

[0058] Next, the FFT processing section 15 multiplies the result of the 2-dimensional FFT of the edge image of the individual region CHK by the result of the 2-dimensional FFT of the edge image of the reference image GRF, and performs line IFFT on the product. Specifically, the FFT processing section 15 reads out the result of the 2-dimensional FFT of the edge image of the individual region CHK and the result of the 2-dimensional FFT of the edge image of the reference image GRF from the input line buffer 13 line by line, and multiplies them by each point. That is, when i is 1, 2, ···, nc, j is 1, 2, ···, nb, the data of i line j column in the result of the 2-dimensional FFT of the edge image of the individual region CHK is multiplied by the data of i line j column in the result of the 2-dimensional FFT of the edge image of the reference image GRF. The FFT processing section 15 performs IFFT processing on the product of each 1 line, and writes the result line by line into the output line buffer 16.

[0059] Next, the FFT processing section 15 performs the following column IFFT on the result of the line IFFT. The FFT processing section 15 reads out the result of the line IFFT column by column from the output line buffer 16, performs IFFT processing on each column, and writes the result column by column into the input line buffer 13. The above line IFFT and column IFFT are combined as a 2-dimensional IFFT. The result of the 2-dimensional IFFT becomes the cross correlation of the edge image of the individual region CHK and the edge image of the reference image GRF, and thus the cross correlation is obtained for each individual region CHK. The cross correlation of each of the individual regions CHK is transferred from the input line buffer 13 to the output line buffer 16, and output from the output line buffer 16 to the position detection section 17.

[0060] The position detection section 17 generates a cross-correlation of the display image GIN and the reference image GRF by synthesizing the cross-correlations of each individual area CHK. Figure 6 An example of the cross-correlation fcor of the display image GIN and the reference image GRF is shown. x is a coordinate in the horizontal direction in the display image GIN, and y is a coordinate in the vertical direction in the display image GIN. Figure 3 Figure 3 The cross-correlation fcor has a correlation value at each position (x, y). The position detection section 17 compares the cross-correlation fcor and the detection threshold value TH, and judges whether there is a position (x, y) for which fcor ≥ TH. In the example shown in FIG. 9, there is a position (x', y') for which fcor ≥ TH. Figure 6

[0061] Figure 7 is a diagram illustrating the operation of the error detection circuit 150. HPS is a display position detected by the search circuit 10. Here, an example in which the center of an icon corresponding to the center of the reference image GRF is detected as the display position is shown. The error detection circuit 150 sets an error detection area SAR including the foreground image GZN in accordance with the display position HPS. In the example shown in FIG. 10, the error detection circuit 150 sets a rectangular area centered on the display position HPS and capable of including the foreground image GZN as the error detection area SAR. However, the method of setting the error detection area SAR is not limited to this, and the error detection area SAR can be an area including the foreground image GZN and narrower than the display image GIN. In addition, the display position HPS can not be the center of an icon. The error detection circuit 150 calculates a visibility index or a shape index described later in accordance with the image of the error detection area SAR set, and judges whether the foreground image GZN is displayed properly using the visibility index or the shape index. Figure 7

[0062] In the above embodiment, the search circuit 10 calculates a cross-correlation of the display image GIN and the reference image GRF, and searches for whether there is a correlation value exceeding a detection threshold value TH in the cross-correlation. When there is a correlation value exceeding the detection threshold value, the search circuit 10 detects a position (x', y') corresponding to the correlation value exceeding the detection threshold value as a display position, and when there is no correlation value exceeding the detection threshold value, the search circuit 10 does not detect a display position.

[0063] ​​​According to the present embodiment, the search circuit 10 can detect a position at which the correlation of the display image GIN and the reference image GRF is large as a display position of the foreground image GZN. Also, it is possible to realize a series of actions in which the error detection circuit 150 performs error detection when the search circuit 10 detects the display position and the error detection circuit 150 does not perform error detection when the search circuit 10 does not detect the display position.

[0064] Further, in the present embodiment, the search circuit 10 divides the display image GIN into a plurality of band-shaped regions, divides each of the plurality of band-shaped regions into a plurality of individual regions CHK, and searches each of the plurality of individual regions CHK. The search circuit 10 processes the search for the plurality of individual regions in parallel.

[0065] According to the present embodiment, by dividing the display image GIN into a plurality of individual regions CHK, calculating the cross-correlation with the reference image GRF for each individual region CHK, and synthesizing the cross-correlation calculated for each individual region CHK, it is possible to obtain the cross-correlation of the display image GIN and the reference image GRF. Also, by calculating the cross-correlation in parallel for each individual region CHK, it is possible to improve the processing speed. For example, in the case of performing error detection on a moving image, it is desirable that the delay until the end of error detection is small, and by improving the processing speed of the cross-correlation, it is possible to speed up the error detection.

[0066] Further, in the present embodiment, the search circuit 10 uses Fourier transform and inverse Fourier transform to calculate the cross-correlation of the display image GIN and the reference image GRF, thereby performing the search. In the present embodiment, FFT is used as the Fourier transform and IFFT is used as the inverse Fourier transform, but the algorithm of the Fourier transform and the inverse Fourier transform is not limited thereto.

[0067] According to the present embodiment, the display image GIN and the reference image GRF after the Fourier transform are multiplied in the frequency domain, and the result is inverse Fourier transformed, thereby obtaining the cross-correlation. Also, by performing the Fourier transform and the inverse Fourier transform for each individual region CHK, it is possible to improve the processing speed of the Fourier transform and the inverse Fourier transform.

[0068] 3. Detailed configuration example of circuit device

[0069] Figure 8 is a configuration example of the circuit device of the present embodiment. Here, a case in which the circuit device 100 is a display controller is described as an example, but the application target of the search circuit 10 and the error detection circuit 150 is not limited to the display controller. Also, the display controller is also called TCON, that is, Timing Controller.

[0070] The circuit device 100 includes an interface 110, a preprocessing circuit 120, an image acquisition circuit 130, an interface 140, an error detection circuit 150, a CRC circuit 160, a register circuit 170, an icon processing circuit 180, an interface 190, a memory 195. Hereinafter, a display image will also be referred to as an analysis image, and an error detection region will also be referred to as a region of interest or ROI. ROI is an abbreviation of Region Of Interest.

[0071] The interface 110 receives, for example, image data transmitted from a processing device 200 or the like to the circuit device 100, and converts the received image data into a form used inside the circuit device 100. The interface 110 is, for example, OpenLDI, and converts a serial signal received in LVDS into a parallel signal of RGB. The processing device 200 is, for example, an MCU, a CPU, or the like. OpenLDI is an abbreviation of Open LVDS Display Interface, LVDS is an abbreviation of Low Voltage Differential Signaling, and MCU is an abbreviation of Micro Control Unit, and CPU is an abbreviation of Central Processing Unit.

[0072] The preprocessing circuit 120 performs various image processing on the image data input from the interface 110. For example, the preprocessing circuit 120 performs gamma correction, FRC, white balance processing, or the like. FRC is an abbreviation of Frame Rate Control. For example, a one-dimensional lookup table for each of the R channel, the G channel, and the B channel is stored in the memory 195 or the register circuit 170 or a nonvolatile memory not shown, and each channel is subjected to gamma correction using the lookup table. In the FRC, processing of expressing intermediate gray levels in a pseudo manner by switching the gray levels between frames is performed. In the white balance processing, for example, a one-dimensional lookup table for adjusting the white balance is stored in the memory 195 or the register circuit 170 or a nonvolatile memory not shown, and the RGB channels are adjusted using the lookup table.

[0073] The icon processing circuit 180 performs generation or acquisition of an icon image. The icon image corresponds to the reference image GRF described above. For example, a mask image of an icon is stored in the memory 195, and the icon image is generated by converting the mask image into an RGB image. The mask image is a k-bit image in which each pixel has k-bit data. k is an integer of 1 or more.2 kAn index color table is stored in the memory 195 or the register circuit 170 or a nonvolatile memory not shown, and according to this color table, k-bit data is converted into RGB data. For example, in the case of k = 2, the color table is a lookup table that corresponds 4 colors to 2-bit indexes. Alternatively, in the case of k = 1, the color table is a lookup table that corresponds 2 colors to 1-bit indexes, a specific color corresponding to "0" of a pixel indicating a background, and the other color corresponding to "1" of a pixel indicating a foreground. The other color is a specific color different from the background. The memory 195 is, for example, a RAM such as an SRAM.

[0074] The image inputted from the preprocessing circuit 120 to the image acquisition circuit 130 is called an input image. The image acquisition circuit 130 composites the icon image on the input image by superimposing the icon image on the input image, and outputs the composite image as a display image to the interface 140. The icon image that is composited with the input image corresponds to the above-mentioned foreground image GZN. In addition, the portion of the input image that remains without superimposing the icon image corresponds to the above-mentioned background image. For example, the image acquisition circuit 130 superimposes the icon image on the input image in such a manner that the icon image completely masks the background. Alternatively, the icon image and the background can be mixed in accordance with a given mixing ratio. The position at which the icon image is superimposed on the input image is set, for example, in the register circuit 170 or the memory 195 or a nonvolatile memory not shown.

[0075] In the above, the case where the image acquisition circuit 130 superimposes the icon image as a foreground image on the input image as a background image is described, but is not limited thereto. That is, the input image on which the icon image as a foreground image has been superimposed can be inputted to the image acquisition circuit 130. For example, as described above, the interface 110 receives image data from the processing device 200 or the like, and this image data can also be image data on which an icon image has been superimposed. In any case, the display image outputted by the image acquisition circuit 130 is an image that contains the icon image as a foreground image.

[0076] The search circuit 10 searches for the display position of the foreground image from the display image. The search method is as described above. The error detection circuit 150 judges whether the icon image is appropriately displayed in the error detection region based on the image of the error detection region. The details of the error detection method are described later.

[0077] The interface 140 outputs the display image to the outside of the circuit device 100. For example, the interface 140 outputs the display image to a display driver that drives a display panel. For example, the interface 140 is an interface of LVDS, and converts a parallel signal of RGB from the image acquisition circuit 130 into a serial signal of LVDS. When the error detection circuit 150 detects an error, the interface 140 stops outputting the display image. Alternatively, the display image can be output together with error information detected by the error detection circuit 150, and a display driver that receives the error information performs an action based on the error information. The error information is, for example, an error determination flag or an index. The action based on the error information is a stop of display or the like.

[0078] The interface 190 communicates setting information, control information, and the like between the circuit device 100 and the processing device 200. For example, the interface 190 is a serial communication interface of an SPI system or an I2C system or the like. SPI is an abbreviation of Serial Peripheral Interface. The setting information and the control information from the processing device 200 are written, for example, into the register circuit 170, and the circuit device 100 performs an action corresponding to the setting information and the control information.

[0079] The CRC circuit 160 performs error detection based on CRC on the image data received by the interface 110. That is, a reference CRC value input from the processing device 200 via the interface 190 and a CRC value calculated from the image data received by the interface 110 are compared, and it is detected whether they are identical.

[0080] In addition, the pre-processing circuit 120, the image acquisition circuit 130, the error detection circuit 150, the CRC circuit 160, and the icon processing circuit 180 are logical circuits. These logical circuits can be configured as individual circuits, or can be configured as integrated circuits by automatic configuration wiring or the like. Furthermore, part or all of these logical circuits can be implemented by a processor such as a DSP. DSP is an abbreviation of Digital Signal Processor. In this case, a program or an instruction set that describes the function of each circuit is stored in a memory, and the function of each circuit is implemented by the processor executing the program or the instruction set.

[0081] 4. Error detection processing

[0082] Next, the error detection processing performed by the error detection circuit 150 will be described.

[0083] In an image processing system that displays content on a display, it is sometimes necessary to confirm whether a prescribed region of an image is consistent with the original intention. For example, consider a case in which an important image is displayed in a cluster display of a system for a car. At this time, it is necessary to display prescribed important information by means of a visible image that overlaps the existing content displayed on the screen. Below, several methods for detecting whether an image is correctly displayed are described. Detection is performed by analyzing an error detection region and deriving several main indicators that indicate the degree to which the region is correctly displayed.

[0084] Below, a method and a concept for verifying a display image with respect to a reference are used. This is achieved by performing an operation on the degree of agreement of a display image with respect to a reference image. Below, an example in which the method of the present application is applied to a color image is described, but the method of the present application can also be applied to a grayscale image or a binary image.

[0085] Figure 9 is a flowchart that shows the processing flow of error detection processing. In steps S1, S2, in the error detection processing, an icon image is overlaid on an input image to obtain a display image. Next, in step S4, the error detection processing is performed by comparing the reference image obtained in step S3 and the display image obtained in step S2, and an indicator is derived in step S5. In addition, the reference image is not necessarily obtained. For example, sometimes the reference image is not used when the visibility indicator is derived, and the indicator is calculated using only the display image.

[0086] In the error detection processing, the appropriateness of the display image in the error detection region is confirmed by comparing the display image with the reference image. In the above comparison, color shifts, changes in brightness, and changes due to scaling or a prescribed intentional image conversion are not detected as errors, but other important errors such as errors due to deformation based on an unintended rotation, cropping that makes it impossible for the user to recognize the image, or noise are detected.

[0087] Therefore, two indicators, the visibility indicator and the shape indicator, are used. The visibility indicator is a numerical value that indicates the degree to which the image of the error detection region does not blend into the background and can be visually confirmed. In addition, the visibility indicator is also referred to as a first indicator, and the shape indicator is also referred to as a second indicator.

[0088] As described above, the error detection processing is applied to a color image. It is also possible to apply it to a black-and-white or grayscale image by using one channel for grayscale or using only two values in one channel for a binary image. Therefore, the pixels of the error detection region in the display image are converted from RGB format to YCbCr format. However, the method of the present application can also be applied to other color spaces such as Lab or Hsv.

[0089] 4.1. First operation method for deriving a visibility indicator

[0090] Figure 10 is a histogram of each channel of YCbCr in the error detection region. In addition, Figure 11 is an autocorrelation value obtained by autocorrelation operation on the histogram.

[0091] As Figure 10 shown, for each channel of the YCbCr image, a histogram is obtained using n bins. For example, 256 bins can be used to generate a histogram having different bin groups.

[0092] The histogram counts the number of times a specific value is generated in the error detection region. That is, for each channel of the YCbCr image, the number of pixels having a value shown by each bin within the error detection region is counted. Then, the histogram is normalized to a value between 0 and a. The value "a" can be selected in consideration of ease of installation, and for example, 1 or 255 or the like can be selected. In Figure 10 , a = 1. Then, the histogram of each channel is autocorrelated with itself, and the autocorrelation signal is used for analysis thereafter. As Figure 11 shown, the autocorrelation signal is normalized so that the peak at zero delay is 1 or a predetermined value.

[0093] The autocorrelation value is obtained by the following formula (1). f and g represent functions on which the correlation operation is performed, and in the case of autocorrelation, f = g. The function here is the signal of the histogram. f * g represents the correlation operation of the function f and the function g. f * represents the complex conjugate of the function f, and in the present embodiment, f * = f. m represents the number of bins of the histogram. n represents the delay, and in the present embodiment, n is an integer of -255 to +255. The delay is also referred to as lag. Figure 11

[0094] [Formula 1]

[0095]

[0096] In addition, in the histogram of Figure 10 , 256 bins are normalized between 0 and 1, and therefore the horizontal axis is 0 to 1. Regarding Figure 11 the correlation value, since the correlation value is obtained by changing the delay for each bin, the horizontal axis is -(256-1) to +(256-1).

[0097] As Figure 11 ​As shown, when two colors of an image exist in the error detection region, the sideband is obtained through autocorrelation. The distance of the delay from the zero delay that produces the aforementioned peak represents the contrast between colors. Contrast includes luminance contrast and color contrast. The human eye can identify image features through contrast, therefore all peaks of the three channels are checked. Figure 11 In the diagram, the Y channel is represented by a dashed line, the Cb channel by a thin solid line, and the Cr channel by a thick solid line. The inspection is performed by setting a threshold for peak retrieval to prevent the picking up of noise in the autocorrelation signal. For example, the minimum peak threshold is set to 0.05. Peaks larger than the threshold are retrieved from the signal, and local maxima are identified.

[0098] Additionally, to avoid signal peaks within the frequency band, the minimum distance between consecutive peaks can be set to a specified value. These thresholds are adjustable and can be selected according to the application.

[0099] To determine the primary metric for whether a recognizable image appears on a background with two or more colors, after identifying all peaks of the autocorrelation signal exceeding the noise threshold across all channels, the maximum distance that produces these peaks is calculated. The maximum distance is the maximum delay. The maximum delay among the peak-producing delays in the three channels is selected as the metric for visibility.

[0100] exist Figure 11 In the correlation graph shown, peaks are represented by circles. In the example illustrated, the Cr channel shows the largest separation, with a distance of 184. This value is normalized to the maximum conceivable delay. For example, the maximum conceivable delay is the number of intervals in the histogram, 256. Therefore, the index value is 184 / 255 = 0.722. Figure 12 In the image shown, the above index values ​​are represented as Vis parameters. An example is provided to illustrate the above operation.

[0101] Figure 12 This is the first example of an image display. A1 is the error detection area, and A2 is the icon. Additionally, the dashed lines representing the error detection area are not actually drawn in the displayed image. For example, in... Figure 12 The icon A2, represented in black, has a red interior. Figure 12 The background represented by white in the middle is green.

[0102] exist Figure 12 In the image, there are groups of red and green pixels within the error detection area, therefore... Figure 10In the histogram shown, two peaks are generated in each channel of YCbCr. For example, in the Cr channel, peaks are generated in the intervals Ba and Bb. The distance between these two peaks represents the contrast between the color of the icon as the foreground and the color of the background, and the greater the distance means the more different the colors of the foreground and the background. The distance between the two peaks in the histogram becomes the index of visibility in the image. Figure 11 The distance of the delay at which a peak is generated in the autocorrelation value shown. In the image shown, Figure 12 In the image shown, the foreground is red and the background is green, so in the histogram shown, Figure 10 In the histogram shown, the distance between the two peaks of the Cr channel is the maximum distance, which is |Ba-Bb| x 255. This is detected as the foreground and the background in the image shown. Figure 11 The maximum distance at which a peak is generated in the autocorrelation value shown, and the index value after normalization is |Ba-Bb|. Therefore, the greater the contrast between the color of the foreground and the color of the background, the greater the index value of visibility.

[0103] The error detection circuit 150 performs error detection based on the visibility index calculated as described above. For example, the visibility index is compared with a given threshold value, and in the case where the visibility index is smaller than the given threshold value, it is determined to be an error. Alternatively, the visibility index can be output to the outside of the circuit device 100 as an error detection result.

[0104] 4.2. 2nd to 4th calculation methods of the visibility index

[0105] In the 2nd calculation method, the visibility index is calculated using cross-correlation operation.

[0106] In the 1st calculation method, autocorrelation operation is used to check the visibility of the reference within the error detection region. The reference image in this case does not contain information such as color related to the background image. Therefore, only the display image is analyzed to investigate whether the composite image contains two or more colors.

[0107] In the 2nd calculation method, the case where the reference image contains the entire information can be assumed. For example, the case where the source image is changed by display processing can be assumed. In this case, the histogram of the reference image is generated by the same method as the histogram of the display image, and cross-correlation operation between the histogram signals of the reference image and the display image can be performed instead of autocorrelation operation. In mathematics, autocorrelation operation is cross-correlation operation of the signal itself. Therefore, cross-correlation operation or autocorrelation operation can be performed by changing only the input signal to the correlation operation. That is, in the above equation (1), the histogram of the reference image is set to one of f and g, and the histogram of the display image is set to the other of f and g.

[0108] In the case of the cross-correlation operation, instead of finding the distance from the center of the peak, it is investigated whether or not a peak exceeding a prescribed threshold exists in the cross-correlation signal. In the case where such a peak exists, if the distribution of the pixels is considered, the reference image and the display image are in good agreement. Thus, the display image can be subjected to the first-level error detection. This parameter does not indicate the spatial correlation, but only the pixel distribution correlation. The index in this case can be the peak itself, not the distance from the center of the peak as in the case of the autocorrelation operation.

[0109] Figure 13 is an example of a histogram in the case where the foreground of an icon or the like is multi tone of two or more colors. Figure 14 is an example of a histogram of Figure 13 is an example of a cross-correlation value of a histogram of

[0110] As shown in Figure 13 , three or more peaks are generated in the histograms of the display image and the reference image. It is assumed that the peaks of the histogram of the display image are shifted from the peaks of the histogram of the reference image by Bn. In this case, as shown in Figure 14 , a large peak appears at a delay of Bn in the cross-correlation value. In the case where the peak value of this peak is larger than a threshold value Thr, for example, this peak value is adopted as the index value of the visibility.

[0111] In the third operation method, the ratio of the contrast of the foreground to the background is found as the index value of the visibility.

[0112] In the first operation method, the difference |Ba-Bb| of the interval Ba, Bb in which a peak is generated in the histogram of the Cr channel is used as the index value of the visibility.

[0113] In the third operation method, the ratio of the contrast |Ba-Bb| / Ba or |Ba-Bb| / Bb is found, and is set as the index value of the visibility. Or, in the case where a reference image like that of the second operation method is used, C1 = |Ba-Bb| in the display image and C2 = |Ba-Bb| in the reference image are found, and the ratio of the contrast C1 / C2 or C2 / C1 is found as the index value of the visibility.

[0114] In the fourth operation method, a multi-dimensional histogram is generated, and the visibility index is found.

[0115] In the first operation method, the one-dimensional histogram of each channel is used in the analysis of the visibility.

[0116] In the fourth operation method, a multi-dimensional histogram is generated from the signals of the plurality of channels, and a multi-dimensional autocorrelation operation or a multi-dimensional cross-correlation operation is performed on the multi-dimensional histogram to calculate the visibility index. Thus, it is possible to more accurately simulate the contrast detection by the human eye. By using a 3D color histogram, it is sometimes possible to achieve better performance.

[0117] According to the above embodiment, the error detection circuit 150 calculates a histogram of pixel values of the display image and performs a correlation operation using the histogram. In the present embodiment, Figure 10 The histograms of the respective channels of YCbCr shown in FIG. 6 correspond to histograms of pixel values, and the correlation operation described in Figure 11 is performed on the histograms. The error detection circuit 150 calculates an index indicating the degree of dissimilarity between an image of a given region in the display image, i.e., a foreground image, and a background image corresponding to the background of the foreground image in the display image, based on the result of the correlation operation, and performs error detection based on the index. The index here is a visibility index.

[0118] Thus, error detection of the display image can be performed based on an index indicating the degree of dissimilarity between a foreground image and a background image of the display image, instead of bit-based error detection such as CRC. In a case where the degree of dissimilarity of the foreground image with respect to the background image is high, it is considered that the foreground image is highly likely to be visually distinguished from the background image, and thus the visibility of the foreground image is high. That is, according to the present method, it is possible to determine that there is an error in a case where the visibility of the foreground image is low. For example, an icon or the like for warning a user is displayed in a meter panel or the like mounted on a vehicle. According to the present embodiment, such an icon does not stop being displayed due to a one-bit error or the like, and can be displayed as much as possible while ensuring the visibility, thereby warning the user.

[0119] Here, in Figure 8 , the image acquisition circuit 130 is an OSD, but is not limited thereto, and the image acquisition circuit 130 can be a circuit that acquires an arbitrary display image. The OSD is an abbreviation of On-Screen Display. Details are described in "5. Modified Example". In addition, the display image refers to an image generated or the like for causing a display to perform display. In the above embodiment, the display image is an image after rendering by the OSD, but is not limited thereto, and can be, for example, an image generated through certain image processing, or an image received through communication, or an image read from a memory, or the like.

[0120] Further, the error detection is outputting an index-based error detection result, for example, determining whether or not there is an error in the display image based on the index. Alternatively, the index can also be outputted as the error detection result. For example, the higher the degree of non-similarity between the foreground image and the background image, the larger the value of the index. In this case, it is determined that the display image contains an error in a case where the index is smaller than a given value.

[0121] Further, the foreground image is an image of a region in the display image in which the degree of non-similarity with the background image is to be determined based on the index. Further, the region is a given region. For example, a mask image in which a foreground is specified is stored in a memory or the like. For example, pixels of "1" in a 1-bit mask become pixels defining the foreground, and pixels of the foreground image are determined by the pixels defining the foreground. More specifically, a position on the display image to which the mask image defining the foreground is applied is specified, and the foreground is determined based on the position and the mask image.

[0122] Further, the background image is the display image other than the foreground image. That is, a region in the display image other than the foreground image in which the error detection region is set is the background image. For example, pixels of "0" in a 1-bit mask become pixels defining the background in the mask image, and pixels of the background image are determined by the pixels defining the background.

[0123] Further, the degree of non-similarity refers to the degree of non-similarity in each component of the structural components of the color space. The structural components of the color space are also referred to as channels. For example, in the YCbCr space, it is the degree to which the luminance of the foreground image and the luminance of the background image differ, or the degree to which the color of the foreground image and the color of the background image differ. Alternatively, in the RGB space, it is the degree to which the color of the foreground image and the color of the background image differ.

[0124] Further, in the present embodiment, as described in Figure 10 the error detection circuit 150 calculates a histogram of each component of the structural components of the color space. As described in Figure 11 the error detection circuit 150 performs autocorrelation on the histogram of each component, calculates a distance of a peak of autocorrelation for each component, and calculates the index based on the largest distance (|Ba-Bb|) among the calculated distances.

[0125] Thus, the index can be calculated based on the component in which the foreground image and the background image differ the most among the components of the structural components of the color space. The component in which the foreground image and the background image differ the most can be considered to be the component in which the visual difference appears to be large, and thus, by calculating the index based on this component, the visibility of the foreground can be appropriately evaluated. The visibility here is the degree of non-similarity with the background.

[0126] Here, the index is a value calculated from the maximum distance |Ba-Bb|. For example, in the first calculation method, the index is the maximum distance |Ba-Bb| itself. Also, in the second calculation method, the index is the contrast ratio based on the maximum distance |Ba-Bb|. The contrast ratio is |Ba-Bb| / Ba or the like.

[0127] Also, in the present embodiment, as explained in Figure 13 the error detection circuit 150 calculates the first histogram of each component of the structural component of the color space from the display image, and calculates the second histogram of each component from the reference image corresponding to the foreground image. As explained in Figure 14 the error detection circuit 150 performs cross-correlation operation of the first histogram and the second histogram for each component, and calculates the index from the peak value of the peak of the cross-correlation.

[0128] Thus, even in the case where the reference image contains colors of two or more colors, it is possible to calculate the index indicating the degree of dissimilarity between the foreground image and the background image. That is, two or more peaks are generated in the histogram of the reference image, but in the case where the same pattern as the histogram is contained in the histogram of the display image, the image similar to the reference image in at least the pattern of color or brightness is contained in the display image. In this case, since a large peak should be generated in the result of the cross-correlation operation, by using the peak value thereof to calculate the index, it is possible to appropriately evaluate the visibility of the foreground.

[0129] Here, the reference image is an image corresponding to the foreground image in the case where the foreground image is correctly displayed in the display image. More specifically, at least the pattern of the peak in the histogram is the same image as the foreground image. At this time, as long as the relative positional relationship between the peaks is the same as Figure 13 the pattern as a whole can be shifted.

[0130] Further, in the present embodiment, the error detection circuit 150 calculates a second index indicating the degree of agreement of the foreground image and the reference image from the pixel value of the display image and the pixel value of the reference image serving as the reference of the foreground image, or the edge of the display image and the edge of the reference image. The error detection is performed based on the index and the second index. The index here is the visibility index, and the second index is the shape index.

[0131] Thus, it is possible to combine two indexes evaluating mutually different properties and perform error detection of the display image. That is, by combining the index indicating the degree of non-similarity of the brightness or color between the foreground image and the reference image and the second index indicating the degree of agreement of the shape between the foreground image and the reference image, it is possible to perform error detection of the display image with higher accuracy. Further, the second index will be described later.

[0132] Further, in the present embodiment, the image acquisition circuit 130 generates the display image by superimposing the second image on the first image in the given region. The background image is an image corresponding to the first image in the display image. In the present embodiment, the input image corresponds to the first image, and the reference image corresponds to the second image.

[0133] Thus, it is possible to generate the display image by superimposing, for example, an icon or a character and the like on the input image using the OSD. In this case, the superimposed character or icon corresponds to the foreground image, and the portion of the original input image other than this corresponds to the background image. In the present embodiment, by performing error detection of the display image, it is possible to determine an error when the icon or character is not properly superimposed in a manner visible to the user in the OSD. On the other hand, even if a processing error of about one bit occurs in the superimposition, it is not determined to be an error in a case where visibility can be ensured, and thus, it is possible to present the icon or character to the user.

[0134] Further, the present embodiment can be implemented as the following error detection method. That is, in this method, a histogram of the pixel values of the display image is found, a correlation operation using the histogram is performed, an index indicating the degree of non-similarity between the foreground image and the background image is found from the result of the correlation operation, and error detection of the display image is performed based on the index, wherein the foreground image is an image of a given region in the display image, and the background image corresponds to the background of the foreground image in the display image.

[0135] Further, the present embodiment can be implemented as the following error detection method. That is, in this method, the error detection region of the display image is analyzed, and an index describing the visibility of the foreground with respect to the background is calculated. At this time, it is also possible to use the reference image as a mask in order to determine the foreground. The index is calculated using the following (a) or (b) technique. (a) The separation between the foreground and the background is explored using the histogram of the display image. (b) The ratio of the contrast of the foreground to the background is calculated.

[0136] 4.3. First calculation method of the shape index

[0137] The shape index is an index indicating whether the shape of the image included in the error detection region agrees with the reference image. Hereinafter, the calculation method of the shape index will be described.

[0138] First, the pixel block of the error detection region of the display image is averaged in a manner that the final averaged image is m x n pixels. This down-sampling process is performed in order that a few pixel errors are not detected as important errors, and the overall shape of the reference image and the display image is confirmed, ignoring errors such as color shift or small distortion. In order to get a perfect match, the resolution of the down-sampled image can be increased. The value of m x n can be selected according to the use. As described below, in the case of use in association with the reference image, the value of m x n is selected based on the sample data observation.

[0139] In the case where the error detection region of the display image is u x v pixels, the averaged block size is u / m x v / n pixels. In the case where the reference background information cannot be utilized, the pixels of the display image of the portion where the reference pixel does not exist are deleted. This is equivalent to the reference foreground mask. This is performed because it is necessary to baseline the background pixels between the reference image and the display image. Baseline means to align or place in the same condition. Therefore, the value of the background pixel is set to the same value in both the display image and the reference image.

[0140] The averaging of the reference image is also performed in a manner that becomes m x n pixels. The averaging is performed for each channel separately. Figure 15 is an example of the reference image. The foreground Fl of the reference image RIA is an icon, and is colored. The background is the area other than the icon, and is, for example, achromatic such as black. In Figure 15 In, the size of the reference image RIA is 256 x 256 pixels. Figure 16 is an averaged image of the reference image. In Figure 16 In, m = n = 16, and the size of the averaged image SRef is 16 x 16 pixels. Since the background of this reference image and its averaged image is achromatic, the error detection region of the display image also converts the background to achromatic, and the averaged image of this error detection region is obtained.

[0141] Next, using the distance reference, the averaged image SRef of the reference image and the averaged image SAnz of the error detection region of the display image are compared for each pixel, and the distance D is obtained as in the following formula (2). SRef and SAnz are m x n pixels. The distance D is a three-dimensional distance. In the present embodiment, the distance reference is the square of the Cartesian distance, but the same parameters can be obtained even if it is other distance reference.

[0142] [Formula 2]

[0143]

[0144] c denotes a channel, x denotes a pixel position in a horizontal direction in the averaged image, and y denotes a pixel position in a vertical direction in the averaged image. The horizontal direction is the horizontal direction, and the vertical direction is the vertical direction. m and n are the sizes of the averaged image. R xyc denotes a pixel value at a position (x, y) of the averaged image of the reference image in a channel c. c denotes a pixel value at a position (x, y) of the averaged image of the reference image in a channel c. xy denotes an average value of the pixels. The average value is obtained by averaging R xyc in the averaged image. xyc denotes a pixel value at a position (x, y) of the averaged image of the display image in a channel c. c denotes a pixel value at a position (x, y) of the averaged image of the display image in a channel c. xy denotes an average value of the pixels. The average value is obtained by averaging A xyc in the averaged image.

[0145] The reason for subtracting the average value in each channel is to not regard a small color shift between the reference image and the display image as an error. In a case where complete agreement is required, the average value can be set to 0. In this case, the shape and the color agreement are checked by the distance reference.

[0146] The display image is, for example, Figure 12 The error detection region Al is extracted around the region after the reference image is synthesized with the input image. In Figure 12 , the error detection region is indicated by a dotted quadrangle.

[0147] The shape index S is derived from the distance parameter according to the following equations (3) and (4). S is also called a shape parameter. T is a threshold value, and can take an arbitrary value. In a case where D < T, T / D = 1, and the shape index S does not change.

[0148] [Equation 3]

[0149]

[0150] [Equation 4]

[0151] D = T if D < T (4)

[0152] The function f is selected in a manner such that it is easy to install to hardware. For example, the function f can be a scaling function K that scales the range 0 to 1 to 0 to k. In the example described below, the function f is a unit function. That is, S = T / D. The shape index S indicates the degree of agreement of the shape between the reference image and the display image. In a case where the images do not agree, the value decreases, and there is a tendency to become 0. The example is described below.

[0153] In Figure 12In this case, the icon of the reference image is correctly displayed on the display image. In this case, the shape index is S = 1. In Figure 12 In this case, Shape: 1.000 is indicated.

[0154] Figure 17 is the second example of the display image. Bl indicates the false detection region. As shown in B2 of Figure 17 , the icon of the reference image is unclear in the display image. That is, some of the pixels of the reference pixels do not exist in the display image, and in the case where the function f is the unit function, the shape index S is less than 1. In the case of such unclear foreground, both the visibility index and the shape index are small values.

[0155] Figure 18 is the third example of the display image. El indicates the false detection region. As shown in E2 of Figure 18 , the icon of the reference image is rotated in the display image. In this example, since the shape is rotated from the reference, in the case where the function f is the unit function, the shape index S is less than 1. In the case where the foreground is rotated like this, the visibility index is a large value and the shape index is a small value. In this way, by combining the visibility index and the shape index, appropriate false detection can be performed in various states of the foreground, and the accuracy of the false detection can be improved.

[0156] In addition, the above describes the case where the display image and the reference image can be used as images, but the application object of the present application is not limited to this. For example, the same operation can be easily performed in the case where the image is a line, a pixel, or a sub image streaming.

[0157] The above shape index only checks the coincidence of the basic signal. In the case of an image with low visibility, after a first gradient image is generated by convolving an edge detection kernel with the false detection region of the display image and the reference image, the parameters can be found by a shape operation algorithm. The edge detection kernel is Laplacian or Sobel, or the like. According to the found parameters, the false detection obtained by the shape index can be removed. In this way, a correct false detection result can be obtained in the case of an image with low visibility.

[0158] 4.4. Second operation method of shape index

[0159] Figure 19 is the fourth example of the display image. In Figure 19In the case of Fig. 13, an example in which the icon ICA is superimposed on the dashboard image DIM is shown. The foreground and the background of the icon image are alpha-blended with the dashboard image in a separate manner. In the above case, the foreground of the icon image is partially blended, on the other hand, the background of the icon image is completely blended. Here, the foreground of the icon image is the icon portion of the icon image. In the case of Fig. 13, the icon portion is indicated by the black portion, which is a pixel of bit "1" of the mask image MSB. The background of the icon image is the portion other than the icon of the icon image. In the case of Fig. 13, the portion other than the icon is indicated by the white portion, which is a pixel of bit "0" of the mask image MSB. In the background of the icon image, blending is performed in accordance with the ratio of the icon image being 0 and the dashboard image DIM being 1. In the foreground of the icon image, blending is performed in accordance with the ratio of the icon image being a and the dashboard image DIM being (1 - a). 0 < a < 1. The icon that is blended in accordance with the ratio a is the foreground in the display image, and the area other than the icon is the background in the display image. Figure 20 In the case of Fig. 13, the icon portion is indicated by the black portion, which is a pixel of bit "1" of the mask image MSB, is the mask image MSB. The background of the icon image is the portion other than the icon of the icon image. In the case of Fig. 13, the portion other than the icon is indicated by the white portion, which is a pixel of bit "0" of the mask image MSB. In the background of the icon image, blending is performed in accordance with the ratio of the icon image being 0 and the dashboard image DIM being 1. In the foreground of the icon image, blending is performed in accordance with the ratio of the icon image being a and the dashboard image DIM being (1 - a). 0 < a < 1. The icon that is blended in accordance with the ratio a is the foreground in the display image, and the area other than the icon is the background in the display image. Figure 20 In the case of Fig. 13, the portion other than the icon is indicated by the white portion, which is a pixel of bit "0" of the mask image MSB, is the mask image MSB. In the background of the icon image, blending is performed in accordance with the ratio of the icon image being 0 and the dashboard image DIM being 1. In the foreground of the icon image, blending is performed in accordance with the ratio of the icon image being a and the dashboard image DIM being (1 - a). 0 < a < 1. The icon that is blended in accordance with the ratio a is the foreground in the display image, and the area other than the icon is the background in the display image.

[0160] In the present embodiment, the relationship between the icon and the original icon is analyzed to check whether the icon is displayed correctly. Specifically, using an edge detection technique such as the Sobel edge detection convolution operator, an edge is detected in the error detection area as in the case of the reference.

[0161] Figure 20 is the first example of the error detection area of the reference image and the display image and the mask image. The mask image MSB is a mask that indicates the foreground and the background of the reference image ICB, and the black color indicates the foreground pixel and the white color indicates the background pixel. The reference image ICB that is the reference icon is an image in which the foreground of the icon is colored. The display image CIB that is the display icon is an image in which the error detection area of the reference image ICB is blended in the dashboard image DIM. In the icon portion, the dashboard image DIM is seen through by blending.

[0162] Figure 21 is an example of the edge values calculated from the reference image and the display image. EICB is the edge image of the reference image ICB, and ECIB is the edge image of the display image CIB. In the illustrated relationship, the edges are indicated by the black line and the gray line, but in reality, the intensity of the edge can be indicated by the gradation. The white color indicates a high-intensity edge, and the black color indicates no edge. The edge detection is performed on the luminance channel. Also, the edge detection is performed on the color channel or in the color space such as YCbCr.

[0163] The edge of the foreground region and the background region is calculated for the reference image and the display image, and the shape index is calculated by calculating the similarity as shown in the following equations (5) to (16). Match of the following equation (16) is the shape index. Hereinafter, it is assumed that the reference image is of m x n pixels in size, and the error detection region of the display image is also m x n pixels.

[0164] The following equation (5) is a horizontal Sobel kernel. The horizontal Sobel kernel is an operator of a Sobel filter that detects an edge in a horizontal direction. The following equation (6) is a vertical Sobel kernel. The vertical Sobel kernel is an operator of a Sobel filter that detects an edge in a vertical direction.

[0165] [Equation 5]

[0166]

[0167] [Equation 6]

[0168]

[0169] The edge value is calculated for each pixel position in the error detection region of the display image and the reference image as shown in the following equations (7) to (12). "*" is a convolution operator. N is a normalization coefficient for keeping the value between 0 and 1, and here N = 4. IRef is the luminance channel of the reference image. IRef (x，y) is the pixel at position x, y of the luminance channel of the reference image. x is an integer of 0 < x < m, and y is an integer of 0 < y < n. IRen is the luminance channel of the display image in the error detection region. IRen (x，y) is a 3 x 3 pixels centered on the position x, y of the luminance channel of the display image in the error detection region.

[0170] [Equation 7]

[0171]

[0172] [Equation 8]

[0173] E1 H(x，y) = F H * IRef (x，y) (8)

[0174] [Equation 9]

[0175] E1 V(x，y) = F V * IRef (x，y) (9)

[0176] [Equation 10]

[0177]

[0178] [Formula 11]

[0179] E2 H(x，y) = F H *IRen (x，y) (11)

[0180] [Formula 12]

[0181] E2 V(x，y) = F V *IRen (x，y) (12)

[0182] The shape index Match is calculated from the above edge values as shown in the following formulas (13) to (16). The shape index Match is also called a fitness value. The "•" indicates an inner product operator.

[0183] [Formula 13]

[0184]

[0185] [Formula 14]

[0186]

[0187] [Formula 15]

[0188]

[0189] [Formula 16]

[0190]

[0191] When the above operation is applied in Figure 20 , Figure 21 , Match = 0.78.

[0192] In the case where the fitness value is calculated without analyzing the background, the calculation shown in the following formulas (17) to (22) is used.

[0193] [Formula 17]

[0194]

[0195] [Formula 18]

[0196]

[0197] [Formula 19]

[0198]

[0199] [Formula 20]

[0200]

[0201] [Formula 21]

[0202]

[0203] [Formula 22]

[0204]

[0205] M (x，y) is a mask pixel that defines which pixels belong to the background and which pixels belong to the foreground. The mask can be implemented by a simple 1-bit mask that defines the background as 0 and the foreground as 1. Alternatively, it can be a mask of more than 1 bit that supports edges against anti-aliasing. In the mask, values between 0 and 1 are used as partial background and partial foreground. For example, a value of 0.25 that is "01" in 2-bit representation means 25% foreground and 75% background.

[0206] Figure 22 is a second example of the error detection region and the mask image of the reference image and the display image. The reference image ICC, the display image CIC, and the mask image MSC are the same as the reference image ICB, the display image CIB, and the mask image MSB of Figure 20 . Figure 23 is an example of the edge value calculated from the reference image and the display image. EICC is the edge image of the reference image ICC, and ECIC is the edge image of the display image CIC. In the edge image ECIC of the display image, the edge component of the background of the icon is masked by M (x，y) as shown in FIG. 18. In the above operation, the shape index Match rises to 0.82. Figure 22

[0207] According to the above embodiment, the error detection circuit 150 calculates an index indicating the degree of agreement between the foreground image and the reference image from the pixel value of the display image and the pixel value of the reference image that is the reference of the foreground image, or from the pixel value of the edge image of the display image and the pixel value of the edge image of the reference image, and performs error detection of the display image based on the index, where the foreground image is the image of a given region in the display image. The pixel value of the edge image corresponds to the amount of edge. The index here is the shape index.

[0208] ​Thus, the error detection of the display image can be performed based on the index indicating the degree of coincidence between the display image and the reference image, instead of the error detection in units of bits such as CRC. In a case where the degree of coincidence of the foreground image with respect to the reference image is high, it is likely that the foreground image is visually regarded as the same shape as the reference image. That is, according to the present method, it is possible to determine that there is an error in a case where the shape of the foreground image is not displayed correctly. For example, an icon or the like for warning a user is displayed in a meter panel or the like mounted on a vehicle. According to the present embodiment, such an icon does not stop being displayed due to a 1-bit error or the like, and is displayed as much as possible in a case where the shape can be correctly recognized, so that the user can be warned.

[0209] Here, the first calculation method shown in the above equations (3) to (5) corresponds to a case where the index S is calculated based on the pixel value of the display image and the pixel value of the reference image serving as a reference for the foreground image. Also, the second calculation method shown in the above equations (5) to (22) corresponds to a case where the index Match is calculated based on the pixel value of the edge image of the display image and the pixel value of the edge image of the reference image. The pixel value of the edge image corresponds to the edge amount of the above equations (7), (10), (17), and (18).

[0210] In addition, the degree of coincidence refers to the degree of coincidence of the shape of an icon or the like. More specifically, it refers to the degree of coincidence of the outline and the orientation of the icon or the like. The icon or the like can be an icon, a character, a figure, a mark, or the like. Further, it can also include the degree of coincidence of the state of the inside of the outline of the icon or the like. The state is, for example, whether it is filled or not. For example, the higher the degree of coincidence of the foreground image with respect to the background image, the larger the value of the index indicating the degree of coincidence.

[0211] Further, in the present embodiment, as explained in Figure 16 the error detection circuit 150 performs down-sampling that reduces the number of pixels or the resolution of the display image and the reference image. As explained in the above equation (2), the error detection circuit 150 calculates distance information indicating the distance in the color space between the pixel value of the down-sampled display image and the pixel value of the down-sampled reference image. As explained in the above equations (3) and (4), the error detection circuit 150 calculates the index based on the above distance information.

[0212] Since the pixel values are averaged by performing the down-sampling, the influence of noise such as a 1-bit error can be reduced when the index is calculated. The noise here is a slight error that does not affect the shape. Also, the distance in the color space between the pixel value of the display image and the pixel value of the reference image should be small in a case where the shapes coincide. Thus, by using the distance in the color space, the degree of coincidence of the shapes can be appropriately evaluated.

[0213] Further, in the present embodiment, the error detection circuit 150 calculates the index from a value obtained by dividing a given threshold value by the distance information. In the above equations (3), (4), the given threshold value is T, the distance information is D, and the index is S.

[0214] The higher the degree of coincidence of the shapes, the smaller the distance, and thus by dividing the given threshold value by the distance information, the index can be calculated which is larger the higher the degree of coincidence of the shapes.

[0215] Further, in the present embodiment, as explained in the above equations (13), (16), the error detection circuit 150 performs a product-sum operation of the pixel values of the edge image of the display image and the pixel values of the edge image of the reference image, and calculates the index from the result of the product-sum operation.

[0216] The edge image is an image in which the edge amount is defined as the pixel value of each pixel. In the case where the shapes coincide, when the edge image of the display image and the edge image of the reference image are compared with the same pixels, the edge amounts should be approximately the same. Conversely, in the case where the shapes do not coincide, since the positions of the edges do not coincide between the display image and the reference image, for example, even if there is a large edge amount in the edge image of the display image, the edge amount is zero in the same pixel of the edge image of the reference image. Therefore, when the edge amounts of the same pixels are product-summed, the result of the product-sum is a larger value in the case where the shapes coincide, and the result of the product-sum is a smaller value in the case where the shapes do not coincide. Therefore, by using the product-sum operation of the edge amounts, the degree of coincidence of the shapes can be appropriately evaluated.

[0217] Here, in the above equation (13), the "product" of the "product-sum" is the inner product of vectors, but the "product" is not limited to this. For example, in the case where the edge amount is defined by a scalar, the "product" is the product of the scalars.

[0218] Further, in the present embodiment, as explained in the above equations (18), (19), the error detection circuit 150 masks the region corresponding to the background image in the edge image of the display image, and performs the product-sum operation using the masked edge image of the display image.

[0219] Thus, even in the case where the edge is included in the background, the edge can be masked and the product-sum operation of the edge amounts can be performed. That is, the degree of coincidence of the edges of the display image and the reference image can be evaluated without being affected by the edge of the background, and the accuracy of the error detection can be further improved.

[0220] Furthermore, this embodiment can be implemented as the following error detection method. That is, in this method, an index representing the degree of consistency between the foreground image and the reference image is obtained based on the pixel values ​​of the displayed image and the pixel values ​​of the reference image serving as the reference for the foreground image, or based on the pixel values ​​of the edge image of the displayed image and the pixel values ​​of the edge image of the reference image, and error detection of the displayed image is performed based on this index, wherein the foreground image is an image of a given region in the displayed image.

[0221] Furthermore, this embodiment can be implemented as the following error detection method. That is, in this method, the error detection region of the displayed image is analyzed, and an index describing the similarity to a reference image is calculated. The index is calculated using the following techniques (a) or (b): (a) Comparing the three-dimensional distance errors of the downsampled pixels of the displayed image and the reference image. (b) Comparing the three-dimensional distance errors of the edges of the displayed image and the reference image.

[0222] 5. Variations

[0223] You can also from Figure 8 The processing device 200 and other systems input the icon display status to the circuit device 100. The icon display status indicates whether an icon is displayed, or which icon is displayed, etc. Thus, the circuit device 100 can detect the following situations: an icon of a different type than the icon that the system wants to display is displayed, or an icon is not actually displayed even though the system wants to display an icon, or an icon is actually displayed even though the system does not want to display an icon.

[0224] Furthermore, the above embodiments were described using the application of the error detection method of the present invention to a display controller as an example, but the application of the present invention is not limited to this. That is, the present invention can be applied at any stage of the path in which the display image is processed or transmitted.

[0225] For example, a display driver that drives a display panel can also be used as a circuit device to which the present invention is applied. In this case, for example, the interface for inputting image data in the display driver is equivalent to an image acquisition circuit, and an error detection circuit is provided between the interface and the driving circuit. For example, without overlay, the error detection circuit performs error detection of the present invention on a given area of ​​icons, etc., in the image received through the interface.

[0226] Furthermore, in the above embodiments, the example described is that the image acquisition circuit 130 generates a display image by overlaying icons onto an input image that has been preprocessed and input to the display controller. However, the application of the present invention is not limited to this. That is, the image acquisition circuit 130 can acquire any image as a display image.

[0227] For example, the image acquisition circuit 130 can also use the input image itself as the display image. In this case, an icon, for example, is already included in the input image, and a region including the icon or the like is set as the error detection region.

[0228] Alternatively, the image acquisition circuit 130 can also use an image obtained by performing scaling processing on the input image as the display image. In this case, an icon, for example, is included in the input image, the icon is scaled, and a region including the scaled icon is set as the error detection region.

[0229] Alternatively, the image acquisition circuit 130 can also use an image obtained by performing gamma conversion processing on the input image as the display image. In this case, an icon, for example, is included in the input image, the icon is gamma-converted, and a region including the gamma-converted icon is set as the error detection region.

[0230] Alternatively, the image acquisition circuit 130 can also use an image obtained by performing deformation processing on the input image as the display image. In this case, an icon, for example, is included in the input image, the icon or the entire input image is deformed, and a region including the deformed icon is set as the error detection region. For example, the image is sometimes deformed in order to be displayed on a head-mounted display or the like.

[0231] Alternatively, the image acquisition circuit 130 can also read out an image stored in a memory and use the image as the display image. In this case, the memory controller corresponds to the image acquisition circuit 130. Alternatively, the image acquisition circuit 130 can also use an image received through an interface as the display image. In this case, the interface corresponds to the image acquisition circuit 130.

[0232] 6. Electronic device

[0233] Figure 24 is a configuration example of an electronic device including the circuit device of the present embodiment. The electronic device 300 includes a processing device 310, a circuit device 320, a display driver 330, a display panel 340, a storage device 350, an operation device 360, and a communication device 370. The processing device 310 is, for example, an MCU or the like. The circuit device 320 is, for example, a TCON.

[0234] The processing device 310 transfers the image data stored in the storage device 350 or the image data received by the communication device 370 to the circuit device 320. The circuit device 320 performs image processing of the image data, display timing control, error detection processing of the image data transferred to the display driver, and the like. The display driver 330 drives the display panel 340 in accordance with the image data transferred from the circuit device 320 and the display timing control of the circuit device 320 to make an image. The display panel 340 is, for example, a liquid crystal display panel or an EL display panel, or the like. The storage device 350 is, for example, a memory or a hard disk drive or an optical disk drive, or the like. The operation device 360 is a device for a user to operate the electronic device 300, and is, for example, a button or a touch panel or a keyboard, or the like. The communication device 370 is, for example, a device that performs wired communication or a device that performs wireless communication. The wired communication is, for example, LAN or USB, or the like. The wireless communication is, for example, Wifi, Bluetooth (registered trademark), or the like.

[0235] As the electronic device including the circuit device of the present embodiment, various devices such as a display terminal of a factory equipment or the like, a display device mounted on a robot, an information processing device, and a mobile information processing terminal can be conceived. The in-vehicle electronic device is, for example, an instrument panel or the like. The structure of the electronic device is not limited to Figure 24 , and various structures can be adopted according to the use. For example, in the in-vehicle electronic device, the circuit device 320, the display driver 330, the display panel 340, and the operation device 360 are assembled into an instrument panel, and the processing device 310, the storage device 350, and the communication device 370 are assembled into an ECU. In this case, the instrument panel corresponds to the electronic device including the circuit device of the present embodiment. The ECU is an abbreviation of Electronic Control Unit.

[0236] The circuit device of the present embodiment described above includes an image acquisition circuit that acquires a display image, an error detection circuit, and a search circuit. The error detection circuit calculates a histogram of pixel values of the display image, performs a correlation operation using the histogram, calculates an index indicating a degree of dissimilarity between a foreground image and a background image from a result of the correlation operation, and performs error detection of the display image based on the index, the foreground image being an image of a given region in the display image, the background image corresponding to a background of the foreground image in the display image. The search circuit searches for a display position of the foreground image in the display image based on a reference image that is a reference of the foreground image. The error detection circuit performs error detection based on the display position searched for by the search circuit.

[0237] According to the present embodiment, it is possible to appropriately detect such an error that the user makes a misrecognition. That is, in a case where the degree of non-similarity of the foreground image with respect to the background image is high, the foreground image is highly likely to be visually distinguished with respect to the background image, and thus it is possible to determine that there is an error in a case where the visibility of the foreground image is low by using the index described above. At this time, the error detection circuit can perform error detection in accordance with the display position searched by the search circuit, and thus it is possible to perform error detection even if the display position of the foreground image is not determined in advance. A series of processes for performing such error detection is not disclosed in the related art such as Patent Literature 1.

[0238] Further, in the present embodiment, it can be that the error detection circuit performs error detection when the search circuit detects the display position in the search, and does not perform error detection when the search circuit does not detect the display position in the search.

[0239] According to the present embodiment, in a case where the foreground image is included in the display image, error detection is performed, and in a case where the foreground image is not included in the display image, error detection is not performed. For example, in a case where a frame image of a dynamic image is input as the display image, the error detection circuit performs error detection of an icon or the like when the icon or the like is displayed in the dynamic image, and does not perform error detection when the icon or the like is not displayed in the dynamic image. A technique of performing such image search and error detection as a series of processes is not disclosed in the related art such as Patent Literature 1.

[0240] Further, in the present embodiment, it can be that the search circuit calculates the cross-correlation of the display image with the reference image, and detects whether or not there is a correlation value that exceeds a detection threshold in the cross-correlation. It can be that the search circuit detects a position corresponding to the correlation value that exceeds the detection threshold as the display position in a case where there is the correlation value that exceeds the detection threshold, and does not detect the display position in a case where there is no correlation value that exceeds the detection threshold.

[0241] According to the present embodiment, the search circuit can detect a position at which the correlation of the display image with the reference image is large as the display position of the foreground image. Also, it is possible to realize a series of actions of causing the error detection circuit to perform error detection when the search circuit detects the display position, and not causing the error detection circuit to perform error detection when the search circuit does not detect the display position.

[0242] Further, in the present embodiment, it can be that the search circuit divides the display image into a plurality of belt-shaped regions, divides each of the plurality of belt-shaped regions into a plurality of individual regions, and searches each of the plurality of individual regions.

[0243] According to the present embodiment, the cross-correlation with the reference image is calculated for each individual region, and the cross-correlations calculated for each of the individual regions are synthesized, whereby the cross-correlation of the display image and the reference image can be obtained. In addition, by appropriately operating the cross-correlation in parallel for each individual region, the speed of the search processing can be improved. For example, in the case of error detection for a moving image, it is desirable that the delay until the end of the error detection be small, and the error detection can be speeded up by improving the processing speed of the cross-correlation.

[0244] Further, in the present embodiment, the search circuit can perform the search by calculating the cross-correlation of the display image and the reference image using a Fourier transform and an inverse Fourier transform.

[0245] According to the present embodiment, the display image and the reference image after the Fourier transform are multiplied in the frequency domain, and the result is subjected to an inverse Fourier transform, whereby the cross-correlation is obtained. In addition, by performing the Fourier transform and the inverse Fourier transform for each individual region, the processing speed of the Fourier transform and the inverse Fourier transform can be improved.

[0246] Further, in the present embodiment, the error detection circuit can obtain a histogram of each component of the structural component of the color space, perform an autocorrelation operation on the histogram of each component, obtain a distance of a peak of the autocorrelation for each component, and obtain the index based on the maximum distance among the obtained distances.

[0247] Thus, the index can be obtained by the component of the structural component of the color space that is most different between the foreground image and the background image. The component that is most different between the foreground image and the background image can be considered to be visually different and thus, by obtaining the index using this component, the degree of dissimilarity of the foreground and the background can be appropriately evaluated.

[0248] Further, in the present embodiment, the error detection circuit can obtain a first histogram of each component of the structural component of the color space as a histogram based on the display image, obtain a second histogram of each component based on the reference image, perform a cross-correlation operation of the first histogram and the second histogram for each component, and obtain the index based on the peak value of the peak of the cross-correlation.

[0249] In the case where the histogram of the display image contains the same pattern as the histogram of the reference image, the display image contains an image similar to the reference image in at least the pattern of color or brightness. In this case, since a large peak should be generated in the result of the cross-correlation operation, by obtaining the index using the peak value, the degree of dissimilarity of the foreground and the background can be appropriately evaluated.

[0250] Further, in this embodiment, it can be that the error detection circuit calculates a second index indicating the degree of coincidence of the foreground image, which is an image of a given region in the display image, with the reference image, from the pixel values of the display image and the pixel values of the reference image, or the pixel values of the edge image of the display image and the pixel values of the edge image of the reference image, and performs error detection based on the index and the second index.

[0251] In this way, by combining the index indicating the degree of non-similarity of the brightness or color between the foreground image and the reference image and the second index indicating the degree of coincidence of the shape between the foreground image and the reference image, error detection of the display image can be performed with higher accuracy.

[0252] Further, in this embodiment, it can be that the image acquisition circuit generates the display image by superimposing the second image on the first image in the given region, and the background image is an image corresponding to the first image in the display image.

[0253] In this way, for example, a display image can be generated by superimposing an icon, a character, or the like on an input image. In this case, the superimposed character or icon corresponds to the foreground image, and the portion of the original input image other than this corresponds to the background image. According to one embodiment of the present application, by performing error detection of such a display image, it is possible to determine that there is an error in a case where an icon or a character is not superimposed in an appropriate manner that can be recognized by the user.

[0254] Further, the circuit device of this embodiment includes an image acquisition circuit that acquires a display image, an error detection circuit, and a search circuit. The error detection circuit calculates an index indicating the degree of coincidence of a foreground image, which is an image of a given region in the display image, with a reference image that is a reference for the foreground image, from the pixel values of the display image and the pixel values of the reference image, or the pixel values of the edge image of the display image and the pixel values of the edge image of the reference image, and performs error detection of the display image based on the index. The search circuit searches for a display position of the foreground image in the display image based on the reference image. The error detection circuit performs error detection based on the display position searched for by the search circuit.

[0255] According to this embodiment, it is possible to appropriately detect an error in which the user makes a misrecognition. That is, in a case where the degree of coincidence of the foreground image with the reference image is high, the likelihood that the foreground image looks like a visually identical shape with respect to the reference image is high, and thus by using the index described above, it is possible to determine that there is an error in a case where the shape of the foreground image is not displayed correctly. At this time, the error detection circuit can perform error detection based on the display position searched for by the search circuit, and thus it is possible to perform error detection even if the display position of the foreground image is not decided in advance. A series of processes for performing such error detection is not disclosed in the related art such as Patent Document 1.

[0256] Further, in the present embodiment, it can be that the error detection circuit performs error detection when the search circuit detects a display position in the search, and does not perform error detection when the search circuit does not detect a display position in the search.

[0257] According to the present embodiment, error detection is performed when a foreground image is included in a display image, and error detection is not performed when a foreground image is not included in a display image. For example, in a case where a frame image of a dynamic image is input as a display image, the error detection circuit performs error detection of an icon or the like when the icon or the like is displayed in the dynamic image, and does not perform error detection when the icon or the like is not displayed in the dynamic image. A technique of performing such image search and error detection as a series of processes is not disclosed in the related art such as Patent Literature 1.

[0258] Further, in the present embodiment, it can be that the search circuit calculates a cross-correlation between the display image and the reference image, and detects whether or not there is a correlation value that exceeds a detection threshold in the cross-correlation. It can be that the search circuit detects a position corresponding to the correlation value that exceeds the detection threshold as a display position in a case where there is the correlation value that exceeds the detection threshold, and does not detect a display position in a case where there is not the correlation value that exceeds the detection threshold.

[0259] According to the present embodiment, the search circuit can detect a position at which a correlation between the display image and the reference image is large as a display position of a foreground image. Also, it is possible to realize a series of actions of causing the error detection circuit to perform error detection when the search circuit detects a display position, and not causing the error detection circuit to perform error detection when the search circuit does not detect a display position.

[0260] Further, in the present embodiment, it can be that the search circuit divides the display image into a plurality of belt-shaped regions, divides each of the plurality of belt-shaped regions into a plurality of individual regions, and performs search on each of the plurality of individual regions.

[0261] According to the present embodiment, a cross-correlation with the reference image is calculated for each individual region, and the cross-correlations calculated for each of the individual regions are synthesized, whereby it is possible to obtain a cross-correlation between the display image and the reference image. By appropriately operating the cross-correlation in parallel for each individual region, it is possible to improve the speed of search processing. For example, in a case where a dynamic image is subjected to error detection, it is desirable that a delay until the end of error detection is small, and it is possible to speed up error detection by improving the processing speed of the cross-correlation.

[0262] Further, in the present embodiment, it can be that the search circuit performs search by calculating a cross-correlation between the display image and the reference image using a Fourier transform and an inverse Fourier transform.

[0263] According to the present embodiment, the display image and the reference image after the Fourier transform are multiplied in the frequency domain, and the result is inverse-Fourier transformed, whereby the cross-correlation is obtained. Further, by performing the Fourier transform and the inverse-Fourier transform for each individual region, the processing speed of the Fourier transform and the inverse-Fourier transform can be improved.

[0264] Further, in the present embodiment, the error detection circuit can perform down-sampling that reduces the number of pixels or the resolution of the display image and the reference image, obtain distance information that indicates the distance in the color space between the pixel values of the down-sampled display image and the pixel values of the down-sampled reference image, and obtain the index from the distance information.

[0265] The distance in the color space between the pixel values of the down-sampled display image and the pixel values of the down-sampled reference image should be small in the case where the shapes are consistent. Therefore, by using the distance in the color space, the degree of consistency of the shapes can be appropriately evaluated. Further, by performing the down-sampling, the pixel values are averaged, and thus the influence of slight errors that do not affect the shapes can be reduced.

[0266] Further, in the present embodiment, the error detection circuit can obtain the index from a value obtained by dividing a given threshold value by the distance information.

[0267] The higher the degree of consistency of the shapes, the smaller the distance indicated by the distance information. Therefore, by dividing the given threshold value by the distance information, the index that is larger the higher the degree of consistency of the shapes can be obtained.

[0268] Further, in the present embodiment, the error detection circuit can perform a product-sum operation of the pixel values of the edge image of the display image and the pixel values of the edge image of the reference image, and obtain the index from the result of the product-sum operation.

[0269] The edge image is an image in which the edge amount is defined as the pixel values of the respective pixels. In the case where the shapes are consistent, when the edge image of the display image and the edge image of the reference image are compared at the same pixels, the edge amounts should be approximately the same. In contrast, in the case where the shapes are inconsistent, the positions of the edges are inconsistent between the display image and the reference image. Therefore, if the edge amounts of the same pixels are product-summed, the result of the product-sum becomes a larger value in the case where the shapes are consistent. By using such a product-sum operation of the edge amounts, the degree of consistency of the shapes can be appropriately evaluated.

[0270] Further, in the present embodiment, the error detection circuit can mask the region corresponding to the background image in the edge image of the display image, and perform the product-sum operation using the edge image of the display image after the masking.

[0271] Thus, even in a case where the edge is included in the background, the edge can be masked to perform the sum and product operations of the edge amount. That is, the degree of coincidence of the edges of the display image and the reference image can be evaluated without being affected by the edge of the background, and the accuracy of error detection can be further improved.

[0272] Further, in the present embodiment, the image acquisition circuit can generate the display image by superimposing the second image on the first image in the given region.

[0273] Thus, for example, an icon, a character, or the like can be superimposed on the input image to generate the display image. In this case, the superimposed character or icon corresponds to the foreground image, and the portion of the original input image other than this corresponds to the background image. According to other modes of the present application, by performing error detection of such a display image, it is possible to determine as an error in a case where the icon or character is not superimposed in an appropriate manner that can be recognized by the user.

[0274] Further, the electronic device of the present embodiment includes the circuit device of any one of the above.

[0275] Further, in the error detection method of the present embodiment, a display position of a foreground image in a display image is searched for from a reference image that is a reference of the foreground image, the foreground image being an image of a given region in the display image. In the error detection method, a histogram of pixel values of the display image is calculated from the searched display position. In the error detection method, a correlation operation using the histogram is performed. In the error detection method, an index indicating a degree of dissimilarity between the foreground image and a background image that corresponds to a background of the foreground image is calculated from a result of the correlation operation. In the error detection method, error detection of the display image is performed based on the index.

[0276] Further, in the error detection method of the present embodiment, a display position of a foreground image in a display image is searched for from a reference image that is a reference of the foreground image, the foreground image being an image of a given region in the display image. In the error detection method, a histogram of pixel values of the display image is calculated from the searched display position. In the error detection method, a correlation operation using the histogram is performed. In the error detection method, an index indicating a degree of dissimilarity between the foreground image and a background image that corresponds to a background of the foreground image is calculated from a result of the correlation operation. In the error detection method, error detection of the display image is performed based on the index.

[0277] In addition, although the present embodiment has been described in detail as described above, those skilled in the art will be able to easily understand that various modifications can be implemented without departing substantially from the new matters and effects of the present application. Therefore, such modifications are all included in the scope of the present application. For example, a term recited together with a different term of broader meaning or synonymous term at least once in the specification or drawings can be replaced with the different term or the synonymous term at any place in the specification or drawings. Also, all combinations of the present embodiment and the modifications are included in the scope of the present application. Also, the structure and the operation and the like of the circuit device and the electronic apparatus and the like are not limited to those described in the present embodiment, and various modifications can be implemented.

Claims

1. A circuit device, characterized in that, The circuit device includes: Image acquisition circuit, which acquires and displays images; A retrieval circuit retrieves the display position of the foreground image in the display image based on a reference image that serves as a reference for the foreground image; the foreground image is an image of a given region in the display image. as well as An error detection circuit, based on the display position, sets an error detection region containing the foreground image onto the display image, calculates a histogram of pixel values ​​in the error detection region, performs correlation operations using the histogram, and calculates an index representing the degree of dissimilarity between the foreground image and the background image based on the result of the correlation operations. Error detection is performed when the index is less than a predetermined threshold, indicating an error in the display image. This background image corresponds to the background of the foreground image in the display image. When the retrieval circuit detects the display position in the retrieval, the error detection circuit performs the error detection; when the retrieval circuit does not detect the display position in the retrieval, the error detection circuit does not perform the error detection.

2. The circuit device according to claim 1, characterized in that, The retrieval circuit calculates the cross-correlation between the displayed image and the reference image, and searches for whether there is a correlation value exceeding a detection threshold in the cross-correlation. If there is a correlation value exceeding the detection threshold, the position corresponding to the correlation value exceeding the detection threshold is detected as the display position. If there is no correlation value exceeding the detection threshold, the display position is not detected.

3. The circuit device according to claim 1 or 2, characterized in that, The retrieval circuit divides the displayed image into multiple strip regions, further divides each of the multiple strip regions into multiple individual regions, and performs the retrieval on each of the multiple individual regions.

4. The circuit device according to claim 1 or 2, characterized in that, The retrieval circuit performs the retrieval by calculating the cross-correlation between the displayed image and the reference image using Fourier transform and inverse Fourier transform.

5. The circuit device according to claim 1 or 2, characterized in that, The error detection circuit calculates the histogram of each component of the structural components of the color space, performs autocorrelation calculation on the histogram of each component, calculates the distance of the peak that generates autocorrelation for each component, and calculates the index based on the maximum distance among the calculated distances.

6. The circuit device according to claim 1 or 2, characterized in that, The error detection circuit calculates the first histogram of each component of the color space structure based on the displayed image, calculates the second histogram of each component based on the reference image, performs cross-correlation operation on the first histogram and the second histogram of each component, and calculates the index based on the peak value of the cross-correlation peak.

7. The circuit device according to claim 1 or 2, characterized in that, The error detection circuit calculates a second index representing the degree of consistency between the foreground image of the image representing the given region in the displayed image and the reference image, based on the pixel values ​​of the displayed image and the reference image, or the pixel values ​​of the edge image of the displayed image and the edge image of the reference image, and performs the error detection based on the index and the second index.

8. The circuit device according to claim 1 or 2, characterized in that, The image acquisition circuit generates the display image by overlaying the second image with the first image in the given area. The background image is the image in the displayed image that corresponds to the first image.

9. A circuit device, characterized in that, The circuit device includes: Image acquisition circuit, which acquires and displays images; A retrieval circuit retrieves the display position of the foreground image in the display image based on a reference image that serves as a reference for the foreground image; the foreground image is an image of a given region in the display image. as well as An error detection circuit defines an error detection region containing the foreground image within the displayed image based on the display position. It calculates an index representing the degree of consistency between the foreground image and the reference image based on the pixel values ​​of the error detection region and the pixel values ​​of the reference image, or based on the pixel values ​​of the edge image of the error detection region and the pixel values ​​of the edge image of the reference image. The circuit then performs error detection, determining an error in the displayed image when the index is less than a predetermined threshold. When the retrieval circuit detects the display position in the retrieval, the error detection circuit performs the error detection; when the retrieval circuit does not detect the display position in the retrieval, the error detection circuit does not perform the error detection.

10. The circuit device according to claim 9, characterized in that, The retrieval circuit calculates the cross-correlation between the displayed image and the reference image, detects whether there is a correlation value exceeding a detection threshold in the cross-correlation, and if there is a correlation value exceeding the detection threshold, the position corresponding to the correlation value exceeding the detection threshold is detected as the display position; if there is no correlation value exceeding the detection threshold, the display position is not detected.

11. The circuit device according to claim 9 or 10, characterized in that, The retrieval circuit divides the displayed image into multiple strip regions, further divides each of the multiple strip regions into multiple individual regions, and performs the retrieval on each of the multiple individual regions.

12. The circuit device according to claim 9 or 10, characterized in that, The retrieval circuit performs the retrieval by calculating the cross-correlation between the displayed image and the reference image using Fourier transform and inverse Fourier transform.

13. The circuit device according to claim 9 or 10, characterized in that, The error detection circuit performs downsampling to reduce the number of pixels or resolution of the displayed image and the reference image, calculates distance information representing the distance between the pixel values ​​of the downsampled displayed image and the pixel values ​​of the downsampled reference image in the color space, and calculates the index based on the distance information.

14. The circuit device according to claim 13, characterized in that, The error detection circuit calculates the index based on the value obtained by dividing a given threshold by the distance information.

15. The circuit device according to claim 9 or 10, characterized in that, The error detection circuit performs a product-sum operation on the pixel values ​​of the edge image of the displayed image and the pixel values ​​of the edge image of the reference image, and calculates the index based on the result of the product-sum operation.

16. The circuit device according to claim 15, characterized in that, The error detection circuit masks the region in the edge image of the displayed image that corresponds to the background image, and uses the masked edge image of the displayed image to perform the product sum operation.

17. The circuit device according to claim 9 or 10, characterized in that, The image acquisition circuit generates the display image by overlaying the second image with the first image in the given area.

18. An electronic device, characterized in that, The electronic device includes the circuit arrangement as described in any one of claims 1 to 17.

19. An error detection method, characterized in that, The display position of the foreground image in the displayed image is retrieved based on a reference image that serves as a reference for the foreground image. The foreground image is an image of a given region within the displayed image. Based on the retrieved display position, the error detection region containing the foreground image is set in the display image. Calculate the histogram of pixel values ​​in the error detection region, and perform correlation operations using the histogram. Based on the results of the relevant calculations, an index representing the degree of dissimilarity between the foreground image and a background image equivalent to the foreground image is obtained. When the display position is detected in the search, error detection is performed to determine that the displayed image is incorrect if the index is less than a predetermined threshold. When the display position is not detected in the search, error detection is not performed.

20. An error detection method, characterized in that, The display position of the foreground image in the displayed image is retrieved based on a reference image that serves as a reference for the foreground image. The foreground image is an image of a given region within the displayed image. Based on the retrieved display position, the error detection region containing the foreground image is set in the display image. Based on the pixel values ​​of the error detection region and the pixel values ​​of the reference image, or based on the pixel values ​​of the edge image of the error detection region and the pixel values ​​of the edge image of the reference image, an index representing the degree of consistency between the foreground image and the reference image is calculated. When the display position is detected in the search, error detection is performed to determine that the displayed image is incorrect if the index is less than a predetermined threshold. When the display position is not detected in the search, error detection is not performed.

Images