Image processing circuit

Abstract

An image processing circuit (1000) includes: a first processing circuit that performs noise reduction processing that reduces noise of an image; a second processing circuit that performs super-resolution processing that increases resolution of the image; a calculation circuit that calculates a synthesis ratio that indicates a ratio of a plurality of images to be synthesized, based on respective strengths of the noise reduction processing and the super-resolution processing; a neural network circuit (1007) that performs the noise reduction processing or the super-resolution processing based on the strengths; a first synthesis circuit (1008) that synthesizes an image on which the noise reduction processing is performed by the first processing circuit and an image on which image processing is performed by the neural network circuit (1007), based on the synthesis ratio; and a second synthesis circuit (1010) that synthesizes an image on which the super-resolution processing is performed by the second processing circuit and an image on which image processing is performed by the neural network circuit (1007), based on the synthesis ratio.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to Japanese Patent Application No. 2022-003991, filed on January 13, 2022, entitled "Image Processing Circuit", the entire contents of which are incorporated herein by reference. Technical Field

[0003] The embodiments of this application relate to image processing circuits. Background Technology

[0004] There exist neural network circuits that circuitize neural networks. These circuits, for example, perform noise reduction processing to decrease image noise. Compared to conventional noise reduction circuits that circuitize noise reduction algorithms, neural network circuits reduce noise more effectively.

[0005] Therefore, neural network circuits are not limited to noise reduction processing, but are also intended for various image processing applications such as super-resolution processing. Furthermore, when performing multiple image processing operations, it is preferable that the intensity of each image processing operation can be varied according to each individual image processing operation.

[0006] Prior art literature

[0007] Patent documents

[0008] Patent Document 1: Japanese Patent Application Publication No. 2020-191046 Summary of the Invention

[0009] However, neural network circuits are large in scale, making it difficult to configure them for each image processing task. Furthermore, the sheer number of coefficients assigned to each neuron in a neural network circuit makes it difficult to prepare intensity settings for each of multiple image processing tasks.

[0010] The technical problem to be solved by this application is to provide an image processing circuit that, while using a neural network circuit, can change the intensity setting of each of multiple image processing operations.

[0011] The image processing circuit of the embodiment includes: a first processing circuit that performs image processing to reduce noise in an image, i.e., noise reduction processing; a second processing circuit that performs image processing to increase the resolution of an image, i.e., super-resolution processing; a calculation circuit that calculates a synthesis ratio representing the ratio of multiple images to be synthesized based on the respective intensity settings of the noise reduction processing and the super-resolution processing; a neural network circuit formed by a neural network that performs the noise reduction processing or the super-resolution processing based on the intensity settings; a first synthesis circuit that synthesizes an image for which the noise reduction processing was performed by the first processing circuit and an image for which the image processing was performed by the neural network circuit based on the synthesis ratio; and a second synthesis circuit that synthesizes an image for which the super-resolution processing was performed by the second processing circuit and an image for which the image processing was performed by the neural network circuit based on the synthesis ratio. Attached Figure Description

[0012] Figure 1 This is a diagram illustrating an example of the hardware configuration of a television device according to an embodiment;

[0013] Figure 2 This is a block diagram illustrating an example of the circuit configuration of an image processing circuit.

[0014] Figure 3 This is a diagram illustrating an example of how the synthesis ratio is calculated using a synthesis ratio calculation circuit;

[0015] Figure 4 This is a diagram illustrating an example of how the synthesis ratio is calculated using a synthesis ratio calculation circuit;

[0016] Figure 5 This is a diagram illustrating an example of how the synthesis ratio is calculated using a synthesis ratio calculation circuit.

[0017] Explanation of reference numerals in the attached figures

[0018] 10…Television device, 108…Signal processing unit, 1000…Image processing circuit, 1001…Level output circuit, 1002…Coefficient selection circuit, 1003…First coefficient group, 1004…Second coefficient group, 1005…Synthesis ratio calculation circuit, 1006…NR (noise reduction) circuit, 1007…Neural network circuit, 1008…First synthesis circuit, 1009…SR (Super Resolution) circuit, 1010…Second synthesis circuit, L1…First arrow, L2…Second arrow, P1, P2…Matching points, R1…First diagonal line, R2…Second diagonal line, R3…Third diagonal line, R4…Fourth diagonal line, R5…Fifth diagonal line, R6…Sixth diagonal line, R7…Seventh diagonal line. Detailed Implementation

[0019] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings.

[0020] Figure 1 This diagram illustrates an example of the hardware configuration of a television device 10 according to an embodiment. The television device 10 performs image processing on images input via broadcasting or the like. Then, the television device 10 displays the image after image processing.

[0021] like Figure 1 As shown, the television device 10 includes an antenna 101, input terminals 102a to 102c, a tuner 103, a demodulator 104, a demultiplexer 105, an A / D (analog-to-digital) converter 106, a selector 107, a signal processing unit 108, a speaker 109, a display panel 110, an operation unit 111, a light receiving unit 112, an IP communication unit 113, a CPU (Central Processing Unit) 114, a memory 115, a storage unit 116, a microphone 117, and an audio I / F (interface) 118.

[0022] Antenna 101 receives the broadcast signal from digital broadcasting and supplies the received broadcast signal to tuner 103 via input terminal 102a.

[0023] Tuner 103 selects the desired channel from the broadcast signal supplied by antenna 101 and supplies the selected broadcast signal to demodulator 104.

[0024] Demodulator 104 demodulates the broadcast signal supplied from tuner 103 and supplies the demodulated broadcast signal to demultiplexer 105.

[0025] The demultiplexer 105 separates the broadcast signal supplied from the demodulator 104 to generate video and audio signals, and supplies the generated video and audio signals to the selector 107.

[0026] Selector 107 selects one of the multiple signals supplied by demultiplexer 105, A / D converter 106, and input terminal 102c, and supplies the selected signal to signal processing unit 108.

[0027] The signal processing unit 108 performs prescribed signal processing on the video signal supplied from the selector 107 and supplies the processed video signal to the display panel 110. Additionally, the signal processing unit 108 performs prescribed signal processing on the audio signal supplied from the selector 107 and supplies the processed audio signal to the speaker 109. The signal processing unit 108 has... Figure 2 The image processing circuit 1000 shown is shown.

[0028] The speaker 109 outputs speech or various sounds based on the speech signal supplied from the signal processing unit 108. In addition, the speaker 109 changes the volume of the speech or various sounds to be output based on the control performed by the CPU 114.

[0029] The display panel 110, which serves as the display unit, displays images such as static images and dynamic images, other images, and text information based on the video signal supplied from the signal processing unit 108 or the control performed by the CPU 114.

[0030] Input terminal 102b receives analog signals such as video and audio signals input from external sources. Input terminal 102c receives digital signals such as video and audio signals input from external sources. For example, input terminal 102c can receive signals from a video recorder equipped with a drive unit that drives recording media such as BD (Blu-ray Disc) for recording and playback.

[0031] The A / D converter 106 supplies a digital signal to the selector 107, which is a signal generated by performing A / D conversion on the analog signal supplied from the input terminal 102b.

[0032] The operation unit 111 receives user operation input.

[0033] The light-receiving unit 112 receives infrared light from the remote control 119.

[0034] IP Communication Unit 113 is a communication interface used for IP (Internet Protocol) communication via network 40.

[0035] CPU114 controls the entire television device 10.

[0036] Memory 115 includes ROM for storing various computer programs executed by CPU 114, and RAM for providing work partitions for CPU 114.

[0037] The memory 116 is an HDD (Hard Disk Drive) or SSD (Solid State Drive), etc. The memory 116 records, for example, the signal selected by the selector 107 as video recording data.

[0038] The microphone 117, which serves as the voice input unit, acquires the user's voice and sends it to the audio I / F 118.

[0039] The audio I / F118 converts the voice acquired by the microphone 117 from analog to digital and sends it out as a voice signal to the CPU 114.

[0040] Next, the image processing circuit 1000 of the signal processing unit 108 will be described.

[0041] Figure 2 This is a block diagram illustrating an example of the circuit configuration of the image processing circuit 1000. For example... Figure 2 As shown, the image processing circuit 1000 includes a level output circuit 1001, a coefficient selection circuit 1002, a synthesis ratio calculation circuit 1005, an NR (noise reduction) circuit 1006, a neural network circuit 1007, a first synthesis circuit 1008, an SR (Super Resolution) circuit 1009, and a second synthesis circuit 1010.

[0042] The image processing circuit 1000 performs various types of image processing at different intensity levels on an input image. For example, the image processing circuit 1000 can set four intensity levels. As an example of multiple image processing operations, the image processing circuit 1000 performs noise reduction and super-resolution processing. However, the image processing circuit 1000 may also perform image processing different from noise reduction or super-resolution processing.

[0043] The intensity output circuit 1001 is a circuit that outputs the intensity levels of the noise reduction processing and the super-resolution processing, respectively. The intensity level is an intensity setting representing the intensity of each of the noise reduction and super-resolution processing. Furthermore, the intensity level can be a value determined by a user setting, a value determined based on various settings, a calculated value, or a value other than these. Moreover, if the intensity level is input externally from the image processing circuit 1000, the image processing circuit 1000 may not have the intensity output circuit 1001.

[0044] The NR circuit 1006 is a circuit that performs image processing, i.e., noise reduction processing, to reduce noise in an image. The NR circuit 1006 is an example of a first processing circuit. More specifically, the NR circuit 1006 is a circuit that implements a noise reduction processing algorithm. Furthermore, the NR circuit 1006 performs noise reduction processing corresponding to the intensity level of the noise reduction processing output from the level output circuit 1001.

[0045] Specifically, the NR circuit 1006 performs noise reduction processing on the input image input to the image processing circuit 1000, corresponding to the intensity level of the noise reduction processing. Then, the NR circuit 1006 outputs the noise-reduced image, i.e., the NR image, to the neural network circuit 1007 and the first synthesis circuit 1008.

[0046] The SR circuit 1009 is a circuit that performs image processing, i.e., super-resolution processing, to improve the resolution of an image. The SR circuit 1009 is an example of a second processing circuit. For example, super-resolution processing involves making fine lines in an image appear to float or emphasizing edges. More specifically, the SR circuit 1009 is a circuit that implements the super-resolution processing algorithm. Furthermore, the SR circuit 1009 performs super-resolution processing corresponding to the intensity level of the super-resolution processing output from the level output circuit 1001.

[0047] Specifically, the SR circuit 1009 performs super-resolution processing on the first composite image synthesized by the first synthesis circuit 1008, corresponding to the intensity level of the super-resolution processing. Then, the NR circuit 1006 outputs the super-resolution processed image, i.e., the SR image, to the second synthesis circuit 1010.

[0048] The coefficient selection circuit 1002 selects either a first coefficient group 1003 for enabling the neural network circuit 1007 to perform noise reduction processing, or a second coefficient group 1004 for enabling the neural network circuit 1007 to perform super-resolution processing, based on the intensity level. The coefficient selection circuit 1002 is an example of a selection circuit. The first coefficient group 1003 is a group of coefficients set for each neuron of the neural network circuit 1007. For example, the first coefficient group 1003 is a group of coefficients used for noise reduction processing with an intensity level of 4. Furthermore, the first coefficient group 1003 is an example of first setting information. The second coefficient group 1004 is a group of coefficients set for each neuron of the neural network circuit 1007. For example, the second coefficient group 1004 is a group of coefficients used for super-resolution processing with an intensity level of 4. Furthermore, the second coefficient group 1004 is an example of second setting information.

[0049] More specifically, the coefficient selection circuit 1002 compares the intensity level of the noise reduction process with the intensity level of the super-resolution process. If the intensity level of the noise reduction process is higher than that of the super-resolution process, the coefficient selection circuit 1002 selects the coefficient group for the noise reduction process, i.e., the first coefficient group 1003. Conversely, if the intensity level of the super-resolution process is higher than that of the noise reduction process, the coefficient selection circuit 1002 selects the coefficient group for the super-resolution process, i.e., the second coefficient group 1004. Then, the coefficient selection circuit 1002 outputs either the first coefficient group 1003 or the second coefficient group 1004.

[0050] Furthermore, when the intensity level of the noise reduction processing is the same as the intensity level of the super-resolution processing, the coefficient selection circuit 1002 selects the first coefficient group 1003 to enable the neural network circuit 1007 to perform noise reduction processing at an intensity level of 4. It should be noted that the coefficient selection circuit 1002 could also select the second coefficient group 1004 to enable the neural network circuit 1007 to perform super-resolution processing, but the first coefficient group 1003 is preferred. Super-resolution processing causes fine lines in the image to float or emphasizes edges. Additionally, the neural network circuit 1007 performs image processing more powerfully than the SR circuit 1009 or the NR circuit 1006. Therefore, when super-resolution processing is more powerful than noise reduction processing, there is a possibility that noise may be emphasized. Therefore, when the intensity level of the noise reduction processing is the same as the intensity level of the super-resolution processing, the coefficient selection circuit 1002 selects the first coefficient group 1003.

[0051] also, Figure 2 The coefficient selection circuit 1002 shown stores either a first coefficient group 1003 or a second coefficient group 1004. However, the coefficient selection circuit 1002 may also not store the first coefficient group 1003 or the second coefficient group 1004. For example, the coefficient selection circuit 1002 may also obtain the first coefficient group 1003 or the second coefficient group 1004 from a storage medium such as RAM and output the obtained first coefficient group 1003 or the second coefficient group 1004.

[0052] Furthermore, the coefficient selection circuit 1002 does not have five coefficient groups with intensity levels from 0 to 4, but rather one coefficient group with an intensity level of 4. Thus, the coefficient selection circuit 1002 selects a coefficient group representing a subset of the intensity levels from multiple stages. Therefore, the storage medium does not need to store coefficient groups representing all combinations of intensity levels for each image processing step. Here, the coefficient groups are set for each neuron of the neural network circuit 1007, resulting in a large data capacity. Therefore, if it were possible to prepare coefficient groups representing all combinations of intensity levels for each image processing step, a very large storage medium would be required. The coefficient selection circuit 1002 selects coefficient groups from either the first coefficient group 1003 or the second coefficient group 1004, thereby also avoiding the need for a very large storage medium.

[0053] The neural network circuit 1007 is a circuit formed by a neural network that performs noise reduction or super-resolution processing based on the intensity levels of noise reduction and super-resolution processing output from the level output circuit 1001. The neural network circuit 1007 is an example of a neural network circuit. More specifically, the neural network circuit 1007 performs image processing on either a first coefficient group 1003 or a second coefficient group 1004 selected by the coefficient selection circuit 1002 according to the intensity levels of the noise reduction and super-resolution processing.

[0054] That is, when the first coefficient group 1003 is output from the coefficient selection circuit 1002, the neural network circuit 1007 performs noise reduction processing with an intensity level of 4 as shown by the first coefficient group 1003. Furthermore, when the second coefficient group 1004 is output from the coefficient selection circuit 1002, the neural network circuit 1007 performs super-resolution processing with an intensity level of 4 as shown by the second coefficient group 1004.

[0055] Furthermore, the neural network circuit 1007 performs image processing corresponding to the intensity level for the image that has undergone noise reduction processing by the NR circuit 1006. That is, the neural network circuit 1007 performs noise reduction processing at an intensity level of 4, or super-resolution processing at an intensity level of 4. Then, the neural network circuit 1007 outputs the image that has undergone image processing, i.e., the neural network image, to the first synthesis circuit 1008 and the second synthesis circuit 1010.

[0056] The first synthesis circuit 1008 synthesizes an NR image (which has undergone noise reduction processing by the NR circuit 1006) and a neural network image (which has undergone image processing by the neural network circuit 1007) based on a synthesis ratio calculated by the synthesis ratio calculation circuit 1005. The first synthesis circuit 1008 is one example of a first synthesis circuit. Furthermore, the first synthesis circuit 1008 can synthesize images using any method. For example, the first synthesis circuit 1008 can synthesize an image by multiplying the synthesis ratio as a weighting coefficient. Then, the first synthesis circuit 1008 outputs the first synthesized image generated by synthesizing the NR image and the neural network image to the SR circuit 1009.

[0057] The second synthesis circuit 1010 synthesizes an SR image (which has undergone super-resolution processing by the SR circuit 1009) and a neural network image (which has undergone image processing by the neural network circuit 1007) based on a synthesis ratio calculated by the synthesis ratio calculation circuit 1005. The second synthesis circuit 1010 is one example of a second synthesis circuit. Alternatively, the second synthesis circuit 1010 can synthesize images using any method. For example, the second synthesis circuit 1010 can synthesize an image by multiplying the synthesis ratio as a weighting coefficient. Then, the second synthesis circuit 1010 outputs the resulting image generated by synthesizing the SR image and the neural network image.

[0058] The synthesis ratio calculation circuit 1005 calculates the synthesis ratio, representing the ratio of the multiple images to be synthesized, based on the respective intensity levels of the noise reduction processing and the super-resolution processing. The synthesis ratio calculation circuit 1005 is an example of a calculation circuit. More specifically, the synthesis ratio calculation circuit 1005 calculates the synthesis ratio of the image synthesized from the image processed by the NR circuit 1006 or the SR circuit 1009 and the neural network image processed by the neural network circuit 1007, based on the difference between the intensity levels of the noise reduction processing and the super-resolution processing.

[0059] Figure 3 This diagram illustrates an example of the method for calculating the composite ratio using the composite ratio calculation circuit 1005. The composite ratio calculation circuit 1005 utilizes... Figure 3 The graph shown is used to calculate the synthesis ratio. The vertical axis of the graph represents the intensity level of the noise reduction process. The horizontal axis of the graph represents the intensity level of the super-resolution process. The first diagonal line R1, the second diagonal line R2, the third diagonal line R3, and the fourth diagonal line R4 are diagonal lines used to determine the synthesis ratio when the neural network circuit 1007 performs noise reduction processing. The fifth diagonal line R5, the sixth diagonal line R6, and the seventh diagonal line R7 are diagonal lines used to determine the synthesis ratio when the neural network circuit 1007 performs super-resolution processing.

[0060] exist Figure 3 In the diagram shown, the synthesis ratio calculation circuit 1005 detects the intersection point between the intensity level of the noise reduction process output from the level output circuit 1001 and the intensity level of the super-resolution process. If the intersection point exists on either the first arrow L1 or the second arrow L2, the synthesis ratio calculation circuit 1005 plots the intersection point as matching points P1 and P2 (refer to...). Figure 4 , Figure 5 If there is no intersection point on the first arrow L1 or the second arrow L2, the composite ratio calculation circuit 1005 determines the diagonal line that passes through the intersection point. That is, the composite ratio calculation circuit 1005 selects the diagonal line that passes through the intersection point from the first diagonal line R1, the second diagonal line R2, the third diagonal line R3, the fourth diagonal line R4, the fifth diagonal line R5, the sixth diagonal line R6, and the seventh diagonal line R7.

[0061] In addition, the synthesis ratio calculation circuit 1005 plots the points where the first arrow L1 or the second arrow L2 is orthogonal to the selected diagonal line as matching points P1 and P2 (refer to...). Figure 4 , Figure 5 Then, the synthesis ratio calculation circuit 1005 calculates the ratio based on the matching points P1 and P2 in the first arrow L1 or the second arrow L2 (refer to...). Figure 4 , Figure 5 The synthesis ratio is determined by the amount of material used.

[0062] Here, use Figure 4 and Figure 5 A specific example is given to illustrate the method of calculating the synthesis ratio by the synthesis ratio calculation circuit 1005. Figure 4 This is a diagram illustrating an example of the method for calculating the synthesis ratio by the synthesis ratio calculation circuit 1005. Figure 4 The diagram shows the state where the noise reduction intensity level is 3 and the super-resolution processing intensity level is 2.

[0063] The synthesis ratio calculation circuit 1005 detects the orthogonal point between the intensity level of the noise reduction process and the intensity level of the super-resolution process. Since no orthogonal point is detected on either the first arrow L1 or the second arrow L2, the synthesis ratio calculation circuit 1005 detects the third diagonal line R3 passing through the orthogonal point. The synthesis ratio calculation circuit 1005 identifies the point where the first arrow L1 and the third diagonal line R3 are orthogonal as the matching point P1. Then, the synthesis ratio calculation circuit 1005 determines the synthesis ratio of the image based on the matching point P1. Specifically, the matching point P1 divides the first arrow L1 into a 1:3 ratio. Therefore, the synthesis ratio calculation circuit 1005 determines the ratio of the NR image to be 75% and the ratio of the neural network image to be 25%.

[0064] Figure 5 This is a diagram illustrating an example of the method for calculating the synthesis ratio by the synthesis ratio calculation circuit 1005. Figure 5 The diagram illustrates a state where the noise reduction intensity level is 1 and the super-resolution processing intensity level is 3. The synthesis ratio calculation circuit 1005 detects the orthogonal point between the noise reduction intensity level and the super-resolution processing intensity level. Because a detected orthogonal point exists on either the first arrow L1 or the second arrow L2, the synthesis ratio calculation circuit 1005 identifies the orthogonal point as a matching point P2. Then, the synthesis ratio calculation circuit 1005 determines the image synthesis ratio based on the matching point P2. Specifically, the matching point P2 divides the second arrow L2 into a 1:1 ratio. Therefore, the synthesis ratio calculation circuit 1005 determines the ratio of the NR image to 50% and the ratio of the neural network image to 50%.

[0065] With matching points P1 and P2 (reference) Figure 4 , Figure 5 Approaching the fourth diagonal line R4, the synthesis ratio operation circuit 1005 reduces the proportion of the neural network image in the image synthesis. In other words, as the difference between the intensity level of the noise reduction processing and the intensity level of the super-resolution processing decreases, the synthesis ratio operation circuit 1005 reduces the proportion of the neural network image processed by the neural network circuit 1007 in the NR image or SR image processed by the NR circuit 1006 or SR circuit 1009, compared to the proportion of the neural network image processed by the neural network circuit 1007.

[0066] exist Figure 3 In the diagram shown, matching points P1 and P2 are marked on the first arrow L1 (refer to...). Figure 4 , Figure 5 In the case of matching points P1 and P2 (refer to...), Figure 4 , Figure 5 Approaching the fourth diagonal line R4, the synthesis ratio calculation circuit 1005 reduces the proportion of the neural network image that has undergone noise reduction processing by the neural network circuit 1007. On the other hand, matching points P1 and P2 are marked on the second arrow L2 (refer to...). Figure 4 , Figure 5 In the case of matching points P1 and P2 (refer to...), Figure 4 , Figure 5 Approaching the fourth diagonal line R4, the synthesis ratio operation circuit 1005 reduces the ratio of the neural network image that has undergone super-resolution processing by the neural network circuit 1007.

[0067] In this way, the synthesis ratio calculation circuit 1005 reduces the proportion of the neural network image based on the difference between the intensity level of the noise reduction processing and the intensity level of the super-resolution processing, thereby maintaining the continuity of the image processing effect when the intensity level is changed. Here, the neural network circuit 1007 achieves a higher image processing effect compared to the NR circuit 1006 and the SR circuit 1009. Therefore, the image processing effect will change when the content of the image processing performed by the neural network circuit 1007 is switched.

[0068] For example, when the neural network circuit 1007 performs noise reduction processing, the first synthesis circuit 1008 generates a first synthesized image by synthesizing the NR image and the neural network image. When the image processing performed by the neural network circuit 1007 is switched from noise reduction processing to super-resolution processing, the first synthesis circuit 1008 generates the first synthesized image based on the NR image. In this case, since the neural network image is not used for image synthesis, if the proportion of the neural network image in the first synthesized image is high, the effect of the noise reduction processing will change significantly. On the other hand, if the proportion of the neural network image in the first synthesized image is low, the effect of the noise reduction processing will change less significantly.

[0069] Furthermore, when the difference in image processing setting levels is small, the neural network circuit 1007 switches the content of the image processing to be performed with minute changes in setting levels. Therefore, the synthesis ratio calculation circuit 1005 can maintain the continuity of the image processing effect by reducing the proportion of the neural network image as the difference in image processing setting levels decreases.

[0070] Furthermore, the synthesis ratio calculation circuit 1005 is not limited to Figure 3 The chart shown can also be used to calculate the blending ratio. For example, the blending ratio calculation circuit 1005 calculates the difference between the intensity level of the noise reduction process and the intensity level of the super-resolution process. Then, the blending ratio calculation circuit 1005 calculates the blending ratio of the image based on the difference.

[0071] For example, when the difference is 0, the ratio calculation circuit 1005 sets the ratio of the image generated by the neural network circuit 1007 to 0%, and the ratio of the image generated by the NR circuit 1006 or the SR circuit 1009 to 100%. Furthermore, when the difference is 1, the ratio calculation circuit 1005 sets the ratio of the image generated by the neural network circuit 1007 to 25%, and the ratio of the image generated by the NR circuit 1006 or the SR circuit 1009 to 75%. Furthermore, when the difference is 2, the ratio calculation circuit 1005 sets the ratio of the image generated by the neural network circuit 1007 to 50%, and the ratio of the image generated by the NR circuit 1006 or the SR circuit 1009 to 50%. Furthermore, when the difference is 3, the ratio calculation circuit 1005 sets the ratio of the image generated by the neural network circuit 1007 to 75%, and the ratio of the image generated by the NR circuit 1006 or the SR circuit 1009 to 25%. Furthermore, when the difference is 4, the synthesis ratio calculation circuit 1005 sets the ratio of the image generated by the neural network circuit 1007 to 100%, and sets the ratio of the image generated by the NR circuit 1006 or the SR circuit 1009 to 0%. In addition, the ratio of each image can be arbitrarily changed.

[0072] As described above, the image processing circuit 1000 of the embodiment includes: an NR circuit 1006 that performs noise reduction processing; an SR circuit 1009 that performs super-resolution processing; and a neural network circuit 1007 that performs noise reduction processing or super-resolution processing based on intensity levels. Furthermore, the synthesis ratio calculation circuit 1005 calculates a synthesis ratio, representing the ratio of the multiple images to be synthesized, based on the respective intensity levels of the noise reduction processing and the super-resolution processing. The first synthesis circuit 1008 synthesizes an NR image that has undergone noise reduction processing by the NR circuit 1006 and a neural network image that has undergone image processing by the neural network circuit 1007, based on the synthesis ratio. The second synthesis circuit 1010 synthesizes an SR image that has undergone super-resolution processing by the SR circuit 1009 and a neural network image that has undergone image processing by the neural network circuit 1007, based on the synthesis ratio. Then, the image processing circuit 1000 outputs the neural network image synthesized by the second synthesis circuit 1010.

[0073] In this way, the image processing circuit 1000 causes the neural network circuit 1007 to perform noise reduction or super-resolution processing based on the intensity level. Additionally, the image processing circuit 1000 calculates the synthesis ratio based on the intensity level. Then, the image processing circuit 1000 synthesizes an NR image and a neural network image based on the synthesis ratio, and also synthesizes an SR image and a neural network image based on the synthesis ratio. Therefore, the image processing circuit 1000 can utilize the neural network circuit 1007 and can change the respective intensity levels of multiple image processing operations.

[0074] Furthermore, in the above embodiment, the signal processing unit 108 is described as having an image processing circuit 1000. However, components other than the signal processing unit 108 may also have an image processing circuit 1000.

[0075] Furthermore, in the above-described embodiment, the television device 10 is described as having an image processing circuit 1000. However, devices other than the television device 10 may also have an image processing circuit 1000. For example, personal computers, smartphones, tablet computers, video recorders, and display devices may also have an image processing circuit 1000.

[0076] The embodiments described herein are illustrated, but are provided as examples and are not intended to limit the scope of the application. These new embodiments can be implemented in a wide variety of other forms, with various omissions, substitutions, and modifications possible without departing from the spirit of the invention. These embodiments or variations thereof are included within the scope and spirit of the application, and within the scope of the technical solutions described in the claims and their equivalents.

Claims

1. An image processing circuit, comprising: The first processing circuit performs noise reduction processing, which is image processing to reduce noise in the image. The second processing circuit performs super-resolution processing, which is image processing that improves the resolution of an image. The computing circuit calculates the synthesis ratio, which represents the ratio of the multiple images to be synthesized, based on the respective intensity settings of the noise reduction process and the super-resolution process. A neural network circuit, formed by a neural network, which performs the noise reduction process or the super-resolution process based on the intensity setting; A first synthesis circuit, based on the synthesis ratio, synthesizes an image for which the noise reduction processing was performed by the first processing circuit and an image for which the image processing was performed by the neural network circuit; as well as The second synthesis circuit, based on the synthesis ratio, synthesizes the image that has undergone the super-resolution processing by the second processing circuit and the image that has undergone the image processing by the neural network circuit; The computing circuit calculates the synthesis ratio for synthesizing two images based on the difference between the intensity setting of the noise reduction processing and the intensity setting of the super-resolution processing: one is an image processed by the first processing circuit or the second processing circuit, and the other is an image processed by the neural network circuit.

2. The image processing circuit according to claim 1, wherein, The image processing circuit further includes a selection circuit that selects either first setting information or second setting information based on the intensity setting. The first setting information is used to cause the neural network circuit to perform the noise reduction processing, and the second setting information is used to cause the neural network circuit to perform the super-resolution processing. The neural network circuit performs the image processing based on the first setting information or the second setting information selected by the selection circuit.

3. The image processing circuit according to claim 2, wherein, The selection circuit selects either the first setting information or the second setting information of a portion of the intensity settings in multiple stages.

4. The image processing circuit according to claim 1, wherein, As the difference between the intensity setting of the noise reduction process and the intensity setting of the super-resolution process decreases, the computing circuit reduces the proportion of images processed by the neural network circuit among the images processed by the first processing circuit or the second processing circuit and the images processed by the neural network circuit.

5. The image processing circuit according to any one of claims 1 to 4, wherein, The first processing circuit performs the noise reduction process on the input image. The neural network circuit performs image processing corresponding to the intensity setting on the image for which the noise reduction processing has been performed by the first processing circuit. The first synthesis circuit synthesizes an image that has undergone the noise reduction process by the first processing circuit and an image that has undergone the image processing process by the neural network circuit. The second processing circuit performs the super-resolution processing on the image synthesized by the first synthesis circuit. The second synthesis circuit synthesizes the image that has undergone the super-resolution processing by the second processing circuit and the image that has undergone the image processing by the neural network circuit.

Images