Semiconductor equipment

Abstract

To reduce the load on the image processing circuit of a semiconductor device. [Solution] The semiconductor device 1 comprises a parameter storage unit 16 and an image processing circuit 15. The image processing circuit 15 includes a scaling calculation unit 121 that changes the size of the input image based on the scaling ratio or reduction ratio set by the scaling ratio setting information. Furthermore, the image processing circuit 15 includes an optimized image area calculation unit 111 that calculates the image area size necessary to output the output image based on the output size of the output image and the scaling ratio, and an input image area comparison unit 112 that compares the required image area size with the input size of the input image, selects the image area of ​​the smaller size among the required image area size and the input size, and uses the selected image area for image processing of the input image.

Description

Technical Field

[0001] The present disclosure relates to a semiconductor device, for example, to a technology of a semiconductor device including an image processing circuit.

Background Art

[0002] Techniques for reducing the scale and power consumption of a circuit for changing the order of image processing are known (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, image quality has been enhanced to high definition. In such a situation, in a semiconductor device including an image processing circuit, the amount of information to be transferred for performing image processing between the semiconductor device and a memory has been increasing. For this reason, the load on the image processing circuit included in the semiconductor device has been increasing.

[0005] Other problems and novel features will become apparent from the description of this specification and the accompanying drawings.

Means for Solving the Problems

[0006] A typical embodiment of the present disclosure has the following configuration. A semiconductor device according to one embodiment includes an image processing circuit. The image processing circuit calculates an image area size necessary for outputting an output image based on an output size of the output image and a magnification factor. The image processing circuit compares the necessary image area size with an input size of an input image, selects an image area having a smaller size between the necessary image area size and the input size, and uses the selected image area for image processing of the input image.

Effects of the Invention

[0007] According to a representative embodiment of this disclosure, a technology can be provided that can reduce the load on the image processing circuit of a semiconductor device. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a schematic diagram showing an example of the configuration of a semiconductor device according to the embodiment. [Figure 2] Figure 2 is a diagram illustrating an example of image processing in a semiconductor device according to an embodiment. [Figure 3] Figure 3 is a flowchart showing an example of image processing for a semiconductor device according to the embodiment. [Figure 4] Figure 4 is a diagram illustrating an example of an image region in the embodiment. [Figure 5] Figure 5 shows an example of an unnecessary image region in the case shown in Figure 4. [Figure 6] Figure 6 shows an example of the storage information for the descriptor list in the embodiment. [Figure 7] Figure 7 shows relationship information, which is an example of the relationship between the parameters of the embodiment and the storage address. [Figure 8] Figure 8 shows the image region of the input image in Example 1. [Figure 9] Figure 9 shows the scaling ratio and the image region of the enlarged image for Example 1. [Figure 10] Figure 10 shows the image region of the output image of Example 1. [Figure 11] Figure 11 shows the image region after the optimization calculation in Example 1. [Figure 12] Figure 12 shows the image region after comparative updating in Example 1. [Figure 13] Figure 13 is a schematic diagram showing an example of the specifications and calculation results of Example 1. [Figure 14] Figure 14 is a comparison of the output image area of ​​Example 1 and the output image area of ​​Example 2. [Figure 15] FIG. 15 is a diagram showing an example of the storage information of the descriptor list in the second embodiment. [Figure 16] FIG. 16 is a schematic diagram showing an example of the specifications and calculation results in the second embodiment.

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the drawings, the same components are generally denoted by the same reference numerals, and repeated descriptions are omitted. In the drawings, the representation of the components may not represent the actual position, size, shape, range, etc. in order to facilitate the understanding of the invention.

[0010] <Embodiment> (Configuration of the Semiconductor Device) FIG. 1 is a diagram schematically showing an example of the configuration of the semiconductor device 1. The semiconductor device 1 is connected to the memory 12. The semiconductor device 1 receives image information from the memory 12 and outputs the image information to the memory. The semiconductor device 1 is a semiconductor device that processes image information. In the present embodiment, the memory 12 is described as being provided separately from the semiconductor device 1, but the memory 12 may be configured to be included in the semiconductor device 1.

[0011] As shown in FIG. 1, the semiconductor device 1 includes an image processing circuit 15, a CPU 11, a memory controller 125, a bus controller 126, and other IP groups 127. The CPU 11 and other IP groups 127 communicate with the image processing circuit 15 via the bus controller 126. The image processing circuit 15 communicates with the memory 12 via the bus controller 126 and the memory controller 125.

[0012] The memory 12 stores descriptor lists 124a to 124n, input image information 13, and output image information 14. The descriptor lists 124a to 124n each include input image setting information, magnification ratio setting information, output image setting information, and a parameter list group. The input image setting information includes the input size of the image area of the input image input from the memory 12 to the image processing circuit 15. The magnification ratio setting information includes the magnification ratio for setting the magnification ratio of the input image. The output image setting information includes the output size of the image area of the output image output from the image processing circuit 15 to the memory 12. The parameter list group includes parameters storing instruction information for the next frame. The instruction information is, for example, a parameter for instructing the frame for which the processing of the input image is to be executed next. The input image information 13 is information on the input image input to the image processing circuit 15. The output image information 14 is information on the output image output from the image processing circuit 15.

[0013] The information stored in the descriptor lists 124a to 124n is set in a setting unit described later in the image processing circuit 15 before the input image is read from the memory 12 by the image processing circuit 15. For example, the input image setting information is set in the input image read information setting unit 17 (first setting unit). The magnification ratio setting information is set in the magnification ratio information setting unit 18 (second setting unit). The output image setting information is set in the output image write information setting unit 19 (third setting unit). The memory 12 includes a plurality of descriptor lists 124. The image processing circuit 15 executes image processing of the input image according to the above instruction information included in the descriptor list. Details of these processes will be described later.

[0014] The image processing circuit 15 processes the input image received from the memory 12 and changes its size according to a specified scaling ratio. The CPU 11 is a processor that instructs the image processing circuit 15 on the necessary setting information. The other IP group 127 is a group of other IPs that make access requests to the memory 12. The memory controller 125 performs read control or write control on the information stored in the memory 12. The bus controller 126 mediates the bus priority between the image processing circuit 15 and various IPs such as the CPU 11, and transmits and receives information with the memory controller 125.

[0015] The image processing circuit 15 includes a parameter storage unit 16, an input image region determination unit 110, an image input / output control unit 113, an input image format conversion unit 120, a scaling calculation unit 121, an output image format conversion unit 122, and a flow control unit 123. The parameter storage unit 16 is composed of, for example, registers that store information. The input image region determination unit 110, the image input / output control unit 113, the input image format conversion unit 120, the scaling calculation unit 121, the output image format conversion unit 122, and the flow control unit 123 are composed of, for example, circuits.

[0016] The parameter storage unit 16 stores various setting parameters in the image processing circuit 15. The parameter storage unit 16 includes an input image reading information setting unit 17 (first setting unit), a scaling ratio information setting unit 18 (second setting unit), and an output image writing information setting unit 19 (third setting unit).

[0017] The input image reading information setting unit 17 is configured with the above-mentioned input image setting information. More specifically, when reading an input image, the input image reading information setting unit 17 is configured with information indicating the starting position, image size, and image format for the input image information 13 in the memory 12.

[0018] The scaling ratio information setting unit 18 is configured to determine the scaling ratio at which the calculation will be performed on the input image (corresponding to the input image information 13).

[0019] The output image writing information setting unit 19 is configured with the above-mentioned output image setting information. More specifically, when writing output image information 14, the output image writing information setting unit 19 is configured with information representing the starting position on the memory 12, the image size, and the image format. The image size (output size) is determined according to the display size of a device connected to the semiconductor device 1, for example, a liquid crystal display device. The image size is also determined according to the processing of the image processing circuit 15, for example. In this embodiment, this subsequent processing corresponds to the processing of the other IP group 127 described later.

[0020] The input image region determination unit 110 performs a reduction determination of the read image region for the input image input to the image processing circuit 15. The input image region determination unit 110 includes an optimized image region calculation unit 111 and an input image region comparison unit 112.

[0021] The optimized image region calculation unit 111 calculates the truly necessary image region based on the output image writing information set in the output image writing information setting unit 19 and the scaling ratio information from the scaling ratio information setting unit 18. For example, the optimized image region calculation unit 111 calculates the image region size required to output the output image based on the output size and scaling ratio of the output image information 14. In this embodiment, the optimized image region calculation unit 111 calculates the required image region size by dividing the output size of the output image information 14 by the scaling ratio. Details of the process will be described later.

[0022] The input image area comparison unit 112 updates the value of the input image read information setting unit 17 if it determines, based on the calculation results of the optimized image area calculation unit 111, that the image area of ​​the output image can be optimized to be reduced. For example, the input image area comparison unit 112 compares the required image area size calculated by the optimized image area calculation unit 111 with the input size of the input image information 13. After comparison, the input image area comparison unit 112 selects the smaller of the required image area size and the input size of the input image information 13, and uses the selected image area for image processing of the input image. Details of the processing will be described later.

[0023] The image input / output control unit 113 manages the internal timing of reading information from a specified address to the memory 12 and writing information to a specified address when the image processing circuit 15 performs image processing. The image input / output control unit 113 includes an information read control unit 114, an input image FIFO 115, an input image FIFO control unit 116, an information write control unit 117, an output image FIFO 118, and an output image FIFO control unit 119.

[0024] The information read control unit 114 controls the reading of input image information 13 from the memory 12. The input image FIFO 115 temporarily stores the information read from the memory 12 under the control of the information read control unit 114, and then transfers the stored information to the subsequent input image format conversion unit 120 at the timing of the internal image processing. The input image FIFO control unit 116 checks the available capacity of the input image FIFO 115 and sends a start command to the information read control unit 114. The information write control unit 117 executes the control to write output image information 14 to the memory 12. The output image FIFO 118 temporarily stores the enlarged or reduced image under the control of the output image FIFO control unit 119, and after a certain amount of information has been accumulated, it sends the information to the memory 12. The output image FIFO control unit 119 checks the available capacity of the output image FIFO 118, temporarily stores the output image information 14, and after a certain amount of information has been accumulated, prompts the information writing control unit 117 to write to the memory 12.

[0025] The input image format conversion unit 120 performs a process to convert the input image information 13 sent from the input image FIFO 115 into an image format suitable for subsequent processing.

[0026] The scaling calculation unit 121 receives scaling ratio setting information from the scaling ratio information setting unit 18 and calculates the scaling of the input image. More specifically, the scaling calculation unit 121 changes the image size of the input image information 13 sent from the input image format conversion unit 120 according to the specified scaling ratio. The scaling calculation unit 121 uses the changed input image information 13 as output image information 14 and sends out the output image information 14 according to the output image area.

[0027] The output image format conversion unit 122 converts the output image information 14 sent from the scaling calculation unit 121 into an image format appropriate for subsequent processing.

[0028] The Flow control unit 123 controls the entire image processing circuit 15 based on instructions from the CPU 11. The Flow control unit 123 also reads the appropriate descriptor list from descriptor lists 124a to 124n in the memory 12, decodes the contents of the descriptor list, and sets various registers in the specified image processing circuit 15.

[0029] Furthermore, Figure 1 shows paths S1 through S10. Path S1 represents the path for setting parameters from the CPU 11 to the image processing circuit 15.

[0030] Path S2 is a path for setting parameters in the input image area comparison unit 112. The parameters set in path S2 are, for example, IMG_IN_HSIZE and IMG_IN_VSIZE. IMG_IN_HSIZE indicates the number of pixels or required bytes in the horizontal direction of the input image. IMG_IN_VSIZE indicates the number of pixels or required bytes in the vertical direction of the input image.

[0031] Route S3 is a route for setting parameters in the optimized image region calculation unit 111. The parameters set in route S3 are, for example, IMG_HSCALE and IMG_VSCALE. IMG_HSCALE indicates the horizontal scaling factor of the input image. IMG_VSCALE indicates the vertical scaling factor of the input image.

[0032] Route S4 is a route for setting parameters in the optimized image region calculation unit 111. The parameters set in route S4 are, for example, IMG_OUT_HSIZE and IMG_OUT_VSIZE. IMG_OUT_HSIZE indicates the number of pixels or required bytes in the horizontal direction of the output image. IMG_OUT_VSIZE indicates the number of pixels or required bytes in the vertical direction of the output image.

[0033] Route S5 is a route for setting parameters indicating the size of the updated input image after comparison, as updated by the input image area comparison unit 112, to the information readout control unit 114. The parameters set in route S5 are, for example, IMG_IN_HSIZE_RENEW and IMG_IN_VSIZE_RENEW. IMG_IN_HSIZE_RENEW indicates the number of pixels or required bytes in the horizontal direction of the updated input image. IMG_IN_VSIZE_RENEW indicates the number of pixels or required bytes in the vertical direction of the updated input image.

[0034] Route S6 is a route for setting parameters in the information readout control unit 114. The parameter set in route S6 is, for example, IMG_IN_STADD. IMG_IN_STADD indicates the starting address when reading the input image information 13 from the memory 12.

[0035] Route S7 is a route for setting parameters in the input image format conversion unit 120. The parameter set in route S7 is, for example, IMG_IN_FMT. IMG_IN_FMT indicates the image format of the input image.

[0036] Route S8 is a route for setting parameters in the output image format conversion unit 122. The parameter set in route S8 is, for example, IMG_OUT_FMT. IMG_OUT_FMT indicates the image format of the output image.

[0037] Route S9 is a route for setting parameters in the information writing control unit 117. The parameter set in route S9 is, for example, IMG_OUT_STADD. IMG_OUT_STADD indicates the starting address when writing the output image.

[0038] Route S10 is a route for setting the calculation results calculated by the optimized image region calculation unit 111 to the input image region comparison unit 112. The calculation results set in route S10 are, for example, CAL_IN_HSIZE and CAL_IN_VSIZE. CAL_IN_HSIZE indicates the number of pixels or required bytes in the horizontal direction of the input image after processing by the optimized image region calculation unit 111. CAL_IN_VSIZE indicates the number of pixels or required bytes in the vertical direction of the input image after processing by the optimized image region calculation unit 111.

[0039] (Image processing for semiconductor devices) Figure 2 is a diagram illustrating an example of image processing for semiconductor device 1. The semiconductor device 1 includes, as an example of IP group 127, CRU127a, 3DGE127b, VCD127c, LCDC127d, and DRP127e. In the semiconductor device 1, in addition to the image processing circuit 15, CRU127a, 3DGE127b, VCD127c, LCDC127d, and DRP127e can access the memory 12 via the bus controller 126 and the memory controller 125, respectively. The processing of steps ST1 to ST7 shown in Figure 2 will be described later with reference to Figure 3.

[0040] CRU127a is a camera device for capturing images. 3DGE127b is a device for GPU processing. VCD127c is a device for image compression. LCDC127d is a liquid crystal display device for displaying images. DRP127e is a device for performing conversions for AI processing.

[0041] Next, an example of image processing when the semiconductor device 1 outputs an input image captured by the camera device to a liquid crystal display device will be described. Figure 3 is a flowchart of an example of image processing by the semiconductor device 1. As shown in Figure 3, in step ST1, the input image captured by the CRU127a is stored in the memory 12. Next, in step ST2, the input image is read from the memory 12 by the image processing circuit 15, and subsequent processing, i.e., image processing for VCD (image compression) conversion, is performed, and the processed image is stored in the memory 12.

[0042] Next, in step ST3, the input image stored in memory 12 is read by the image processing circuit 15, and subsequent processing, namely image processing for DRP (for AI processing) conversion, is performed, and the processed image is stored in memory 12. Next, in step ST4, the converted image is read from memory 12 by DRP 127e, DRP recognition processing is performed on the read input image, and the recognition result is stored in memory 12. This recognition result is, for example, the coordinates of the vertices where an object was recognized.

[0043] Next, in step ST5, the recognition results from step ST4 stored in memory 12, such as the coordinates of vertices (vertex coordinate information), are read by 3DGE127b, and GPU processing is performed to draw rectangles and other shapes by connecting vertices from the read vertex coordinate information, and the image after this GPU processing is stored in memory 12. Next, in step ST6, the input image stored in memory 12 is read by the image processing circuit 15, and processing is performed to convert the image size so that the GPU-processed image can be overlaid for LCD display, and the image after this processing is stored in memory 12.

[0044] Next, in step ST7, the input image stored in memory 12 processed in step ST1 and the input image processed in step ST6 are read by LCDC127d, and the two images are combined into one image within LCDC127d and displayed as a combined image on the liquid crystal display device.

[0045] Thus, in the image processing of the semiconductor device 1, the image processing circuit 15 frequently accesses the memory 12. Therefore, as the amount of information increases due to the higher resolution of the input image, it becomes necessary to read or write a large amount of information from the memory 12, which increases the load on the image processing circuit 15.

[0046] (Image area used for image processing) Next, we will describe the image region used for image processing when the image processing circuit 15 performs image processing. In this embodiment, we will explain using an example where the input image is enlarged. Figure 4 is a diagram illustrating an example of an image region.

[0047] As shown in Figure 4, the input image area 21, the enlarged input image area 22, the output image area (Crop Area) 23, and the input image area 24 after comparison and updating are shown. The relative sizes of the input image area 21, the enlarged input image area 22, the output image area (Crop Area) 23, and the input image area 24 after comparison and updating are as follows: input image area 24 after comparison and updating < input image area 21 < output image area 23 < input image area 22 after enlargement. In Figure 4, the left-right direction in the illustration represents the horizontal direction, and the up-down direction represents the vertical direction. The same applies to the following cases.

[0048] For the input image area 21, the horizontal size is indicated by IMG_IN_HSIZE and the vertical size by IMG_IN_VSIZE. For the input image area 22, the horizontal size is indicated by IMG_IN_HSIZE*IMG_HSCALE and the vertical size by IMG_IN_VSIZE*IMG_VSCALE. For the output image area (Crop Area) 23, the horizontal size is indicated by IMG_OUT_HSIZE and the vertical size by IMG_OUT_VSIZE. For the input image area 24, the horizontal size is indicated by IMG_OUT_HSIZE / IMG_HSCALE and the vertical size by IMG_OUT_VSIZE / IMG_VSCALE.

[0049] The input image area 21 represents the image area of ​​the input image information 13. The area required for the output image is the output image area 23. When the input image area 21 is enlarged by the set magnification ratio, the input image area becomes the enlarged input image area 22. On the other hand, even if the image area is enlarged to the extent of the input image area 22, the area of ​​the output image remains the output image area 23. From the above, it can be said that in order to output the output image of the output image area 23, the entire range of the input image area 22 may be unnecessary. In other words, when enlarging the input image, if the output image is to be output in the output image area 23, there may be an image area in the input image area 21 that is unnecessary for image processing. Note that although this embodiment describes the case of enlarging the input image, similarly, when reducing the input image, there may be an image area in the input image area 21 that is unnecessary for image processing.

[0050] Figure 5 shows an example of an unnecessary image region in the case shown in Figure 4. As shown in Figure 5, in this example, the unnecessary image region 25 is the area outside the input image region 24 after comparison and update within the input image region 21. In Figure 5, the unnecessary image region 25 is indicated by a dot.

[0051] (Before image processing begins) Next, the setting process for parameters and other settings before the image processing circuit 15 executes image processing in the semiconductor device 1 will be described. This setting process is performed by the CPU 11, parameter storage unit 16, flow control unit 123, memory controller 125, bus controller 126, and descriptor list 124a, as shown in Figure 1.

[0052] ((Explanation of settings and descriptor list before image processing)) In the semiconductor device 1, various parameters are set and control instructions are given to the image processing circuit 15 in response to the input and output of a single image.

[0053] First, before the image processing circuit 15 starts processing, the CPU 11 initializes the Flow control unit 123. During this initialization, the CPU 11 sets the ON / OFF status of interrupt signals generated within the image processing circuit 15, and sets the addresses stored in the descriptor list 124a. Once initialization is complete, the CPU 11 instructs the Flow control unit 123 to start. This series of processes is performed via path S1. Parameter writing via path S1 is performed via the bus controller 126. Note that writing may also be performed according to protocols such as APB / AHB / AXI as defined in the AMBA standard.

[0054] When the Flow control unit 123 receives a start command, it reads the descriptor list 124a stored in memory 12. Since the CPU 11 has previously stored the address of the descriptor list 124a in the Flow control unit 123, the Flow control unit 123 begins accessing that address.

[0055] When access is initiated, first, an access request is sent from the Flow control unit 123 to the bus controller 126. If the bus controller 126 has access requests for memory 12 from other IP groups 127, for example, from IP groups other than the Flow control unit 123, it prioritizes the access. If other IP groups are already accessing the memory, the bus controller 126 waits until those transactions are finished. Then, when it is the bus controller 126's turn, it sends a read access request to the memory controller 125.

[0056] The memory controller 125 performs read control on the memory 12. This read control is performed according to the type of memory 12. Generally, the memory 12 can be SRAM or DRAM. Signal control is performed according to the specifications of such memory 12. The information in the descriptor list 124a read from the memory 12 is taken up by the Flow control unit 123 via the memory controller 125 and the bus controller 126.

[0057] The Flow control unit 123 analyzes the information in the header portion of the descriptor list 124a, then refers to the stored contents and stores the information in the specified register. The descriptor list 124a is described below. Figure 6 shows an example of the stored information T10 in the descriptor list 124a.

[0058] As shown in Figure 6, the stored information T10 is associated with a category, the number of bytes, and the content to be set. The categories are header, body, and footer. The number of bytes indicates the size of the information to be set. The content to be set is defined according to the category.

[0059] The header portion of the storage information T10 shown in Figure 6 specifies, for example, the total number of bytes for the main body portion of the descriptor list 124a. The main body portion specifies a combination of an address for specifying a register in the image processing circuit 15 and information defining the parameters. Based on this combination, the parameter settings of the specified register are changed. The footer portion specifies the upper address, lower address, and control bit group of the next descriptor list. The control bit group specifies, for example, whether or not to perform automatic frame processing. This setting selects whether or not to issue an interrupt after this frame processing. Other settings may be specified as the control bit group. Also, the example shown in Figure 6 shows the case where 10 parameters to be set in the parameter storage unit 16 are set.

[0060] If the main body contains address and parameter information, the Flow control unit 123 stores the parameters in the register at the specified address based on the address and parameter information.

[0061] The relationship between parameters and their destination addresses will be explained in more detail. Figure 7 shows an example of relationship information T20, which represents the relationship between parameters and their destination addresses. As shown in Figure 7, relationship information T20 associates the destination, route, parameters, and description. Note that the description portion does not necessarily have to be included in relationship information T20.

[0062] The storage location indicates a register or circuit for setting parameters within the image processing circuit 15. In the example shown in Figure 7, the input image read information setting unit 17, the scaling ratio information setting unit 18, and the output image write information setting unit 19 are specified as storage locations. The path indicates the path for setting the parameters. The path is one of the paths S1 to S10 described above, depending on the parameters. The parameters are those described above. The description shows the content of the corresponding parameters.

[0063] For example, the scaling calculation unit 121 is configured to set IMG_HSCALE and IMG_VSCALE via path S3.

[0064] (Explanation of parameters before image processing) The parameter storage unit 16 is used to set various parameters. In this embodiment, as shown in Figure 1, the parameter storage unit 16 includes an input image reading information setting unit 17, a scaling ratio information setting unit 18, and an output image writing information setting unit 19, which are the minimum necessary for image processing.

[0065] The input image reading information setting unit 17 is configured with, for example, the starting address of the memory 12 when reading the input image information 13, IMG_IN_STADD, the number of pixels or required bytes in the horizontal direction, IMG_IN_HSIZE, the number of pixels or required bytes in the vertical direction, IMG_IN_VSIZE, and image format information IMG_IN_FMT representing the components of the input image information 13.

[0066] The scaling ratio information setting unit 18 is configured with setting information related to the horizontal scaling ratio, such as IMG_HSCALE, and information related to the vertical scaling ratio, such as IMG_VSCALE.

[0067] The output image writing information setting unit 19 is configured with settings such as the horizontal pixel count or required byte count IMG_OUT_HSIZE, which indicates which part of the input image area 22 after scaling will be cropped and output; the vertical pixel count or required byte count IMG_OUT_VSIZE; and the starting address IMG_OUT_STADD when writing these images to memory 12. If the scaled image is smaller than the values ​​set in this way, a process is performed to generate and fill pixels using a pre-specified method. On the other hand, if the scaled image is larger than the specified area, a process is performed to output only the specified area. Details of the process will be described later.

[0068] (Footer processing for descriptor list) Once the information specified in the main body of descriptor list 124a has been stored, footer processing begins. Here, after processing one image, the address of the next descriptor list 24b is temporarily stored in the Flow control unit 123.

[0069] Furthermore, if "Yes" is selected for the setting to enable automatic frame processing in the footer section, the image processing circuit 15 starts processing. If "No" is selected for the setting to enable automatic frame processing, the system will wait for a certain period of time. In this case, if a start request is received from the CPU 11, the image processing circuit 15 will start processing. Once a start command is given, the necessary image calculation process begins.

[0070] (Calculation of required image area) Below, we will describe two cases, Example 1 and Example 2, regarding the process of calculating the image area necessary for outputting the output image.

[0071] <Example 1> First, the calculation process for the required image area will be explained in the case of Example 1. Figures 8 to 12 are diagrams illustrating the calculation of the required image area. Figure 8 shows the image area of ​​the input image. Figure 9 shows the scaling factor and the image area of ​​the enlarged image. Figure 10 shows the image area of ​​the output image. Figure 11 shows the image area after optimization calculation. Figure 12 shows the image area after comparison update.

[0072] <<Calculation process for required images: See Figure 8>> When a processing start command is given, the process moves to the input image area determination unit 110. The input image region determination unit 110 includes an optimized image region calculation unit 111 and an input image region comparison unit 112, as described above. To explain the processing of the input image region determination unit 110, please refer to Figures 4, 5, and 8 to 12 above. Figures 8 to 12 show the sizes of the image regions according to the processing order. Figure 4 above can also be described as an image diagram showing the superimposed state of the image regions in Figures 8 to 12. Figure 5 above can also be described as an image diagram showing the image regions that are unnecessary to output as output image information 14 for the input image information 13.

[0073] The input image area 21 represents the area corresponding to the input image information 13 stored in the memory 12. In this embodiment 1, the input image information 13 consists of parameters IMG_IN_HSIZE, which is the number of pixels in the horizontal direction, and IMG_IN_VSIZE, which is the number of pixels in the vertical direction. The parameters of the input image information 13 are stored in the input image read information setting unit 17. The parameters of the input image information 13 are set from the input image read information setting unit 17 to the input image area comparison unit 112 via the path S2 shown in Figure 1.

[0074] <<Calculation process for required images: See Figure 9>> The enlarged input image region 22 is represented by equations (1) and (2) below, where IMG_HSCALE is the horizontal scaling factor and IMG_VSCALE is the vertical scaling factor. The parameters of the enlarged input image region 22 are set from the scaling factor information setting unit 18 to the optimized image region calculation unit 111 via the path S3 shown in Figure 1.

[0075] Horizontal size after scaling = IMG_IN_HSIZE * IMG_HSCALE …(1) Vertical size after scaling = IMG_IN_VSIZE * IMG_VSCALE …(2)

[0076] The size of the input image region 22 is determined using equations (1) and (2) above. The input image region 22 is represented by the horizontal size after scaling multiplied by the vertical size after scaling.

[0077] <<Calculation process for required images: See Figure 10>> Next, we will explain the case where an output image area 23 is set for an enlarged or reduced input image area 22. Normally, a device that outputs an input image as an output image, such as a liquid crystal display device, has a predetermined image size (output image area 23) to display. In this case, if the enlarged input image area 22 is larger than the output device image size, the image area is cropped to match the image size of the output device. The same applies even if no output device is provided, if other image processing is performed in the subsequent stage of the image processing circuit 15. For example, other cases include when the image size (output image area 23) for image processing is predetermined, and when the image size (output image area 23) is restricted to being an integer multiple of 16.

[0078] Therefore, in this embodiment 1, the size of the output image area 23 is set in the output image writing information setting unit 19. When outputting the output image information 14, if the horizontal image size is represented by IMG_OUT_HSIZE and the vertical image size by IMG_OUT_VSIZE, the size of the output image information 14 is represented by the output image area (Crop Area) 23 in Figure 4 above. The parameters for outputting the output image information 14 are set from the output image writing information setting unit 19 to the optimized image area calculation unit 111 via the path S4 shown in Figure 1. The output image area 23 shown in Figure 10 corresponds to the area obtained by cutting out the unnecessary image area 25 from the enlarged input image area 22 shown in Figure 5 above. Comparing the two figures, since there is no unnecessary image area in the vertical direction, in Figure 10, the horizontal image area is the unnecessary image area 25.

[0079] If the enlarged input image area 22 is smaller than the output image area 23, the missing pixel information is filled in and output. In this embodiment 1, the missing pixel information to be filled in is generated within the image processing circuit 15. Another method is to set a separate register in the parameter storage unit 16 in advance and set the color information of the pixels to be filled in that register. Generally, this type of processing is called padding.

[0080] <<Calculation process for required images: See Figure 11>> Next, the function of the optimized image region calculation unit 111 will be described. By calculating the value obtained by dividing the output image region 23 set by the output image writing information setting unit 19 by the horizontal scaling ratio IMG_HSCALE and the vertical scaling ratio IMG_VSCALE set by the scaling ratio information setting unit 18, the optimized image region calculation unit 111 can calculate the input image region 24 after comparison and update. If the horizontal image size of the input image region 24 after the optimized image region calculation is CAL_IN_HSIZE and the vertical image size is CAL_IN_VSIZE, then it can be expressed by the following equations (3) and (4).

[0081] CAL_IN_HSIZE = IMG_OUT_HSIZE / IMG_HSCALE …(3) CAL_IN_VSIZE = IMG_OUT_VSIZE / IMG_VSCALE …(4)

[0082] The above calculation determines how much of the input image area 21 is needed to output the output image of output image area 23, considering the scaling factor. Furthermore, the calculated input image area 24 shown in Figure 11 corresponds to Figure 5 above.

[0083] In this way, the optimized image region calculation unit 111 divides the scaling ratios IMG_HSCALE and IMG_VSCALE by the horizontal and vertical directions, respectively, based on the output image region sizes IMG_OUT_HSIZE and IMG_OUT_VSIZE. As a result, the optimized image region calculation unit 111 can obtain the minimum input image region required for the image output, that is, the input image region 24 after comparison and update. The calculation results CAL_IN_HSIZE and CAL_IN_VSIZE calculated by the optimized image region calculation unit 111 are sent to the input image region comparison unit 112 via the path S10.

[0084] <<Calculation process for necessary images after the initial processing: See Figure 12>> Next, the function of the input image area comparison unit 112 will be explained. The parameters (IMG_IN_HSIZE and IMG_IN_VSIZE) set in the input image reading information setting unit 17 are input to the input image region comparison unit 112 via path S2. Meanwhile, the calculation results calculated by the optimized image region calculation unit 111, i.e., the sizes after optimized image calculation (CAL_IN_HSIZE and CAL_IN_VSIZE), are input to the input image region comparison unit 112 via path S10.

[0085] The input image region comparison unit 112 compares the horizontal and vertical sizes based on the input parameters and the calculation results. For the horizontal direction, the input image region comparison unit 112 compares IMG_IN_HSIZE and CAL_IN_HSIZE and stores the smaller value as IMG_IN_HSIZE_RENEW. Similarly, for the vertical direction, the input image region comparison unit 112 compares IMG_IN_VSIZE and CAL_IN_VSIZE and stores the smaller value as IMG_IN_VSIZE_RENEW. This process means that if the calculation result of the optimized image region calculation unit 111 is smaller than the parameters set in the input image reading information setting unit 17, it is only necessary to read the portion of the updated input image region 24 that is smaller than the input image region 21 from memory 12.

[0086] In Figure 12, dotted frames F1 and F2 are shown. Dotted frame F1 shows the input image region 21 defined by CAL_IN_HSIZE and CAL_IN_VSIZE. Dotted frame F2 shows the input image region 24 after comparison and update defined by IMG_IN_HSIZE and IMG_IN_VSIZE. The input image region comparison unit 112 selects the one with the smaller result from the calculation of the two image regions. In this embodiment 1, the input image region 24 after comparison and update shown in frame F1 is selected. Here, the stored CAL_IN_HSIZE and CAL_IN_VSIZE become IMG_IN_HSIZE_RENEW and IMG_IN_VSIZE_RENEW. IMG_IN_HSIZE_RENEW and IMG_IN_VSIZE_RENEW are sent to the information readout control unit 114 via path S5 and used for subsequent processing.

[0087] Thus, in the image processing circuit 15, the input image region 21 is replaced with the input image region 24 after comparison and update, which is the required image input region. This eliminates the need to read the unnecessary image region 25 from the memory 12, thereby reducing bandwidth. The reduction amount RA, which indicates the amount of bandwidth reduction, can be calculated using equation (5) below.

[0088]

number

[0089] <<Calculation example based on actual image size examples for the required image calculation process>> The specific calculations for step ST7, shown in Figure 4, will now be explained. Step ST7 is image processing when the image is displayed by the LCDC127d.

[0090] First, let's explain the specifications. The input image area of ​​input image information 13 is 1920 x 1080 pixels (IMG_IN_HSIZE=1920, and IMG_IN_VSIZE=1080), the output image area is a 2560 x 1600 image (IMG_OUT_HSIZE=2560, and IMG_OUT_VSIZE=1600), and the magnification is 1.4815x x 1.4815 (IMG_HSCALE=1.4815, and IMG_HSCALE=1.4815). The magnification is calculated by determining the ratio in the vertical direction, and the aspect ratio of the width and height is fixed.

[0091] <<Example of calculation process for determining the required images>> First, let's explain the calculation in the horizontal direction. The size of the input image is a parameter set in the input image reading information setting unit 17, and the size of the calculation result (CAL_IN_HSIZE) is the result calculated by the optimized image region calculation unit 111 using equations (1) and (3). Horizontal size after scaling: IMG_IN_HSIZE * IMG_HSCALE = 1920 * 1.4815 ≈ 2844 Calculation result size: IMG_OUT_HSIZE / IMG_HSCALE = 2560 / 1.4815 ≈ 1728 As a result, the calculated size CAL_IN_HSIZE (1728) is smaller than the input image size IMG_IN_HSIZE (1920), so the input image area comparison unit 112 can optimize the horizontal readout size and update the parameters. In other words, the input image area comparison unit 112 updates the calculated value CAL_IN_HSIZE (1728) from the value IMG_IN_HSIZE (1920) to IMG_IN_HSIZE_RENEW (1728).

[0092] Next, we will explain the calculation in the vertical direction. The size of the input image is a parameter set in the input image reading information setting unit 17, and the size of the calculation result (CAL_IN_VSIZE) is the result calculated by the optimized image region calculation unit 111 using equations (2) and (4). Vertical size after scaling: IMG_IN_VSIZE * IMG_VSCALE = 1080 * 1.4815 ≈ 1600 Calculation result size: IMG_OUT_VSIZE / IMG_VSCALE=1600 / 1.4815≈1080 As a result, the input image area comparison unit 112 keeps the vertical read size at 1080 because the size of the input image's IMG_IN_VSIZE (1080) and the size of the calculation result's CAL_IN_VSIZE (1080) are the same. In other words, the input image area comparison unit 112 keeps the value of the calculation result's CAL_IN_VSIZE (1080) as the value of IMG_IN_VSIZE (1080) and sets it to IMG_IN_VSIZE_RENEW (1080).

[0093] Figure 13 is a schematic diagram showing an example of specifications and calculation results. As shown in Figure 13, the size of the input image area 21 is "IMG_IN_HSIZE=1920" horizontally and "IMG_IN_VSIZE=1080" vertically. The size of the enlarged input image area 22 is "IMG_IN_HSIZE*IMG_HSCALE=1920*1.4815=2844" horizontally and "IMG_IN_VSIZE*IMG_VSCALE=1600" vertically. The size of the output image area 23 is "IMG_OUT_HSIZE=2560" horizontally and "IMG_OUT_VSIZE=1600" vertically. The size of the input image area 24 after comparison and update is "IMG_IN_HSIZE_RENEW=1728" horizontally and "IMG_IN_HSIZE_RENEW=1728" vertically. The unnecessary image region 25 is the area enclosed by the dashed line frame in the illustration.

[0094] The enlarged input image area 22 is enlarged by 1.4815 times in both the horizontal and vertical directions compared to the input image area 21. Furthermore, the input image area 24 is reduced to 1 / 1.4815 times in both the horizontal and vertical directions compared to the enlarged input image area 22.

[0095] By performing the above processing, the image processing circuit 15 can reduce the number of pixels in the input image information 13 read from the memory 12 by (1920-1728) × 1080 = 207,360 pixels. In addition, the bandwidth becomes (1920-1728) / 1920 = 0.10, so the image processing circuit 15 can reduce the bandwidth of the input image information 13 read from the memory 12 by 10%. Therefore, the semiconductor device 1 including the image processing circuit 15 can reduce the amount of access to the memory 12 and reduce the load on the image processing circuit 15.

[0096] <Example 2> Next, we will describe Example 2. In Example 1, we described an example in which the semiconductor device 1 processed according to the descriptor list 124a shown in Figure 6. The footer portion at the end of the frame of descriptor list 124a contained an address for reading the next descriptor list. In Example 2, we will describe the case in which descriptor list 124n is stored as the next descriptor list. Below, we will explain how the size of the input image area used as the output image changes when descriptor list 124n is used.

[0097] In Example 2, we will explain using an example in which the size of the input image information 13 is enlarged by 125% in both the horizontal and vertical directions compared to Example 1. Figure 14 is a comparison of the output image area 23 of Example 1 and the output image area 23a of Example 2. Since the monitor size of the LCDC127d (liquid crystal display device) is the same as in Example 1, as shown in Figure 14, the output image area 23 within the frame F3, which is the same size as the output image area 23 of Example 1, becomes the output image area that needs to be output in Example 2. In other words, the image of the same output image area 23 as in Example 1 is displayed on the liquid crystal display device, and the image outside the frame F3 is not displayed. Therefore, in the input image area 22 that is enlarged by 125%, unnecessary image areas are generated in both the horizontal and vertical directions.

[0098] <<Calculation example based on actual size examples for the necessary image calculation process after the initial processing>> Similar to Example 1, the specific calculations for step ST7 in Figure 7 will be explained. Step ST7 is the image processing when performing the display processing of LCDC127d.

[0099] First, let's explain the specifications. The input image area of ​​input image information 13 is 1920 x 1080 pixels (IMG_IN_HSIZE=1920, and IMG_IN_VSIZE=1080), the output image area is a 2560 x 1600 image (IMG_OUT_HSIZE=2560, and IMG_OUT_VSIZE=1600), and the magnification is 1.8519 times x 1.8519 times (IMG_HSCALE=1.8519, and IMG_HSCALE=1.8519). Compared to Example 1, the horizontal and vertical directions are each magnified by 1.25 times (1.4181 x 1.25 compared to the original input image). In other words, this corresponds to a magnification of 125% on the LCD monitor.

[0100] <<Descriptor List 124n>> In descriptor list 124n, in Example 2, by modifying the following four parameters, it becomes possible to enlarge the input image information 13 by 1.25 times compared to Example 1. Parameter "IMG_IN_STADD": The starting address of memory 12 when reading the input image. Parameter "IMG_OUT_STADD": The starting address of memory 12 when writing the output image. Parameter "IMG_HSCALE": Horizontal scaling factor Parameter "IMG_VSCALE": Vertical scaling factor

[0101] Since no parameters other than those mentioned above are modified, the descriptor list 124n is represented as shown in Figure 15. Figure 15 shows an example of the stored information T30 of the descriptor list 124n in Example 2. Except for the parameters that are changed as described above, it is the same as in Figure 6. For the descriptor list 124n, a column titled "Updates from Example 1" is stored to explain the differences from the descriptor list 124a in Figure 6, but this column does not need to be included.

[0102] <<Example of calculation process for determining the required images>> First, let's explain the calculation in the horizontal direction. The size of the input image is a parameter set in the input image reading information setting unit 17, and the size of the calculation result (CAL_IN_HSIZE) is the result calculated by the optimized image region calculation unit 111 using equations (1) and (3). Horizontal size after scaling: IMG_IN_HSIZE * IMG_HSCALE = 1920 * 1.8519 ≈ 3556 Calculation result size: IMG_OUT_HSIZE / IMG_HSCALE = 2560 / 1.8519 ≈ 1382 As a result, the size of the calculation result, CAL_IN_HSIZE (1382), is smaller than the input image size, IMG_IN_HSIZE (1920). Therefore, the input image area comparison unit 112 can optimize the horizontal readout size and update the parameters. In other words, the input image area comparison unit 112 updates the value of the calculation result CAL_IN_HSIZE (1382) from the value of IMG_IN_HSIZE (1920) to IMG_IN_HSIZE_RENEW (1382).

[0103] Next, we will explain the calculation in the vertical direction. The size of the input image is a parameter set in the input image reading information setting unit 17, and the size of the calculation result (CAL_IN_VSIZE) is the result calculated by the optimized image region calculation unit 111 using equations (2) and (4). Vertical size after scaling: CAL_IN_VSIZE * IMG_VSCALE = 1080 * 1.8519 ≈ 2000 Calculation result size: IMG_OUT_VSIZE / IMG_VSCALE=1600 / 1.8519 ≈ 863 As a result, the size of the calculation result CAL_IN_VSIZE (863) is smaller than the size of the input image IMG_IN_VSIZE (1080), so the input image region comparison unit 112 can optimize the vertical readout size and update the parameters. In other words, the input image region comparison unit 112 updates the value of the calculation result CAL_IN_VSIZE (863) from the value of IMG_IN_VSIZE (1080) to IMG_IN_HSIZE_RENEW (863).

[0104] Figure 16 is a schematic diagram showing an example of specifications and calculation results. As shown in Figure 16, the size of the input image area 21 is "IMG_IN_HSIZE=1920" horizontally and "IMG_IN_VSIZE=1080" vertically. The size of the enlarged input image area 22 is "IMG_IN_HSIZE*IMG_HSCALE=1920*1.8519=3555" horizontally and "IMG_IN_VSIZE*IMG_VSCALE=2000" vertically. The size of the output image area 32 is "IMG_OUT_HSIZE=2560" horizontally and "IMG_OUT_VSIZE=1600" vertically. The size of the input image area 24 after comparison and update is "IMG_IN_HSIZE_RENEW=1382" horizontally and "IMG_OUT_VSIZE / IMG_VSCALE=863" vertically. The unnecessary image region 25 is the area enclosed by the dashed line frame in the illustration.

[0105] The enlarged input image area 22 is 1.8519 times larger horizontally than the input image area 21. The input image area 24 is reduced to 1 / 1.8519 times its original size horizontally and vertically compared to the enlarged input image area 22.

[0106] By performing the above processing, the image processing circuit 15 can reduce the number of pixels in the input image information 13 read from the memory 12 by (1920-1382)×863+1920×(1080-863)=880,934 pixels. In terms of bandwidth, 880934 / (1920-1080)=0.42, meaning the image processing circuit 15 can reduce the bandwidth of the input image by 42%. Therefore, the semiconductor device 1 including the image processing circuit 15 can reduce the load on the image processing circuit 15.

[0107] (effect) The semiconductor device 1, which includes an image processing circuit 15 that performs image processing such as scaling, can calculate an optimized input image area 24 from the ratio of the scaling ratio of the output image area 23 and the input image area 21. As a result, the image processing circuit 15 can access the pixels of the truly necessary image area 25 from the input image information 13, and the bandwidth required to access the memory 12 can be reduced.

[0108] Furthermore, by reducing the access bandwidth to memory 12, the image processing circuit 15 can improve the processing speed from image input to outputting the processed image.

[0109] Furthermore, the image processing circuit 15 can automatically calculate the image size to be read simply by adding a process to change the scaling ratio setting, without placing a load on the CPU 11. In this way, the processing within the image processing circuit 15 can be reduced without placing a load on the CPU 11, thereby improving the overall processing efficiency of the semiconductor device 1.

[0110] As the resolution of images to be processed has increased to 4K and 8K in recent years, the bandwidth of the bus has become extremely strained. By applying the technology disclosed above, the semiconductor device 1 can significantly reduce the amount of information in the input image information 13 it accesses, thereby improving the efficiency of the bus used.

[0111] Although the present invention has been specifically described above based on embodiments, it goes without saying that the present invention is not limited to the above embodiments and can be modified in various ways without departing from its essence. [Explanation of Symbols]

[0112] 1...Semiconductor device, 11...CPU, 12...Memory, 13...Input image information, 14...Output image information, 15...Image processing circuit, 16...Parameter storage unit, 17...Input image reading information setting unit, 18...Scaling ratio information setting unit, 19...Output image writing information setting unit, 110...Input image area determination unit, 111...Optimized image area calculation unit, 112...Input image area comparison unit, 113...Image input / output control unit, 114...Information reading control unit, 115...FI for input image FO, 116...FIFO control unit for input image, 117...Information writing control unit, 118...FIFO for output image, 119...FIFO control unit for output image, 120...Input image format conversion unit, 121...Scale calculation unit, 122...Output image format conversion unit, 123...Flow control unit, 124a to 124n...Descriptor list, 125...Memory controller, 126...Bus controller, 127...Other IPs, 127a...CRU, 127b...3DGE, 127c...VCD, 127d...LCDC, 127e...DRP, S1 to S10...Route

Claims

1. A register in which input image setting information including the input size of the image area of ​​the input image to be read from memory, scaling ratio setting information for setting the scaling ratio of the input image, and output image setting information including the output size of the image area of ​​the output image to be output to memory are set, An image processing circuit includes a scaling unit that changes the size of the input image based on the scaling ratio set by the scaling ratio setting information, and performs image processing on the input image, Equipped with, The aforementioned image processing circuit is A calculation unit that calculates the image area size necessary to output the output image based on the output size and the scaling ratio, Includes a comparison unit that compares the required image area size with the input size, selects the smaller of the required image area size and the input size, and uses the selected image area for the image processing of the input image. Semiconductor equipment.

2. A semiconductor device according to claim 1, The comparison unit calculates the required image area size by dividing the output size by the scaling ratio. Semiconductor equipment.

3. A semiconductor device according to claim 1, The aforementioned image processing circuit is The system further includes an information readout control unit that performs control to read the input image from the memory, The information reading control unit, The input image of the image region selected by the comparison unit from the image region of the input image is read from the memory. Semiconductor equipment.

4. A semiconductor device according to claim 3, The system further includes a scaling calculation unit that receives the scaling ratio setting information from the register and calculates the scaling process for the input image selected by the comparison unit, The aforementioned scaling calculation unit, The input image of the selected image region is enlarged or reduced by the enlargement ratio included in the enlargement ratio setting information. Semiconductor equipment.

5. A semiconductor device according to claim 4, The system includes an information writing control unit that executes control to write the output image to the memory, The information writing control unit, The input image of the enlarged or reduced image region is output to the memory as the output image. Semiconductor equipment.

6. A semiconductor device according to claim 1, The register includes a first setting unit, a second setting unit, and a third setting unit. The input image setting information is set in the first setting unit. The second setting unit is set with the scaling ratio setting information. The output image setting information is set in the third setting unit. The comparison unit receives the input image setting information from the first setting unit, The calculation unit receives the scaling ratio setting information from the second setting unit and the output image setting information from the third setting unit. Semiconductor equipment.

7. A semiconductor device according to claim 6, The aforementioned memory is A list including the input image setting information, the scaling ratio setting information, and the output image setting information is stored. The aforementioned image processing circuit is Before reading the input image of the image region from the memory, the input image setting information, the scaling ratio setting information, and the output image setting information are read from the memory. The input image setting information, the scaling ratio setting information, and the output image setting information read from the first setting unit, the second setting unit, and the third setting unit are respectively set. Semiconductor equipment.

8. A semiconductor device according to claim 7, The memory includes multiple of the lists, The list includes instruction information that directs the list to perform the next processing of the input image, The image processing circuit performs image processing on the input image according to the instruction information included in the list. Semiconductor equipment.

9. A semiconductor device according to claim 1, The output size of the output image is determined according to the device connected to the semiconductor device. Semiconductor equipment.

10. A semiconductor device according to claim 1, The output size of the output image is determined according to the processing in the subsequent stage of the image processing circuit. Semiconductor equipment.

Images