Semiconductor device

Abstract

Embodiments of this disclosure relate to a semiconductor device. The semiconductor device includes a parameter storage unit and an image processing circuit. The image processing circuit includes a scaling calculation unit that changes the size of an input image based on a magnification ratio or reduction ratio set in scaling factor setting information. The image processing circuit further includes: an optimized image region calculation unit that calculates a desired image region size for outputting the image based on the output size of the output image and a scaling factor; and an input image region comparison unit that compares the desired image region size with the input size of the input image, selects an image region having the smaller size between the desired image region size and the input size, and uses the selected image region for image processing of the input image.

Description

Cross-reference to related applications

[0001] The disclosure of Japanese Patent Application No. 2024-220840, filed on December 17, 2024, including the specification, drawings and abstract, is incorporated herein by reference in its entirety. Technical Field

[0002] This disclosure relates to semiconductor devices, and, for example, to technologies for semiconductor devices including image processing circuitry. Background Technology

[0003] The disclosed technologies are listed below.

[0004] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2011-59911

[0005] Techniques for reducing the size and power consumption of circuits used to change the order of image processing are known (see Patent Document 1). Summary of the Invention

[0006] In recent years, high-resolution image quality has been achieved. Consequently, the amount of information transferred to semiconductor devices, including image processing circuitry, for image processing using memory, is increasing. For this reason, the load on the image processing circuitry included in semiconductor devices is increasing.

[0007] Other issues and novel features will become apparent from the description and accompanying drawings in this specification.

[0008] Representative embodiments of this disclosure have the following configuration. A semiconductor device according to an embodiment includes an image processing circuit. The image processing circuit calculates a desired image region size for outputting the output image based on the output size of the output image and a scaling factor. The image processing circuit compares the desired image region size with the input size of the input image, selects an image region having the smaller size between the desired image region size and the input size, and uses the selected image region for image processing of the input image.

[0009] According to representative embodiments of this disclosure, techniques can be provided that can reduce the load on image processing circuitry in semiconductor devices. Attached Figure Description

[0010] Figure 1 This is a schematic diagram illustrating an example configuration of a semiconductor device according to an embodiment.

[0011] Figure 2 This is a diagram used to explain an example of image processing in a semiconductor device according to an embodiment.

[0012] Figure 3 This is a flowchart illustrating an example of image processing in a semiconductor device according to an embodiment.

[0013] Figure 4 This is an example diagram used to explain the image regions in the embodiments.

[0014] Figure 5 It is illustrated in Figure 4 The diagram shows an example of an unnecessary image region in the case shown.

[0015] Figure 6 This is a diagram illustrating an example of information about the stored list of descriptors in an embodiment.

[0016] Figure 7 This is a diagram illustrating the relationship information in the embodiment, which is an example of the relationship between parameters and the address of the storage destination.

[0017] Figure 8 This is a diagram illustrating the image region of the input image in the first example.

[0018] Figure 9 It is a diagram illustrating the scale factor and image region of the magnified image in the first example.

[0019] Figure 10 This is a diagram illustrating the image region of the output image in the first example.

[0020] Figure 11 This is a diagram illustrating the image region after optimization calculation in the first example.

[0021] Figure 12 This is a diagram illustrating the image region after comparison and updating in the first example.

[0022] Figure 13 This is a schematic diagram illustrating an example of the specifications and calculation results in the first example.

[0023] Figure 14 This is a graph that compares the output image region in the first example with the output image region in the second example.

[0024] Figure 15 This is a diagram illustrating an example of the information in the stored list of descriptors in the second example.

[0025] Figure 16 This is a schematic diagram illustrating an example of the specifications and calculation results in the second example. Detailed Implementation

[0026] In the following, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the drawings, identical parts are generally indicated by the same reference numerals, and repeated descriptions are omitted. In the drawings, for ease of understanding of the invention, the depiction of components may not indicate actual location, size, shape, extent, etc.

[0027] Example

[0028] Configuration of semiconductor equipment

[0029] Figure 1 This diagram schematically illustrates an example configuration of semiconductor device 1. Semiconductor device 1 is connected to memory 12. Semiconductor device 1 receives image information from memory 12 and outputs the image information to memory. Semiconductor device 1 is a semiconductor device for processing image information. In this embodiment, the case where memory 12 is provided separately from semiconductor device 1 will be described; however, memory 12 may be configured to be included within semiconductor device 1.

[0030] like Figure 1 As shown, semiconductor device 1 includes image processing circuitry 15, CPU 11, memory controller 125, bus controller 126, and other IPs 127. CPU 11 and other IPs 127 communicate with image processing circuitry 15 via bus controller 126. Image processing circuitry 15 communicates with memory 12 via bus controller 126 and memory controller 125.

[0031] Memory 12 stores descriptor lists 124a to 124n, input image information 13, and output image information 14. Each descriptor list 124a to 124n includes input image setting information, scale factor setting information, output image setting information, and a parameter list. The input image setting information includes the input size of the image region of the input image input from memory 12 to image processing circuit 15. The scale factor setting information includes a scale factor used to set the scale factor of the input image. The output image setting information includes the output size of the image region of the output image output from image processing circuit 15 to memory 12. The parameter list includes parameters storing information specifying the next frame. The specifying information is, for example, a parameter used to specify the frame from which processing of the input image will be performed next. Input image information 13 is information about the input image input to image processing circuit 15. Output image information 14 is information about the output image output from image processing circuit 15.

[0032] Before the input image is read from memory 12, the information stored in descriptor lists 124a to 124n is set by image processing circuit 15 in setting units, which will be described later in the section on image processing circuit 15. For example, input image setting information is set in input image read information setting unit 17 (first setting unit). Scale factor setting information is set in scale factor setting unit 18 (second setting unit). Output image setting information is set in output image write information setting unit 19 (third setting unit). Memory 12 includes multiple descriptor lists 124. Image processing circuit 15 performs image processing on the input image based on the specified information included in the descriptor lists. Details of these processing segments will be described later.

[0033] Image processing circuit 15 performs processing on the input image from memory 12, changing the image size based on a specified scaling factor. CPU 11 is a processor that specifies the required setup information for image processing circuit 15. Other IPs 127 are other IPs that make access requests to memory 12. Memory controller 125 performs read or write control on the information stored in memory 12. Bus controller 126 adjusts the priority order of the bus between image processing circuit 15 and various IPs (such as CPU 11), and sends and receives information from memory controller 125.

[0034] The image processing circuit 15 includes a parameter storage unit 16, an input image region determination unit 110, an image input and output control unit 113, an input image format conversion unit 120, a scaling calculation unit 121, an output image format conversion unit 122, and a stream control unit 123. The parameter storage unit 16 includes, for example, a register for storing information. The input image region determination unit 110, the image input and output control unit 113, the input image format conversion unit 120, the scaling calculation unit 121, the output image format conversion unit 122, and the stream control unit 123 include, for example, circuitry.

[0035] The parameter storage unit 16 stores various setting parameters in the image processing circuit 15. The parameter storage unit 16 includes an input image reading information setting unit 17 (first setting unit), a scaling factor information setting unit 18 (second setting unit), and an output image writing information setting unit 19 (third setting unit).

[0036] Input image setting information is set in the input image reading information setting unit 17. More specifically, for input image information 13 on memory 12 when the input image is read, information such as the start position, image size, and image format is set in the input image reading information setting unit 17.

[0037] In the scaling factor information setting unit 18, the magnification ratio or reduction ratio used to perform calculations on the input image (corresponding to the input image information 13) is set.

[0038] Output image setting information is set in the output image write information setting unit 19. More specifically, when output image information 14 is written, information indicating the start position, image size, image format, etc., on the memory 12 is set in the output image write information setting unit 19. The image size (output size) is determined based on the display size of the device connected to the semiconductor device 1 (e.g., a liquid crystal display device). Furthermore, the image size is determined, for example, based on the processing at a subsequent stage of the image processing circuit 15. The subsequent processing corresponds to the processing of other IP 127, which will be described later in this embodiment.

[0039] The input image region determination unit 110 determines the image region to be reduced based on the input image input to the image processing circuit 15. The input image region determination unit 110 includes an optimized image region calculation unit 111 and an input image region comparison unit 112.

[0040] The optimized image region calculation unit 111 calculates the actual required image region based on the output image writing information set in the output image writing information setting unit 19 and the scale factor information from the scale factor information setting unit 18. For example, the optimized image region calculation unit 111 calculates the required image region size for outputting the output image based on the output size of the output image information 14 and the scale factor. In this embodiment, the optimized image region calculation unit 111 calculates the required image region size by dividing the output size of the output image information 14 by the scale factor. The details of the processing will be described later.

[0041] When it is determined that optimization can be performed such that the image region of the output image can be reduced based on the calculation result of the optimized image region calculation unit 111, the input image region comparison unit 112 updates the value of the input image reading information setting unit 17. For example, the input image region comparison unit 112 compares the desired image region size calculated by the optimized image region calculation unit 111 with the input size of the input image information 13. In the comparison, the input image region comparison unit 112 selects the image region with the smaller size between the desired image region size and the input size of the input image information 13, and uses the selected image region for image processing of the input image. The details of the processing will be described later.

[0042] When image processing is performed in the image processing circuit 15, the image input and output control unit 113 manages the internal timing for reading information from and writing information to specified addresses in the memory 12. The image input and output control unit 113 includes an information reading control unit 114, an input image FIFO 115, an input image FIFO control unit 116, an information writing control unit 117, an output image FIFO 118, and an output image FIFO control unit 119.

[0043] Information reading control unit 114 executes control for reading input image information 13 from memory 12. Input image FIFO 115, under the control of information reading control unit 114, temporarily stores the information read from memory 12 and transmits the stored information to subsequent input image format conversion unit 120 based on the timing of internal image processing. Input image FIFO control unit 116 checks the free space in input image FIFO 115 and transmits a start command to information reading control unit 114. Information writing control unit 117 executes control for writing output image information 14 into memory 12. Output image FIFO 118, under the control of output image FIFO control unit 119, temporarily stores magnified or reduced images and transmits information to memory 12 after a certain amount of information has been stored. Output image FIFO control unit 119 temporarily stores output image information 14 while checking the free space in output image FIFO 118, and after a certain amount of information has been stored, prompts information writing control unit 117 to execute write control to memory 12.

[0044] The input image format conversion unit 120 performs a process to convert the input image information 13 transmitted from the input image FIFO 115 into an image format based on subsequent processing.

[0045] The scaling calculation unit 121 receives scaling factor setting information from the scaling factor information setting unit 18 and calculates the scaling process for the input image. More specifically, the scaling calculation unit 121 changes the image size of the input image information 13 transmitted from the input image format conversion unit 120 based on a specified magnification or reduction ratio. The scaling calculation unit 121 sets the modified input image information 13 as output image information 14 and transmits the output image information 14 based on the output image region.

[0046] The output image format conversion unit 122 converts the output image information 14 transmitted from the scaling calculation unit 121 into an image format based on subsequent processing.

[0047] The flow control unit 123 controls the entire image processing circuit 15 based on instructions from the CPU 11. Additionally, the flow control unit 123 reads appropriate descriptor lists from descriptor lists 124a to 124n in the memory 12, decodes the contents of the descriptor lists, and sets various registers specified in the image processing circuit 15.

[0048] also, Figure 1 The diagram illustrates paths S1 to S10. Path S1 indicates the path used to set parameters for the image processing circuitry 15 from the CPU 11.

[0049] Path S2 is a path used to set parameters in the input image region comparison unit 112. The parameters set in path S2 are, for example, IMG_IN_HSIZE and IMG_IN_VSIZE. IMG_IN_HSIZE indicates the number of pixels or the required number of bytes in the horizontal direction of the input image. IMG_IN_VSIZE indicates the number of pixels or the required number of bytes in the vertical direction of the input image.

[0050] Path S3 is used to set parameters in the optimized image region calculation unit 111. The parameters set in path S3 are, for example, IMG_HSCALE and IMG_VSCALE. IMG_HSCALE indicates the horizontal magnification or reduction ratio of the input image. IMG_VSCALE indicates the vertical magnification or reduction ratio of the input image.

[0051] Path S4 is used to set parameters in the optimized image region calculation unit 111. Parameters set in path S4 include, for example, IMG_OUT_HSIZE and IMG_OUT_VSIZE. IMG_OUT_HSIZE indicates the number of pixels or the number of bytes required in the horizontal direction of the output image. IMG_OUT_VSIZE indicates the number of pixels or the number of bytes required in the vertical direction of the output image.

[0052] Path S5 is a path used to set parameters in the information reading control unit 114, which indicate the size of the input image after a comparison update by the input image region comparison unit 112. Parameters set in path S5 include, for example, IMG_IN_HSIZE_RENEW and IMG_IN_VSIZE_RENEW. IMG_IN_HSIZE_RENEW indicates the number of pixels or the required number of bytes in the updated input image in the horizontal direction. IMG_IN_VSIZE_RENEW indicates the number of pixels or the required number of bytes in the updated input image in the vertical direction.

[0053] Path S6 is a path used to set parameters in the information reading control unit 114. Parameters set in path S6 include, for example, IMG_IN_STADD. IMG_IN_STADD indicates the starting address when the input image information 13 is read from memory 12.

[0054] Path S7 is a path used to set parameters in the input image format conversion unit 120. Parameters set in path S7 include, for example, IMG_IN_FMT. IMG_IN_FMT indicates the image format of the input image.

[0055] Path S8 is a path used to set parameters in the output image format conversion unit 122. Parameters set in path S8 include, for example, IMG_OUT_FMT. IMG_OUT_FMT indicates the image format of the output image.

[0056] Path S9 is used to set parameters in the information writing control unit 117. Parameters set in path S9 include, for example, IMG_OUT_STADD. IMG_OUT_STADD indicates the starting address when the output image is written.

[0057] Path S10 is used to set the calculation result calculated by the optimized image region calculation unit 111 in the input image region comparison unit 112. The calculation result set in path S10 is, for example, CAL_IN_HSIZE and CAL_IN_VSIZE. CAL_IN_HSIZE indicates the number of pixels or the number of bytes required in the horizontal direction of the input image after processing by the optimized image region calculation unit 111. CAL_IN_VSIZE indicates the number of pixels or the number of bytes required in the vertical direction of the input image after processing by the optimized image region calculation unit 111.

[0058] Image processing of semiconductor devices

[0059] Figure 2 This is a diagram used to explain an example of image processing for semiconductor device 1.

[0060] Semiconductor device 1 includes CRU 127a, 3DGE 127b, VCD 127c, LCDC 127d, and DRP 127e, as examples of other IPs 127. In semiconductor device 1, in addition to image processing circuitry 15, CRU 127a, 3DGE 127b, VCD 127c, LCDC 127d, and DRP 127e can access memory 12 via bus controller 126 and memory controller 125. Note that... Figure 2 The processing of steps ST1 to ST7 shown will be referenced later. Figure 3To describe.

[0061] CRU 127a is a camera device for capturing images. 3DGE 127b is a device for performing GPU processing. VCD 127c is a device for performing image compression. LCDC 127d is a liquid crystal display device for displaying images. DRP 127e is a device for performing conversions for AI processing.

[0062] Next, an example of image processing will be described when semiconductor device 1 outputs an input image captured by camera device to liquid crystal display device. Figure 3 This is a flowchart illustrating an example of image processing in semiconductor device 1. (Example) Figure 3 As shown, in step ST1, the input image captured by CRU 127a is stored in memory 12. Next, in step ST2, the input image is read from memory 12 by image processing circuit 15, and subsequent processing, i.e., image processing for VCD (image compression) conversion, is performed, and the processed image is stored in memory 12.

[0063] Next, in step ST3, the input image stored in memory 12 is read by image processing circuit 15, and subsequent processing, i.e., image processing for DRP (AI processing) conversion, is performed, and the processed image is stored in memory 12. Next, in step ST4, the converted image is read from memory 12 by DRP 127e, DRP recognition processing is performed on the read input image, and the recognition result is stored in memory 12. The recognition result is, for example, the coordinates of the vertices of the identified object.

[0064] Next, in step ST5, the recognition results from step ST4 stored in memory 12, such as vertex coordinates (vertex coordinate information), are read by the 3DGE 127b. GPU processing, which obtains quadrilaterals or similar objects connecting the vertices from the read vertex coordinate information, is performed, and the GPU-processed image is stored in memory 12. Next, in step ST6, the input image stored in memory 12 is read by the image processing circuit 15. Processing, which converts the image size to overlay the GPU-processed image onto the LCDC display image, is performed, and the processed image is stored in memory 12.

[0065] Next, in step ST7, the input image processed in step ST1 and stored in memory 12, as well as the input image processed in step ST6, are read by LCDC 127d, and the combination process is performed in LCDC 127d in such a way that the two images are combined into one image, and the resulting image is displayed on the liquid crystal display device as a combined image.

[0066] As described above, in the image processing of semiconductor device 1, memory 12 is frequently accessed by image processing circuit 15. For this reason, when the amount of information increases due to the high resolution of the input image, it is necessary to read a large amount of information from memory 12 or write a large amount of information to memory 12, and the load on image processing circuit 15 increases.

[0067] Image region used for image processing

[0068] Next, the image region used for image processing will be described when image processing is performed by the image processing circuit 15. In this embodiment, an example of an input image being magnified will be described. Figure 4 This is a diagram used to illustrate an example of an image region.

[0069] like Figure 4 As shown, the input image region 21, the magnified input image region 22, the output image region (cropped region) 23, and the compared and updated input image region 24 are illustrated. The magnitude relationship among the input image region 21, the magnified input image region 22, the output image region (cropped region) 23, and the compared and updated input image region 24 is: compared and updated input image region 24 < input image region 21 < output image region 23 < magnified input image region 22. Figure 4 In the center, left and right indicate the horizontal direction, and up and down indicate the vertical direction. The following situations also apply.

[0070] In input image region 21, the horizontal dimension is indicated by IMG_IN_HSIZE, and the vertical dimension is indicated by IMG_IN_VSIZE. In input image region 22, the horizontal dimension is indicated by... Indicated, and the vertical dimension is determined by Indicators: In the output image region (cropped region) 23, the horizontal dimension is indicated by IMG_OUT_HSIZE, and the vertical dimension is indicated by IMG_OUT_VSIZE. In the input image region 24, the horizontal dimension is indicated by IMG_OUT_HSIZE / IMG_HSCALE, and the vertical dimension is indicated by IMG_OUT_VSIZE / IMG_VSCALE.

[0071] Input image region 21 indicates the image region of input image information 13. The region required for the output image is output image region 23. When input image region 21 is magnified at a set magnification ratio, the input image region becomes magnified input image region 22. On the other hand, even if the image region is magnified like input image region 22, the region of the output image remains output image region 23. Therefore, the entire range of input image region 22 may not be necessary in order to output the output image of output image region 23. That is, when the input image is magnified and the output image is output as output image region 23, there is an image region in input image region 21 that is not necessary for image processing. Note that in this embodiment, the case where the input image is magnified will be described. However, similarly, when the input image is reduced, there may also be an image region in input image region 21 that is not necessary for image processing.

[0072] Figure 5 It is illustrated in Figure 4 The diagram illustrates an example of an unnecessary image region in the case shown. Figure 5 As shown, in this example, the region that forms the outer layer of the input image region 24 after comparison and update within the input image region 21 is the unnecessary image region 25. Figure 5 In the image, the unnecessary image region 25 is represented by points.

[0073] Before starting image processing

[0074] Next, the setup process, such as setting parameters, will be described before the image processing circuit 15 performs image processing in the semiconductor device 1. The setup process is performed by, for example... Figure 1 The CPU 11, parameter storage unit 16, flow control unit 123, memory controller 125, bus controller 126, and descriptor list 124a shown are used for execution.

[0075] Description of settings and descriptor list before image processing

[0076] In semiconductor device 1, various parameters are set and control commands are given to image processing circuit 15 for the input and output of an image.

[0077] First, before the image processing circuit 15 begins processing, the CPU 11 initializes the flow control unit 123. During this initialization, the CPU 11 performs ON and OFF settings for interrupt signals generated in the image processing circuit 15, and sets the addresses stored in the descriptor list 124a, etc. Upon completion of initialization, the CPU 11 instructs the flow control unit 123 to start. A series of processes are executed in path S1. Parameters are written to path S1 via the bus controller 126. Note that the writing can be performed according to protocols such as APB / AHB / AXI as defined in the AMBA standard.

[0078] Upon receiving a start command, the flow control unit 123 reads the descriptor list 124a stored in the memory 12. Since the CPU 11 had previously stored the address of the descriptor list 124a in the flow control unit 123 as the read destination, the flow control unit 123 begins accessing the address.

[0079] Upon initiation of an access request, the access request is first transmitted from the flow control unit 123 to the bus controller 126. The bus controller 126 performs access prioritization if access requests from IPs other than the flow control unit 123 (e.g., from other IPs 127) overlap with memory 12. If other IPs have already been accessed, the bus controller 126 waits until the transaction is complete. Upon arrival of a command, the bus controller 126 then issues a read access request to the memory controller 125.

[0080] The memory controller 125 performs read control on the memory 12. This read control is performed based on the type of memory 12. Generally, SRAM, DRAM, etc., are considered memory 12. Signal control is performed according to the specifications of the memory 12. Information from the descriptor list 124a read from the memory 12 is temporarily acquired by the flow control unit 123 via the memory controller 125 and the bus controller 126.

[0081] After analyzing the information in the header section of descriptor list 124a, the flow control unit 123 references the stored content and stores the information in the designated register. Here, descriptor list 124a is described. Figure 6 This is a diagram illustrating an example of the information T10 stored in descriptor list 124a.

[0082] like Figure 6 As shown, the category, the number of bytes, and the content to be set are associated with each other in the stored information T10. The category consists of the header, body, and footer. The number of bytes indicates the size of the information to be set. The information based on the category is limited to the content to be set.

[0083] In such Figure 6 In the header portion of the stored information T10 shown, for example, the total number of bytes related to the main body portion of descriptor list 124a is limited. In the main body portion, a combination of information specifying the addresses of registers in image processing circuit 15 and defining parameters is defined. Based on this combination, the settings of the parameters of the specified registers are changed. In the tail portion, the upper and lower addresses of the next descriptor list, as well as control bits, are defined. Among the control bits, for example, settings regarding whether frame processing is performed automatically are defined. This is a setting that selects whether an interrupt is issued after the current frame processing. Other settings can be defined as control bits. Additionally, Figure 6 The example shown illustrates the setting of ten parameters that will be set in parameter storage unit 16.

[0084] When address and parameter information exist in the main body, the flow control unit 123 stores the parameters at the address specified in the register based on the address and parameter information.

[0085] The relationship between the parameters and the storage destination address will be described in more detail below. Figure 7 This is a diagram illustrating relational information T20, which is an example of the relationship between parameters and storage destination addresses. For example... Figure 7 As shown, the storage destination, path, parameters, and description are associated with each other in the relational information T20. Note that the description portion does not need to be included in the relational information T20.

[0086] The storage destination indicates the register or circuit in the image processing circuit 15 where parameters are set. Figure 7 In the example shown, the input image reading information setting unit 17, the scale factor information setting unit 18, and the output image writing information setting unit 19 are defined as storage destinations. The path indicates the path used to set the parameters. Any of the paths S1 to S10 is defined based on the parameters. The parameters are defined as follows. The description indicates the content of the corresponding parameter.

[0087] For example, IMG_HSCALE and IMG_VSCALE are set in the scaling calculation unit 121 via path S3.

[0088] Description of parameters prior to image processing

[0089] Various parameters are stored in parameter storage unit 16. In this embodiment, such as... Figure 1As shown, the parameter storage unit 16 includes an input image reading information setting unit 17, a scale factor information setting unit 18, and an output image writing information setting unit 19. The input image reading information setting unit 17, the scale factor information setting unit 18, and the output image writing information setting unit 19 are the smallest units required for image processing.

[0090] In the input image reading information setting unit 17, for example, the starting address of the memory 12 when reading the input image information 13, IMG_IN_STADD, the number of pixels or the number of bytes required in the horizontal direction, IMG_IN_HSIZE, the number of pixels or the number of bytes required in the vertical direction, IMG_IN_VSIZE, and the image format information indicating the components of the input image information 13, IMG_IN_FMT, are set.

[0091] In the scale factor information setting unit 18, setting information related to the scale factor in the horizontal direction, such as IMG_HSCALE, and setting information related to the scale factor in the vertical direction, such as IMG_VSCALE, are set.

[0092] In the output image write information setting unit 19, the number of pixels or required bytes in the horizontal direction (IMG_OUT_HSIZE) and the number of pixels or required bytes in the vertical direction (IMG_OUT_VSIZE) are set. These indicate which portions are cropped from the input image area 22 and output after scaling. The starting address (IMG_OUT_STADD) when these images are written to memory 12 is also set. If the scaled image is smaller than the value set in this way, a process of generating and filling pixels using a pre-specified method is performed. On the other hand, if the scaled image is larger than the specified area, a process of outputting only the specified area is performed. The details of the process will be described later.

[0093] End processing of descriptor list

[0094] When the storage of the information defined in the main body of descriptor list 124a ends, the end-of-list processing begins. Here, after image processing of an image is completed, the address of the storage destination of the next descriptor list 24b is temporarily stored in the flow control unit 123.

[0095] If "Yes" is selected in the setting regarding whether to automatically perform frame processing in the table footer section, processing by the image processing circuit 15 begins. If "No" is selected in the setting regarding whether to automatically perform frame processing, processing enters a standby state for a certain period of time. In this case, processing by the image processing circuit 15 begins when a start request is received from the CPU 11. When a start command is given, the calculation processing of the required image begins.

[0096] Calculation of the required image region

[0097] In the following text, the process of calculating the desired image region for outputting the image will be described in two cases, namely, the first example and the second example.

[0098] First Example

[0099] First, the computational processing of the desired image region, as described in the first example, will be explained. Figures 8 to 12 It is a graph used to interpret the calculations for the desired image region. Figure 8 It is a diagram illustrating the image region of the input image. Figure 9 It is a diagram illustrating the scale factor and image region of a magnified image. Figure 10 This is a diagram illustrating the image region of the output image. Figure 11 This is a diagram illustrating the image region after optimization calculations. Figure 12 This is a diagram illustrating the image region after the comparison and update.

[0100] Required image processing: See Figure 8

[0101] When the instruction to start processing is given, the processing proceeds to the input image region determination unit 110.

[0102] As previously described, the input image region determination unit 110 includes an optimized image region calculation unit 111 and an input image region comparison unit 112. When describing the processing of the input image region determination unit 110, refer to the above... Figure 4 , Figure 5 and Figures 8 to 12 . Figures 8 to 12 The dimensions of the image regions are illustrated in the processing order. Alternatively, it can be said that... Figure 4 It's a diagram. Figures 8 to 12 The image is a superimposed image of the image region. Alternatively, it can be said that the above... Figure 5 This is an image diagram illustrating the image region that does not need to be output as output image information 14 relative to the input image information 13.

[0103] Input image region 21 indicates the region corresponding to the input image information 13 stored in memory 12. In the first example, the input image information 13 includes parameters for the number of pixels in the horizontal direction (IMG_IN_HSIZE) and the number of pixels in the vertical direction (IMG_IN_VSIZE). The parameter settings for the input image information 13 are stored in the input image reading information setting unit 17. The parameters of the input image information 13 are obtained from the input image reading information setting unit 17 via… Figure 1 The path S2 shown is set in the input image region comparison unit 112.

[0104] Required image processing: See Figure 9

[0105] The magnified input image region 22 is represented by the following formulas (1) and (2), where the scale factor in the horizontal direction is IMG_HSCALE and the scale factor in the vertical direction is IMG_VSCALE. The parameters of the magnified input image region 22 are obtained from the scale factor information setting unit 18 via... Figure 1 The path S3 shown is set in the optimized image region calculation unit 111.

[0106] Horizontal size after scaling ..(1)

[0107] Vertical dimensions after scaling (2)

[0108] The size of the input image region 22 can be obtained using the above formulas (1) and (2). The input image region 22 is represented by the scaled horizontal dimension × the scaled vertical dimension.

[0109] Required image processing: See Figure 10

[0110] Next, the case where the output image area 23 is set relative to the scaled input image area 22 will be described. In devices that output an input image as an output image, such as liquid crystal displays, the size of the image to be displayed (output image area 23) is usually predetermined. Here, if the enlarged input image area 22 is larger than the output device image size, the image area is cropped based on the output device image size. Furthermore, this also applies even if no output device is provided, in cases where another image processing is performed at a subsequent stage of the image processing circuit 15. For example, as other examples, there are cases where the image size (output image area 23) for image processing is determined, or where the image size (output image area 23) is limited to an integer multiple of 16, etc.

[0111] Therefore, in the first example, the size of the output image region 23 is set in the output image writing information setting unit 19. When the output image information 14 is output, and the image size in the horizontal direction is represented by IMG_OUT_HSIZE and the image size in the vertical direction is represented by IMG_OUT_VSIZE, the size of the output image information 14 is set as described above. Figure 4 The output image region (cropping area) 23 is used to represent this. Parameters for outputting output image information 14 are written from the output image to the information setting unit 19 via... Figure 1 The path S4 shown is set in the optimized image region calculation unit 111. Figure 10 The output image region 23 shown corresponds to the region obtained from the image source. Figure 5 The region shown is obtained by cropping the unnecessary image region 25 from the magnified input image region 22. When comparing the two images, since there is no unnecessary image region in the vertical direction, Figure 10 In the image, the horizontal region is an unnecessary image region 25.

[0112] When the magnified input image region 22 is smaller than the output image region 23, the insufficient pixel information is filled in and output. In the first example, the insufficient pixel information for filling is generated in the image processing circuit 15. As another method, for example, a method can be considered in which a register is provided separately in advance in the parameter storage unit 16, and the color information of the pixels to be filled is set in that register. Generally, such processing is called PADDING processing.

[0113] Required image computational processing: see Figure 11

[0114] Next, the function of the optimized image region calculation unit 111 will be described. By calculating the output image region 23 set by the output image writing information setting unit 19 and dividing it by the horizontal scaling factor IMG_HSCALE and the vertical scaling factor IMG_VSCALE set by the scaling factor information setting unit 18, the optimized image region calculation unit 111 can calculate and compare the updated input image region 24. The input image region 24 after the optimized image region calculation can be represented by the following formulas (3) and (4), where the horizontal image size is CAL_IN_HSIZE and the vertical image size is CAL_IN_VSIZE.

[0115] CAL_IN_HSIZE = IMG_OUT_HSIZE / IMG_HSCALE......(3)

[0116] CAL_IN_VSIZE = IMG_OUT_VSIZE / IMG_VSCALE......(4)

[0117] In the above calculations, when the output image of output image region 23 is output, the size of the required image region in input image region 21 is calculated based on a scaling factor. Figure 11 The input image region 24 after the calculation shown corresponds to the above. Figure 5 .

[0118] As described above, the optimized image region calculation unit 111 divides the horizontal and vertical scaling factors IMG_HSCALE and IMG_VSCALE based on the output image region sizes IMG_OUT_HSIZE and IMG_OUT_VSIZE. As a result, the optimized image region calculation unit 111 can obtain the minimum required input image region for the image output, i.e., the compared and updated input image region 24. CAL_IN_HSIZE and CAL_IN_VSIZE, as the calculation results calculated by the optimized image region calculation unit 111, are transmitted to the input image region comparison unit 112 via path S10.

[0119] The computational processing of the required image after the processing begins: See [link / reference] Figure 12

[0120] Next, the function of the input image region comparison unit 112 will be described.

[0121] The parameters (IMG_IN_HSIZE and IMG_IN_VSIZE) set in the input image reading information setting unit 17 are input to the input image region comparison unit 112 via path S2. On the other hand, the calculation results calculated by the optimized image region calculation unit 111, that is, the optimized image calculation dimensions (CAL_IN_HSIZE and CAL_IN_VSIZE), are input to the input image region comparison unit 112 via path S10.

[0122] The input image region comparison unit 112 compares the horizontal and vertical dimensions based on the input parameters and calculation results. In the horizontal direction, the input image region comparison unit 112 compares IMG_IN_HSIZE with CAL_IN_HSIZE and stores the smaller value as IMG_IN_HSIZE_RENEW. In the vertical direction, the input image region comparison unit 112 compares IMG_IN_VSIZE with CAL_IN_VSIZE and stores the smaller value as IMG_IN_VSIZE_RENEW. This process means that if the calculation result of the optimized image region calculation unit 111 is less than the parameters set in the input image reading information setting unit 17, it is only necessary to read the portion of the image region smaller than the input image region 21 from the updated input image region 24 after comparison from the memory 12.

[0123] exist Figure 12 The diagram illustrates dashed boxes F1 and F2. Dashed box F1 indicates the input image region 21 defined by CAL_IN_HSIZE and CAL_IN_VSIZE. Dashed box F2 indicates the compared and updated input image region 24 defined by IMG_IN_HSIZE and IMG_IN_VSIZE. The input image region comparison unit 112 selects the image region with the smaller calculation result from these two image regions. In the first example, the compared and updated input image region 24 indicated by box F2 is selected. Here, the stored CAL_IN_HSIZE and CAL_IN_VSIZE become IMG_IN_HSIZE_RENEW and IMG_IN_VSIZE_RENEW. IMG_IN_HSIZE_RENEW and IMG_IN_VSIZE_RENEW are transmitted to the information reading control unit 114 via path S5 and used for subsequent processing.

[0124] As described above, in the image processing circuit 15, the input image region 21 is replaced with the updated input image region 24 (which is the required image input region), so that it is not necessary to read the unnecessary image region 25 from the memory 12, and the bandwidth can be reduced. The amount of bandwidth reduction RA is obtained by the following formula (5).

[0125] [Mathematical Formula 1]

[0126]

[0127] Calculation example based on actual size example in the computational processing of required images

[0128] will describe Figure 3The specific calculations in step ST7 are shown. Step ST7 is image processing performed when the image is displayed by LCDC127d.

[0129] First, the specifications will be described. The input image region of input image information 13 has 1920 × 1080 pixels (IMG_IN_HSIZE = 1920, and IMG_IN_VSIZE = 1080), the output image region has 2560 × 1600 pixels (IMG_OUT_HSIZE = 2560, and IMG_OUT_VSIZE = 1600), and the magnification ratio is 1.4815 × 1.4815 (IMG_HSCALE = 1.4815, and IMG_VSCALE = 1.4815). As the magnification ratio, the ratio in the vertical direction is obtained, and the aspect ratio in both the vertical and horizontal directions is fixed.

[0130] Calculation examples of computational processing of required images

[0131] First, the calculation in the horizontal direction will be described. The size of the input image is a parameter set in the input image reading information setting unit 17, and the size of the calculated result (CAL_IN_HSIZE) is the result calculated by the optimized image region calculation unit 111 using formula (1) and formula (3).

[0132] Dimensions in the horizontal direction after scaling: ≈2844

[0133] The calculated size is: IMG_OUT_HSIZE / IMG_HSCALE = 2560 / 1.4815 ≈ 1728

[0134] As a result, since the size of the calculated result CAL_IN_HSIZE (1728) is smaller than the size of the input image IMG_IN_HSIZE (1920), the input image region comparison unit 112 can optimize the reading size in the horizontal direction and update the parameters. That is, the input image region comparison unit 112 updates the value of the calculated result CAL_IN_HSIZE (1728) from the value of IMG_IN_HSIZE (1920) to IMG_IN_HSIZE_RENEW (1728).

[0135] Next, the calculation in the vertical direction will be described. The size of the input image is a parameter set in the input image reading information setting unit 17, and the size of the calculated result (CAL_IN_VSIZE) is the result calculated by the optimized image region calculation unit 111 using formula (2) and formula (4).

[0136] Vertical dimensions after scaling: ≈ 1600

[0137] The calculated size is: IMG_OUT_VSIZE / IMG_VSCALE = 1600 / 1.4815 ≈ 1080

[0138] As a result, since the size (1080) of the input image's IMG_IN_VSIZE is the same as the size (1080) of the calculated CAL_IN_VSIZE, the input image region comparison unit 112 maintains the vertical reading size at 1080. That is, the input image region comparison unit 112 maintains the calculated CAL_IN_VSIZE value (1080) at the IMG_IN_VSIZE value (1080) and obtains IMG_IN_VSIZE_RENEW (1080).

[0139] Figure 13 This is a schematic diagram illustrating an example of the specifications and calculation results. For example... Figure 13 As shown, the size of the input image region 21 is "IMG_IN_HSIZE = 1920" in the horizontal direction and "IMG_IN_VSIZE = 1080" in the vertical direction. The size of the magnified input image region 22 is "IMG_IN_HSIZE = 1920" in the horizontal direction. "And in the vertical direction is " The output image region 23 has dimensions of "IMG_OUT_HSIZE = 2560" horizontally and "IMG_OUT_VSIZE = 1600" vertically. The updated input image region 24 has dimensions of "IMG_IN_HSIZE_RENEW = 1728" horizontally and "IMG_IN_VSIZE_RENEW = 1080" vertically. The unnecessary image region 25 is the area enclosed by a broken frame in the figure.

[0140] The magnified input image region 22 is magnified from the input image region 21 by a factor of 1.4815 in both the horizontal and vertical directions. Conversely, the input image region 24 is reduced from the magnified input image region 22 by a factor of 1 / 1.4815 in both the horizontal and vertical directions.

[0141] By performing the above processing, the image processing circuit 15 can reduce the number of pixels in the input image information 13 read from the memory 12 by (1920 - 1728) × 1080 = 207360 pixels. Furthermore, the bandwidth is (1920 - 1728) / 1920 = 0.10, and the image processing circuit 15 can reduce the bandwidth of the input image information 13 read from the memory 12 by 10%. Therefore, the semiconductor device 1 including the image processing circuit 15 can reduce the amount of access to the memory 12 and reduce the load on the image processing circuit 15.

[0142] Second example

[0143] Next, a second example will be described. In the first example, semiconductor device 1 has already been described according to... Figure 6 The example shown is of descriptor list 124a used to perform processing. The address used to read the next descriptor list is stored in the last part of the frame of descriptor list 124a. In the second example, descriptor list 124n is stored as the next descriptor list. The following will describe how the size of the input image region used as the output image changes when descriptor list 124n is used.

[0144] The second example will be described by using the following example, in which the size of the input image information 13 is magnified to 125% in each of the horizontal and vertical directions compared to the first example. Figure 14 This diagram compares the output image area 23 of the first example with the output image area 23a of the second example. Since the monitor size of the LCDC 127d (Liquid Crystal Display Device) is the same as the monitor size in the first example, as... Figure 14 As shown, the output image area 23 in frame F3, which has the same size as the output image area 23 in the first example, is the output image area that needs to be output in the second example. That is, the image in output image area 23 that is the same as the image in the first example is displayed on the liquid crystal display device, and the image outside frame F3 is not displayed. Therefore, in the input image area 22, which is magnified by 125%, unnecessary image areas are generated in both the horizontal and vertical directions.

[0145] In the computational processing of the required image after the processing begins, a computational example based on the actual size example is provided.

[0146] As in the first example, the description Figure 3 The specific calculations for step ST7 are as follows. Step ST7 is image processing performed when the display processing of LCDC 127d is executed.

[0147] First, the specifications will be described. The input image area of ​​input image information 13 has 1920 × 1080 pixels (IMG_IN_HSIZE = 1920, and IMG_IN_VSIZE = 1080), the output image area has 2560 × 1600 pixels (IMG_OUT_HSIZE = 2560, and IMG_OUT_VSIZE = 1600), and the magnification ratio is 1.8519 × 1.8519 (IMG_HSCALE = 1.8519, and IMG_VSCALE = 1.8519). Compared to the first example, the magnification in the horizontal and vertical directions is performed with a factor of 1.25 (1.4181 × 1.25 for the original input image). That is, the magnification ratio corresponds to 125% on the monitor of the liquid crystal display device.

[0148] Description list 124n

[0149] In the second example, by correcting the following four parameters in the descriptor list 124n, the input image information 13 can be magnified by a factor of 1.25 compared to the first example.

[0150] Parameter "IMG_IN_STADD": The starting address of memory 12 when the input image is read.

[0151] Parameter "IMG_OUT_STADD": The starting address of memory 12 when the output image is written.

[0152] Parameter "IMG_HSCALE": Horizontal scaling factor

[0153] Parameter "IMG_VSCALE": Vertical scaling factor

[0154] Since other parameters besides those mentioned above have not been corrected, descriptor list 124n is as follows: Figure 15 As shown in the figure. Figure 15 This is a diagram illustrating an example of the information T30 stored in descriptor list 124n in the second example. The parameters other than those changed above are... Figure 6 The parameters are the same. In descriptor list 124n, it is used to describe... Figure 6 The field "Update from the first example" of the difference in descriptor list 124a is stored for description, but the field does not need to be provided.

[0155] Calculation example of computational processing of the required image

[0156] First, the calculation in the horizontal direction will be described. The size of the input image is a parameter set in the input image reading information setting unit 17, and the size of the calculated result (CAL_IN_HSIZE) is the result calculated by the optimized image region calculation unit 111 using formula (1) and formula (3).

[0157] Horizontal dimensions after scaling: ≈ 3556

[0158] The calculated size is: IMG_OUT_HSIZE / IMG_HSCALE = 2560 / 1.8519 ≈ 1382

[0159] As a result, since the size of the calculated result CAL_IN_HSIZE (1382) is smaller than the size of the input image IMG_IN_HSIZE (1920), the input image region comparison unit 112 can optimize the reading size in the horizontal direction and update the parameters. That is, the input image region comparison unit 112 updates the value of the calculated result CAL_IN_HSIZE (1382) from the value of IMG_IN_HSIZE (1920) to IMG_IN_HSIZE_RENEW (1382).

[0160] Next, the calculation in the vertical direction will be described. The size of the input image is a parameter set in the input image reading information setting unit 17, and the size of the calculated result (CAL_IN_VSIZE) is the result calculated by the optimized image region calculation unit 111 using formula (2) and formula (4).

[0161] Vertical dimensions after scaling: ≈2000

[0162] The calculated size is: IMG_OUT_VSIZE / IMG_VSCALE = 1600 / 1.8519 ≈ 863

[0163] As a result, since the size (863) of the calculated CAL_IN_VSIZE is smaller than the size (1080) of the input image IMG_IN_VSIZE, the input image region comparison unit 112 can optimize the reading size in the vertical direction and update the parameters. That is, the input image region comparison unit 112 updates the value (863) of the calculated CAL_IN_VSIZE from the value (1080) of IMG_IN_VSIZE to IMG_IN_HSIZE_RENEW (863).

[0164] Figure 16 This is a schematic diagram illustrating an example of the specifications and calculation results. For example... Figure 16As shown, the size of the input image region 21 is "IMG_IN_HSIZE=1920" in the horizontal direction and "IMG_IN_VSIZE=1080" in the vertical direction. The size of the magnified input image region 22 is "IMG_IN_HSIZE=1920" in the horizontal direction. "And in the vertical direction is " The output image region 23 has dimensions of "IMG_OUT_HSIZE = 2560" horizontally and "IMG_OUT_VSIZE = 1600" vertically. The updated input image region 24 has dimensions of "IMG_IN_HSIZE_RENEW = 1382" horizontally and "IMG_OUT_VSIZE / IMG_VSCALE = 863" vertically. The unnecessary image region 25 is the area enclosed by a broken frame in the figure.

[0165] The magnified input image region 22 is magnified horizontally from the input image region 21 by a factor of 1.8519. Conversely, the input image region 24 is reduced horizontally and vertically from the magnified input image region 22 by a factor of 1 / 1.8519.

[0166] By performing the above processing, the image processing circuit 15 can reduce the number of pixels in the input image information 13 read from the memory 12 by (1920-1382)×863 + 1920 × (1080-863) = 880934 pixels. Furthermore, the bandwidth is 880934 / (1920-1080) = 0.42, and the image processing circuit 15 can reduce the bandwidth of the input image by 42%. Therefore, the semiconductor device 1 including the image processing circuit 15 can reduce the load on the image processing circuit 15.

[0167] Effect

[0168] The semiconductor device 1, including image processing circuitry 15 that performs image processing such as scaling, can calculate an optimized input image region 24 based on the ratio of the scaling factor between the output image region 23 and the input image region 21. As a result, the image processing circuitry 15 can access the pixels of the truly needed image region 25 from the input image information 13 and can reduce the bandwidth used to access the memory 12.

[0169] Furthermore, by reducing the access bandwidth to memory 12, image processing circuit 15 is able to improve the processing speed from image input to processed image output.

[0170] Furthermore, the image processing circuit 15 can automatically calculate an easily readable image size simply by adding a process of changing the scaling factor setting without placing a load on the CPU 11. As described above, the processing of the image processing circuit 15 can also be reduced without placing a load on the CPU 11, and the processing efficiency of the entire semiconductor device 1 can be improved.

[0171] Although the resolution of images to be processed has been improved to 4K and 8K in recent years, bus bandwidth is extremely limited. Therefore, by applying the technology disclosed herein, semiconductor device 1 can significantly reduce the amount of input image information 13 to be accessed, and the efficiency of the bus to be used can be improved.

[0172] Although the present invention has been specifically described based on the embodiments, the present invention is not limited to the above embodiments, and it is obvious that various modifications can be made without departing from the spirit of the present invention.

Claims

1. A semiconductor device, comprising: The register contains input image setting information, scale factor setting information, and output image setting information. The input image setting information includes the input size of the image region of the input image to be read from the memory. The scale factor setting information is used to set the scale factor of the input image. The output image setting information includes the output size of the image region of the output image to be output to the memory. as well as An image processing circuit includes a scaling unit that changes the size of the input image based on a scaling factor set in the scaling factor setting information, and performs image processing on the input image. The image processing circuit includes: The calculation unit calculates the required image region size for outputting the output image based on the output size and the scaling factor. The comparison unit compares the desired image region size with the input size, selects an image region having the smaller size between the desired image region size and the input size, and uses the selected image region for image processing of the input image.

2. The semiconductor device according to claim 1, The comparison unit calculates the desired image region size by dividing the output size by the scaling factor.

3. The semiconductor device according to claim 1, The image processing circuit further includes an information reading control unit, which performs control for reading the input image from the memory. The information reading control unit reads the input image from the memory the image region selected by the comparison unit in the image region of the input image.

4. The semiconductor device of claim 3, further comprising a scaling calculation unit, the scaling calculation unit receiving the scaling factor setting information from the register and calculating the scaling processing of the input image selected by the comparison unit. The scaling calculation unit scales the input image of the selected image region using the scaling factor included in the scaling factor setting information.

5. The semiconductor device of claim 4, further comprising an information writing control unit, the information writing control unit performing control for writing the output image to the memory. The information writing control unit outputs a scaled input image of the image region to the memory as the output image.

6. The semiconductor device according to claim 1, The register includes a first setting unit, a second setting unit, and a third setting unit. The input image settings information is set in the first settings unit. The scaling factor setting information is set in the second setting unit. The output image settings information is set in the third setting unit. The comparison unit receives the input image setting information from the first setting unit, and The calculation unit receives the scaling factor setting information from the second setting unit and the output image setting information from the third setting unit.

7. The semiconductor device according to claim 6, The memory stores a list, which includes the input image settings information, the scaling factor settings information, and the output image settings information. Before reading the input image of the image region from the memory, the image processing circuit reads the input image setting information, the scaling factor setting information, and the output image setting information from the memory, and sets the read input image setting information, scaling factor setting information, and output image setting information in the first setting unit, the second setting unit, and the third setting unit, respectively.

8. The semiconductor device according to claim 7, The memory mentioned therein includes multiple lists. The list includes specified information that specifies the list for subsequent processing of the input image. The image processing circuit described therein performs image processing on the input image based on the specified information included in the list.

9. The semiconductor device according to claim 1, The output size of the output image is determined based on the device connected to the semiconductor device.

10. The semiconductor device according to claim 1, The output size of the output image is determined based on processing at a subsequent stage of the image processing circuit.

Images