Method and system for one-dimensional signal extraction for various computer processors

Abstract

Methods and systems for extracting a one-dimensional (1D) signal from a two-dimensional (2D) digital image along a projection line are provided herein. The method and system store the digital image in a memory hierarchy, where a non-blocking prefetch operation may fetch pixels from a main memory to a data cache. In response to the direction of the projection line, a prefetch plan, a pixel processing plan, and a prefetch distance are selected. The prefetch plan uses a first address order designed to facilitate efficient extraction of pixels from the main memory to the data cache for a given direction. The pixel processing plan uses a second address order designed to facilitate calculation of a one-dimensional signal along the projection line. A pixel processing plan is used in coordination with a prefetch plan to compute the one-dimensional signal such that the pixel is fetched from the main memory to the data cache in advance in response to an amount of time of the prefetch distance before the pixel operation uses the pixel.

Description

Cross-references to related applications

[0001] This application claims priority to U.S. Provisional Patent Application Serial No. 63 / 500,415, filed May 5, 2023, entitled “Method and System for One-Dimensional Signal Extraction for Various Computing Processors,” the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0002] This technical field generally relates to digital electronic methods and systems, including computer program products for extracting one-dimensional digital signals from two-dimensional digital images. Background Technology

[0003] In digital image processing applications, it is desirable to extract a one-dimensional (1D) signal along a line in a two-dimensional (2D) digital image. Such applications can include, for example, inspection, measurement, and guidance in electronics, semiconductors, and general manufacturing, as well as barcode and other symbol reading.

[0004] The term "projection" is sometimes used to refer to the act of extracting a one-dimensional signal along a line in a two-dimensional image (referred to as the projection line in this paper). The term is sometimes also applied to the one-dimensional signal itself, and sometimes has other meanings in the field of digital image processing.

[0005] In some methods or systems, projection lines are constrained to lie on rows, columns, or diagonals of a digital image (parallel to or diagonally opposite the pixel grid of the digital image). In this case, a one-dimensional signal can be extracted from pixel values ​​that lie precisely on the projection lines. For rows and columns, the sampling points of this one-dimensional signal can be spaced one pixel apart, and for diagonals, the sampling points of the one-dimensional signal can be spaced... 1 pixel (assuming it is a square pixel).

[0006] In other methods or systems where projection lines are parallel to or diagonally opposite to the grid, a one-dimensional signal is extracted by summing or averaging pixel values ​​along a direction perpendicular to the projection lines. For example, if the projection follows the 20th row of a digital image, each sample point of the one-dimensional signal along that line could be the sum or average of pixel values ​​on a portion of columns including rows 18, 19, 20, 21, and 22.

[0007] When the projection lines are not parallel or diagonal, pixels that lie precisely on or perpendicular to the lines are typically sparsely distributed or almost nonexistent. Therefore, in some methods or systems where the projection lines are not limited to rows, columns, or diagonals, a one-dimensional signal is extracted from a set of pixels that roughly follow the projection lines. An example of this approach is the so-called Bresenham line-following method, which typically proceeds one pixel at a time along rows, columns, or diagonals, such that the accessed pixels roughly follow the projection lines.

[0008] In another approach (referred to in this paper as linear convolution), a one-dimensional signal is extracted by convolving the digital image with a two-dimensional filter kernel at a location selected by the Bresenham line-following method. This filter kernel is designed to provide a summation or average approximately perpendicular to the projection lines. The filter kernel can have uniform weights, or the weights can become smaller for pixels farther from the projection lines.

[0009] In another approach (referred to in this paper as tilt projection), pixels in a parallelogram pattern are used to extract a one-dimensional signal. This parallelogram has two sides along the rows of the image, and two other sides at a tilt angle (typically not along the columns). Therefore, the parallelogram consists of a specific number of consecutive pixels from each of a number of consecutive rows whose starting columns are offset to approximately follow the tilt angle. The one-dimensional signal is formed by summing or averaging along the tilt direction.

[0010] In another approach (referred to in this paper as nearest neighbor projection), a grid of points is selected, distributed at intervals (usually one pixel) along the projection line and at intervals (also usually one pixel) in a direction perpendicular to the projection line. The image coordinates of these points are rounded to the nearest integer so that they fall on pixel coordinates, and the pixels thus specified are used to extract the one-dimensional signal by summing or averaging approximately (in the nearest neighbor sense) perpendicular to the projection line.

[0011] In other methods (referred to herein as bilinear interpolation and bicubic interpolation), the point grid is selected in a manner similar to that used for nearest-neighbor projection. However, instead of rounding the point coordinates to integers, the coordinates are used to calculate the interpolated pixel values. These interpolations are used to extract the one-dimensional signal by summing or averaging perpendicular to the projection lines. The formulas for bilinear and bicubic interpolation are well known in the art. Summary of the Invention

[0012] This disclosure relates to a digital electronic method and system for extracting one-dimensional (1D) digital signals from a two-dimensional (2D) digital image along projection lines.

[0013] It can receive two-dimensional digital images (e.g., from a camera, scanner, or computer rendering). The image can include pixels arranged on a pixel grid. Each pixel can be a single value or a set of values, such as a complex number or a vector, including, for example, a color vector. The values ​​can represent any type of information (e.g., numbers or symbols) and can be encoded in various formats (e.g., binary integers or floating-point values).

[0014] Digital images can be stored using a memory hierarchy that includes main memory and a data cache, where non-blocking prefetch operations fetch pixels from main memory into the data cache. Main memory can be large enough to hold a digital image of a useful size, but may be much slower than a digital processor used to compute a one-dimensional signal, making pixel acquisition time a significant bottleneck. The data cache can be smaller and faster than main memory to provide high-speed access to selected portions of the digital image. The data cache can have any number of levels of varying sizes and speeds. While the prefetch operation is in progress, the non-blocking prefetch operation does not prevent the digital processor or memory hierarchy from continuing to perform other operations (e.g., acquiring pixels and using them to compute a one-dimensional signal). Pixels can be fetched from main memory into the data cache in units called cache lines. The memory hierarchy can be any digital memory system, or any memory system that includes main memory and a data cache that supports non-blocking prefetch operations.

[0015] Information describing the projection line can be received, from which the direction of the projection line can be obtained. This direction may be one of a set of allowed directions, which includes directions that are not parallel to the pixel grid and are not diagonally opposite to the pixel grid. In some embodiments, the set of allowed directions may include directions with rational slopes. In some embodiments, the set of allowed directions may be predetermined.

[0016] A prefetch plan can be selected in response to the direction, the prefetch plan specifying a sequence of prefetch operations in a first address order, the sequence of prefetch operations including a row sequence. The first address order and the row sequence can be designed to facilitate efficient fetching of pixels from main memory to the data cache for a given direction. A row can occupy one or more adjacent cache lines or portions of cache lines.

[0017] A pixel processing plan can be selected in response to this direction, specifying a sequence of pixel operations with a second address order different from the first address order. This second address order can be designed to facilitate the computation of a one-dimensional signal along the projection line.

[0018] The prefetch distance can be selected in response to this direction. The prefetch distance can include a time quantity or other unit of work, such as the number of loop iterations, which lies between the prefetch operation of fetching pixels from main memory to the data cache and the operation of the processor fetching those pixels from the data cache for computing a one-dimensional signal.

[0019] The pixel processing plan is used in coordination with the prefetch plan to compute the one-dimensional signal, including: executing the prefetch operation sequence to fetch the pixel from the main memory into the data cache in advance of the amount of time corresponding to the prefetch distance before the pixel operation uses the pixel.

[0020] In some embodiments, at least one of the prefetch schedule, the pixel processing schedule, or the prefetch distance may be pre-calculated and stored in a table memory.

[0021] In some embodiments, the pixel processing plan may specify a repeating sequence of pixel weight templates.

[0022] In some embodiments, the prefetching plan may include a first phase for initialization and a second phase for cyclic execution.

[0023] In some embodiments, the row sequence may include an initial, possibly empty, partial row sequence, and a cyclic, complete row sequence. In some embodiments, the first phase for initialization may specify prefetching a partial row. In some embodiments, the second phase may specify prefetching the complete row.

[0024] In some embodiments, the prefetch distance can be a parametric function of the direction. In some embodiments, the prefetch distance can be determined by measuring the execution time.

[0025] In some embodiments, the orientation can be distinguished as follows: one orientation is where no row in the corresponding prefetch schedule exceeds the cache line size by more than one addressable memory cell, and another orientation is where at least one row in the corresponding prefetch schedule exceeds the cache line size by more than one addressable memory cell. Typically, the latter orientations are closer to horizontal, and therefore these are called near-horizontal orientations, while the former, and usually more numerous, orientations are called non-horizontal orientations. In these embodiments, the prefetch schedules for near-horizontal orientations can use a different style than those for non-horizontal orientations.

[0026] In some embodiments, for a prefetching plan with multiple allowed directions, it may be specified to prefetch only the first and last pixels of each row. These directions may include non-horizontal directions, wherein fetching only the first and last pixels of each row ensures that all cached rows containing a portion of that row are prefetched, while other cached rows are not prefetched.

[0027] In some embodiments, the near-horizontal prefetching plan may specify prefetching three pixels per row. In some embodiments, the near-horizontal prefetching plan may specify prefetching exactly three pixels per row.

[0028] Some embodiments relate to an electronic device for extracting a one-dimensional signal from a two-dimensional image along a projection line. The device may include: a memory hierarchy including a main memory and a data cache, wherein the two-dimensional image comprises pixels arranged on a pixel grid and stored in the memory hierarchy, and pixels of the two-dimensional image are fetched from the main memory to the data cache via a non-blocking prefetch operation; and at least one processor configured to execute computer-executable instructions, wherein the computer-executable instructions include instructions for: receiving information describing the projection line, the information including the direction of the projection line, wherein the direction is one of a set of allowed directions, the set of allowed directions including those not parallel to the pixel. The grid is not diagonally opposite to the pixel grid; in response to the direction, a prefetch plan is selected, the prefetch plan specifies a prefetch operation sequence in a first address order, the prefetch operation sequence including a row sequence; in response to the direction, a pixel processing plan is selected, the pixel processing plan specifies a pixel operation sequence in a second address order different from the first address order; in response to the direction, a prefetch distance is selected; and the pixel processing plan is coordinated with the prefetch plan to compute the one-dimensional signal, including: executing the prefetch operation sequence to fetch the pixel from the main memory into the data cache an amount of time corresponding to the prefetch distance before the pixel operation sequence uses the pixel.

[0029] In some embodiments, at least one of the prefetch schedule, the pixel processing schedule, or the prefetch distance may be pre-calculated and stored in a table memory.

[0030] In some embodiments, the pixel processing plan may specify a repeating sequence of pixel weight templates.

[0031] In some embodiments, the prefetch schedule includes a first phase for initialization and a second phase for cyclic execution.

[0032] In some embodiments, the first phase may specify a prefetch of a subset of rows.

[0033] In some embodiments, the second phase may specify prefetching of complete rows.

[0034] In some embodiments, the prefetch distance can be a parameter function of the direction.

[0035] In some embodiments, the prefetch distance can be determined by measuring the execution time of the pixel processing schedule.

[0036] In some embodiments, for a prefetching plan with multiple allowed directions, it may be specified to prefetch only the first and last pixels of each row.

[0037] In some embodiments, the near-horizontal prefetching plan can specify prefetching three pixels per row.

[0038] In some embodiments, the near-horizontal prefetching schedule may specify prefetching exactly three pixels per row.

[0039] Some embodiments relate to a non-transitory computer-readable medium storing computer-executable instructions configured to, when executed by at least one processor, perform a method for extracting a one-dimensional signal from a two-dimensional image along projection lines. The method may include: accessing the two-dimensional image, the two-dimensional image comprising pixels arranged on a pixel grid; storing the two-dimensional image using a memory hierarchy comprising main memory and a data cache, wherein a non-blocking prefetch operation is configured to fetch pixels from the main memory into the data cache; receiving information describing the projection line, the information including the direction of the projection line, wherein the direction is one of a set of allowed directions, the set of allowed directions including directions that are not parallel to the pixel grid and not diagonally opposite to the pixel grid; in response to the direction, selecting a prefetch schedule, the prefetch schedule specifying a sequence of prefetch operations in a first address order, the prefetch operation sequence including a row sequence; in response to the direction, selecting a pixel processing schedule, the pixel processing schedule specifying a sequence of pixel operations in a second address order different from the first address order; in response to the direction, selecting a prefetch distance; and coordinating the pixel processing schedule with the prefetch schedule to compute the one-dimensional signal, including: executing the prefetch operation sequence to fetch pixels from the main memory into the data cache an amount of time in advance, in response to the prefetch distance, before the pixel operation sequence uses the pixels.

[0040] Therefore, the features of the disclosed subject matter have been outlined rather broadly in order to better understand the detailed description that follows and to better understand the present contribution to the art. Of course, the invention also contains additional features of the disclosed subject matter that will be described thereafter and form the subject matter of the appended claims. It should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered limiting. Attached Figure Description

[0041] The foregoing and other aspects, features, and advantages of this digital electronic method and system, as well as the digital electronic method and system itself, will be more fully understood from the following description of various embodiments taken in conjunction with the accompanying drawings.

[0042] Figure 1 This is a schematic illustration of an exemplary system operation for one-dimensional signal extraction.

[0043] Figure 2 It shows Figure 1This is part of an exemplary system, which shows in more detail elements that are generally relevant to this disclosure.

[0044] Figure 3 An exemplary pixel processing scheme according to the signal extraction method is shown, illustrating the projection line with a slope of 3 / 5, the module sequence, and the illustrative pixel access order for signal extraction.

[0045] Figure 4 It shows Figure 3 An exemplary pixel processing scheme is shown, which illustrates a portion of the row and row cycle according to an illustrative embodiment of this disclosure.

[0046] Figure 5 An exemplary program in the C programming language is shown for prefetching a row of pixels.

[0047] Figure 6 It shows the use of Figure 3 and Figure 4 The exemplary partial row and row cycle phase of the 3 / 5 slope projection line.

[0048] Figure 7 An exemplary prefetching scheme for a projection line with a slope of 3 / 5 is shown.

[0049] Figure 8 An alternative exemplary prefetching scheme for a projection line with a slope of 3 / 5 is shown.

[0050] Figure 9 An exemplary pixel processing scheme according to the signal extraction method is shown, which illustrates the projection line with a slope of 7 / 3, the module sequence, and the illustrative pixel access order for signal extraction.

[0051] Figure 10 An exemplary prefetching scheme for a 7 / 3 slope projection line is shown.

[0052] Figure 11 An exemplary pixel processing scheme according to the signal extraction method is shown, illustrating the projection line with a slope of 1 / 7, the module sequence, and the illustrative pixel access order for signal extraction.

[0053] Figure 12 An exemplary near-horizontal prefetching scheme for a projection line with a slope of 1 / 7 is shown.

[0054] Figure 13 This is an exemplary flowchart illustrating the process of coordinating pixel processing plans with prefetching plans.

[0055] Figure 14 A diagram is shown illustrating an exemplary method for selecting a prefetch distance in response to a projection direction. Detailed Implementation

[0056] In the following detailed description of illustrative embodiments, reference is made to the accompanying drawings, which form a part of this document, illustrating specific embodiments in which the methods or systems described herein may be practiced. It should be understood that other embodiments may be utilized and structural changes may be made without departing from the scope of this disclosure.

[0057] It is expected that the time required to extract a one-dimensional (1D) signal from a two-dimensional (2D) image on a given digital processing device can be reduced, or the operating speed can be increased equivalently. It is worth noting that, regardless of the method or system used, a significant portion of the time required for one-dimensional signal extraction may be consumed by the time required to retrieve image pixels from memory. Memory large enough to hold a useful digital image (such as those commonly referred to as dynamic random access memory) is typically much slower than the processor used, making pixel acquisition time a significant bottleneck.

[0058] One way to reduce this bottleneck is to fetch pixels from a larger, slower memory to a smaller, faster memory before the processor needs them, allowing pixel acquisition time to overlap with other operations (such as computing and outputting one-dimensional signals). This overlap partially hides the pixel acquisition time, thereby reducing the overall signal extraction time. Since the smaller, faster memory may be too small to hold the entire image, it is important to carefully choose which pixels to acquire, when to acquire them, and when to replace previously acquired pixels that are no longer needed.

[0059] For example, a repeating sequence of pixel weight templates can be placed at a series of relative positions in a digital image. The use of such pixel weight templates can provide high photometric and geometric accuracy while allowing independent control of blurring along and perpendicular to the projection lines. The time required to fetch pixels from memory can significantly impact operating speed. A direct memory access controller (DMAC) can be used to extract pixels into a small, fast temporary memory for access by a digital signal processor. A DMAC can be used to overlap pixel acquisition with the computation of one-dimensional signal extraction, suitable for devices equipped with a DMAC, such as digital signal processors. An example of generating a one-dimensional signal is shown in U.S. Patent No. 9,122,952 (hereinafter referred to as the '952 Patent), the contents of which are incorporated herein by reference in their entirety. However, many contemporary digital processors intended for one-dimensional signal extraction lack a DMAC to overlap pixel acquisition with the computation of one-dimensional signal extraction.

[0060] Many modern digital processors employ various memory hierarchy designs that can include larger, slower memories working in conjunction with smaller, faster memories (called data cache levels) to provide a system with the performance of a larger memory and some speed advantages of a smaller memory. These systems may include the ability to perform prefetch operations, which fetch data from slower memory to faster memory before the processor needs it, allowing pixel acquisition to overlap with other computations performed by the processor. The effectiveness of the speedup achieved by prefetch operations depends critically on the order and timing of these operations relative to the operations performed by the processor.

[0061] Prefetching can be initiated, for example, by the processor under program control (software prefetching) or automatically by a hardware prefetching engine. In the case of software prefetching, some compilers can identify certain simple memory access patterns in the source code and insert prefetch instructions into the object code. A hardware prefetching engine can attempt to infer the order in which the processor accesses memory during program runtime.

[0062] Compilers or hardware prefetch engines can attempt to predict which parts of slower memory to prefetch and when to prefetch. These methods are limited because they are general-purpose algorithms that do not take advantage of the specific requirements and properties of one-dimensional signal extraction, including complex memory access patterns that are highly dependent on the projection line orientation, and where the desired pixel order for prefetching may be very different from the desired pixel processing order for signal extraction.

[0063] This disclosure provides methods and systems for creating, selecting, and using prefetch plans that employ direct knowledge of memory access patterns specific to a given projection line orientation. This disclosure further provides methods and systems for coordinating the use of these prefetch plans with pixel processing to control the timing between pixel prefetching and pixel usage to achieve desired overlap. In some embodiments, plans for a subset or all of the allowed orientation sets can be pre-computed and stored in a table memory.

[0064] Figure 1 This is a schematic illustration of exemplary system operation for one-dimensional signal extraction. A two-dimensional digital image 100 (e.g., from a camera, scanner, or computer rendering) is received. The digital image 100 may contain features, such as a barcode 110. Information describing a projection line 120 is received, along which a one-dimensional digital signal (e.g., signal 140) is expected to be extracted, corresponding to the example feature barcode 110. The digital electronic system 130 extracts signal 140 from image 100 along projection line 120.

[0065] A one-dimensional digital signal typically comprises a series of values ​​(often referred to as sampling points or projection intervals). Values ​​can be single digits or groups of digits, such as complex numbers, or vectors, such as color vectors. The digits can be encoded in various formats, such as binary integers or floating-point values.

[0066] The direction of the projection line 120 relative to the digital image 100 is one of a set of allowed directions, which may include all directions that can be encoded in the received information describing the projection line 120, and may further: be restricted to a range (e.g., 0–45°); be restricted to a direction with a rational slope; be restricted to a direction that is chosen as favorable according to some metric; be restricted to a direction that is randomly selected; or any other suitable restriction or combination of restrictions.

[0067] It should be noted that while barcodes are used as examples in some parts of this article, it is well known that extracting one-dimensional signals along projection lines in a digital image is useful for a variety of applications (e.g., in the manufacture of printed circuit boards, solar panels, and integrated circuits). One such application in integrated circuit manufacturing is positioning leads on a lead frame during wire bonding. Therefore, the barcode examples in this article are for illustrative purposes only and should not be considered limiting.

[0068] Figure 2 A portion of an exemplary digital electronic system 130 is shown, illustrating elements generally associated with this disclosure.

[0069] In the example shown, the received digital image 230 is stored in main memory 220, which is an element of memory hierarchy 270. The image 230 is logically a two-dimensional array of pixels, each of which can be identified by an integer coordinate (x, y) and an address in memory 220 that can be derived from the (x, y) coordinates. Although a byte-addressable memory system with single-byte pixels is shown, it should be understood that different memory storage units or pixel sizes can be used. Each pixel can be a single value or a set of values, such as a complex number or a vector, including, for example, a color vector. Values ​​can represent any type of information (e.g., numbers or symbols) and can be encoded in various formats (e.g., binary integers or floating-point values).

[0070] The main memory 220 may also accommodate a table memory 280, which can be used to store pixel processing plans and prefetch plans, as further described below.

[0071] In the example illustrated in this article, the image is displayed relative to coordinates (x, y). In some embodiments: The pixel address increases by the pixel size in the +x direction.

[0072] A set of pixels with a constant y-coordinate is called a row, or arranged along a row.

[0073] The pixel address increases in the +y direction with respect to the line spacing, where the line spacing may vary from image to image, but is constant within the image. The line spacing can be positive or negative.

[0074] In the attached diagram, the +x direction is a horizontal direction from left to right.

[0075] In the attached diagram, the +y direction is the vertical direction from top to bottom.

[0076] While the image storage schemes with specific conventions have been described above, it should be understood that other conventions may be adopted. For example, an image storage scheme may not have a constant, image-related line spacing.

[0077] Memory hierarchy 270 may include larger, slower memories (e.g., main memory 220) that cooperate with smaller, faster memories (e.g., data cache 210). The smaller, faster memories may be referred to as cache levels and can provide the performance of larger memories while retaining some of the speed advantages of smaller memories. The smallest and fastest of these memories is typically referred to as a Level 1 (L1) cache. The speed performance of this arrangement depends critically on the order in which pixels are acquired and the ability to overlap pixel processing with the acquisition of pixels that are about to be needed.

[0078] The data cache 210 of the memory hierarchy 270 can be any suitable arrangement of any suitable number of cache levels. For the purposes of this description, the number of levels is not important. Pixels can be fetched from memory 220 to any or all levels of the data cache 210 as needed using the methods and systems illustrated and described herein. Cache levels can be dedicated to processor 200 or shared with other processors (e.g., in a so-called multi-core configuration).

[0079] In the example shown, the address space of memory hierarchy 270 is logically an array of bytes, which is internally organized into an array of memory blocks, referred to as cache lines, and is typically a power of 2 in size (e.g., 64 bytes). Different cache levels can have cache lines of different sizes, although this is not typical.

[0080] Processor 200 operates to (among other activities) acquire pixels from image 230 and calculate signal 140. To acquire pixels, processor 200 performs transactions using byte addresses and memory hierarchy 270. Hierarchy 270 executes methods for acquiring data, which may involve: Identify the cache line containing the requested bytes; Determine whether the cache line is already in the L1 cache. If it is, it is called an L1 hit; if not, it is called an L1 miss. If an L1 cache miss occurs, the cache line is fetched from a higher level or from main memory 220 into the L1 cache, and another cache line may be evicted to free up space; and Provide the requested bytes from the L1 cache to processor 200.

[0081] Memory hierarchy 270 supports non-blocking prefetch operations, which can respond to prefetch requests 250 from processor 200 without preventing processor 200 and / or hierarchy 270 from continuing to perform other operations. Prefetch request 250 specifies the address of a memory cell (e.g., a byte), which hierarchy 270 can interpret as a hint that processor 200 will soon need the specified memory cell. Prefetch request 250 can also specify a cache level target. Hierarchy 270 can respond to a prefetch request by performing a non-blocking prefetch operation that fetches a cache line containing the specified memory cell from a higher level or from main memory 220 to the target cache level, or to the default level if no target is provided. This non-blocking prefetch operation does not prevent processor 200 or hierarchy 270 from continuing to perform other operations while a prefetch operation is in progress (e.g., acquiring pixels and using them to compute signal 140). In this way, the time required to acquire pixels from slower memory can overlap with the time required to process previously acquired pixels, potentially significantly speeding up overall signal acquisition. The effectiveness of prefetching for acceleration depends critically on the order and timing of pixels and prefetch addresses.

[0082] Processor 200 can be any suitable digital electronic processor capable of issuing prefetch requests, such as 64-bit microprocessors designed by Intel and ARM and manufactured and sold by various vendors. Processor 200 can be, but is not limited to, designs commonly referred to as a central processing unit, digital signal processor, or graphics processing unit. It can also be an application-specific integrated circuit or a field-programmable gate array, wherein such devices communicate with memory and can issue prefetch requests. Prefetch requests can originate from machine instructions and / or hardwired logic executed by processor 200.

[0083] Figure 3 An exemplary pixel processing plan according to a signal extraction method is shown. The pixel processing plan can specify a sequence of pixel operations, which in turn can specify a memory address order, and can be selected in response to the direction of the projection lines.

[0084] Pixel grid 300 may correspond to a portion of the received digital image (e.g., image 230). The aforementioned pixel coordinate orientation convention is illustrated by coordinate axis 360. Pixel values ​​are not displayed.

[0085] Information describing the example projection line 310 can be received directly or indirectly. The direction of the projection line 310, corresponding to the slope in this example, is obtained from this information. y / x = 3 / 5 (approximately 31 degrees to the horizontal). In this document and other examples herein, the slope has the general meaning of the ratio of displacement in the y-direction to displacement in the x-direction. This direction is one of a set of allowed directions that are not parallel to or diagonally opposite to the pixel grid 300. For example, an exemplary embodiment of the method includes 171 allowed directions in the range of 0–90°, each with a rational slope.

[0086] A significant aspect of this disclosure is the order in which the memory addresses of pixels are obtained from the memory hierarchy 270. For signal computation purposes, this order is expected to generally follow the direction of the projection lines, as permitted by the discrete mesh geometry.

[0087] Figure 3 A pixel processing scheme characterized by a repetitive pattern, referred to as a module, is shown, placed at a series of locations that are staggered and follow projection line 310, as illustrated. The modules placed at specific locations are module cycles of the sequence, such as the first module cycle 330, the second module cycle 340, and the third module cycle 350. In this example, each cycle is displaced relative to the previous one in the x-direction. x = 5 pixels, displacement in the y direction y = 3 pixels, so that these loops follow the projection line 310.

[0088] Figure 3 The example pattern for each module can be seen in each loop and includes a set of eight disjoint relative pixel positions (called slices, labeled 0–7 in the diagram). The relative pixel position at a particular module loop can be used to obtain a memory address to retrieve that pixel. For each module loop, pixels are retrieved in slice order, from 0 to 7 in this example, and then the process continues to slice 0 of the next loop. The number of slices is the module period. Figure 3 In the example, it is 8. Within a given slice, the pixel access order is not important and can be determined by other criteria. By accessing pixels in the order determined by the module loop and the slice, the projection line 310 can be followed at pixel grid resolution.

[0089] although Figure 3A particular arrangement was used as an example, but it can be seen that any sequence of pixel operations following the projection lines in some cyclic pattern can be used within the scope of this invention. For embodiments of this disclosure using a pixel processing scheme, the pixel processing scheme can use module cycles and slicing order.

[0090] The range of a module in the y-direction is called the module height; for example, a range of 320 indicates the module height. Figure 3 For example, the module height is 6.

[0091] Figure 4 Showing with Figure 3 The same slicing pattern is used, but organized by image rows rather than module cycles to show how pixels are represented in the memory hierarchy. At the beginning of the projection lines are three partial rows 410, followed by repeating complete sequence of rows, also known as row cycles, such as example row cycles 420, 430, and 440. Each row cycle has three phases, derived from the module... y = 3, usually referred to as the row period. A row period can span many slices and multiple module loops. The row period can be an integer, such as 3. A row loop can be a set of rows that can be repeated multiple times. The row period can also be the length of the row loop.

[0092] For each module loop, some pixels are selected in a certain order from a number equal to the row period. The data is retrieved from the rows of y, where the new row contains pixels not needed in the previous module loop. If the height of a module is h, then there must be an h-pole gap between adjacent module loops. The overlap of y-lines. Since the first module loop 330 has no previous module loop, there will be h- y = 3 parts, row 410 has no loops overlapping it.

[0093] This disclosure recognizes that it is expected that rows can be prefetched at some point before the processor 200 uses them, such that pixels are likely to reside in the L1 cache when they are fetched in the address order specified by the pixel processing plan. The amount of time or other unit of work (e.g., the number of loop iterations) between prefetching and using is commonly referred to as the prefetch distance. The prefetch plan specifies the prefetch address sequence and parameters that control the prefetch distance, and can be selected in response to the direction of the projection lines, as shown in subsequent figures.

[0094] This disclosure further recognizes that the desired address order following the projection lines for pixel processing may differ from the desired address order for the row prefetching stage, the desired pixel processing cycle period may differ from the desired row prefetching cycle period, pixel processing and row prefetching may be coordinated to maintain an appropriate prefetch distance, and pixel processing and prefetching schedules may be selected in response to the direction of the projection lines.

[0095] Therefore, the description and illustrations herein provide methods and systems for creating, selecting, and coordinating prefetching schedules with pixel processing schedules, wherein these schedules can be created and selected in response to the direction of the projection lines. In some embodiments, some or all of the schedules for the allowed set of directions can be pre-computed and stored in table memory 280.

[0096] The given line to be prefetched will reside in one or more cache lines. It is expected that all cache lines containing a portion of the given line will be prefetched, and no other cache lines will be prefetched. The prefetch plan (whether or not it is pre-computed) may not know the location of the cache lines, as such a plan may only contain relative pixel locations that can be applied to the image, whose size, line spacing, and memory location are unknown beforehand.

[0097] In some embodiments, once a particular image is specified, appropriate logic can determine which cache lines to prefetch for each line when using a prefetch schedule. In some embodiments, particularly those using software instructions, this logic can be avoided as it may introduce performance overhead due to the need for additional instructions, including expensive conditional branches.

[0098] Figure 5 Computer source code in the C programming language is shown for an exemplary embodiment, wherein a line is prefetched in such a way that all cached lines containing a portion of the line are prefetched without prefetching other cached lines, without needing to know the location of the cached line boundaries.

[0099] A prefetch plan for a given direction may contain information about the rows to be prefetched (e.g., row prefetch information structure 500), which stores the offsets of the first and last bytes of the row relative to row address 510. An exemplary procedure for obtaining row address 510 is given below. Although the offsets in row prefetch information structure 500 are byte offsets, and row address 510 is a byte address (because example memory hierarchy 270 uses byte addresses), it should be understood that a pixel may not be a byte. For larger pixels, both byte offsets and addresses can be obtained.

[0100] The row information pointer 520 stores the address of the row prefetch information structure 500 instance to be used for prefetching a specific row; its value can be obtained from the prefetch plan. The cache line size 530 is an attribute of the memory hierarchy in use.

[0101] Prefetch loop 540 prefetches zero or more cache lines, starting with the cache line containing the first pixel of the line, stepping through cache line size 530 to subsequent cache lines, stopping when the last pixel is reached or exceeded. Note that prefetch loop 540 prefetches zero cache lines only if the offsets of the first and last pixels are equal (meaning the line size is one pixel).

[0102] After prefetch cycle 540 completes, the last pixel prefetch operation 550 prefetches the cache line containing the last pixel of the row. Since the cache line boundaries are unknown, the cache line containing the last pixel of the row may or may not have been prefetched in the last iteration of prefetch cycle 540. Therefore, the last pixel prefetch operation 550 may be unnecessary but can be ignored by memory hierarchy 270 because the cache line has already been prefetched. In these embodiments, performing an unnecessary and ignored prefetch operation can have a lower logical cost than avoiding it.

[0103] It should be understood that the scope of this disclosure may be used. Figure 5 Many variations of exemplary embodiments, regardless of whether computer code is used, any suitable programming language is used, and whether or not they are equivalent to Figure 5 .

[0104] In some embodiments, a distinction is made between two directions: one where no line in the corresponding prefetch schedule exceeds the cache line size by more than one addressable memory cell; and another where at least one line in the corresponding prefetch schedule exceeds the cache line size by more than one addressable memory cell. Typically, the latter directions are closer to horizontal (primarily in the x-coordinate direction), and are therefore referred to as near-horizontal directions, while the former, and usually more numerous, directions are referred to as non-horizontal directions. In these embodiments, the prefetch schedules for near-horizontal directions can use a different style than those for non-horizontal directions.

[0105] For non-horizontal directions, prefetch loop 540 will perform one iteration (or, in rare cases, zero iterations). Only two prefetch operations are needed, one for the first pixel of the row and one for the last pixel, to ensure that all cached rows containing a portion of the row are prefetched, while other cached rows are not prefetched, and potentially one unnecessary prefetch operation is performed and ignored. By differentiating between near-horizontal and non-horizontal implementations, non-horizontal rows can be prefetched without any potentially expensive loop control logic—simply prefetching unconditionally at the first and last pixel addresses. Options for prefetching rows in the near-horizontal direction will be discussed below. Figure 12 As explained in the description.

[0106] Figure 6 It shows the relationship with Figure 3and Figure 4 Same pixel position, but makes part of row 600 and row cycles 610, 620 and 630... The y = 3-line stage becomes explicit. The pixel positions of partial line 600 are indicated by the letters A, B, and C, while the pixel positions of the complete line loop are indicated by stage numbers. Following the conventions used in this paper, the pixels of a line are located at adjacent addresses, which reside in a much smaller number of cache lines. Each Since none of the 3-line stages are larger than a cache line, the slope 3 / 5 in this example is non-horizontal.

[0107] In some embodiments, the prefetching plan includes an initialization phase and a looping phase. The initialization phase prefetches a portion of the rows, followed by a looping phase prefetching zero or more rows, referred to as extra lead rows. Once the initialization phase is complete, a looping pixel processing plan is used in coordination with the looping phase of the prefetching plan to compute a one-dimensional signal, continuing from the row loops and row phases where the extra lead rows stopped. Thus, the prefetch distance responds to the number of extra lead rows, which in turn responds to the direction of the projection lines.

[0108] Figure 7 Displayed corresponding to Figure 6 An exemplary prefetch schedule with rows and prefetch distances responding to zero additional rows is used. Based on information describing the projection lines, a starting pixel position of 700 is determined, and the corresponding row address is obtained from that position. The row address is updated after each row is prefetched and is displayed for each row by a circle within the grid position.

[0109] The prefetch plan includes partial row information 750, which gives the number of partial rows (h- y = 3) and the offsets of the first and last pixels of each partial row relative to the updated row address. In the example, partial row A runs from offset 2 to offset 3, partial row B from 1 to 5, and partial row C from 0 to 6. After each partial row is prefetched, the row address is updated by the row spacing, for example, to grid position 705 for partial row B. After all partial rows are prefetched, the row address will be at grid position 710. Since there are zero extra prefetched rows in this example, the loop phase starts at the row address of grid position 710.

[0110] To coordinate pixel processing schedules and prefetch schedules to maintain an appropriate prefetch distance, an example pixel processing schedule with a period of 8 and a period of... The prefetch plan for y = 3 can be synchronized. For examples with very low logical overhead, the first loop of each module... y = 3 slices can be interleaved with 3 row stages, and then the remaining slices are executed separately. Figure 13The following description provides further details on how this coordination is achieved. It should be understood that any suitable scheme for interleaving or synchronizing pixel planning and prefetching loops can be used. The effective prefetch distance can vary slightly within the module loop and can be kept within a suitable range.

[0111] During the loop phase of the prefetch plan, row addresses can be updated after each row using a loop row update amount of 715, for example, updating the row address from grid position 710 to grid positions 720, 725, and 730 for use in phase 0 of the next loop. The loop row update amount 715 in this exemplary prefetch plan is the sum of the row spacing and the xRowUpdate amount given in prefetch plan parameter 780 (2 in this example). This allows the row address to be updated by 1 in the y-direction and xRowUpdate in the x-direction after each row (roughly following the direction of the projection lines), while keeping row update overhead low because the loop row update for each prefetch plan is constant. It's important to note that using xRowUpdate does not increase overhead because it is combined with the row spacing.

[0112] To maintain the cyclic nature of pixel processing and the prefetch schedule, it is desirable that the row address be at the same relative row position in each row cycle. To achieve this, xCycleAdjust 735 (its example value of -1 is given in prefetch schedule parameter 780) is added to the row address after each row cycle. This moves the row address from, for example, grid position 730 at the end of the first cycle to grid position 740 at the beginning of the second cycle, ensuring that the row address maintains the same relative row position in each row cycle. It can be seen that, to serve this function, xCycleAdjust = x - (row cycle) * xRowUpdate(1) In this example, xCycleAdjust = 5–3 * 2 = -1.

[0113] The overhead of using xCycleAdjust is very low because it is only used once per row loop. The grid position moved by xCycleAdjust 735 at the end of each loop is... Figure 7 In the examples of subsequent diagrams, diamonds are used to represent them.

[0114] for Figure 7The exemplary prefetch plan, with basic cyclic row information 760 providing the offsets of the first and last pixels of each cyclic row stage relative to the update row address, for the case of zero additional advance rows. Active cyclic row information 770 includes adjustable additional advance, as well as offsets of the first and last pixels of each cyclic row stage relative to the update row address, adjusted from basic cyclic row information 760 to reflect the adjustable additional advance, as further described below.

[0115] The additional lead can be adjusted individually for each prefetch plan to achieve the desired prefetch distance in response to the projection line direction. When using a prefetch plan, the adjusted offset in the active cycle line information 770 can be used to obtain the correct prefetch address.

[0116] It should be noted that in some embodiments, a non-zero xRowUpdate is used to approximately follow the projection line within each loop, keeping the first and last pixel offsets small, thereby allowing prefetch schedules to have a smaller memory footprint, which can be stored in table memory 280 (since single-byte offsets can be stored, for example). In other embodiments, xRowUpdate is not used (effectively always being 0), or other values ​​are used, and the offsets in the basic loop row information 760 and the active loop row information 770 are adjusted appropriately.

[0117] In the exemplary embodiments described herein, when the additional lead time is a row period When y is an integer multiple of y (e.g. Figure 7 As shown), the offset in the active cycle line information 770 is the same as that in the basic cycle line information 760. When the additional lead is not an integer multiple of the line period, some adjustments can be made to the offset in the active cycle line information, such as... Figure 8 As shown, it illustrates the corresponding Figure 6 An alternative exemplary prefetch plan for the row, with a prefetch distance of 2 additional advance rows in response.

[0118] use Figure 8 An alternative exemplary prefetching plan prefetches five initial rows 802 during the initialization phase, and prefetches the first row (loop 804), the second row (loop 806), and an appropriate number of subsequent rows (not shown) during the loop phase.

[0119] The initialization phase begins with... Figure 7 The exemplary prefetch plan prefetches portions of rows A, B, and C in the same manner, using portion row information 860, which is identical to portion row information 750. Row addresses begin at grid position 800 and, as indicated by circles within each grid position in the figure, advance by row spacing with each row. After the portion of the rows is prefetched, the row address is located at grid position 810.

[0120] The initialization phase continues to prefetch two additional lead lines, namely stages 0 and 1 within initial line 802, as shown in the figure. The number of additional lead lines is given in the active loop line information 880 and can be adjusted as needed. These additional lead lines utilize [the same methods as] [uncoordinated pixel processing operations]. Figure 7 The same process is performed for prefetching the same rows, with the row offset taken from the basic cycle row information 870. After each such prefetch, the row address is updated by the cycle row update amount 815, which, as previously described, can be the sum of the row spacing and the xRowUpdate amount given in the prefetch plan parameter 890. If additional rows cross the cycle boundary (from stage 2 to stage 0) (which does not occur in this example), the xCycleAdjust value in the prefetch plan parameter 890 is used as previously described. After the initialization phase, the row address is finally located at grid position 825, and the cycle phase begins.

[0121] The expected cycle phases proceed in the same manner (regardless of how the additional lead time is adjusted), i.e., with Figure 7 The method is the same. Rows are prefetched during the cycle phase and their addresses are updated to move from grid position 825 to grid positions 830, 835, and 840 at the end of the first cycle 804. As previously described, at the end of each cycle, the row address is adjusted by xCycleAdjust 845, the value of which is given in the prefetch plan parameter 890. The prefetch operation can be coordinated with the pixel processing operation as previously described.

[0122] In this example, the extra lead is not the row period. The cyclic phase begins with row phase 2, and each cycle of the cyclic phase prefetches row phases in the order 2, 0, and 1, instead of 0, 1, and 2. The row offsets in the basic cyclic row information 870 may be incorrect and can be adjusted to produce the offsets in the active cyclic row information 880. Offsets for row phases greater than or equal to the extra lead (row phase 2 in this example) do not need adjustment. For row phases less than the extra lead, the offsets in the basic cyclic row information 870 can be adjusted by adding xCycleAdjust to each offset to produce the offsets in the active cyclic row information 880. Figure 8 As shown, the adjusted offset correctly identifies the position of the first and last pixels of each row relative to the row address.

[0123] Figure 9 An exemplary pixel processing plan is shown, with the direction determined by the slope. y / The projection line 900, represented by x = 7 / 3, forms an angle of approximately 66.8° with the horizontal direction. The figure shows the first module loop 910, the second module loop 920, the third module loop 930, and the module height 940 (h = 7).

[0124] Figure 10 It shows that it can be used with Figure 9 The example non-horizontal prefetching scheme used for pixel processing plan coordination. Figure 10 This will help to further explain how to create and use prefetch plans, and is related to... Figure 7 and Figure 8 The comparison helps to clarify how the prefetching scheme responds to the direction of the projection line.

[0125] exist Figure 10 In the example, h- y = 0, therefore there are no partial rows, as shown in partial row information 1050. When xRowUpdate = 0, Equation 1 gives xCycleAdjust 1040, which is... x = 3, both of which are given in the prefetching plan parameter 1080.

[0126] Basic loop information 1060 is displayed There are 7 row stages (y = 7), and the offsets of the first and last pixels are shown in the figure. Since xRowUpdate is 0, these basic loop row offsets are relative to the row address that is updated only in the y-direction, as shown by the grid positions containing the circles. For example, offsets 2 and 5 for stage 5 are relative to grid position 1030.

[0127] Since there are no partial rows in this example, the prefetching plan's initialization phase only prefetches 5 initial rows 1000. The row address starts at grid position 1020 and is updated to grid position 1030 at the end of the initialization phase. The first and last pixel offsets of the initial row 1000 are taken from the basic cyclic row information 1060.

[0128] Figure 10 The exemplary prefetch plan prefetches the first row cycle 1005, the second row cycle 1010, and an appropriate number of subsequent row cycles (not shown). At the end of the first row cycle 1005, the row address will be at grid position 1035 and moved to grid position 1045 using row update amount 1040.

[0129] Stages 5 and 6 are greater than or equal to the extra lead, so their row offsets in the active cyclic row information 1070 are the same as in the basic cyclic row information 1060. Stages 0–4 are less than the extra lead, so their row offsets in the active cyclic row information 1070 are adjusted by adding xCycleAdjust to the corresponding offsets in the basic cyclic row information 1060. Looking at the positions of the rows in the first cycle 1005 and the second cycle 1010 relative to the circled grid positions, it can be seen that the adjusted row offsets correctly position the first and last pixels of each row.

[0130] Figure 11 An exemplary pixel processing plan is shown, with the direction determined by the slope. y / The projection line 1160, represented by x = 1 / 7, is at approximately 8.1° to the horizontal direction. It shows the first module loop 1100, the second module loop 1110, the third module loop 1120, the fourth module loop 1130, the fifth module loop 1140, and the module height 1150 (h = 5).

[0131] Figure 12 It shows that it can be used with Figure 11 An example near-horizontal prefetching scheme is used for pixel processing scheme coordination. Figure 12 This helps to further illustrate how to create and use prefetch plans, and the comparison with the previous figures helps to clarify how prefetch plans respond to the direction of the projection lines.

[0132] exist Figure 12 In the example, h- y = 4, therefore there are 4 partial rows 1200, as shown in partial row information 1250. xRowUpdate is 7, and Equation 1 gives xCycleAdjust as 0, both of which are given in the prefetch plan parameter 1280. Since xCycleAdjust is 0, it is not displayed on the grid.

[0133] During the cyclic phase of the prefetch plan, row update 1230 is performed using a cycle, for example, updating the row address from grid position 1225 to grid position 1235. Row Cycle y = 1, therefore the loop phase has only phase 0 rows. The first row of loop 1205 and the second row of loop 1210 are also shown, and subsequent rows of loops extend beyond the portion of the image represented by the illustrated grid.

[0134] exist Figure 12In the prefetching plan, assume an exemplary cache line size of 16. This is typically smaller than the value typical for modern devices and is chosen for simplicity. We can see that a portion of line D contains 22 pixels, and each complete cycle of line stage 0 contains 24 pixels, both larger than the assumed cache line size plus one, making this a near-horizontal orientation. An exemplary non-horizontal strategy (prefetching the addresses of the first and last pixels of each line) may not guarantee prefetching all cache lines containing a portion of that line.

[0135] In some embodiments, Figure 5 The exemplary row prefetching strategy is used for partial rows and circular rows. In some embodiments, a non-horizontal strategy is used for all rows, which could be... Figure 5 The trade-off between the cost of the loop control logic and the cost of potentially failing to prefetch one or more cache lines containing a portion of that line.

[0136] In some embodiments, different strategies can be used for partial rows and complete circular rows. For example, a non-horizontal strategy can be used for partial rows 1200, and a different strategy can be used for circular rows. Since partial rows are typically few and usually smaller than circular rows, the strategy used for them may not be very important. In some embodiments, a non-horizontal strategy is used for partial rows, while a different strategy is used for circular rows (e.g., Figure 5 (strategy).

[0137] The exemplary non-leveling strategy using only two prefetch operations can be generalized to other fixed-number operations. For example, three prefetch operations guarantee that for a line no larger than twice the cache line size plus one, all cache lines containing a portion of that line will be prefetched, and no other lines will be prefetched, even if some of these operations may be unnecessary and ignored. In some embodiments, this strategy can be more efficient than using... Figure 5 The loop control logic is more advantageous.

[0138] Figure 12 The exemplary prefetching plan employs this three-operation strategy for the loop row. Basic loop row information 1260 and active loop row information 1270 provide three pixel offsets. The first and third offsets shown correspond to the first and last pixels of the loop row. The second offset is equal to the first pixel offset plus the cache line size, or the last pixel offset, whichever is smaller.

[0139] For very long lines, this three-operation strategy may fail to retrieve some cached lines containing a portion of that line. However, in many implementations, near-horizontal directions are rare, and very long lines are even rarer.

[0140] Figure 13This is an exemplary flowchart illustrating the process of coordinating a pixel processing plan with a prefetch plan. Rectangular flowchart elements represent pixel operations, rounded rectangular flowchart elements represent prefetch operations, and other flowchart elements correspond to control flow. The diagram shows initialization phase 1300 and loop phase 1310, both as previously described.

[0141] The module loop 1320 iterates along the projection line for each module loop (i.e., position). For each module loop, the nested slice loop 1330 iterates for each slice in the current module loop. For each slice, the pixel processing step 1340 performs pixel operations for that slice according to the pixel processing plan. The pixel processing step 1340 may also compute a portion of the one-dimensional signal to be extracted. Further, for each slice, if the condition 1350 determines that the slice number is less than the row period, the prefetch step 1360 performs a prefetch operation to prefetch the next loop row.

[0142] Those skilled in the art can make various modifications to the flowchart to meet various purposes. If, for a given pixel processing and prefetching plan, the row cycle is greater than the module cycle, then the slice loop 1330 can be replaced by the row loop, and the condition judgment 1350 can be modified appropriately. Alternatively, the loop structure can be changed so that the relationship between the module and the row cycle becomes irrelevant.

[0143] To better control the projection line length, the flowchart can be modified in various obvious ways to handle parts of the last module. The prefetch operation can be completed before the pixel processing operation in slice loop 1330.

[0144] For simplicity, the flowchart does not show the various loop initialization and update steps (including row address initialization and update), which are described in detail above. Filling in these details can be done by those skilled in the art.

[0145] As referenced above Figure 7 , Figure 8 , Figure 10 and Figure 12 The method described above uses an additional advance value to provide a way to control the prefetch distance, which is controlled independently for each prefetch plan and in response to the direction of the projection line.

[0146] Various procedures can be used to select a suitable additional advance value. Ideally, the prefetch distance for each prefetch schedule should be large enough that pixels have time to be brought into the data cache (2^10), but this prefetch distance cannot be too large, otherwise these pixels may be evicted before use. Clearly, the prefetch distance does not need to be chosen precisely to be effective, but a reasonable value should be used.

[0147] exist Figure 14In an exemplary embodiment, the additional lead amount is selected through a parameter formula. Figure 14 Chart 1400 is shown, which plots an additional lead of 1420 as a function of the projection line angle 1410 (from 0 to 90 degrees) 1430. The formula for this example is... Additional lead time = round[V sin(θ)] (2) Where θ is the projection line angle 1410, and V is a parameter, in this example V = 12.

[0148] Equation 2 is based on the observation that projection directions closer to the vertical tend to acquire new pixels at a higher rate than those closer to the horizontal, because more vertical directions tend to have fewer prefetch rows and more prefetch rows per module cycle. These new pixels will be needed by the pixel processing schedule more quickly, and therefore the additional lead should be greater.

[0149] The V parameter of Equation 2 may be responsive to details of the processor 200, the memory hierarchy 270, the projection line direction region, or any suitable feature of any particular embodiment.

[0150] In another exemplary embodiment, for each projection line direction, the pixel processing plan and prefetch plan are analyzed in detail to determine an additional advance value that can achieve a given effective prefetch distance, which is measured by computational steps (e.g., pixel processing and prefetch operations) rather than row stages directly controlled by the additional advance value. The values ​​thus obtained can be stored in table memory 280 along with the prefetch plan.

[0151] In yet another alternative exemplary embodiment, the additional advance value for each projection line direction can be determined by measuring the execution time on a given system. Using this embodiment, various additional advance values ​​can be tried to select a value with a short execution time, which can be combined with other desired characteristics.

[0152] The various methods or processes outlined in this article can be coded as software that can be executed on one or more processors employing any of a variety of operating systems or platforms. Furthermore, such software can be written using any of a variety of suitable programming languages ​​and / or programming or scripting tools, and can also be compiled into executable machine language code or intermediate code that executes on a virtual machine or suitable framework.

[0153] In this regard, various inventive concepts can be embodied in at least one non-transitory computer-readable storage medium (e.g., a computer memory, one or more floppy disks, optical disks, magnetic tapes, flash memory, field-programmable gate arrays, or circuit configurations in other semiconductor devices), which is encoded with one or more programs that, when executed on one or more computers or other processors, implement various embodiments of the present disclosure. The non-transitory computer-readable medium or media may be portable, such that programs stored thereon or programs can be loaded onto any computer resource to implement the various aspects of the present disclosure discussed above.

[0154] The terms "program," "software," and / or "application" are used herein in a general sense to refer to any type of computer code or set of computer-executable instructions that can be used to program a computer or other processor to implement the various aspects of the embodiments discussed above. Furthermore, it should be understood that, according to one aspect, one or more computer programs that perform the methods of this disclosure when executed do not need to reside on a single computer or processor, but can be distributed in a modular manner across different computers or processors to implement the various aspects of this disclosure.

[0155] Computer-executable instructions can take many forms, such as program modules that are executed by one or more computers or other devices. Typically, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. The functionality of program modules can usually be combined or distributed as needed in various embodiments.

[0156] Similarly, data structures can be stored in any suitable form on a non-transitory computer-readable storage medium. A data structure may have fields that are related to each other by their respective positions within the data structure. Such relationships can also be achieved by assigning storage locations in a non-transitory computer-readable medium to each field, where these locations represent the relationships between the fields. However, any suitable mechanism can be used to establish relationships between information fields within a data structure, including the use of pointers, labels, or other mechanisms for establishing relationships between data elements.

[0157] Various inventive concepts can be embodied in one or more methods, examples of which have been provided above. Actions performed as part of a method can be ordered in any suitable manner. Accordingly, embodiments can be constructed in which the order of action execution differs from that illustrated; such embodiments may include the simultaneous execution of certain actions, even if these actions are shown to be performed sequentially in the exemplary embodiments.

[0158] Unless explicitly stated otherwise, the indefinite articles "a" and "an" used in this specification and claims shall be understood to mean "at least one". As used in this specification and claims, the phrase "at least one" when referring to a list of one or more elements shall be understood to mean at least one element selected from any one or more elements in that list, but does not necessarily include every single element specifically listed in the list and at least one of each element, and does not exclude any combination of elements in the list. This allows for the optional presence of elements other than those specifically identified in the list of elements referred to by the phrase "at least one", whether or not such other elements are related to those specifically identified.

[0159] As used in this specification and claims, the phrase "and / or" should be understood to mean "any one or both" of the elements so connected, that is, elements that are parallel in some cases and disjunctive in others. Multiple elements listed with "and / or" should be interpreted in the same way, that is, "one or more" of the elements so connected. Other elements besides those specifically identified by the "and / or" clause may optionally be present, whether related to or unrelated to those specifically identified. Thus, as a non-limiting example, when referring to "A and / or B" in conjunction with open-ended language (e.g., "comprising"), in one embodiment it may refer to only A (optionally including elements other than B); in another embodiment it may refer to only B (optionally including elements other than A); in yet another embodiment it may refer to both A and B (optionally including other elements); and so on.

[0160] The word “or” as used in this specification and claims shall be understood to have the same meaning as “and / or” as defined above. For example, when separating items in a list, “or” or “and / or” shall be interpreted as inclusive, meaning that it includes multiple elements or at least one of the items in the list, but also includes more than one, and may optionally include additional items not listed. Only terms that explicitly indicate the opposite (e.g., “only one” or “exactly one”, or “consisting of” as used in the claims) refer to multiple elements or exactly one element in the list. Generally, the term “or” as used herein shall only be interpreted as indicating an exclusive alternative (i.e., “one or the other, but not both”) when modified by exclusive terms (e.g., “any,” “one of,” “only one,” or “exactly one”). The term “consisting substantially of” as used in the claims shall have its ordinary meaning as used in the field of patent law.

[0161] The use of ordinal numbers such as "first," "second," and "third" to modify claim elements does not imply any priority, order, or precedence of any claim element relative to another element, nor does it imply the temporal order of the actions performed in the method. Such terms are used merely as labels to distinguish a claim element with a certain name from another element with the same name (but without the ordinal number).

[0162] The wording and terminology used herein are intended to be descriptive and should not be considered limiting. The use of "including," "containing," "having," "containing," "involving," and variations thereof is intended to cover the items listed thereafter and any additional items thereto.

[0163] After describing several embodiments in detail, various modifications and improvements will readily occur to those skilled in the art. Such modifications and improvements are intended to fall within the scope and spirit of this disclosure. Therefore, the foregoing description is merely illustrative and is not intended to be limiting.

[0164] This disclosure describes various aspects, including but not limited to the following: 1. A method for extracting a one-dimensional signal from a two-dimensional image along a projection line, comprising: accessing the two-dimensional image, the two-dimensional image including pixels arranged on a pixel grid; storing the two-dimensional image using a memory hierarchy including main memory and a data cache, wherein a non-blocking prefetch operation is configured to fetch pixels from the main memory to the data cache; receiving information describing the projection line, the information including the direction of the projection line, wherein the direction is one of a set of allowed directions, the set of allowed directions including directions that are not parallel to the pixel grid and not diagonally opposite to the pixel grid; and responding to the... The method includes: selecting a direction; selecting a prefetch plan that specifies a prefetch operation sequence in a first address order, the prefetch operation sequence including a row sequence; selecting a pixel processing plan that specifies a pixel operation sequence in a second address order different from the first address order in response to the direction; selecting a prefetch distance in response to the direction; and coordinating the pixel processing plan with the prefetch plan to compute the one-dimensional signal, including: executing the prefetch operation sequence to fetch pixels from the main memory into the data cache an amount of time corresponding to the prefetch distance before the pixel operation sequence uses the pixels.

[0165] 2. The method according to aspect 1 or any other aspect, wherein at least one of the prefetch plan, the pixel processing plan, or the prefetch distance is pre-calculated and stored in a table memory.

[0166] 3. The method according to aspect 1 or aspect 2 or any other aspect, wherein the pixel processing plan specifies a repeating sequence of pixel weight templates.

[0167] 4. The method described in any one of aspects 1-3 or any other aspect, wherein the prefetching plan includes a first phase for initialization and a second phase for cyclical execution.

[0168] 5. The method described in aspect 4 or any other aspect, wherein the first phase specifies a prefetch portion of the rows.

[0169] 6. The method described in aspect 4 or any other aspect, wherein the second phase specifies prefetching complete rows.

[0170] 7. The method described in any one of aspects 1-6 or any other aspect, wherein the prefetch distance is a parametric function of the direction.

[0171] 8. The method described in any one of aspects 1-7 or any other aspect, wherein the prefetch distance is determined by measuring the execution time of the pixel processing schedule.

[0172] 9. The method described in any one of aspects 1-8 or any other aspect, wherein for a prefetching plan with multiple permissible directions, it is specified that only the first and last pixels of each row are prefetched.

[0173] 10. The method according to aspect 9 or any other aspect, wherein the prefetching plan in the near-horizontal direction specifies prefetching three pixels per row.

[0174] 11. The method described in aspect 9 or any other aspect, wherein the prefetching schedule in the near-horizontal direction specifies that exactly three pixels of each row are prefetched.

[0175] 12. An electronic device for extracting a one-dimensional signal from a two-dimensional image along a projection line, the device comprising: a memory hierarchy including a main memory and a data cache, wherein the two-dimensional image includes pixels arranged on a pixel grid and stored in the memory hierarchy, and pixels of the two-dimensional image are fetched from the main memory to the data cache via a non-blocking prefetch operation; and at least one processor configured to execute computer-executable instructions, wherein the computer-executable instructions include instructions for: receiving information describing the projection line, the information including a direction of the projection line, wherein the direction is one of a set of allowed directions, the allowed... The set of possible directions includes directions that are not parallel to the pixel grid and are not diagonally opposite to the pixel grid; in response to the direction, a prefetch plan is selected, the prefetch plan specifying a prefetch operation sequence in a first address order, the prefetch operation sequence including a row sequence; in response to the direction, a pixel processing plan is selected, the pixel processing plan specifying a pixel operation sequence in a second address order different from the first address order; in response to the direction, a prefetch distance is selected; and the pixel processing plan is coordinated with the prefetch plan to compute the one-dimensional signal, including: executing the prefetch operation sequence to fetch the pixel from the main memory into the data cache an amount of time corresponding to the prefetch distance before the pixel operation sequence uses the pixel.

[0176] 13. The method according to aspect 12 or any other aspect, wherein at least one of the prefetch schedule, the pixel processing schedule or the prefetch distance is pre-calculated and stored in a table memory.

[0177] 14. The apparatus described in aspect 12 or aspect 13 or any other aspect, wherein the pixel processing scheme specifies a repeating sequence of pixel weight templates.

[0178] 15. The apparatus described in any one of aspects 12-14 or any other aspect, wherein the prefetching plan includes a first phase for initialization and a second phase for cyclic execution.

[0179] 16. The apparatus described in aspect 15 or any other aspect, wherein the first stage designates a prefetch portion of the row.

[0180] 17. The apparatus described in aspect 15 or any other aspect, wherein the second phase specifies the prefetching of complete rows.

[0181] 18. The apparatus described in any one of aspects 12-17 or any other aspect, wherein the prefetch distance is a parametric function of the direction.

[0182] 19. The apparatus described in any one of aspects 12-18 or any other aspect, wherein the prefetch distance is determined by measuring the execution time of the pixel processing schedule.

[0183] 20. The apparatus described in any one of aspects 12-19 or any other aspect, wherein, for a prefetching scheme of multiple permissible directions, only the first and last pixels of each row are prefetched.

[0184] 21. The method as described in aspect 20 or any other aspect, wherein the prefetching plan in the near-horizontal direction specifies prefetching three pixels per row.

[0185] 22. The method described in aspect 20 or any other aspect, wherein the prefetching schedule in the near-horizontal direction specifies that exactly three pixels of each row are prefetched.

[0186] 23. A non-transitory computer-readable medium storing computer-executable instructions configured to, when executed by at least one processor, perform a method for extracting a one-dimensional signal from a two-dimensional image along a projection line, the method comprising: accessing the two-dimensional image, the two-dimensional image comprising pixels arranged on a pixel grid; storing the two-dimensional image using a memory hierarchy including main memory and a data cache, wherein a non-blocking prefetch operation is configured to fetch pixels from the main memory to the data cache; receiving information describing the projection line, the information including the direction of the projection line, wherein the direction is one of a set of allowed directions, the allowed directions The process includes: selecting a prefetch plan in response to the direction of the set, the prefetch plan specifying a prefetch operation sequence in a first address order, the prefetch operation sequence including a row sequence; selecting a pixel processing plan in response to the direction, the pixel processing plan specifying a pixel operation sequence in a second address order different from the first address order; selecting a prefetch distance in response to the direction; and coordinating the pixel processing plan with the prefetch plan to compute the one-dimensional signal, including: executing the prefetch operation sequence to fetch pixels from the main memory into the data cache an amount of time prior to the use of pixels in the pixel operation sequence, in response to the prefetch distance.

Claims

1. A method for extracting a one-dimensional signal from a two-dimensional image along a projection line, comprising: Access the two-dimensional image, which includes pixels arranged on a pixel grid; The two-dimensional image is stored using a memory hierarchy that includes main memory and a data cache, wherein a non-blocking prefetch operation is configured to fetch pixels from the main memory to the data cache. Receive information describing the projection line, the information including the direction of the projection line, wherein the direction is one of a set of allowed directions, the set of allowed directions including directions that are not parallel to the pixel grid and are not diagonal to the pixel grid; In response to the direction, a prefetch plan is selected, the prefetch plan specifying a prefetch operation sequence in a first address order, the prefetch operation sequence including a row sequence; In response to the direction, a pixel processing plan is selected, the pixel processing plan specifying a sequence of pixel operations in a second address order different from the first address order; In response to the stated direction, a prefetched distance is selected; and Using the pixel processing plan in conjunction with the prefetch plan to compute the one-dimensional signal includes: executing the prefetch operation sequence to fetch the pixel from the main memory into the data cache in advance of the amount of time corresponding to the prefetch distance before the pixel operation sequence uses the pixel.

2. The method according to claim 1, wherein, At least one of the prefetch plan, the pixel processing plan, or the prefetch distance is pre-calculated and stored in the table memory.

3. The method according to claim 1 or 2, wherein, The pixel processing plan specifies a repeating sequence of pixel weight templates.

4. The method according to any one of claims 1-3, wherein, The prefetch plan includes a first phase for initialization and a second phase for cyclical execution.

5. The method according to claim 4, wherein, The first phase specifies the prefetching of a subset of rows.

6. The method according to claim 4, wherein, The second stage specifies the prefetching of complete rows.

7. The method according to any one of claims 1-6, wherein, The pre-fetched distance is a parametric function of the direction.

8. The method according to any one of claims 1-7, wherein, The prefetch distance is determined by measuring the execution time of the pixel processing plan.

9. The method according to any one of claims 1-8, wherein, Multiple allowed orientation prefetching plans specify that only the first and last pixels of each row are prefetched.

10. The method according to claim 9, wherein, The near-horizontal prefetching plan specifies that three pixels per row should be prefetched.

11. The method according to claim 9, wherein, The near-horizontal prefetching plan specifies that exactly three pixels of each row should be prefetched.

12. An electronic device for extracting a one-dimensional signal from a two-dimensional image along a projection line, the device comprising: A memory hierarchy including main memory and a data cache, wherein the two-dimensional image comprises pixels arranged on a pixel grid and stored in the memory hierarchy, and the pixels of the two-dimensional image are fetched from the main memory to the data cache via a non-blocking prefetch operation; and At least one processor, the at least one processor being configured to execute computer-executable instructions, wherein the computer-executable instructions include instructions for performing the following operations: Receive information describing the projection line, the information including the direction of the projection line, wherein the direction is one of a set of allowed directions, the set of allowed directions including directions that are not parallel to the pixel grid and are not diagonal to the pixel grid; In response to the direction, a prefetch plan is selected, the prefetch plan specifying a prefetch operation sequence in a first address order, the prefetch operation sequence including a row sequence; In response to the direction, a pixel processing plan is selected, the pixel processing plan specifying a sequence of pixel operations in a second address order different from the first address order; In response to the stated direction, a prefetched distance is selected; and Using the pixel processing plan in conjunction with the prefetch plan to compute the one-dimensional signal includes: executing the prefetch operation sequence to fetch the pixel from the main memory into the data cache in advance of the amount of time corresponding to the prefetch distance before the pixel operation sequence uses the pixel.

13. The method according to claim 12, wherein, At least one of the prefetch plan, the pixel processing plan, or the prefetch distance is pre-calculated and stored in the table memory.

14. The apparatus according to claim 12 or claim 13, wherein, The pixel processing plan specifies a repeating sequence of pixel weight templates.

15. The apparatus according to any one of claims 12-14, wherein, The prefetch plan includes a first phase for initialization and a second phase for cyclical execution.

16. The apparatus according to claim 15, wherein, The first phase specifies the prefetching of a subset of rows.

17. The apparatus according to claim 15, wherein, The second stage specifies the prefetching of complete rows.

18. The apparatus according to any one of claims 12-17, wherein, The pre-fetched distance is a parametric function of the direction.

19. The apparatus according to any one of claims 12-18, wherein, The prefetch distance is determined by measuring the execution time of the pixel processing plan.

20. The apparatus according to any one of claims 12-19, wherein, Multiple allowed orientation prefetching plans specify that only the first and last pixels of each row are prefetched.

21. The method according to claim 20, wherein, The near-horizontal prefetching plan specifies that three pixels per row should be prefetched.

22. The method according to claim 20, wherein, The near-horizontal prefetching plan specifies that exactly three pixels of each row should be prefetched.

23. A non-transitory computer-readable medium storing computer-executable instructions configured to, when executed by at least one processor, perform a method for extracting a one-dimensional signal from a two-dimensional image along projection lines, the method comprising: Access the two-dimensional image, which includes pixels arranged on a pixel grid; The two-dimensional image is stored using a memory hierarchy that includes main memory and a data cache, wherein a non-blocking prefetch operation is configured to fetch pixels from the main memory to the data cache. Receive information describing the projection line, the information including the direction of the projection line, wherein the direction is one of a set of allowed directions, the set of allowed directions including directions that are not parallel to the pixel grid and are not aligned with the pixel grid; In response to the direction, a prefetch plan is selected, the prefetch plan specifying a prefetch operation sequence in a first address order, the prefetch operation sequence including a row sequence; In response to the direction, a pixel processing plan is selected, the pixel processing plan specifying a sequence of pixel operations in a second address order different from the first address order; In response to the stated direction, a prefetched distance is selected; and Using the pixel processing plan in conjunction with the prefetch plan to compute the one-dimensional signal includes: executing the prefetch operation sequence to fetch the pixel from the main memory into the data cache in advance of the amount of time corresponding to the prefetch distance before the pixel operation sequence uses the pixel.

Images