A warp method, device, sending card and display control system based on FPGA
By processing video images in blocks on an FPGA, and acquiring and calculating the pixel data of the sub-images, the problem of low efficiency in graphics card processing is solved, and a high-efficiency real-time video frame rate improvement is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA FILM EQUIP
- Filing Date
- 2023-03-31
- Publication Date
- 2026-06-02
Smart Images
Figure CN116389770B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of image processing technology, and more particularly to an FPGA-based warp method, apparatus, sending card, and display control system. Background Technology
[0002] The higher the frame rate of a video, the smoother the visual experience for the human eye. There are generally two ways to increase the frame rate: one is by copying the previous frame to create an interpolated frame. While this method can improve the visual experience to some extent, the improvement is limited. The other method involves first determining the motion information of objects within the video and then predicting the content of the interpolated frame based on that motion information. Since developing the code directly is extremely complex, this method is usually performed on the graphics card. However, graphics card processing can only perform offline transcoding and cannot handle real-time frame interpolation.
[0003] Therefore, it is necessary to improve one or more of the problems existing in the above-mentioned related technical solutions.
[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention
[0005] The purpose of this disclosure is to provide an FPGA-based warp method, apparatus, sending card, and display control system, thereby overcoming at least to some extent one or more problems caused by limitations and defects in related technologies.
[0006] In a first aspect, this disclosure provides an FPGA-based warp method, which includes:
[0007] The target image to be inserted is divided into multiple blocks of equal size, each block having a preset number of pixels;
[0008] Acquire and store the position information of all pixels in the block on the original image;
[0009] Determine the boundaries of the location information;
[0010] Acquire and store pixel data of the sub-image that matches the boundary of the position information on the original image;
[0011] Based on the location information and the pixel data of the sub-image, the pixel values of each target interpolation point in the block are calculated and stored sequentially.
[0012] Optionally, the step of determining the boundary of the location information includes:
[0013] Calculate the maximum and minimum values in the horizontal and vertical directions of the location information respectively;
[0014] The boundaries of the location information are determined based on the maximum and minimum values in the horizontal and vertical directions.
[0015] Optionally, the step of acquiring and storing pixel data of a sub-image that matches the boundary of the position information on the original image includes:
[0016] The sub-image is obtained by extending the boundary of the position information on the original image outward by one pixel in each of the four directions: up, down, left, and right.
[0017] Acquire and store the pixel data in the sub-image.
[0018] Optionally, the step of sequentially calculating and storing the pixel values of each target interpolation point in the block based on the location information and the pixel data of the sub-image includes:
[0019] Based on the location information, obtain the position of the target interpolation point in the sub-image, which is 4 pixels around its original position.
[0020] Based on the pixel data of the sub-image, the pixel values of the four pixels are obtained, and bilinear interpolation is performed to obtain the pixel value of the target interpolation point.
[0021] Store the pixel values of each target interpolation point in the block.
[0022] Secondly, this disclosure provides an FPGA-based warp device, which includes: a partitioning module, a position information acquisition module, a boundary determination module, a sub-image data acquisition module, and a calculation module mounted on the FPGA, wherein...
[0023] The segmentation module is used to divide the target image to be inserted into multiple blocks of equal size, each block having a preset number of pixels;
[0024] The location information acquisition module is used to acquire and store the location information of all pixels in the block on the original image;
[0025] The boundary determination module is used to determine the boundary of the location information;
[0026] The sub-image data acquisition module is used to acquire and store pixel data of sub-images that match the boundaries of the position information on the original image;
[0027] The calculation module is used to calculate and store the pixel values of each target interpolation point in the block sequentially based on the location information and the pixel data of the sub-image.
[0028] Optionally, the boundary determination module includes:
[0029] The statistical unit is used to calculate the maximum and minimum values in the horizontal and vertical directions of the location information, respectively.
[0030] The determining unit is used to determine the boundary of the location information based on the maximum and minimum values of the horizontal and vertical directions in the location information.
[0031] Optionally, the sub-image data acquisition module includes:
[0032] An extension unit is used to extend the boundary of the position information on the original image outward by one pixel in each of the four directions (up, down, left, and right) to obtain the sub-image;
[0033] The first storage unit is used to acquire and store pixel data in the sub-image.
[0034] Optionally, the computing module includes:
[0035] The original position acquisition unit is used to acquire the position of 4 pixels around the original position of the target interpolation point on the sub-image based on the position information;
[0036] An interpolation calculation unit is used to obtain the pixel values of the four pixels based on the pixel data of the sub-image, and perform bilinear interpolation calculation to obtain the pixel value of the target interpolation point;
[0037] The second storage unit is used to store the pixel values of each target interpolation point in the block.
[0038] Thirdly, this disclosure provides a sending card, including:
[0039] A processor that generates interpolated video image information according to the FPGA-based warp method described in any of the preceding claims.
[0040] Fourthly, this disclosure provides a display control system, including:
[0041] The host computer is used to send video image data;
[0042] A transmitting card is used to receive the video image data, perform data conversion, and transmit the video image information;
[0043] A receiving card, used to receive the video image information and drive the display device to display it;
[0044] The sending card mentioned above is the same as the sending card described earlier.
[0045] The technical solutions provided in the embodiments of this disclosure may include the following beneficial effects:
[0046] In the embodiments of this disclosure, on an FPGA processing platform, the target image to be inserted is first divided into multiple blocks. Then, on a block-by-block basis, the positions of pixels in that block corresponding to the positions in the original image are obtained. Pixel data of the corresponding sub-images are then read from the original image based on these positions to obtain the pixel values of each target interpolation point in each block. Finally, the pixel values of the entire target image are obtained, thereby generating the estimated video frame. Compared to reading all pixel data from the original image, reading only the pixel data of the sub-images significantly improves data reading efficiency. Furthermore, this method of fusing sub-images from the original image on a block-by-block basis on the FPGA compensates for the burstiness of memory read / write operations with the parallelism of the FPGA, achieving real-time video processing.
[0047] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.
[0049] Figure 1 This diagram illustrates the steps of an FPGA-based warp method according to an exemplary embodiment of this disclosure.
[0050] Figure 2 A schematic diagram of the FPGA processing architecture in an exemplary embodiment of this disclosure is shown;
[0051] Figure 3 This diagram illustrates the principle of image motion in an exemplary embodiment of the present disclosure.
[0052] Figure 4 This illustration shows a block diagram of a target image in an exemplary embodiment of the present disclosure;
[0053] Figure 5 This diagram illustrates the position comparison between the target image and the original image in an exemplary embodiment of this disclosure.
[0054] Figure 6 This diagram illustrates the position of an interpolation point in the original image according to an exemplary embodiment of the present disclosure.
[0055] Figure 7This illustration shows a schematic diagram of the module structure of an FPGA-based warp device according to an exemplary embodiment of the present disclosure;
[0056] Figure 8 This diagram illustrates the structure of a sending card according to an exemplary embodiment of the present disclosure;
[0057] Figure 9 This diagram illustrates the structure of a display control system according to an exemplary embodiment of the present disclosure. Detailed Implementation
[0058] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0059] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.
[0060] Warp is a key technique in Motion Estimation and Motion Compensation (MEMC) used to fuse motion information with the original image to obtain the predicted video frame. During the fusion process, it's necessary to calculate the position of each pixel in the target image relative to the original image. Before this calculation, it's crucial to locate the position of each pixel in the target image corresponding to its position in the original image. However, due to the burst nature of memory and the randomness of the pixel's position relative to the original image, reading data pixel-by-pixel from the original image to generate a new frame is extremely inefficient, and the memory bandwidth cannot support real-time processing of high-resolution images.
[0061] Therefore, this example implementation first provides an FPGA-based warp method to achieve real-time video processing. (Reference) Figure 1 As shown, the method may include the following steps S101-S105:
[0062] Step S101: Divide the target image to be inserted into multiple blocks of equal size, each block having a preset number of pixels.
[0063] Step S102: Obtain and store the position information of all pixels in the block on the original image.
[0064] Step S103: Determine the boundaries of the location information.
[0065] Step S104: Obtain and store the pixel data of the sub-image that matches the boundary of the position information on the original image.
[0066] Step S105: Read the pixel data of the sub-image according to the location information, and calculate and store the pixel values of each target interpolation point in the block in sequence.
[0067] In the above method, on the FPGA processing platform, the target image to be inserted is first divided into multiple blocks. Then, on a block-by-block basis, the positions of pixels in that block corresponding to the positions in the original image are obtained. Pixel data of the corresponding sub-images are then read from the original image based on these positions to obtain the pixel values of each target interpolation point in each block. Finally, the pixel values of the entire target image are obtained, thereby generating the estimated video frame. Compared to reading all pixel data from the original image, reading only the pixel data of the sub-images significantly improves data reading efficiency. Furthermore, this method of fusing sub-images from the original image on a block-by-block basis on the FPGA leverages the parallelism of the FPGA to compensate for the burstiness of memory read / write operations, achieving real-time video processing.
[0068] Building such on FPGA Figure 2 The processing architecture shown executes steps S101-S105 as described above. The entire processing architecture can consist of a control module, a core processing module, a DDR (DDR memory), and a DDR arbitration module. The control module is used for system scheduling, the core processing module initiates DDR read requests and stores the settlement results in the DDR, and the DDR arbitration module arbitrates DDR read / write requests from multiple processing modules.
[0069] Below, we will combine Figures 1 to 6 The steps of the method described above in this example embodiment will be explained in more detail.
[0070] In step S101, the target image to be inserted is the motion image to be inserted from two adjacent original frames.
[0071] refer to Figure 3To further explain the principle of image motion, the object in the image is at position A in frame 1 of the original video and at position B in frame 2. Simple frame interpolation would duplicate each frame, for example, duplicating frame 1 to create frame 1.5, and inserting it between frame 1 and frame 2, but this has limited improvement. Motion estimation, on the other hand, inserts a frame of moving image between frame 1 and frame 2, where the image is at position A and B. Figure 2 The motion image at position AB is the target image to be inserted.
[0072] In this disclosure, when dividing the target image to be inserted into blocks, there is no limitation on the size or number of blocks. In some embodiments, the size of each block can be 64×64 pixels, 32×32 pixels, or 16×16 pixels. The total number of blocks is related to the overall resolution of the target image to be inserted and the size of each block. For example, as... Figure 4 In a target image with a resolution of 4096×2160, if each block is 64×64 pixels, then the entire target image has 64 horizontal blocks and 34 vertical blocks. During subsequent image processing, the target pixels of each block can be calculated sequentially from top to bottom and from left to right.
[0073] In step S102, the control module sends the initial address of the block to be processed to the corresponding processing core module. Of course, there can be multiple processing core modules, meaning multiple processing core modules can process multiple blocks simultaneously, thus further improving the efficiency of image data processing.
[0074] Specifically, the core processing module, based on the initial address of the corresponding block, sends a request to the DDR to read the position information of all pixels in that block on the original image. The DDR arbitration module responds accordingly. When multiple core processing modules process multiple blocks simultaneously and send multiple requests to the DDR, the DDR arbitration module responds sequentially according to the chronological order of the requests. After reading the position information of all pixels in the block on the original image from the DDR, this position information is fed back to the core processing module. Finally, the core processing module stores this position information in a first register, which in this specific example can be RAM_Pos.
[0075] In step S103, the location information stored in RAM_Pos is traversed to determine the boundaries of the location information.
[0076] Specifically, the process of determining the boundary includes: calculating the maximum and minimum values in the horizontal and vertical directions of the location information respectively; and determining the boundary of the location information based on the maximum and minimum values in the horizontal and vertical directions of the location information.
[0077] It is understandable that the positions of some pixels in the block may not be integers but decimals in the original image. Therefore, by statistically analyzing the maximum and minimum values in the horizontal and vertical directions of the position information and determining the boundaries based on these values, we can ensure that all pixels are located within the final determined boundaries and avoid missing the position information of some pixels that are not integers.
[0078] In step S104, after the boundary is determined, the core processing module sends a request to the DDR to read the original image data. Similarly, when multiple core processing modules send multiple requests, the DDR arbitration module responds in the order of the requests.
[0079] During the process of reading the original image based on the boundary value, the location of the boundary in the original image is first expanded outward by one pixel in each of the four directions (up, down, left, and right), resulting in the sub-image contained within the expanded region. This expansion process is also to further ensure that the positions of all pixels contained within the block in the original image are not missed.
[0080] After reading the pixel data of the sub-image from DDR, this data is fed back to the core processing module. Finally, the core processing module stores the pixel data of the sub-image in the second register. In a specific example, the second register can be RAM_Src.
[0081] In step S105, the core processing module reads the position information in the first register according to the order of the interpolation points, then reads the corresponding pixel data in the second register according to the position information, and finally calculates the pixel value of each target interpolation point.
[0082] It is understandable that the position of the target interpolation point in the original image is not necessarily an integer and may also contain decimals. For example... Figure 5 As shown, the hollow circle on the left represents the original image, and the solid circle on the right represents the target interpolated image. Each target pixel is calculated from the original pixel. The optical flow estimation algorithm calculates the original position of each target point. This position may contain decimals, meaning it may not coincide with the original image.
[0083] Therefore, the original position of the target point consists of two parts: the horizontal position PosH and the vertical position PosV. For example, in Figure 6In the diagram illustrating the positions of the solid circles, PosH contains a 12-bit integer part HPosInt and a 4-bit fractional part HPosFrac; HPosInt represents the position of the pixel to the left of the interpolation point, and HPosFrac represents the distance of the interpolation point from the original pixel to the left. PosV contains a 12-bit integer part VPosInt and a 4-bit fractional part VPosFrac; VPosInt represents the position of the pixel above the interpolation point, and VPosFrac represents the distance of the interpolation point from the original pixel above it.
[0084] Therefore, the pixel value calculation method for the target interpolation point includes:
[0085] Based on the location information, the positions of the target interpolation point around its original position on the sub-image are obtained. For example, if the position is represented by (PosH, PosV), then the four positions (HPosInt, VPosInt), (HPosInt+1, VPosInt), (HPosInt, VPosInt+1), and (HPosInt+1, VPosInt+1) are read from the first register.
[0086] Then, the pixel values at the four positions mentioned above are read from the second register, and bilinear interpolation is performed to obtain the pixel value of the target interpolation point. Finally, the calculated pixel values of each target interpolation point are stored in the third register, which in this specific example can be RAM_Dst.
[0087] Finally, the processing control module sends a write request to the DDR, writing the data in the third register into the DDR, thus completing the processing of one data block. The processing of other blocks is the same as above, and will not be described in detail here.
[0088] It should be noted that although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order. Of course, in addition to the possibility of changing the order of the steps, multiple steps can be combined into one step, and / or one step can be broken down into multiple steps.
[0089] refer to Figure 7As shown, this disclosure also provides an FPGA-based warp device, which includes a partitioning module, a position information acquisition module, a boundary determination module, a sub-image data acquisition module, and a calculation module mounted on the FPGA. The partitioning module is used to divide the target image to be inserted into multiple blocks of equal size, each block having a preset number of pixels. The position information acquisition module is used to acquire and store the position information of all pixels in the block on the original image. The boundary determination module is used to determine the boundary of the position information. The sub-image data acquisition module is used to acquire and store the pixel data of the sub-images on the original image that match the boundary of the position information. The calculation module is used to sequentially calculate and store the pixel values of each target interpolation point in the block based on the position information and the pixel data of the sub-images.
[0090] In one embodiment, the boundary determination module includes:
[0091] The statistical unit is used to calculate the maximum and minimum values in the horizontal and vertical directions of the location information, respectively.
[0092] The determining unit is used to determine the boundary of the location information based on the maximum and minimum values of the horizontal and vertical directions in the location information.
[0093] In one embodiment, the sub-image data acquisition module includes:
[0094] An extension unit is used to extend the boundary of the position information on the original image outward by one pixel in each of the four directions (up, down, left, and right) to obtain the sub-image;
[0095] The first storage unit is used to acquire and store pixel data in the sub-image.
[0096] In one embodiment, the computing module includes:
[0097] The original position acquisition unit is used to acquire the position of 4 pixels around the original position of the target interpolation point on the sub-image based on the position information;
[0098] An interpolation calculation unit is used to obtain the pixel values of the four pixels based on the pixel data of the sub-image, and perform bilinear interpolation calculation to obtain the pixel value of the target interpolation point;
[0099] The second storage unit is used to store the pixel values of each target interpolation point in the block.
[0100] The FPGA-based warp device disclosed herein divides the target image to be inserted into multiple blocks through various modules. Then, on a block-by-block basis, it obtains the position of pixels in the original image corresponding to those pixels in the block. Based on these positions, it reads the pixel data of the corresponding sub-image from the original image to obtain the pixel values of each target interpolation point in each block. Finally, it obtains the pixel values of the entire target image, thereby obtaining the estimated video frame. Compared to reading all pixel data from the original image, reading only the pixel data of the sub-image significantly improves data reading efficiency. Furthermore, this method of fusing sub-images from the original image on a block-by-block basis on the FPGA compensates for the burstiness of memory read / write operations with the parallelism of the FPGA, achieving real-time video processing.
[0101] Regarding the aforementioned FPGA-based warp device, the specific methods by which each module or unit performs operations have been described in detail in the embodiments of the bit error detection method, and will not be elaborated upon here.
[0102] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units. Components shown as modules or units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the disclosed solution according to actual needs. Those skilled in the art can understand and implement this without any inventive effort.
[0103] Furthermore, in this example implementation, reference is made to... Figure 8 As shown, a sending card is also provided. It includes:
[0104] The processor generates interpolated video image information according to any of the FPGA-based warp methods in the above embodiments.
[0105] Furthermore, in this example implementation, reference is made to... Figure 9 As shown, a display control system is also provided. It includes:
[0106] The host computer is used to send video image data;
[0107] The transmitting card is used to receive video image data, convert it into video image information, and then transmit the video image information.
[0108] A receiver card is used to receive video image information and drive the display device to display it.
[0109] The sending card is the sending card in the above embodiment.
[0110] It should also be understood that the specific manner in which the sending card and display control system are implemented has been described in detail in the embodiments of the FPGA-based warp method, and will not be elaborated here.
[0111] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.
Claims
1. A warp method based on FPGA, characterized in that, include: On the FPGA processing platform, the target image to be inserted is divided into multiple blocks of equal size, each block having a preset number of pixels; The core processing module of the FPGA initiates a request to the DDR to read the position information of all pixels in the block on the original image according to the initial address of the block to be processed. After the DDR arbitration module responds to the request, it reads the position information of all pixels in the block on the original image from the DDR and stores the position information in the first register. Multiple core processing modules process multiple blocks simultaneously. Determine the boundaries of the location information; The pixel data of the sub-image that matches the boundary of the position information on the original image is obtained, and the pixel data of the sub-image is stored in the second register by the core processing module; The core processing module reads the position information in the first register according to the order of the interpolation points, and then reads the pixel data of the sub-image in the second register according to the position information, and calculates and stores the pixel value of each target interpolation point in the block in sequence.
2. The FPGA-based warp method according to claim 1, characterized in that, The step of determining the boundary of the location information includes: Calculate the maximum and minimum values in the horizontal and vertical directions of the location information respectively; The boundaries of the location information are determined based on the maximum and minimum values in the horizontal and vertical directions.
3. The FPGA-based warp method according to claim 1, characterized in that, The step of obtaining pixel data of a sub-image that matches the boundary of the position information on the original image includes: The sub-image is obtained by extending the boundary of the position information on the original image outward by one pixel in each of the four directions: up, down, left, and right. Obtain pixel data from the sub-image.
4. The FPGA-based warp method according to claim 1, characterized in that, The core processing module reads the position information from the first register according to the order of the interpolation points, then reads the pixel data of the sub-image from the second register according to the position information, and sequentially calculates and stores the pixel values of each target interpolation point in the block. The steps include: Based on the location information, obtain the position of the target interpolation point in the sub-image, which is 4 pixels around its original position. Read the pixel data of the sub-image, obtain the pixel values of the four pixels, and perform bilinear interpolation to obtain the pixel value of the target interpolation point; Store the pixel values of each target interpolation point in the block.
5. A FPGA-based warp device, characterized in that, This includes a partitioning module, a location information acquisition module, a boundary determination module, a sub-image data acquisition module, and a calculation module, all mounted on an FPGA. The segmentation module is used to divide the target image to be inserted into multiple blocks of equal size, each block having a preset number of pixels; The location information acquisition module is used to initiate a request to the DDR to read the location information of all pixels in the block on the original image according to the initial address of the block to be processed. After the DDR arbitration module responds to the request, it reads the location information of all pixels in the block on the original image from the DDR and stores the location information in the first register. The boundary determination module is used to determine the boundary of the location information; The sub-image data acquisition module is used to acquire pixel data of the sub-image that matches the boundary of the position information on the original image, and the core processing module of the FPGA stores the pixel data of the sub-image into the second register. The calculation module is used to read the position information in the first register according to the order of the interpolation points, and then read the pixel data of the sub-image in the second register according to the position information, and sequentially calculate and store the pixel values of each target interpolation point in the block.
6. The FPGA-based warp device according to claim 5, characterized in that, The boundary determination module includes: The statistical unit is used to calculate the maximum and minimum values in the horizontal and vertical directions of the location information, respectively. The determining unit is used to determine the boundary of the location information based on the maximum and minimum values of the horizontal and vertical directions in the location information.
7. The FPGA-based warp device according to claim 5, characterized in that, The sub-image data acquisition module includes: An extension unit is used to extend the boundary of the position information on the original image outward by one pixel in each of the four directions (up, down, left, and right) to obtain the sub-image; The first storage unit is used to acquire and store pixel data in the sub-image.
8. The FPGA-based warp device according to claim 5, characterized in that, The computing module includes: The original position acquisition unit is used to acquire the position of 4 pixels around the original position of the target interpolation point on the sub-image based on the position information; An interpolation calculation unit is used to obtain the pixel values of the four pixels based on the pixel data of the sub-image, and perform bilinear interpolation calculation to obtain the pixel value of the target interpolation point; The second storage unit is used to store the pixel values of each target interpolation point in the block.
9. A sending card, characterized in that, include: A processor that generates interpolated video image information using the FPGA-based warp method according to any one of claims 1 to 4.
10. A display control system, characterized in that, include: The host computer is used to send video image data; A transmitting card is used to receive the video image data, perform data conversion, and transmit the video image information; A receiving card, used to receive the video image information and drive the display device to display it; The sending card is the sending card as described in claim 9.