Software and hardware parallel decoding method and device, terminal equipment, chip and chip module

Abstract

The invention relates to a software and hardware parallel decoding method and device, terminal equipment, a chip and a chip module. The method comprises the following steps: controlling a software decoding unit to decode a slice header of a current frame; controlling the software decoding unit to generate a reference frame list according to the current slice header information and the decoded image cache, and configuring a register of the hardware decoding unit to enable the hardware decoding unit to decode the slice data of the current frame; in the process of decoding the slice data, controlling a software decoding unit to decode a slice header of a next frame, generating a reference frame list of the next frame according to the information of the next slice header, and configuring a register; and when the decoding of the piece data is completed, controlling the software decoding unit to update the decoded image cache based on the decoded current frame and the current piece header information, taking the next frame as a new current frame, and returning to the step of enabling the hardware decoding unit to decode the piece data of the current frame until the to-be-decoded image frame does not exist. By adopting the method, efficient parallel decoding of the slice header and the slice data can be realized, and the method has decoding standard universality.

Description

Technical Field

[0001] This application relates to the field of video decoding technology, and in particular to a software and hardware parallel decoding method, apparatus, terminal equipment, chip and chip module. Background Technology

[0002] In recent years, with the continuous advancement of mobile internet and video applications, people's demand for high-definition real-time video has been increasing. However, due to the huge amount of data and the increased encoding complexity of high-definition video, the time required to decode a single frame of video data has also increased. To address this problem, researchers have proposed several decoding schemes to reduce video decoding time, including but not limited to: 1. For example, CN105992008B proposes a multi-level, multi-task parallel decoding method on a multi-core processor platform, which divides HEVC decoding into two types of tasks: frame-level entropy decoding and CTU-level data decoding. The entropy decoding task is parallelized at the frame level, and the CTU data decoding task is parallelized at the CTU data level; 2. For example, the paper "High-Quality H.264 Parallel Decoder Design and Implementation Based on Homogeneous Multi-Core Processor Platform" (Zhejiang University, 2013) proposes a parallel algorithm based on a multi-core processor and combined with the h264 standard for all macroblock rows of a single frame of video data.

[0003] However, most current decoding schemes perform parallel decoding between slices or between CTUs / macroblock lines within a slice, failing to achieve parallel decoding of the header and data within a slice. Furthermore, most are based on specific decoding standards and are not applicable to most mainstream video decoding standards. Even those decoding architectures that involve header and data decoding perform serial decoding. For example, software first decodes the header, which is time-consuming due to system scheduling limitations. After software decoding, hardware decodes the data and writes the resulting frame to DDR (Double Data Rate Dynamic Random Access Memory). This serial decoding method results in resource inefficiency, with software idle when hardware is active and vice versa, leading to long decoding times for a single frame and hindering efficient parallel decoding of headers and data.

[0004] Therefore, current decoding schemes cannot achieve efficient parallel decoding of the title and slice data, and lack universality of decoding standards. Summary of the Invention

[0005] Therefore, it is necessary to provide a software and hardware parallel decoding method, device, terminal equipment, chip, chip module, computer-readable storage medium, or computer program product that can achieve efficient parallel decoding of chip header and chip data and has universality of decoding standards to address the above-mentioned technical problems.

[0006] In a first aspect, this application provides a software and hardware parallel decoding method, applied to a terminal side including a software decoding unit and a hardware decoding unit, comprising:

[0007] The system acquires the image buffer and multiple image frames to be decoded, controls the software decoding unit to decode the header of the current frame in the image frame, and obtains the current header information.

[0008] The control software decoding unit generates a reference frame list for the current frame based on the current header information and the decoded image buffer, and configures the registers of the hardware decoding unit so that the hardware decoding unit can decode the slice data of the current frame according to the configured registers.

[0009] During the decoding of chip data, the control software decoding unit decodes the chip header of the next frame of the current frame, and generates a reference frame list for the next frame and configures the registers based on the decoded chip header information.

[0010] When the chip data decoding is complete, the decoded current frame is obtained. The control software decoding unit updates the decoded image buffer based on the decoded current frame and the current chip header information, and takes the next frame as the new current frame. Then, it returns to the step of enabling the hardware decoding unit to decode the chip data of the current frame according to the configured registers, until there are no more image frames to be decoded.

[0011] In one embodiment, the current title sequence information includes: an image sequence count of the current frame and reference frame rearrangement information; the control software decoding unit generates a reference frame list for the current frame based on the current title sequence information and the decoded image buffer, and configures the registers of the hardware decoding unit, including:

[0012] The control software decoding unit sorts multiple reference frames stored in the decoded image buffer based on image sequence counting to generate an initial reference frame list for the current frame.

[0013] Based on the reference frame rearrangement information, adjust the initial reference frame list to obtain the reference frame list for the current frame;

[0014] Write the list of reference frames for the current frame into the register of the hardware decoding unit.

[0015] In one embodiment, the control software decoding unit sorts multiple reference frames stored in the decoded image buffer based on image sequence counting to generate an initial reference frame list for the current frame, including:

[0016] The control software decoding unit scans the image sequence counts of multiple reference frames to determine multiple candidate reference frames that are closest to the image sequence count of the current frame;

[0017] Multiple candidate reference frames are sorted according to their image order to generate an initial reference frame list for the current frame.

[0018] In one embodiment, writing the reference frame list of the current frame into the register of the hardware decoding unit includes:

[0019] Determine the storage address information of each candidate reference frame in the reference frame list of the current frame, and determine the target data format that is compatible with the registers of the hardware decoding unit;

[0020] The storage address information is encapsulated into target data in the target data format, and the target data is written into the register.

[0021] In one embodiment, the current title sequence information includes: reference frame image identification information for the current frame; the control software decoding unit updates the decoded image buffer based on the decoded current frame and the current title sequence information, including:

[0022] The control software decoding unit parses the marking instructions from the reference frame image identification information;

[0023] According to the marking instructions, the decoded current frame is marked as a reference frame and stored in the decoded image buffer.

[0024] In one embodiment, the hardware and software parallel decoding method further includes:

[0025] When the removable frame information is parsed from the reference frame image identification information, the control software decoding unit locates the removable reference frame in the decoded image buffer based on the removable frame information.

[0026] Remove removable reference frames from the decoded image buffer.

[0027] Secondly, this application also provides a software and hardware parallel decoding device, applied to a terminal side including a software decoding unit and a hardware decoding unit, comprising:

[0028] The software decoding module is used to acquire the decoded image buffer and multiple image frames to be decoded, and to control the software decoding unit to decode the header of the current frame in the image frame to obtain the current header information;

[0029] The hardware decoding module is used to control the software decoding unit to generate a reference frame list for the current frame based on the current chip header information and the decoded image buffer, and to configure the registers of the hardware decoding unit so that the hardware decoding unit can decode the chip data of the current frame according to the configured registers.

[0030] The hardware and software parallel module is used to control the software decoding unit to decode the chip header of the next frame of the current frame during the decoding of chip data, and to generate a reference frame list for the next frame and configure registers based on the decoded chip header information.

[0031] The loop decoding module is used to obtain the decoded current frame when the chip data decoding is completed, control the software decoding unit to update the decoded image buffer based on the decoded current frame and the current chip header information, take the next frame as the new current frame, and return to the steps of the hardware decoding unit to decode the chip data of the current frame according to the configured registers, until there are no more image frames to be decoded.

[0032] Thirdly, this application also provides a terminal device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of the above-mentioned software and hardware parallel decoding method.

[0033] Fourthly, this application also provides a chip, including a processor and a communication interface, wherein the processor is configured to cause the chip to perform the steps of the method provided in the first aspect above.

[0034] Fifthly, this application also provides a chip module, including a communication module, a power module, a storage module, and a chip, wherein:

[0035] The power module is used to provide power to the chip module;

[0036] Storage modules are used to store data and instructions;

[0037] The communication module is used for internal communication within the chip module, or for communication between the chip module and external devices;

[0038] The chip is used to perform the steps of the method provided in the first aspect above.

[0039] Sixthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the first aspect above.

[0040] In a seventh aspect, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method provided in the first aspect above.

[0041] The aforementioned hardware and software parallel decoding method, apparatus, terminal device, chip, chip module, computer-readable storage medium, and computer program product, after acquiring the decoding image buffer and multiple image frames to be decoded, can control the software decoding unit to decode the header of the current frame in the image frame, obtain the current header information, and then control the software decoding unit to generate a reference frame list for the current frame based on the current header information and the decoding image buffer, and configure the registers of the hardware decoding unit, so that the hardware decoding unit decodes the slice data of the current frame according to the configured registers. During the decoding process of the hardware decoding unit, the software decoding unit is controlled to decode the header of the next frame of the current frame, and generates a reference frame list for the next frame and configures the registers according to the decoded next header information, so that the software decoding unit is not idle when the hardware decoding unit is decoding. When the slice data decoding is completed, the decoded current frame is obtained, the software decoding unit is controlled to update the decoding image buffer based on the decoded current frame and the current header information, and takes the next frame as the new current frame, and returns to the step of having the hardware decoding unit decode the slice data of the current frame according to the configured registers, until there are no more image frames to be decoded. Using the above process, while the hardware decodes the slice data of the current frame, the software can decode the chip header of the next frame in parallel and configure registers. This allows the hardware to immediately decode the slice data of the next frame after decoding the current frame. Under this parallel decoding process, the hardware is almost never idle, and the implementation of the above parallel decoding process is not limited to a specific decoding standard. Based on this, the parallel decoding of hardware and software can be fully utilized to achieve efficient parallel decoding of the chip header and slice data, and it has universality in decoding standards. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a diagram illustrating the application environment of a hardware / software parallel decoding method in one embodiment.

[0044] Figure 2 This is a flowchart illustrating a hardware-software parallel decoding method in one embodiment;

[0045] Figure 3 This is a schematic diagram of a parallel decoding architecture for Header and Data in one embodiment;

[0046] Figure 4 This is a flowchart illustrating a hardware and software parallel decoding method suitable for multiple decoding standards in one embodiment.

[0047] Figure 5 This is a structural block diagram of a hardware and software parallel decoding device in one embodiment;

[0048] Figure 6 This is an internal structural diagram of a computer device in one embodiment;

[0049] Figure 7 This is an internal structure diagram of a chip module in one embodiment. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] The hardware and software parallel decoding method provided in this application embodiment can be applied to... Figure 1 The terminal-side / chip / chip module in the application environment shown is compatible with most mainstream video decoding standards, including but not limited to H.264, HEVC, AV1, and VP9. Most mainstream video decoding standards have a layered structure for their frame-level bitstreams. Each slice in this structure consists of a Header and Data, and each image frame can include one or more slices. Each slice is a basic unit for independent encoding and decoding. The Header is a syntactic structure located at the beginning of each slice, containing the parameters and control information necessary for decoding the entire slice. This information defines how to interpret the subsequent Data portion.

[0052] by Figure 1 Taking the illustrated application environment as an example, the terminal / chip / chip module can first acquire the decoded image buffer and multiple image frames to be decoded. It then controls the software decoding unit to decode the header of the current frame in the image frame, obtaining the current header information. Next, the control software decoding unit generates a reference frame list for the current frame based on the current header information and the decoded image buffer, and configures the registers of the hardware decoding unit, enabling the hardware decoding unit to decode the slice data of the current frame according to the configured registers. During the decoding of the slice data, the control software decoding unit decodes the header of the next frame of the current frame, and generates a reference frame list for the next frame and configures the registers based on the decoded next header information. When the slice data decoding is complete, the decoded current frame is obtained. The control software decoding unit updates the decoded image buffer based on the decoded current frame and the current header information, and uses the next frame as the new current frame. It then returns to the step where the hardware decoding unit decodes the slice data of the current frame according to the configured registers, until there are no more image frames to be decoded.

[0053] The decoded image cache can be stored in the internal storage space of the terminal / chip / chip module, or in the cloud storage space connected to the terminal / chip / chip module. The terminal / chip / chip module can receive image frames through network communication, or retrieve image frames from its internal storage space or the connected cloud storage space. The terminal can be various devices with video decoding requirements, including but not limited to personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart TVs, projectors, monitoring equipment, and vehicle dashcams. Portable wearable devices can include smartwatches and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. The chip module is a fully functional independent module formed by integrating other components around a chip. Both the chip and the chip module have data processing capabilities and can decode image frames through parallel control of the software decoding unit and the hardware decoding unit.

[0054] In one exemplary embodiment, such as Figure 2 As shown, a hardware and software parallel decoding method is provided, which can be applied to... Figure 1 Taking a terminal-side / chip / chip module, which includes both software and hardware decoding units, as an example, the explanation includes the following steps 202 to 208. Wherein:

[0055] Step 202: Obtain the decoded image buffer and multiple image frames to be decoded, and control the software decoding unit to decode the header of the current frame in the image frame to obtain the current header information.

[0056] The image frame to be decoded can exist in the form of a bitstream buffer, which contains binary data of one or more encoded frames. The software decoding unit is programmable control logic, complex in logic, highly serializable, and flexible. Its functions can exist in the form of a driver, firmware code, or decoding library, stored on the terminal side / chip / chip module, and can be read and executed. The hardware decoding unit can be composed of dedicated computing circuits and can perform decoding operations strictly according to the register parameters configured by the software decoding unit. In the embodiments provided in this application, the software decoding unit can configure registers for the hardware decoding unit through the decoding chip header to control the hardware decoding unit to decode chip data. Furthermore, the software decoding unit can simultaneously prepare for the hardware decoding unit to decode the chip data of the next frame.

[0057] Optionally, the decoded image buffer and multiple image frames to be decoded can be obtained from internal storage space or connected cloud storage space. Further, the control software decoding unit locates the current frame from the multiple image frames and calls the title parsing function of the corresponding decoding standard (including but not limited to any of the most mainstream video decoding standards) to decode the title of the current frame and obtain the current title information.

[0058] Step 204: The control software decoding unit generates a reference frame list for the current frame based on the current header information and the decoded image buffer, and configures the registers of the hardware decoding unit so that the hardware decoding unit can decode the slice data of the current frame according to the configured registers.

[0059] Optionally, the control software decoding unit traverses all reference frames marked as "decoded" and "available for reference" in the decoded image buffer, and selects a portion of these reference frames based on the current title information, sorts them, and constructs a reference frame list for the current frame. Subsequently, this reference frame list is written to the register of the hardware decoding unit, enabling the hardware decoding unit to read and decode the slice data of the current frame according to the configuration in the register.

[0060] For example, the software decoding unit can write the list of reference frames into the registers of the hardware decoding unit via memory-mapped I / O or a configuration bus.

[0061] Step 206: During the decoding of chip data, the control software decoding unit decodes the chip header of the next frame of the current frame, and generates a reference frame list for the next frame and configures the registers based on the decoded chip header information.

[0062] Optionally, when the hardware decoding unit begins decoding the slice data of the current frame, the software decoding unit does not need to wait for it to complete and can be immediately scheduled to execute the task for the next frame. Based on this, the software decoding unit and the hardware decoding unit can perform decoding work concurrently: the hardware decoding unit decodes the slice data of the current frame, while the software decoding unit decodes the slice header of the next frame, and generates a reference frame list for the next frame based on the decoded slice header information and the decoded image buffer, thereby configuring the registers of the hardware decoding unit according to the reference frame list of the next frame.

[0063] Step 208: When the chip data decoding is completed, the decoded current frame is obtained. The control software decoding unit updates the decoded image buffer based on the decoded current frame and the current chip header information, and takes the next frame as the new current frame. Then, the step of having the hardware decoding unit decode the chip data of the current frame according to the configured register is returned, until there are no more image frames to be decoded.

[0064] Optionally, when the hardware decoding unit completes the decoding of the slice data of the current frame, the software decoding unit and the hardware decoding unit finish decoding the current frame, obtaining the decoded current frame. The software decoding unit then updates the decoding image buffer based on the decoded current frame and the current chip header information. For example, it decides whether to add the decoded current frame as a new reference frame to the decoding image buffer, or whether to delete the old reference frame from the decoding image buffer, to maintain the correct state of the decoding image buffer and ensure that the correct reference frame can be referenced subsequently. When the update of the decoding image buffer is complete, the next frame can be used as the new current frame, and based on the registers pre-configured by the software decoding unit, the steps for the hardware decoding unit to decode the slice data of the current frame according to the configured registers are returned. During the hardware decoding process, the software decoding unit is also controlled to decode the chip header of the new next frame, and based on the decoded next chip header information, a reference frame list for the next frame is generated and the registers are configured, ensuring that the software decoding unit is not idle while the hardware decoding unit is decoding. Based on this, the above parallel hardware and software decoding process is repeated until there are no more image frames to be decoded.

[0065] Based on steps 202 to 208 above, as Figure 3 The diagram illustrates a parallel decoding architecture for Headers and Data, applicable to most current mainstream video decoding standards. The example uses 'sw' to represent the software decoding unit and 'hw' to represent the hardware decoding unit.

[0066] First, the decoded image buffer and N+1 image frames (Frame0 to FrameN) to be decoded are acquired. The first image frame, Frame0, is taken as the current frame. The switch (sw) decodes the header (i.e., the slice header) of Frame0 to obtain the current slice header information. Then, the switch generates a reference frame list for Frame0 based on the current slice header information and the decoded image buffer. The reference frame list of Frame0 is used to configure the registers of the image controller (hw), so that the hw decodes the data (i.e., slice data) of Frame0 according to the configured registers. While the hw decodes the data of Frame0, the switch decodes the header of the next frame, Frame1, and generates a reference frame list for Frame1 and configures the registers according to the decoded next slice header information, so that the switch is not idle while the hw is decoding. When the hw finishes decoding the data of Frame0, it can obtain the decoded current frame and update the decoded image buffer based on the decoded current frame and the current slice header information, and takes Frame1 as the new current frame. The hw can then immediately decode the data of Frame1 according to the configured registers. Repeat the above process until N+1 image frames have been decoded. Using the above process, while hw is decoding the Data of the current frame, sw can decode the Header of the next frame in parallel and configure the registers, so that hw can immediately decode the Data of the next frame after decoding the current frame.

[0067] The aforementioned hardware and software parallel decoding method, after acquiring the decoded image buffer and multiple image frames to be decoded, controls the software decoding unit to decode the header of the current frame in the image frame, obtaining the current header information. Then, the software decoding unit generates a reference frame list for the current frame based on the current header information and the decoded image buffer, and configures the registers of the hardware decoding unit, enabling the hardware decoding unit to decode the slice data of the current frame according to the configured registers. During the hardware decoding process, the software decoding unit decodes the header of the next frame of the current frame, and generates a reference frame list for the next frame and configures the registers based on the decoded next header information, ensuring that the software decoding unit is not idle while the hardware decoding unit is decoding. When the slice data decoding is complete, the decoded current frame is obtained. The software decoding unit updates the decoded image buffer based on the decoded current frame and the current header information, and uses the next frame as the new current frame. The process then returns to the step where the hardware decoding unit decodes the slice data of the current frame according to the configured registers, until there are no more image frames to be decoded. Using the above process, while the hardware decodes the slice data of the current frame, the software can decode the chip header of the next frame in parallel and configure registers. This allows the hardware to immediately decode the slice data of the next frame after decoding the current frame. Under this parallel decoding process, the hardware is almost never idle, and the implementation of the above parallel decoding process is not limited to a specific decoding standard. Based on this, the parallel decoding of hardware and software can be fully utilized to achieve efficient parallel decoding of the chip header and slice data, and it has universality in decoding standards.

[0068] In one embodiment, the current title sequence information includes: an image sequence count of the current frame and reference frame rearrangement information. Based on this, the control software decoding unit generates a reference frame list for the current frame according to the current title sequence information and the decoded image buffer, and configures the registers of the hardware decoding unit, including:

[0069] The control software decoding unit sorts multiple reference frames stored in the decoded image buffer based on image sequence counting to generate an initial reference frame list for the current frame.

[0070] Based on the reference frame rearrangement information, adjust the initial reference frame list to obtain the reference frame list for the current frame;

[0071] Write the list of reference frames for the current frame into the register of the hardware decoding unit.

[0072] Specifically, the Image Order Count (POC) of the current frame in the current title sequence information refers to a unique, globally comparable logical sequence number assigned to each image frame in a video stream containing multiple image frames. This sequence number is used to determine the temporal relationship and display order between image frames. The Reference Frame Reordering Information of the current frame in the current title sequence information refers to an explicit control command used to overwrite or modify the order of reference frames in the reference frame list. The order of reference frames in the reference frame list is determined based on the image order count of the reference frames.

[0073] Optionally, the software decoding unit can be controlled to scan the image sequence counts of multiple reference frames in the decoding image buffer based on the image sequence count of the current frame, select multiple candidate reference frames from the decoding image buffer whose image sequence counts are closest to those of the current frame, sort the multiple candidate reference frames according to their image sequence counts, and load them into a list, thereby generating the initial reference frame list for the current frame. Further, based on the adjustment instructions for image sequence counts in the reference frame rearrangement information, the sorting of multiple candidate reference frames in the initial reference frame list can be adjusted to obtain the reference frame list for the current frame. This reference frame list is then written into the registers of the hardware decoding unit, completing the register configuration of the hardware decoding unit.

[0074] For example, the reference frame rearrangement information may indicate "to swap the position of the candidate reference frame with POC X in the initial reference frame list with that of the candidate reference frame with POC Y", and the software decoding unit may perform the list rearrangement operation indicated by the reference frame rearrangement information to adjust the initial reference frame list and generate the final reference frame list that meets the requirements.

[0075] In this embodiment, the software decoding unit can construct and rearrange the logically complex reference frame list, providing accurate and efficient support for the complex reference frame management mechanism in most mainstream video coding standards. At the same time, it can provide the hardware decoding unit with a clear and direct reference frame list index, laying the foundation for subsequent parallel software and hardware decoding.

[0076] In one possible implementation, the control software decoding unit sorts multiple reference frames stored in the decoded image buffer based on image sequence counting to generate an initial reference frame list for the current frame, including:

[0077] The control software decoding unit scans the image sequence counts of multiple reference frames to determine multiple candidate reference frames that are closest to the image sequence count of the current frame;

[0078] Multiple candidate reference frames are sorted according to their image order to generate an initial reference frame list for the current frame.

[0079] Optionally, the software decoding unit can be controlled to scan the image sequence counts of multiple reference frames in the decoded image buffer, and select reference frames whose image sequence count is less than that of the current frame as the forward reference frame candidate set, and select reference frames whose image sequence count is greater than that of the current frame as the backward reference frame candidate set. The reference frames in the forward reference frame candidate set are displayed before the current frame, and the reference frames in the backward reference frame candidate set are displayed after the current frame.

[0080] Furthermore, the reference frames in the forward reference frame candidate set can be sorted in descending order of image sequence count, and at least one reference frame with the largest image sequence count (most recent in time to the current frame) can be selected as a candidate reference frame. Similarly, the reference frames in the backward reference frame candidate set can be sorted in ascending order of image sequence count, and at least one reference frame with the smallest image sequence count (most recent in time to the current frame) can also be selected as a candidate reference frame. Based on this, multiple candidate reference frames that are closest to the current frame in image sequence count can be identified. These multiple candidate reference frames are then sorted according to their image sequence count and populated into a list in order to generate the initial reference frame list for the current frame.

[0081] For example, the initial reference frame list may have empty slots or be too long. The software decoding unit can "trim" the initial reference frame list according to the rules specified in the corresponding decoding standard, such as specifying the target length of each reference frame list, so as to form the required initial reference frame list.

[0082] In this embodiment, the software decoding unit can intelligently filter and sort multiple reference frames in the decoding image cache based on image sequence counting, thereby efficiently generating a logically reasonable initial reference frame list.

[0083] In some embodiments, writing the reference frame list of the current frame into the register of the hardware decoding unit includes:

[0084] Determine the storage address information of each candidate reference frame in the reference frame list of the current frame, and determine the target data format that is compatible with the registers of the hardware decoding unit;

[0085] The storage address information is encapsulated into target data in the target data format, and the target data is written into the register.

[0086] The storage address information may include: the starting physical address or bus address of the candidate reference frame in the decoded image buffer, the image span, and the image dimensions (such as width and height). The target data format needs to conform to the bit width constraints of the hardware decoding unit's registers, such as a 64-bit or 128-bit descriptor.

[0087] Optionally, the software decoding unit can determine the storage address information of each candidate reference frame in the decoded image buffer within the reference frame list of the current frame, and determine the target data format compatible with the registers of the hardware decoding unit. This encapsulates the storage address information of all candidate reference frames in the reference frame list into target data in the target data format. Furthermore, the hardware decoding unit's registers may include a dedicated "reference frame list descriptor table" register area. The software decoding unit can write the encapsulated target data into this register area to complete the register configuration of the hardware decoding unit.

[0088] For example, the software decoding unit can write the target data (which may be in the form of a physical address descriptor) in the target data format into the register area of ​​the "reference frame list descriptor table" in the order of the candidate reference frames in the reference frame list through a series of memory write operations, so that the subsequent hardware decoding unit can quickly obtain the reference frame list data from the register area.

[0089] In this embodiment, by mapping and encapsulating the logical reference frame list into physical address descriptors directly usable by the hardware decoding unit, efficient and low-latency connection between software control flow and hardware data flow is achieved. Using register configuration avoids complex address translation during the decoding process, reducing decoding latency and ensuring efficient real-time decoding of high-definition video.

[0090] In one embodiment, the current title sequence information includes: reference frame image identification information for the current frame. Based on this, the control software decoding unit updates the decoded image buffer based on the already decoded current frame and the current title sequence information, including:

[0091] The control software decoding unit parses the marking instructions from the reference frame image identification information;

[0092] According to the marking instructions, the decoded current frame is marked as a reference frame and stored in the decoded image buffer.

[0093] The reference frame image identification information for the current frame in the current title sequence can contain explicit management instructions for reference frames in the decoded image buffer. It is a complete set of syntax elements and instructions used to uniquely identify, locate, and manage reference frames. The marking instruction can indicate whether the decoded current frame is marked as a new reference frame as a "short-term reference" or a "long-term reference".

[0094] Optionally, the software decoding unit can be controlled to parse the marking instruction from the reference frame image identification information, mark the decoded current frame as a new reference frame according to the marking instruction, allocate an empty or reusable frame buffer for the new reference frame in the decoded image buffer, and then store the new reference frame in the allocated frame buffer to complete the update of the decoded image buffer.

[0095] For example, the software decoding unit can create or update an entry for a new reference frame in the decoded image buffer according to the marking instruction, recording its: storage status as "for short-term reference" or "for long-term reference"; unique identifier as the corresponding short-term period or long-term period; sequence information as the image sequence count (POC) of the new reference frame; and other attributes including at least image size, pixel format, etc.

[0096] In this embodiment, the reference frames in the decoded image buffer can be precisely managed based on the marker instructions parsed from the reference frame image identification information, ensuring the correctness and consistency of the reference frame resources in the decoded image buffer. This lays the foundation for generating a reference frame list when decoding image frames in the future, and is a key maintenance operation to maintain the long-term stable operation of the entire parallel decoding process.

[0097] In one embodiment, the hardware and software parallel decoding method further includes:

[0098] When the removable frame information is parsed from the reference frame image identification information, the control software decoding unit locates the removable reference frame in the decoded image buffer based on the removable frame information.

[0099] Remove removable reference frames from the decoded image buffer.

[0100] Specifically, removable reference frames can be those that are marked as "no longer used for reference" or "can be removed" in the reference frame image identification information. These reference frames will not be used in future frame decoding processes.

[0101] Optionally, the software decoding unit can parse the reference frame image identification information to determine if removable frame information exists. If removable frame information is parsed from the reference frame image identification information, the software decoding unit can be controlled to search for a frame entry matching the removable frame information in the decoded image buffer, thereby locating the removable reference frame in the decoded image buffer. Furthermore, the software decoding unit can remove the removable reference frame from the decoded image buffer.

[0102] For example, the memory space occupied by the removable reference frame in the decoded image buffer may not be cleared immediately, allowing it to be overwritten and reused by the new reference frame that is marked.

[0103] In this embodiment, the space in the decoded image cache can be efficiently recycled by actively and promptly removing reference frames that are no longer needed. This not only avoids cache overflow but also ensures the ability to continuously decode long video sequences with limited memory resources. It helps reduce invalid cache (removable reference frames), improves the efficiency of the software decoding unit in managing the decoded image cache and generating the reference frame list, and supports the smooth operation of the software and hardware parallel decoding architecture.

[0104] In a specific application scenario, such as Figure 4 As shown, a software and hardware parallel decoding method suitable for multiple decoding standards is provided. This method can be applied to various mainstream video decoding standards such as H264, HEVC, av1, and vp9. Among them: the software decoding unit processes the following: (1) decodes the header of the current frame to obtain the image sequence count and reference frame rearrangement information; (2) generates the reference frame list of the current frame according to the image sequence count, reference frame rearrangement information and the decoded image buffer, and configures the registers of the hardware decoding unit; (3) updates the decoded image buffer. The hardware decoding unit processes the following: (4) decodes the slice data of the current frame according to the configured registers.

[0105] Based on this, the above-mentioned hardware and software parallel decoding method suitable for multiple decoding standards mainly includes the following process:

[0106] (1) The software decoding unit decodes the first current frame's header to obtain the image sequence count, reference frame rearrangement information, and reference frame image identification information;

[0107] (2) The software decoding unit sorts the multiple reference frames stored in the decoded image buffer based on the image sequence counting and generates the initial reference frame list for the current frame;

[0108] (3) The software decoding unit adjusts the initial reference frame list according to the reference frame rearrangement information to obtain the reference frame list of the current frame;

[0109] (4) The software decoding unit writes the reference frame list of the current frame into the register of the hardware decoding unit;

[0110] (5) The hardware decoding unit decodes the slice data of the current frame according to the configured registers, and the software decoding unit simultaneously decodes the slice header of the next frame, generates the reference frame list of the next frame and configures the registers.

[0111] When the slice data is decoded, (6) the software decoding unit adds or deletes reference frames in the decoded image buffer; further, if there are still image frames to be decoded, (7) the next frame is taken as the new current frame, and the above steps are repeated until there are no more image frames to be decoded.

[0112] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0113] Based on the same inventive concept, this application also provides a hardware-software parallel decoding apparatus for implementing the aforementioned hardware-software parallel decoding method. This apparatus can be applied to or integrated into a chip or chip module, for example. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more hardware-software parallel decoding apparatus embodiments provided below can be found in the limitations of the hardware-software parallel decoding method described above, and will not be repeated here.

[0114] In one exemplary embodiment, such as Figure 5 As shown, a software and hardware parallel decoding device is provided, applied to a terminal side including a software decoding unit and a hardware decoding unit, comprising: a software decoding module 502, a hardware decoding module 504, a software and hardware parallel module 506, and a loop decoding module 508, wherein:

[0115] The software decoding module is used to acquire the decoded image buffer and multiple image frames to be decoded, and to control the software decoding unit to decode the header of the current frame in the image frame to obtain the current header information;

[0116] The hardware decoding module is used to control the software decoding unit to generate a reference frame list for the current frame based on the current chip header information and the decoded image buffer, and to configure the registers of the hardware decoding unit so that the hardware decoding unit can decode the chip data of the current frame according to the configured registers.

[0117] The hardware and software parallel module is used to control the software decoding unit to decode the chip header of the next frame of the current frame during the decoding of chip data, and to generate a reference frame list for the next frame and configure registers based on the decoded chip header information.

[0118] The loop decoding module is used to obtain the decoded current frame when the chip data decoding is completed, control the software decoding unit to update the decoded image buffer based on the decoded current frame and the current chip header information, take the next frame as the new current frame, and return to the steps of the hardware decoding unit to decode the chip data of the current frame according to the configured registers, until there are no more image frames to be decoded.

[0119] The aforementioned hardware and software parallel decoding device, after acquiring the decoded image buffer and multiple image frames to be decoded, can control the software decoding unit to decode the header of the current frame in the image frame, obtain the current header information, and then control the software decoding unit to generate a reference frame list for the current frame based on the current header information and the decoded image buffer, and configure the registers of the hardware decoding unit, so that the hardware decoding unit can decode the slice data of the current frame according to the configured registers. During the decoding process of the hardware decoding unit, the software decoding unit is controlled to decode the header of the next frame of the current frame, and generates a reference frame list for the next frame and configures the registers according to the decoded next header information, so that the software decoding unit is not idle when the hardware decoding unit is decoding. When the slice data decoding is completed, the decoded current frame is obtained, and the software decoding unit is controlled to update the decoded image buffer based on the decoded current frame and the current header information, and takes the next frame as the new current frame, and returns to the step of having the hardware decoding unit decode the slice data of the current frame according to the configured registers, until there are no more image frames to be decoded. Using the above process, while the hardware decodes the slice data of the current frame, the software can decode the chip header of the next frame in parallel and configure registers. This allows the hardware to immediately decode the slice data of the next frame after decoding the current frame. Under this parallel decoding process, the hardware is almost never idle, and the implementation of the above parallel decoding process is not limited to a specific decoding standard. Based on this, the parallel decoding of hardware and software can be fully utilized to achieve efficient parallel decoding of the chip header and slice data, and it has universality in decoding standards.

[0120] In one embodiment, the current title sequence information includes: the image sequence count of the current frame and reference frame rearrangement information. The hardware decoding module includes:

[0121] The initial list generation unit controls the software decoding unit to sort multiple reference frames stored in the decoded image buffer based on the image sequence count, and generate an initial reference frame list for the current frame.

[0122] The list adjustment unit is used to adjust the initial reference frame list according to the reference frame rearrangement information to obtain the reference frame list of the current frame.

[0123] The register configuration unit is used to write the reference frame list of the current frame into the register of the hardware decoding unit.

[0124] In one embodiment, the initial list generation unit is specifically used for:

[0125] The control software decoding unit scans the image sequence counts of multiple reference frames to determine multiple candidate reference frames that are closest to the image sequence count of the current frame;

[0126] Multiple candidate reference frames are sorted according to their image order to generate an initial reference frame list for the current frame.

[0127] In one embodiment, the register configuration unit is specifically used for:

[0128] Determine the storage address information of each candidate reference frame in the reference frame list of the current frame, and determine the target data format that is compatible with the registers of the hardware decoding unit;

[0129] The storage address information is encapsulated into target data in the target data format, and the target data is written into the register.

[0130] In one embodiment, the current title sequence information includes: reference frame image identification information for the current frame. The loop decoding module includes:

[0131] The tag instruction parsing unit is used to control the software decoding unit to parse the tag instructions from the reference frame image identification information;

[0132] The reference frame storage unit is used to mark the decoded current frame as a reference frame and store it in the decoded image buffer according to the marking instruction.

[0133] In one embodiment, the hardware and software parallel decoding device further includes:

[0134] The frame removal positioning module is used to control the software decoding unit to locate the removable reference frame in the decoded image buffer based on the removable frame information when the removable frame information is parsed from the reference frame image identification information.

[0135] The reference frame removal module is used to remove removable reference frames from the decoded image buffer.

[0136] Regarding the modules / units included in the various devices and products described in the above embodiments, they can be software modules / units, hardware modules / units, or a combination of both. For example, for various devices and products applied to or integrated into a chip, all of their modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs that run on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits; for various devices and products applied to or integrated into a chip module, all of their modules / units can be implemented using hardware methods such as circuits, and different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The components can be implemented using software programs that run on the processor integrated within the chip module. The remaining (if any) modules / units can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into the terminal, each of its components / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or in different components within the terminal. Alternatively, at least some modules / units can be implemented using software programs that run on the processor integrated within the terminal, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits.

[0137] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data for parallel hardware and software decoding. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a parallel hardware and software decoding method.

[0138] Those skilled in the art will understand that Figure 6The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0139] In one exemplary embodiment, a terminal device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0140] Based on the same inventive concept, this application also provides a chip, including a processor and a communication interface; the communication interface is used to receive or send data; the processor is configured to cause the chip to perform the following steps:

[0141] The system acquires the image buffer and multiple image frames to be decoded, controls the software decoding unit to decode the header of the current frame in the image frame, and obtains the current header information.

[0142] The control software decoding unit generates a reference frame list for the current frame based on the current header information and the decoded image buffer, and configures the registers of the hardware decoding unit so that the hardware decoding unit can decode the slice data of the current frame according to the configured registers.

[0143] During the decoding of chip data, the control software decoding unit decodes the chip header of the next frame of the current frame, and generates a reference frame list for the next frame and configures the registers based on the decoded chip header information.

[0144] When the chip data decoding is complete, the decoded current frame is obtained. The control software decoding unit updates the decoded image buffer based on the decoded current frame and the current chip header information, and takes the next frame as the new current frame. Then, it returns to the step of enabling the hardware decoding unit to decode the chip data of the current frame according to the configured registers, until there are no more image frames to be decoded.

[0145] In one embodiment, the processor is configured to cause the chip to perform the following steps:

[0146] The control software decoding unit sorts multiple reference frames stored in the decoded image buffer based on image sequence counting to generate an initial reference frame list for the current frame.

[0147] Based on the reference frame rearrangement information, adjust the initial reference frame list to obtain the reference frame list for the current frame;

[0148] Write the list of reference frames for the current frame into the register of the hardware decoding unit.

[0149] In one embodiment, the processor is configured to cause the chip to perform the following steps:

[0150] The control software decoding unit scans the image sequence counts of multiple reference frames to determine multiple candidate reference frames that are closest to the image sequence count of the current frame;

[0151] Multiple candidate reference frames are sorted according to their image order to generate an initial reference frame list for the current frame.

[0152] In one embodiment, the processor is configured to cause the chip to perform the following steps:

[0153] Determine the storage address information of each candidate reference frame in the reference frame list of the current frame, and determine the target data format that is compatible with the registers of the hardware decoding unit;

[0154] The storage address information is encapsulated into target data in the target data format, and the target data is written into the register.

[0155] In one embodiment, the processor is configured to cause the chip to perform the following steps:

[0156] The control software decoding unit parses the marking instructions from the reference frame image identification information;

[0157] According to the marking instructions, the decoded current frame is marked as a reference frame and stored in the decoded image buffer.

[0158] In one embodiment, the processor is configured to cause the chip to perform the following steps:

[0159] When the removable frame information is parsed from the reference frame image identification information, the control software decoding unit locates the removable reference frame in the decoded image buffer based on the removable frame information.

[0160] Remove removable reference frames from the decoded image buffer.

[0161] It is understood that the chip involved in the embodiments of this application may be a field-programmable gate array (FPGA), may be an application-specific integrated circuit (ASIC), may be a system on chip (SoC), may be a central processor unit (CPU), may be a network processor (NP), may be a digital signal processor (DSP), may be a microcontroller unit (MCU), may be a programmable logic device (PLD), or other integrated chips, etc.

[0162] Based on the same inventive concept, this application also provides a chip module, such as... Figure 7 As shown, the chip module includes a communication module, a power module, a storage module, and a chip. Among them:

[0163] The power module is used to provide power to the chip module; the storage module is used to store data and instructions; the communication module is used for internal communication within the chip module, or for communication between the chip module and external devices; this chip corresponds to the chip in the above chip embodiment.

[0164] The implementation method of this chip module can be found in the relevant content of the above chip embodiment, and will not be repeated here.

[0165] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0166] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0167] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0168] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0169] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0170] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A hardware and software parallel decoding method, characterized in that, Applied to a terminal side including a software decoding unit and a hardware decoding unit, the method includes: The system acquires the decoded image buffer and multiple image frames to be decoded, and controls the software decoding unit to decode the header of the current frame in the image frames to obtain the current header information. The software decoding unit is controlled to generate a reference frame list for the current frame based on the current title information and the decoded image buffer, and the registers of the hardware decoding unit are configured so that the hardware decoding unit decodes the slice data of the current frame according to the configured registers. During the decoding of the slice data, the software decoding unit is controlled to decode the slice header of the next frame of the current frame, and a reference frame list for the next frame is generated and the register is configured based on the decoded slice header information. When the slice data decoding is completed, the decoded current frame is obtained. The software decoding unit is controlled to update the decoded image buffer based on the decoded current frame and the current slice header information, and the next frame is taken as the new current frame. The process is then returned to the step of having the hardware decoding unit decode the slice data of the current frame according to the configured registers, until there are no more image frames to be decoded.

2. The method according to claim 1, characterized in that, The current title sequence information includes: the image sequence count and reference frame rearrangement information of the current frame; the control of the software decoding unit to generate a reference frame list for the current frame based on the current title sequence information and the decoded image buffer, and to configure the registers of the hardware decoding unit, including: The software decoding unit is controlled to sort multiple reference frames stored in the decoded image buffer based on the image sequence count, and generate an initial reference frame list for the current frame; Based on the reference frame rearrangement information, the initial reference frame list is adjusted to obtain the reference frame list for the current frame; Write the reference frame list of the current frame into the register of the hardware decoding unit.

3. The method according to claim 2, characterized in that, The control unit, based on the image sequence count, sorts multiple reference frames stored in the decoded image buffer to generate an initial reference frame list for the current frame, including: The software decoding unit is controlled to scan the image sequence counts of multiple reference frames to determine multiple candidate reference frames that are closest to the image sequence count of the current frame; The candidate reference frames are sorted according to their image order to generate an initial reference frame list for the current frame.

4. The method according to claim 2, characterized in that, The step of writing the reference frame list of the current frame into the register of the hardware decoding unit includes: The storage address information of each candidate reference frame in the reference frame list of the current frame is determined, and the target data format adapted to the registers of the hardware decoding unit is determined. The storage address information is encapsulated into target data in the target data format, and the target data is written into the register.

5. The method according to claim 1, characterized in that, The current title sequence information includes: reference frame image identification information for the current frame; the control of the software decoding unit to update the decoded image cache based on the decoded current frame and the current title sequence information includes: The software decoding unit is controlled to parse the marking instructions from the reference frame image identification information; According to the marking instruction, the decoded current frame is marked as a reference frame and stored in the decoded image buffer.

6. The method according to claim 5, characterized in that, The method further includes: If removable frame information is parsed from the reference frame image identification information, the software decoding unit is controlled to locate the removable reference frame in the decoded image buffer according to the removable frame information. The removable reference frame is removed from the decoded image buffer.

7. A hardware and software parallel decoding device, characterized in that, Applied to a terminal side including a software decoding unit and a hardware decoding unit, the device includes: The software decoding module is used to acquire the decoded image buffer and multiple image frames to be decoded, and to control the software decoding unit to decode the header of the current frame in the image frame to obtain the current header information; The hardware decoding module is used to control the software decoding unit to generate a reference frame list for the current frame based on the current header information and the decoded image buffer, and to configure the registers of the hardware decoding unit so that the hardware decoding unit can decode the slice data of the current frame according to the configured registers. The hardware and software parallel module is used to control the software decoding unit to decode the chip header of the next frame of the current frame during the decoding of the chip data, and to generate a reference frame list of the next frame and configure the registers based on the decoded chip header information. The loop decoding module is used to obtain the decoded current frame when the slice data decoding is completed, control the software decoding unit to update the decoded image buffer based on the decoded current frame and the current slice header information, and take the next frame as the new current frame, and return to the step of enabling the hardware decoding unit to decode the slice data of the current frame according to the configured registers, until there are no image frames to be decoded.

8. A terminal device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A chip, characterized in that, The device includes a processor and a communication interface, wherein the processor is configured to cause the chip to perform the steps of the method described in any one of claims 1 to 6.

10. A chip module, characterized in that, This includes communication modules, power modules, storage modules, and chips, among which: The power module is used to provide power to the chip module; The storage module is used to store data and instructions; The communication module is used for internal communication within the chip module, or for communication between the chip module and external devices. The chip is used to perform the steps of the method according to any one of claims 1 to 6.

Images