JPEG (Joint Photographic Experts Group) image coding and decoding method based on MCU (Microprogrammed Control Unit), JPEG coding and decoding controller, medium, program product and terminal

Abstract

The invention provides a JPEG (Joint Photographic Experts Group) image coding and decoding method based on an MCU (Microprogrammed Control Unit), a JPEG coding and decoding controller, a medium, a program product and a terminal. The JPEG coding and decoding controller with an RGB (Red, Green, Blue) and YUV format conversion module is added into an MCU controller on hardware, and a queue-based management mode is adopted in software; the problems of overload of the computing power of a CPU and overhigh occupation of memory resources caused by conversion of RGB and YUV formats in the existing JPEG encoding and decoding process are solved, and the effects of remarkably reducing the requirements of the CPU and the memory and improving the system efficiency are achieved. The software and hardware collaborative optimization method reduces the dependence on a high-performance processor and a large-capacity memory, enables the MCU to select a simpler hardware architecture with lower cost, further reduces the overall cost of the system, is especially suitable for MCU application scenarios mainly in RGB formats, greatly optimizes the JPEG encoding and decoding process, and has important practical value.

Description

Technical Field

[0001] The present application relates to the field of image data processing, and in particular to an MCU-based JPEG image encoding and decoding method, a JPEG encoding and decoding controller, a medium, a program product and a terminal. Background Art

[0002] At present, in the image processing application of microcontroller MCU (Microcontroller Unit), the mainstream method is to convert RGB data into YUV format through the central processing unit CPU (Central Processing Unit) software, and then perform JPEG encoding. This process generates YUV data by the CPU parsing and converting the input RGB format, and then the YUV data is processed by dedicated JPEG encoding hardware. When decoding the image, the CPU is responsible for converting the YUV data back to RGB format for image display or other processing.

[0003] This method has significant shortcomings. First, relying on the CPU for data format conversion reduces the system's processing efficiency and resource utilization, and significantly increases the CPU's burden. Second, when processing complex image scenes, the data format conversion takes a long time, which affects the system's response speed. Especially in applications that require real-time image processing, existing solutions often cannot meet the requirements of real-time and efficiency. In addition, the ability to process complex image data streams is limited, which restricts the overall performance of the MCU in image processing, ultimately affecting the application effect. Due to technical limitations, the contradiction between processing speed and image quality is difficult to be effectively resolved. Summary of the invention

[0004] In view of the shortcomings of the prior art described above, the purpose of the present application is to provide a JPEG image encoding and decoding method, a JPEG encoding and decoding controller, a medium, a program product and a terminal based on an MCU, so as to solve the problem that the existing image processing technology relies on the CPU to convert RGB and YUV formats, resulting in low system efficiency, insufficient resource utilization, and difficulty in meeting the needs of efficient image processing.

[0005] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides a JPEG image encoding and decoding method based on an MCU, which is applied to an MCU controller, and the method includes: obtaining one or more encoding and decoding tasks; for each encoding and decoding task, reading the image data to be encoded or the JPEG code stream data to be decoded from the memory space according to the current encoding and decoding task; performing an encoding operation on the image data to be encoded to generate encoded JPEG code stream data, or performing the decoding operation on the JPEG code stream data to be decoded to generate decoded image data; and storing the encoded JPEG code stream data or the decoded image data in the memory space.

[0006] In some embodiments of the first aspect of the present application, the process of obtaining one or more encoding and decoding tasks also performs the following steps: creating a task queue for storing the encoding and decoding tasks, inserting the obtained encoding and decoding tasks into the tail of the queue in sequence, obtaining tasks from the head of the queue in sequence in a first-in-first-out order, and performing encoding operations or decoding operations.

[0007] In some embodiments of the first aspect of the present application, after the current encoding and decoding task is completed, the following steps are also performed: when the task queue is not empty, the next encoding and decoding task is read from the task queue, and the operation type of the next encoding and decoding task is determined; if the operation type of the next encoding and decoding task is an encoding task, a decoding operation is performed based on the header information of the JPEG code stream data to be decoded; otherwise, an encoding operation is performed based on the image data to be encoded.

[0008] In some embodiments of the first aspect of the present application, the field information of the encoding and decoding task includes: one or more of: configuration parameters, storage address, operation type and running status.

[0009] In some embodiments of the first aspect of the present application, if the operation type of the current encoding and decoding task is an encoding operation, the following steps are performed: the running state of the current encoding and decoding task is set to processing; the RGB image to be encoded is read in the memory space according to the storage address of the current encoding and decoding task; an RGB2YUV conversion operation is performed on the RGB image to be encoded to generate a corresponding YUV image to be encoded; and a macroblock encoding operation is performed on the YUV image to be encoded to generate corresponding encoded JPEG code stream data; the encoded JPEG code stream data is stored in the memory space, and the task running state of the current encoding and decoding task is set to processing completed.

[0010] In some embodiments of the first aspect of the present application, if the operation type is a decoding operation, the following steps are performed: the task running state of the current encoding and decoding task is set to processing; the JPEG code stream data to be decoded is read from the memory space based on the storage address; a macroblock decoding operation is performed on the decoded JPEG code stream data to generate decoded YUV image data; and a YUV2RGB conversion operation is performed on the decoded YUV image data to generate decoded RGB image data; the decoded RGB image data is stored in the memory space, and the task running state of the current encoding and decoding task is set to processing completed.

[0011] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides a JPEG codec controller, which includes: an input DMA unit, used to obtain one or more codec tasks; for each codec task, reading the image data to be encoded or the JPEG code stream data to be decoded from the memory space according to the current codec task; an encoding unit, used to perform an encoding operation on the image data to be encoded to generate encoded JPEG code stream data; a decoding unit, used to perform the decoding operation on the JPEG code stream data to be decoded to generate decoded image data; an output DMA unit, used to store the encoded JPEG code stream data or the decoded image data in the memory space; a register module, used to perform parameter configuration operations on the input DMA unit, the encoding unit, the decoding unit and the output DMA unit respectively.

[0012] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the MCU-based JPEG image encoding and decoding method is implemented.

[0013] To achieve the above-mentioned purpose and other related purposes, the fourth aspect of the present application provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer implements the MCU-based JPEG image encoding and decoding method.

[0014] To achieve the above-mentioned purpose and other related purposes, the fifth aspect of the present application provides an electronic terminal, including a memory, a processor and a computer program stored in the memory; the processor executes the computer program to implement the MCU-based JPEG image encoding and decoding method.

[0015] As described above, the JPEG image encoding and decoding method based on MCU, JPEG encoding and decoding controller, medium, program product and terminal of the present application have the following beneficial effects: The present invention proposes a JPEG encoding and decoding method optimized by hardware and software collaboration. Traditional JPEG encoding and decoding requires conversion between the RGB format and the YUV format of the image, and this process greatly consumes the CPU computing power and memory resources in the MCU. The present invention introduces a module for real-time conversion between RGB and YUV in hardware, which effectively reduces the burden of the CPU in data format conversion and improves system efficiency. In terms of software architecture design, the present invention adopts a queue-based management method, and reduces the system's memory occupation requirements by reasonably scheduling the input and output timing of data. This combination of software and hardware enables the entire system to better adapt to application scenarios dominated by RGB pixel format in the MCU field, without occupying a large amount of resources for data format conversion and storage. Through the technical innovation of the present invention, the demand for CPU and memory resources in the JPEG encoding and decoding process is significantly reduced. The dependence on high-performance processors is reduced, so that the MCU can choose a simpler and lower-cost hardware architecture in design. In addition, reducing memory usage means that a smaller capacity memory can be used, thereby reducing the cost of the overall system. For MCU application scenarios requiring JPEG functions, the present invention has important practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flow chart of an embodiment of a JPEG image encoding and decoding method based on MCU of the present application is shown.

[0017] Figure 2 A schematic diagram of the process of executing multiple encoding and decoding tasks in an embodiment of the MCU-based JPEG image encoding and decoding method of the present application is shown.

[0018] Figure 3 A schematic diagram of the process of interrupt processing in a task queue in an embodiment of an MCU-based JPEG image encoding and decoding method of the present application is shown.

[0019] Figure 4 A schematic diagram of the flow of encoding operations in an embodiment of a JPEG image encoding and decoding method based on MCU of the present application is shown.

[0020] Figure 5 The diagram shows the software and hardware interface of the encoding operation in an embodiment of the JPEG image encoding and decoding method based on MCU of the present application.

[0021] Figure 6 A schematic diagram of the process of executing encoding tasks in a task queue in an embodiment of the MCU-based JPEG image encoding and decoding method of the present application is shown.

[0022] Figure 7 A flow chart showing the configuration of the encoding hardware registers in an embodiment of the MCU-based JPEG image encoding and decoding method of the present application is shown.

[0023] Figure 8 A flowchart of the decoding operation in an embodiment of the MCU-based JPEG image encoding and decoding method of the present application is shown.

[0024] Fig. 9 The diagram shows the software and hardware interface of the decoding operation in an embodiment of the JPEG image encoding and decoding method based on MCU of the present application.

[0025] Fig.10 A flow chart of executing a decoding task in a task queue in an embodiment of an MCU-based JPEG image encoding and decoding method of the present application is shown.

[0026] Fig.11 The figure shows a flow chart of the configuration of the decoding hardware registers in an embodiment of the JPEG image encoding and decoding method based on MCU of the present application.

[0027] Fig.12 A structural diagram of an embodiment of a JPEG codec controller of the present application is shown.

[0028] Fig.13 A structural diagram of another embodiment of the JPEG codec controller of the present application is shown.

[0029] Fig.14 A structural diagram of an embodiment of a JPEG image encoding and decoding terminal based on MCU of the present application is shown. DETAILED DESCRIPTION

[0030] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0031] Before further describing the present invention in detail, the nouns and terms involved in the embodiments of the present invention are explained. The nouns and terms involved in the embodiments of the present invention are applicable to the following interpretations:

[0032] <1> JPEG image encoding and decoding: The process of compressing and decompressing JPEG format images to reduce the file size during storage and transmission and restore it to a displayable image when needed.

[0033] <2> MCU controller: A component in a microcontroller unit (MCU) that is responsible for executing instructions and coordinating the operation of various peripheral devices.

[0034] <3> DMA (Direct Memory Access): A mechanism that allows data to be transferred directly between peripherals and memory, without passing through the central processing unit (CPU).

[0035] <4> Queue: A data structure that manages and stores data elements in a first-in-first-out (FIFO) order.

[0036] <5> JPEG header information: Metadata contained in a JPEG image file, such as the image's width, height, and color depth.

[0037] <6> Compression format: refers to a method of reducing the amount of data through a specific algorithm so that it takes up less space when stored and transmitted.

[0038] <7> Resolution: The level of detail in an image, usually expressed as the number of pixels horizontally and vertically.

[0039] <8> Macroblock encoding and decoding operation: The encoding or decoding operation performed on the macroblocks in the image during the video encoding process.

[0040] <9> Huffman decoding: An algorithm based on Huffman coding, used to recover Huffman compressed data.

[0041] <10> DCT coefficient: The coefficient obtained after discrete cosine transform (DCT) processing, which represents the components of the signal at different frequencies.

[0042] <11> Quantization table: A set of standards used to determine how to quantize DCT coefficients in image compression, affecting compression efficiency and image quality.

[0043] <12> Discrete Cosine Transform: A mathematical transform used to convert a signal into the frequency domain, often used in image and video compression.

[0044] <13> Register: A small storage unit in a computer that is used to quickly access data, usually located inside the CPU.

[0045] <14> LPDDR4: Low-power double data rate fourth-generation memory, designed for mobile devices, provides high-speed data transfer and low power consumption.

[0046] <15> DDR5: Double data rate fifth generation memory, providing higher bandwidth and lower power consumption than DDR4, and is used in high-performance computing and gaming devices.

[0047] <16> DRAM (Dynamic Random Access Memory): A type of memory that stores data via capacitors and needs to be refreshed periodically to retain its contents.

[0048] <17> eMMC (embedded Multimedia Card): A storage solution that integrates storage and controller, commonly used in smartphones, tablets and other embedded devices.

[0049] To facilitate understanding of the embodiments of the present application, first Figure 1 Detailed description. Figure 1 The flowchart of a JPEG image encoding and decoding method based on MCU in an embodiment of the present invention is shown. The JPEG image encoding and decoding method based on MCU in this embodiment mainly includes the following steps:

[0050] Step S11: Obtain one or more encoding and decoding tasks.

[0051] In one embodiment of the present application, the process of obtaining one or more encoding and decoding tasks also performs the following steps: creating a task queue for storing the encoding and decoding tasks, inserting the obtained encoding and decoding tasks into the tail of the queue in sequence, obtaining tasks from the head of the queue in sequence in a first-in-first-out order, and performing encoding operations or decoding operations.

[0052] Figure 2 The flowchart of executing multiple encoding and decoding tasks in one embodiment of the present application is shown. The encoding and decoding tasks included in the figure include: JPEG_JOB0, JPEG_JOB1, JPEG_JOB2, JPEG_JOB3, JPEG_JOB4 and JPEG_JOB5. The task queue is managed in a first-in-first-out (FIFO) manner, where JPEG_JOB2 represents the encoding and decoding task being processed; JPEG_JOB3 and JPEG_JOB4 represent the encoding and decoding tasks waiting to be processed; and JPEG_JOB0 and JPEG_JOB1 represent the completed processing work. The scheduling of each encoding and decoding task and the management of the task queue are implemented by the JPEG hardware peripherals of the MCU controller.

[0053] In one embodiment of the present application, after the current encoding and decoding task is completed, the following steps are also performed: when the task queue is not empty, the next encoding and decoding task is read from the task queue, and the operation type of the next encoding and decoding task is determined; if the operation type of the next encoding and decoding task is an encoding task, a decoding operation is performed based on the header information of the JPEG code stream data to be decoded; otherwise, an encoding operation is performed based on the image data to be encoded.

[0054] Figure 3The following is a flow chart showing the process of task queue interruption processing in one embodiment of the present application. Figure 3 The interrupt processing flow can realize the switching between the previous and next encoding and decoding tasks in the above task queue. Figure 3 As shown in the figure, after the JPEG hardware peripheral completes the current encoding and decoding task, it will notify the MCU controller, which will trigger the interrupt processing flow and send a task completion notification to the user layer through the callback function. Then, the MCU controller automatically obtains a new task from the waiting queue. If a new task is obtained, the corresponding hardware parameters are configured according to the task type (encoding or decoding) and the processing is started.

[0055] Exemplarily, for a decoding task, the decoding JOB includes the input JPEG format information (such as JPEG header information) and the register configuration parameters corresponding to the decoded output image information, the original JPEG code stream data, the memory address for storing the output image data after decoding, and the running status information of the encoding and decoding task. The MCU controller will configure the hardware registers of the decoder and start the decoder based on this information; if it is an encoding task, the hardware registers of the encoder will be configured according to the encoding settings and the encoder will be started. If no new task is obtained, the current processing flow is terminated. The entire process above is repeated until all queue tasks are processed.

[0056] It should be explained that the above callback function is a function called through a function pointer so that it can be automatically called when a specific event occurs. Specifically, when the current encoding and decoding task of the JPEG hardware is completed, the callback function will automatically notify the user layer, and there is no need for the user to actively query the task status. In this embodiment, the automation of task processing is achieved through a highly automated interrupt and callback mechanism, which significantly reduces user intervention and improves the overall efficiency of the system. At the same time, a queue management method is adopted to ensure that JPEG hardware resources are fully utilized and idle waiting is avoided. The real-time nature of the interrupt processing mechanism ensures that the processing of the next task can be started in time after the task is completed, thereby optimizing the workflow. In addition, when there are no tasks to be processed, the system can automatically end the processing to avoid unnecessary power consumption.

[0057] Step S12: For each encoding and decoding task, the image data to be encoded or the JPEG code stream data to be decoded is read from the memory space according to the current encoding and decoding task.

[0058] In an embodiment of the present application, the field information of the encoding and decoding task includes: one or more of configuration parameters, storage address, operation type and running status.

[0059] In this embodiment, the effective management and monitoring of the overall life cycle of each encoding and decoding task is achieved through structured field information. Among them, the configuration parameters are specific settings that affect the encoding and decoding. For example, the configuration parameters include but are not limited to: compression format (such as H.264, HEVC, etc.), resolution (such as 1080p, 4K, etc.), etc. Different configuration parameters can be used to achieve different encoding and decoding processing; the storage address is the storage location of the data to be processed in the memory space and the write location of the processed data in the memory space during the encoding and decoding process; the operation type includes JPEG encoding operation and JPEG decoding operation; the running status records the execution status of the encoding and decoding task through a dynamic field, and the running status includes but is not limited to the start time, end time, encoding and decoding progress and whether the encoding and decoding result is successful, etc., so as to monitor the whole process of the encoding and decoding task in detail through more time period information.

[0060] In one embodiment of the present application, the MCU controller reads the image data to be encoded or the JPEG stream data to be decoded from the memory through the DMA controller according to the storage address configured in the current task. This step ensures that the required input data is obtained from the correct memory location. Next, the MCU controller checks the operation type of the current task to determine whether to perform an encoding or decoding operation. According to the operation type, the MCU controller configures the register parameters of the corresponding encoding or decoding module. For example, the encoding module is configured with parameters such as encoding format and resolution, and the decoding module is configured with JPEG decoding parameters, etc. Finally, the MCU controller sends the configured input data to the encoding or decoding module for processing.

[0061] Step S13: performing an encoding operation on the image data to be encoded to generate encoded JPEG code stream data, or performing a decoding operation on the JPEG code stream data to be decoded to generate decoded image data.

[0062] In one embodiment of the present application, if the operation type of the current encoding and decoding task is an encoding operation, the following steps are executed: the running state of the current encoding and decoding task is set to processing; the RGB image to be encoded is read in the memory space according to the storage address of the current encoding and decoding task; an RGB2YUV conversion operation is performed on the RGB image to be encoded to generate a corresponding YUV image to be encoded; and a macroblock encoding operation is performed on the YUV image to be encoded to generate corresponding encoded JPEG code stream data; the encoded JPEG code stream data is stored in the memory space, and the task running state of the current encoding and decoding task is set to processing completed.

[0063] Preferably, the RGB image to be encoded is sequentially converted to YUV by macroblock and encoded each time a macroblock is converted to a color space, so as to generate the corresponding encoded JPEG code stream data. This method shows that the encoding process is performed in batches, that is, the entire image is processed step by step by macroblocks (such as 16x16 or 8x8 pixel blocks) instead of processing the entire image at one time. Specifically, the system will first convert the RGB data of a macroblock into the YUV color space, and then immediately encode the macroblock to generate the corresponding JPEG code stream data. This batch processing method not only reduces the occupancy of memory resources (no need to store the intermediate data of the entire image at one time), but also improves processing efficiency, and is particularly suitable for resource-constrained embedded systems or real-time processing scenarios.

[0064] In this embodiment, the macroblock encoding operation includes the following steps: dividing the YUV image to be encoded into a plurality of 8x8 pixel macroblocks, and applying discrete cosine transform (DCT) to the Y, U and V components of each macroblock, converting them from the spatial domain to the frequency domain, and separating high-frequency and low-frequency information. Subsequently, the coefficients after the DCT transformation are quantized according to a preset quantization table, the quantized coefficients are up-encoded through Huffman coding, and the encoded JPEG code stream is stored in the memory space.

[0065] In one embodiment of the present application, the encoding operation process includes the following steps: the MCU performs parameter configuration operations on the relevant registers of the input DMA unit, and the types of configuration parameters include but are not limited to: source address (i.e., the peripheral register address of the data source), destination address (i.e., the memory address where the data will be stored), amount of data transferred (usually in bytes or words), transfer mode (such as single transfer or cyclic transfer) and transfer trigger conditions (such as peripheral request or timer trigger), etc., and after the configuration is completed, the DMA data transmission is triggered by setting the start peripheral of the DMA controller or the hardware signal of the peripheral.

[0066] Figure 4A flow chart of the encoding operation in one embodiment of the present application is shown. In this embodiment, the input DMA unit reads the original image data from the memory space and stores it in the input buffer of the MCU controller, wherein the size of the single read data is determined according to the size of the buffer space allocated inside the MCU controller. Then the MCU controller determines whether the read original image data is in RGB format. If the input original image data is in RGB format, the image data in RGB format is input into the RGB2YUV converter to be converted into YUV format and stored in the output buffer until the total size of the read data reaches the configuration value of the register. Subsequently, the macroblock processing unit reads the YUV image data from the buffer for encoding, and stores the encoded JPEG code stream data in the output buffer. The output DMA is responsible for obtaining the JPEG code stream data from the output buffer and writing it into the memory. When all image encoding is completed and written into the memory, the encoding operation is stopped and an encoding completion interrupt is generated.

[0067] Figure 5 The software and hardware interface diagram of the encoding operation in one embodiment of the present application is shown. The software system includes two parts: encoding JPEG_JOB and JPEG hardware. Encoding JPEG_JOB contains multiple parameter fields, including configuration parameters, original images, compressed data streams and running status information. The JPEG hardware is responsible for parameter configuration of input DMA registers, RGB2YUV registers, encoding macroblock processing registers and output DMA registers according to the configuration parameters. During the execution process, the JPEG hardware inputs the original image to the input DMA unit. The unit transmits the original image to the RGB2YUV unit to convert the RGB image into a YUV image. Afterwards, the YUV image is sent to the encoding macroblock for processing, thereby generating an encoded JPEG code stream. Finally, the output DMA unit fills the JPEG code stream into the compressed data stream field of encoding JPEG_JOB. At the same time, the running status information field in encoding JPEG_JOB is used to monitor the execution status of the current task of the JPEG hardware.

[0068] Figure 6 The flowchart of executing the encoding task in the task queue in one embodiment of the present application is shown. When creating a new encoding task, the acquired original image data must first be filled into the corresponding fields of the task, and the required encoding format and image quality parameters must be set. After that, it should be checked whether there is an encoding and decoding task being executed in the current state, specifically, whether the task queue is empty. If the task queue is empty, the new encoding task is designated as the current working task, and the necessary register configuration and decoder startup operations are performed. If the task queue is not empty, the newly created encoding and decoding task is added to the waiting queue for subsequent execution.

[0069] Further, Figure 7 The present invention shows a schematic diagram of the configuration flow of the encoder hardware registers for each encoding task in an embodiment of the present invention. First, the encoder hardware registers are configured according to the JPEG header information. When processing the input image data format, if the input format is not YUV but RGB format, the RGB to YUV converter needs to be enabled. At the same time, a series of parameters are set, including but not limited to the configuration of the quantization table, the configuration of the Huffman table, the setting of the macroblock format parameters, and the configuration of the input and output DMA parameters. Finally, the encoding operation is started to complete the preparation of the encoding process.

[0070] Among them, the quantization table configuration is used to balance image quality and data size, and redundant information is reduced by quantizing the DCT coefficients in the frequency domain. The Huffman table configuration realizes the effective encoding of quantized data, and reduces the number of data storage bits through variable-length coding. The macroblock format parameter setting involves the image block method to improve processing and encoding efficiency. The input and output DMA parameter configuration ensures high-speed data transmission between the memory and the encoder. These configurations work together to ensure that the encoding task is executed efficiently and accurately.

[0071] In one embodiment of the present application, if the operation type is a decoding operation, the following steps are performed: the task running state of the current encoding and decoding task is set to processing; the JPEG code stream data to be decoded is read from the memory space based on the storage address; a macroblock decoding operation is performed on the decoded JPEG code stream data to generate decoded YUV image data; and a YUV2RGB conversion operation is performed on the decoded YUV image data to generate decoded RGB image data; the decoded RGB image data is stored in the memory space, and the task running state of the current encoding and decoding task is set to processing completed.

[0072] In this embodiment, the macroblock decoding operation includes: reading the encoded JPEG code stream data from the memory; performing a Huffman decoding operation on the JPEG code stream data to restore the quantized DCT (Discrete Cosine Transform) coefficients; then, performing inverse quantization according to the quantization table to obtain the original DCT frequency domain coefficients; performing an inverse discrete cosine transform (IDCT) on each 8x8 macroblock to convert the frequency domain data back to spatial domain pixel values, and combining the macroblocks in the original order to form a complete YUV image; finally, converting the reconstructed YUV image into RGB format.

[0073] Figure 8A flowchart of the decoding operation in one embodiment of the present application is shown. In this embodiment, the input DMA unit reads the JPEG code stream data from the memory and stores it in the input buffer of the MCU controller. The size of the data read at a single time is determined by the buffer space of the MCU controller. Subsequently, the MCU controller decodes the JPEG code stream and determines whether the decoded image data is in RGB format. If it is in RGB format, it is input to the RGB2YUV converter for YUV format conversion and stored in the output buffer until the total size of the read data reaches the register configuration value. Then, the decoded RGB image data is stored in the output buffer. The output DMA is responsible for writing the decoded image data in the output buffer to the memory. If all data has been read, the task ends; otherwise, continue to read the next JPEG code stream.

[0074] Fig. 9 The software and hardware interface diagram in an embodiment of the present application is shown, and the software system includes two parts: decoding JPEG_JOB and JPEG hardware. Decoding JPEG_JOB contains multiple parameter fields, including configuration parameters, original images, compressed data streams and running status information. The JPEG hardware is responsible for parameter configuration of input DMA registers, RGB2YUV registers, decoding macroblock processing registers and output DMA registers according to the configuration parameters. During execution, the JPEG hardware inputs the JPEG code stream to the input DMA unit. The unit transmits the original image to the decoding macroblock processing to generate a YUV image. The YUV2RGB unit is then used to convert the YUV image into an RGB image. The output DMA unit fills the RGB image into the original image data field of the encoding JPEG_JOB. At the same time, the running status information field in the encoding JPEG_JOB is used to monitor the execution status of the current task of the JPEG hardware.

[0075] Fig.10 The flowchart of executing a decoding task in a task queue in one embodiment of the present application is shown. The following steps are included: obtaining the JPEG code stream data to be decoded from the task queue, filling the JPEG file into the field of the current task, and setting the corresponding decoding format and image quality parameters. Determine whether there is an ongoing decoding task. If the task queue is empty, designate the current decoding task as the task being executed, and perform necessary register configuration and decoder startup; if the task queue is not empty, add the new decoding task to the waiting queue so that it can be executed after the previous task is completed.

[0076] Fig.11The configuration flow diagram of the decoder hardware registers for each decoding task in one embodiment of the present application is shown. The decoder hardware registers are configured according to the JPEG header information. When processing the input image data format, if the input format is not YUV but RGB format, the RGB to YUV converter needs to be enabled. At the same time, a series of parameters are set, including but not limited to the configuration of the quantization table, the configuration of the Huffman table, the setting of the macroblock format parameters, and the configuration of the input and output DMA parameters. Finally, the decoding operation is started to complete the preparation of the decoding process.

[0077] It is worth noting that in one embodiment of the present application, a segmented processing strategy is adopted for a single encoding and decoding task of encoding and decoding. Specifically, only data within the cache capacity is processed each time to achieve efficient image encoding and decoding. During the encoding process, the segmented YUV image is divided into macroblocks of 8x8 pixels and processed block by block. First, a discrete cosine transform (DCT) is performed on each macroblock to convert the data from the spatial domain to the frequency domain. Subsequently, the coefficients after the DCT transformation are quantized according to the quantization table to reduce the number of data bits. Next, the quantized coefficients are entropy encoded (such as Huffman encoding) to generate segmented JPEG code stream data and store it in the target memory.

[0078] Similarly, during the decoding process, the JPEG code stream is read in segments in a similar manner and decoded segment by segment. First, entropy decoding is performed on the JPEG code stream to restore the quantized DCT coefficients. Then, the decoded DCT coefficients are dequantized according to the quantization table to obtain the original DCT frequency domain coefficients. Next, an inverse discrete cosine transform (IDCT) is performed on each 8x8 macroblock to restore the frequency domain data to spatial domain pixel values. Finally, the decoded macroblocks are combined into a complete YUV image in the original order and converted to an RGB image output.

[0079] Through the segmented processing strategy, only the data of the current processing segment needs to be cached, and there is no need to occupy a large memory to buffer the entire image, which significantly reduces memory usage and avoids the memory overhead problem when processing large-size images. The present application realizes the pipeline processing of "reading, converting, encoding / decoding, and outputting", which greatly improves the encoding and decoding efficiency and system response speed. It has strong adaptability and can support large-size or high-resolution image processing, breaking through the limitations of memory and hardware resources. By reducing the waiting time of intermediate data, the utilization rate of storage devices and computing resources is improved, and the overall delay is effectively reduced. In addition, the present application is applicable to both encoding and decoding processes, unifies the encoding and decoding process, and ensures high efficiency and real-time performance.

[0080] Step S14: storing the encoded JPEG code stream data or the decoded image data in the memory space.

[0081] In one embodiment of the present application, the memory space includes an on-chip SRAM in the SOC system for temporary storage of JPEG encoding or decoding data. The encoded JPEG code stream data or the decoded image data will be directly written into the SRAM for quick access or processing by subsequent modules. The on-chip SRAM is fast and suitable for applications that support high frame rates (such as real-time video streaming), but is limited by capacity and is usually only suitable for storing smaller images or processing image blocks.

[0082] In addition, the DMA function can be combined to store the JPEG encoded code stream or decoded image data directly in an external DRAM (such as LPDDR4 or DDR5). External DRAM is typically used to store large image data and is suitable for high-resolution images or batch storage of JPEG files. Using DMA can reduce the CPU burden and achieve high-speed handling and storage through peripherals, but the access latency of DRAM is relatively high compared to SRAM. In actual implementation, on-chip RAM and external storage (such as DRAM or eMMC) may be used in combination. For example, the JPEG code stream can be temporarily stored in the on-chip SRAM and then transferred to the external memory via DMA to optimize storage efficiency.

[0083] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" represent examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0084] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.

[0085] Fig.12 1 is a schematic block diagram of a JPEG codec controller 1200 provided in an embodiment of the present application. Fig.12As shown, the controller includes an input DMA unit 1201 , an encoding unit 1202 , a decoding unit 1203 , an output DMA unit 1204 and a register module 1205 .

[0086] The input DMA unit 1201 is used to obtain one or more encoding and decoding tasks; for each encoding and decoding task, the image data to be encoded or the JPEG code stream data to be decoded is read from the memory space according to the current encoding and decoding task.

[0087] The encoding unit 1202 is used to perform an encoding operation on the image data to be encoded to generate encoded JPEG code stream data.

[0088] The decoding unit 1203 is used to perform the decoding operation on the JPEG code stream data to be decoded to generate decoded image data.

[0089] The output DMA unit 1204 is used to store the encoded JPEG code stream data or the decoded image data into the memory space.

[0090] The register module 1205 is used to perform parameter configuration operations on the input DMA unit, the encoding unit, the decoding unit and the output DMA unit respectively.

[0091] Fig.13The hardware system architecture of the JPEG codec controller in one embodiment of the present invention is shown. The JPEG codec controller is integrated inside the MCU controller and is one of the peripherals of the MCU. It includes: a memory access bus and a register access bus. The memory access bus includes: an input DMA, an input buffer, a macroblock processing, an output buffer, an output DMA, an RGB2YUV converter, a YUV2RGB converter and a register control module. The memory access bus (such as AXI / AHB) is used to access the system memory; the input DMA is responsible for reading the original image data in the encoding mode and reading the JPEG compressed data stream in the decoding mode. If the data read is in RGB format, the RGB2YUV converter converts it to YUV format and stores it in the input buffer. If it is in YUV format, it is directly stored. The input buffer provides data to the macroblock processing unit so that the macroblock processing unit can encode the YUV image into a JPEG code stream when encoding and decode the JPEG code stream into a YUV image when decoding. The output buffer temporarily stores the processing results, stores the JPEG code stream in the encoding mode, and stores the YUV image in the decoding mode. If the RGB format is finally required, the YUV2RGB converter converts YUV into RGB; it is automatically disabled when no conversion is required. The output DMA is responsible for writing the output buffer data to the memory. The register access bus manages the hardware process uniformly through registers, including starting, stopping and configuration of each module, and adopts time-sharing multiplexing technology, allowing multiple modules to share the same set of physical signal lines, and transmit register access requests in turn through time slice rotation, thereby effectively avoiding the monopoly and waste of bus resources, optimizing circuit resources, reducing hardware complexity and power consumption, and improving the flexibility and scalability of the system.

[0092] It should be understood that the specific process of each module executing the above corresponding steps has been described in detail in the above method embodiment, and for the sake of brevity, it will not be repeated here.

[0093] It should also be understood that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application may be integrated into a processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0094] Fig.14 is a schematic block diagram of an electronic terminal provided in an embodiment of the present application. Fig.14As shown, the electronic terminal includes: at least one processor 1401, a memory 1402, at least one network interface 1403 and a user interface 1405. The various components in the device are coupled together through a bus system 1404. It can be understood that the bus system 1404 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 1404 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Fig.14 In the specification, various buses are labeled as bus systems.

[0095] The user interface 1405 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.

[0096] It is understood that the memory 1402 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable categories of memory.

[0097] The memory 1402 in the embodiment of the present invention is used to store various categories of data to support the operation of the electronic terminal 1400. Examples of these data include: any executable program for operating on the electronic terminal 1400, such as an operating system 14021 and an application 14022; the operating system 14021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 14022 may include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The MCU-based JPEG image encoding and decoding method provided in the embodiment of the present invention may be included in the application 14022.

[0098] The method disclosed in the above embodiment of the present invention can be applied to the processor 1401, or implemented by the processor 1401. The processor 1401 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 1401 or the instruction in the form of software. The above processor 1401 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 1401 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor 1401 can be a microprocessor or any conventional processor, etc. In combination with the steps of the accessory optimization method provided in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0099] In an exemplary embodiment, the electronic terminal 1400 may be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD) to execute the aforementioned method.

[0100] According to the method provided in the embodiments of the present application, the present application also provides a computer program product, which includes: computer program code, when the computer program code is run on a computer, the computer executes the MCU-based JPEG image encoding and decoding method of any embodiment shown in the above embodiments.

[0101] According to the method provided in the embodiments of the present application, the present application also provides a computer-readable storage medium, which stores a program code. When the program code runs on a computer, the computer executes the MCU-based JPEG image encoding and decoding method of any one of the embodiments shown above.

[0102] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program and / or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process and / or an execution thread, and a component may be located on a computer and / or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may, for example, communicate through local and / or remote processes according to signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems through signals).

[0103] Those of ordinary skill in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0104] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0105] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0106] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0107] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0108] In the above embodiments, the functions of each functional unit can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When loading and executing computer program instructions (programs) on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. that contains one or more available media integrated. Available media may be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., high-density digital video discs (DVDs), or semiconductor media (e.g., solid state disks (SSDs)).

[0109] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0110] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0111] In summary, the present application provides a JPEG image encoding and decoding method based on MCU, a JPEG encoding and decoding controller, a medium, a program product and a terminal. By adding a JPEG encoding and decoding controller with an RGB and YUV format conversion module to the MCU controller in hardware and adopting a queue-based management method in software, the problems of CPU computing power overload and excessive memory resource occupation caused by RGB and YUV format conversion in the existing JPEG encoding and decoding process are solved, and the CPU and memory requirements are significantly reduced and the system efficiency is improved. The method of hardware and software collaborative optimization reduces the dependence on high-performance processors and large-capacity memories, allowing the MCU to select a lower-cost, simpler hardware architecture, and further reduces the overall system cost. It is particularly suitable for MCU application scenarios based on RGB format, greatly optimizes the implementation of the JPEG function, and has important practical value. Therefore, the present application effectively overcomes the various shortcomings in the prior art and has a high industrial utilization value.

[0112] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A JPEG image encoding and decoding method based on MCU, characterized in that: The method is applied to an MCU controller, and the method comprises: Get one or more encoding and decoding tasks; For each encoding and decoding task, read the image data to be encoded or the JPEG code stream data to be decoded from the memory space according to the current encoding and decoding task; Performing an encoding operation on the image data to be encoded to generate encoded JPEG code stream data, or performing the decoding operation on the JPEG code stream data to be decoded to generate decoded image data; The encoded JPEG code stream data or the decoded image data is stored in the memory space.

2. The MCU-based JPEG image encoding and decoding method according to claim 1, characterized in that: The process of acquiring one or more encoding and decoding tasks also executes the following steps: creating a task queue for storing the encoding and decoding tasks, inserting the acquired encoding and decoding tasks into the tail of the queue in sequence, acquiring tasks from the head of the queue in sequence in a first-in-first-out order, and performing encoding operations or decoding operations.

3. The MCU-based JPEG image encoding and decoding method according to claim 2, characterized in that: After the current encoding and decoding task is completed, the following steps are also performed: When the task queue is not empty, reading the next encoding and decoding task from the task queue, and determining the operation type of the next encoding and decoding task; If the operation type of the next encoding and decoding task is an encoding task, a decoding operation is performed based on the header information of the JPEG code stream data to be decoded; Otherwise, an encoding operation is performed based on the image data to be encoded.

4. The MCU-based JPEG image encoding and decoding method according to claim 1, characterized in that: The field information of the encoding and decoding task includes: one or more of configuration parameters, storage address, operation type and running status.

5. According to the MCU-based JPEG image encoding and decoding method of claim 4, if the operation type of the current encoding and decoding task is an encoding operation, the following steps are performed: Set the running status of the current encoding and decoding task to processing; Reading the RGB image to be encoded in the memory space according to the storage address of the current encoding and decoding task; Performing an RGB2YUV conversion operation on the RGB image to be encoded to generate a corresponding YUV image to be encoded; and performing a macroblock encoding operation on the YUV image to be encoded to generate corresponding encoded JPEG code stream data; The encoded JPEG code stream data is stored in the memory space, and the task running state of the current encoding and decoding task is set to be processed completed.

6. According to the MCU-based JPEG image encoding and decoding method of claim 4, if the operation type is a decoding operation, the following steps are performed: Set the task running status of the current encoding and decoding task to processing; Reading the JPEG code stream data to be decoded from the memory space based on the storage address; Performing a macroblock decoding operation on the decoded JPEG code stream data to generate decoded YUV image data; and performing a YUV2RGB conversion operation on the decoded YUV image data to generate decoded RGB image data; The decoded RGB image data is stored in the memory space, and the task running state of the current encoding and decoding task is set to be processed completed.

7. A JPEG codec controller, characterized in that: The JPEG codec controller comprises: The input DMA unit is used to obtain one or more encoding and decoding tasks; for each encoding and decoding task, the image data to be encoded or the JPEG code stream data to be decoded is read from the memory space according to the current encoding and decoding task; The encoding unit is used to perform an encoding operation on the image data to be encoded to generate encoded JPEG code stream data. The decoding unit is used to perform the decoding operation on the JPEG code stream data to be decoded to generate decoded image data. An output DMA unit, used for storing the encoded JPEG code stream data or the decoded image data into the memory space; The register module is used to perform parameter configuration operations on the input DMA unit, the encoding unit, the decoding unit and the output DMA unit respectively.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the MCU-based JPEG image encoding and decoding method described in any one of claims 1 to 6 is implemented.

9. A computer program product, characterized in that The computer program product includes computer program codes, and when the computer program codes are executed on a computer, the computer is enabled to implement the MCU-based JPEG image encoding and decoding method as claimed in any one of claims 1 to 6.

10. An electronic terminal comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the MCU-based JPEG image encoding and decoding method according to any one of claims 1 to 6.

Images