Apparatus and method for outputting image data, storage medium, and electronic device

Abstract

This disclosure provides apparatus and methods, storage media, and electronic devices for outputting image data. The apparatus includes a first parameter setter, an image signal processor, and an encoder, wherein the first parameter setter is configured to set first configuration parameters for the image signal processor based on complexity associated with an input image frame; the image signal processor is configured to perform image signal processing on at least one image frame based on the first configuration parameters to generate processed image data; and the encoder is configured to encode the processed image data to generate encoded image data. The apparatus and methods, storage media, and electronic devices for outputting image data according to this disclosure adjust the image signal processor parameters and encoder parameters based on the complexity associated with the input image frame, thereby effectively improving the overall performance of image processing and encoding compression.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to apparatus and methods for outputting image data, storage media and electronic devices. Background Technology

[0002] In image processing, images acquired by image acquisition devices are typically processed by an Image Signal Processor (ISP) before being input into an encoder. When the target bitrate is constrained or the application scenario is complex, the encoder's encoding status can be fed back to the ISP, or the encoder's encoding behavior can be statistically analyzed within the ISP. This allows the ISP and encoder to work together to adjust parameters, thereby improving image processing performance under conditions of limited target bitrate or complex application scenarios. Summary of the Invention

[0003] Embodiments of this disclosure provide apparatus and methods for outputting image data, storage media, and electronic devices, which adjust image signal processor parameters and / or encoder parameters according to the complexity associated with input image frames, thereby effectively improving the overall performance of image processing and encoding compression.

[0004] In a first aspect, this disclosure provides an apparatus for outputting image data. The apparatus includes a first parameter setter, an image signal processor, and an encoder, wherein the first parameter setter is configured to set first configuration parameters for the image signal processor based on complexity associated with an input image frame; the image signal processor is configured to perform image signal processing on at least one image frame based on the first configuration parameters to generate processed image data; and the encoder is configured to encode the processed image data to generate encoded image data.

[0005] In one embodiment of this disclosure, the image signal processor is further configured to calculate the current complexity of the current image frame and provide the current complexity to the first parameter setter; the first parameter setter is configured to set a first configuration parameter for the image signal processor corresponding to at least one subsequent image frame based on the current complexity.

[0006] In one embodiment of this disclosure, the encoder is further configured to calculate the current complexity of the currently processed image data corresponding to the current image frame, and provide the current complexity to the first parameter setter; the first parameter setter is configured to set a first configuration parameter for the image signal processor corresponding to at least one subsequent image frame based on the current complexity.

[0007] In one embodiment of this disclosure, an image signal preprocessor is further included, the image signal preprocessor being configured to calculate the current complexity of the current image frame and provide the current complexity to the first parameter setter; the first parameter setter is configured to set a first configuration parameter corresponding to the current image frame or the current image frame and at least one subsequent image frame for the image signal processor based on the current complexity.

[0008] In one embodiment of this disclosure, an image signal preprocessor is further included, the image signal preprocessor being configured to calculate a first complexity of the current image frame and provide the first complexity to the first parameter setter; the image signal preprocessor is further configured to calculate a second complexity of the current image frame and provide the second complexity to the first parameter setter; the encoder is further configured to calculate a third complexity of the currently processed image data corresponding to the current image frame and provide the third complexity to the first parameter setter; the first parameter setter is configured to set the first configuration parameter according to at least two of the first complexity, the second complexity, and the third complexity.

[0009] In one embodiment of this disclosure, a second parameter setter is further included, the second parameter setter being configured to set second configuration parameters for the encoder; the encoder is configured to encode the processed image data based on the second configuration parameters to generate the encoded image data.

[0010] In one embodiment of this disclosure, the image signal processor is further configured to calculate the current complexity of the current image frame and provide the current complexity to the second parameter setter; the second parameter setter is configured to set a second configuration parameter for the encoder corresponding to the currently processed image data or the currently processed image data and at least one subsequent processed image data according to the current complexity.

[0011] In one embodiment of this disclosure, the encoder is further configured to calculate the current complexity of the currently processed image data corresponding to the current image frame, and provide the current complexity to the second parameter setter; the second parameter setter is configured to set a second configuration parameter for the encoder corresponding to at least one subsequently processed image data based on the current complexity.

[0012] In one embodiment of this disclosure, an encoding preprocessor is further included, the encoding preprocessor being configured to calculate the current complexity of the currently processed image data corresponding to the current image frame, and to provide the current complexity to the second parameter setter; the second parameter setter is configured to set a second configuration parameter for the encoder based on the current complexity, corresponding to the currently processed image data or the current processed image data and at least one subsequent processed image data.

[0013] In one embodiment of this disclosure, the system further includes an encoding preprocessor configured to calculate a first complexity of currently processed image data corresponding to a current image frame and provide the first complexity to a second parameter setter; the image signal processor is further configured to calculate a second complexity of currently processed image data corresponding to a current image frame and provide the second complexity to the second parameter setter; the encoder is further configured to calculate a third complexity of currently processed image data corresponding to a current image frame and provide the third complexity to the second parameter setter; the second parameter setter is configured to set a second configuration parameter for the encoder based on at least two of the first complexity, the second complexity, and the third complexity.

[0014] In a second aspect, this disclosure provides a method for outputting image data. The method includes: setting first configuration parameters for an image signal processor based on complexity associated with an input image frame by a first parameter setter; performing image signal processing on at least one image frame according to the first configuration parameters by the image signal processor to generate processed image data; and encoding the processed image data by an encoder to generate encoded image data.

[0015] In one embodiment of this disclosure, setting a first configuration parameter for an image signal processor based on a complexity associated with an input image frame by a first parameter setter includes: the image signal processor calculating a current complexity of the current image frame and providing the current complexity to the first parameter setter; and the first parameter setter setting a first configuration parameter for the image signal processor corresponding to at least one subsequent image frame based on the current complexity.

[0016] In one embodiment of this disclosure, setting a first configuration parameter for an image signal processor based on complexity associated with an input image frame by a first parameter setter includes: the encoder calculating the current complexity of the currently processed image data corresponding to the current image frame and providing the current complexity to the first parameter setter; and the first parameter setter setting a first configuration parameter for the image signal processor corresponding to at least one subsequent image frame based on the current complexity.

[0017] In one embodiment of this disclosure, setting a first configuration parameter for an image signal processor based on a complexity associated with an input image frame by a first parameter setter includes: calculating the current complexity of the current image frame by an image signal preprocessor and providing the current complexity to the first parameter setter; and setting a first configuration parameter for the image signal processor corresponding to the current image frame or the current frame and at least one subsequent image frame according to the current complexity.

[0018] In one embodiment of this disclosure, setting a first configuration parameter for an image signal processor based on a complexity associated with an input image frame by a first parameter setter includes: calculating a first complexity of the current image frame by the image signal preprocessor and providing the first complexity to the first parameter setter; calculating a second complexity of the current image frame by the image signal processor and providing the second complexity to the first parameter setter; calculating a third complexity of the currently processed image data corresponding to the current image frame by the encoder and providing the third complexity to the first parameter setter; and setting the first configuration parameter by the first parameter setter based on at least two of the first complexity, the second complexity, and the third complexity.

[0019] In one embodiment of this disclosure, the method further includes: setting a second configuration parameter for the encoder by a second parameter setter; and encoding the processed image data by the encoder to generate encoded image data, which includes: encoding the processed image data by the encoder based on the second configuration parameter to generate the encoded image data.

[0020] In one embodiment of this disclosure, setting a second configuration parameter for the encoder by a second parameter setter includes: calculating the current complexity of the current image frame by the image signal processor and providing the current complexity to the second parameter setter; and setting a second configuration parameter for the encoder corresponding to the currently processed image data or the currently processed image data and at least one subsequent processed image data according to the current complexity.

[0021] In one embodiment of this disclosure, setting a second configuration parameter for the encoder by a second parameter setter includes: the encoder calculating the current complexity of the currently processed image data corresponding to the current image frame and providing the current complexity to the second parameter setter; and the second parameter setter setting a second configuration parameter for the encoder corresponding to at least one subsequently processed image data based on the current complexity.

[0022] In one embodiment of this disclosure, setting a second configuration parameter for the encoder by a second parameter setter includes: calculating the current complexity of the currently processed image data corresponding to the current image frame by an encoding preprocessor, and providing the current complexity to the second parameter setter; and setting a second configuration parameter for the encoder corresponding to the currently processed image data or the current processed image data and at least one subsequent processed image data according to the current complexity.

[0023] In one embodiment of this disclosure, setting a second configuration parameter for the encoder by a second parameter setter includes: calculating a first complexity of the currently processed image data corresponding to the current image frame by an encoding preprocessor and providing the first complexity to the second parameter setter; calculating a second complexity of the currently processed image data corresponding to the current image frame by the image signal processor and providing the second complexity to the second parameter setter; calculating a third complexity of the currently processed image data corresponding to the current image frame by the encoder and providing the third complexity to the second parameter setter; and setting the second configuration parameter for the encoder by the second parameter setter based on at least two of the first complexity, the second complexity, and the third complexity.

[0024] In a third aspect, this disclosure provides an electronic device. The electronic device includes: a memory configured to store a computer program and image data required to output an input image frame as encoded image data; and a processor communicatively connected to the memory and configured to invoke the computer program to perform the methods described above.

[0025] In a fourth aspect, this disclosure provides a computer-readable storage medium having a computer program stored thereon and image data required to output an input image frame as encoded image data. The computer program is executed by a processor to implement the above-described method.

[0026] According to embodiments of this disclosure, the encoding status is evaluated by the complexity associated with the input image frame, and then the image signal processor parameters and encoder parameters are adjusted based on the encoding status, thereby effectively improving image processing performance. Furthermore, adjustments are made to the video source at the encoder input, achieving source-level adjustment and making the entire image processing system more efficient. In addition, it is applicable to various application scenarios, even those with low or limited target bitrates, or complex application scenarios. Attached Figure Description

[0027] Figure 1 A schematic diagram of the apparatus for outputting image data of this disclosure is shown in one embodiment.

[0028] Figure 2 This is shown as an architecture diagram of the first parameter setter in the first embodiment of the present disclosure, which sets the first configuration parameters of the image signal processor.

[0029] Figure 3 This is shown as an architecture diagram in the second embodiment where the first parameter setter in this disclosure sets the first configuration parameters of the image signal processor;

[0030] Figure 4 This is shown as an architecture diagram of the first parameter setter in this disclosure setting the first configuration parameters of the image signal processor in the third embodiment;

[0031] Figure 5 This is shown as an architecture diagram of the first parameter setter setting the first configuration parameters of the image signal processor in the fourth embodiment of this disclosure;

[0032] Figure 6 A schematic diagram of the apparatus for outputting image data of this disclosure in another embodiment is shown;

[0033] Figure 7 The diagram shows the architecture of the first embodiment in which the second parameter setter sets the second configuration parameters of the encoder in this disclosure;

[0034] Figure 8 This is a diagram showing the architecture of the second embodiment where the second parameter setter sets the second configuration parameters of the encoder in this disclosure;

[0035] Figure 9 This is a diagram showing the architecture of the third embodiment where the second parameter setter in this disclosure sets the second configuration parameters of the encoder;

[0036] Figure 10 The diagram shown is an architecture diagram of the fourth embodiment in which the second parameter setter sets the second configuration parameters of the encoder in this disclosure;

[0037] Figure 11 A flowchart is shown as an embodiment of the method for outputting image data according to this disclosure;

[0038] Figure 12 The diagram shown is a structural schematic of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0039] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. This disclosure can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure.

[0040] It should be noted that the illustrations provided in this embodiment are only schematic representations of the basic concept of this disclosure. Therefore, the drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0041] In existing technologies, when debugging an image signal processor and an encoder in a cascade configuration, the two processes are usually performed separately.

[0042] In one technical solution, the image complexity of the processed image data is received, which is obtained by an image processor connected to an encoder; a preset encoding bit of the processed image data is determined; a quantization parameter of the processed image data is obtained based on the image complexity and the preset encoding bit; and the processed image data is encoded based on the quantization parameter, which can balance the accuracy of bit rate control and the computational complexity of the encoder.

[0043] In another technical solution, the image preprocessing block is configured to: receive image data; process the received image data to provide an image comprising one or more image portions, each comprising multiple pixels; analyze the pixels in each of the one or more image portions in the image to estimate a complexity indicator of the image portion; determine metadata based on one or more estimated complexity indicators of the one or more image portions; and output the determined metadata. Furthermore, the encoder processing block is configured to: receive the metadata determined by the image preprocessing block; use the received metadata to determine the quantization level to be used when encoding the image; and encode the image using the determined quantization level. It can be seen that the above methods all adjust the quantization parameters of the encoder.

[0044] However, the above methods do not present problems when the target bit rate is relatively high or the application scenario is relatively simple. But when the target bit rate is low or limited, or the application scenario is very complex, significant problems arise. This is because when the encoder uses a low bit rate, the image signal processor will unknowingly perform complex image sharpening and contrast adjustments, making it impossible for the encoder to encode complex scenes based on the low bit rate. Conversely, when the encoder uses a high bit rate, the image signal processor will unknowingly simplify the image, resulting in insufficient sharpness, contrast, and clarity.

[0045] To address at least the aforementioned technical problems, this disclosure provides a technical solution for outputting image data. According to embodiments of this disclosure, for a cascaded image signal processor and encoder, the encoding status is evaluated by analyzing the complexity of previous image frames, and then the parameters of the image signal processor are adjusted based on the encoding status. In this way, adjustments are made from the source at the encoder input, achieving source-level adjustment, making the entire image processing system more efficient, and effectively improving the overall performance of image processing and encoding compression. Here, the current complexity includes intra-frame complexity and inter-frame complexity. Intra-frame complexity can be calculated based on the intra-frame data of the current frame, and inter-frame complexity can be calculated based on the current frame and at least one previous frame.

[0046] In the following, embodiments according to the present disclosure will be described with reference to the accompanying drawings through detailed description.

[0047] Figure 1 A schematic diagram of an apparatus for outputting image data according to an embodiment of the present disclosure is shown. Figure 1 As shown, the device includes a first parameter setter 1, an image signal processor 2, and an encoder 3.

[0048] The first parameter setter 1 is configured to set first configuration parameters for the image signal processor based on the complexity associated with the input image frame. In some embodiments, the first configuration parameters include 3D denoising parameters (3DNR), 2D denoising parameters (2DNR), contrast noise parameters (CNR), sharpening parameters, etc., and their adjustment can be either biased or precise.

[0049] The image signal processor 2 is connected to the first parameter setter 1. The image signal processor 2 is configured to perform image signal processing on at least one image frame based on the first configuration parameters to generate processed image data. In some embodiments, continuous image frames acquired by an image acquisition device are input to the image signal processor 2 for image signal processing to obtain the processed image data.

[0050] The encoder 3 is connected to the image signal processor 2. The encoder 3 is configured to encode the processed image data to generate encoded image data. In some embodiments, the encoder 3 outputs a data stream.

[0051] In embodiments according to this disclosure, the first parameter setter 1 may set the first configuration parameters for the image signal processor 2 in any of the following ways. Figures 2 to 5 A schematic diagram of a structure is shown, illustrating the setting of a first configuration parameter for an image signal processor by a first parameter setter according to an embodiment of the present disclosure.

[0052] like Figure 2 In the illustrated embodiment, the image signal processor 2 is configured to calculate the current complexity of the current image frame and provide the current complexity to the first parameter setter 1. The first parameter setter 1 is configured to set a first configuration parameter for the image signal processor 2 corresponding to at least one subsequent image frame based on the current complexity. The current complexity may include motion complexity, image texture complexity, image texture variance, etc. In some embodiments, the current complexity and the first configuration parameter are correlated. A lookup table between the current complexity and the first configuration parameter can be pre-defined. When the first configuration parameter needs to be adjusted, a matching first configuration parameter is found based on the current complexity, and the adjustment is performed based on the matching first configuration parameter.

[0053] like Figure 3 In the illustrated embodiment, the encoder 3 is further configured to calculate the current complexity of the currently processed image data corresponding to the current image frame, and provide the current complexity to the first parameter setter 1. The first parameter setter 1 is configured to set a first configuration parameter for the image signal processor 2 corresponding to at least one subsequent image frame based on the current complexity. The current complexity may include motion complexity, image texture complexity, image texture variance, encoder quantization parameter QP value, encoder structural similarity parameter ssim, etc. In some embodiments, the current complexity and the first configuration parameter are correlated. A lookup table between the current complexity and the first configuration parameter can be preset. When the first configuration parameter needs to be adjusted, a matching first configuration parameter is found based on the current complexity, and the adjustment is performed based on the matching first configuration parameter.

[0054] like Figure 4 In the illustrated embodiment, the apparatus for outputting image data according to this disclosure further includes an image signal preprocessor 4. The image signal preprocessor 4 is connected to the first parameter setter 1. The image signal preprocessor 4 is configured to calculate the current complexity of the current image frame and provide the current complexity to the first parameter setter 1. The first parameter setter 1 is configured to set a first configuration parameter corresponding to the current image frame or the current image frame and at least one subsequent image frame for the image signal processor 2 based on the current complexity. The current complexity may include motion complexity, image texture complexity, image texture variance, etc. In some embodiments, the current complexity and the first configuration parameter are correlated. A lookup table between the current complexity and the first configuration parameter can be preset. When the first configuration parameter needs to be adjusted, a matching first configuration parameter is found based on the current complexity, and the adjustment is performed based on the matching first configuration parameter.

[0055] like Figure 5 In the illustrated embodiment, the image signal preprocessor 4 is configured to calculate a first complexity of the current image frame and provide the first complexity to the first parameter setter 1. The image signal processor 2 is further configured to calculate a second complexity of the current image frame and provide the second complexity to the first parameter setter 1. The encoder 3 is further configured to calculate a third complexity of the currently processed image data corresponding to the current image frame and provide the third complexity to the first parameter setter 1.

[0056] The first parameter setter 1 is connected to the image signal processor 2, the encoder 3, and the image signal preprocessor 4. The first parameter setter 1 is configured to set the first configuration parameter based on at least two of the first complexity, the second complexity, and the third complexity. The second complexity calculated by the image signal processor 2 may include motion complexity, image texture complexity, image texture variance, etc. The third complexity calculated by the encoder 3 may include motion complexity, image texture complexity, image texture variance, encoder quantization parameter QP value, encoder structural similarity parameter ssim, etc.

[0057] In some embodiments, complexity includes intra-frame complexity and inter-frame complexity. Intra-frame complexity can be calculated based on intra-frame data of the current image frame. Furthermore, inter-frame complexity can be calculated based on the current image frame and at least one preceding image frame.

[0058] According to embodiments of this disclosure, configuration parameters for the image signal processor are evaluated by analyzing the complexity of the input image frames, and then the image signal processor is adjusted based on these configuration parameters for subsequent encoding processing. In this way, adjustments are made at the source, from the encoder input, achieving source-level adjustment and making the entire image processing system more efficient, effectively improving the overall performance of image processing and encoding compression.

[0059] Figure 6 A schematic diagram of an apparatus for outputting image data according to an embodiment of the present disclosure is shown. Figure 6 As shown, and as Figure 1 Compared to the aforementioned device, this device further includes a second parameter setter 5. The second parameter setter 5 is connected to the image signal processor 2 and the encoder 3. The second parameter setter 5 is configured to set second configuration parameters for the encoder 3. The encoder 3 is configured to encode the processed image data based on the second configuration parameters to generate the encoded image data. The second configuration parameters may include encoding mode parameters, quantization parameters, etc., and their adjustment can be either biased or precise.

[0060] In embodiments according to this disclosure, the second parameter setter 5 may set the second configuration parameters for the encoder 3 in any of the following ways. Figures 7 to 10 A schematic diagram of a structure showing a second parameter setter setting a second configuration parameter for an encoder according to an embodiment of the present disclosure is shown.

[0061] like Figure 7 In the illustrated embodiment, the image signal processor 2 is further configured to calculate the current complexity of the current image frame and provide the current complexity to the second parameter setter 5. The second parameter setter 5 is configured to set a second configuration parameter for the encoder 3 based on the current complexity, corresponding to the currently processed image data or the currently processed image data and at least one subsequent processed image data. The current complexity may include motion complexity, image texture complexity, image texture variance, etc. In some embodiments, the current complexity and the second configuration parameter are correlated. A lookup table between the current complexity and the second configuration parameter can be pre-set. When the second configuration parameter needs to be adjusted, a matching second configuration parameter is found based on the current complexity, and the adjustment is performed based on the matching second configuration parameter.

[0062] like Figure 8 In the illustrated embodiment, the encoder 3 is further configured to calculate the current complexity of the currently processed image data corresponding to the current image frame, and provide the current complexity to the second parameter setter 5. The second parameter setter 5 is configured to set a second configuration parameter for the encoder 3 corresponding to at least one subsequently processed image data based on the current complexity. The current complexity may include motion complexity, image texture complexity, image texture variance, encoder quantization parameter QP value, encoder structural similarity parameter ssim, etc. In some embodiments, the current complexity and the second configuration parameter are correlated. A lookup table between the current complexity and the second configuration parameter can be preset. When the second configuration parameter needs to be adjusted, a matching second configuration parameter is found based on the current complexity, and the adjustment is performed based on the matching second configuration parameter.

[0063] like Figure 9In the illustrated embodiment, the apparatus for outputting image data according to this disclosure further includes an encoding preprocessor 6. The encoding preprocessor 6 is connected to the image signal processor 2 and the second parameter setter 5. The encoding preprocessor 6 is configured to calculate the current complexity of the currently processed image data corresponding to the current image frame and provide the current complexity to the second parameter setter 5. The second parameter setter 5 is configured to set a second configuration parameter for the encoder 3 based on the current complexity, corresponding to the currently processed image data or the current processed image data and at least one subsequent processed image data. In some embodiments, the current complexity and the second configuration parameter are correlated. A lookup table of the current complexity and the second configuration parameter can be preset. When the second configuration parameter needs to be adjusted, a matching second configuration parameter is found based on the current complexity, and the adjustment is performed based on the matching second configuration parameter.

[0064] like Figure 10 In the illustrated embodiment, the encoding preprocessor 6 is configured to calculate a first complexity of the currently processed image data corresponding to the current image frame and provide the first complexity to the second parameter setter 5. The image signal processor 2 is further configured to calculate a second complexity of the currently processed image data corresponding to the current image frame and provide the second complexity to the second parameter setter 5. The encoder 3 is further configured to calculate a third complexity of the currently processed image data corresponding to the current image frame and provide the third complexity to the second parameter setter 5. The second parameter setter 5 is configured to set the second configuration parameter for the encoder based on at least two of the first complexity, the second complexity, and the third complexity.

[0065] In some embodiments, complexity includes intra-frame complexity and inter-frame complexity. Intra-frame complexity can be calculated based on the processed image data corresponding to the current image frame within the intra-frame data. Furthermore, inter-frame complexity can be calculated based on the processed image data corresponding to the current image frame and at least one previous image frame.

[0066] According to embodiments of this disclosure, configuration parameters for the encoder are further evaluated by analyzing the complexity associated with the input image frame, and the encoder is then adjusted based on these configuration parameters. In this way, the entire image processing system becomes more efficient, effectively improving the overall performance of image processing and encoding compression.

[0067] Figure 11 A flowchart illustrating a method for outputting image data according to an embodiment of the present disclosure is shown. Figure 11 As shown, the method includes steps S111 to S113.

[0068] In step S111, the first parameter setter sets the first configuration parameters for the image signal processor based on the complexity associated with the input image frame.

[0069] In some embodiments, the first configuration parameters include 3D noise reduction parameters (3DNR), 2D noise reduction parameters (2DNR), contrast noise parameters (CNR), sharpening parameters, etc., and their adjustment can be a biased adjustment or a precise adjustment.

[0070] In step S112, the image signal processor performs image signal processing on at least one image frame according to the first configuration parameters to generate processed image data.

[0071] In some embodiments, the continuous image frames acquired by the image acquisition device are input into the image signal processor for image signal processing to obtain the processed image data.

[0072] In step S113, the encoder encodes the processed image data to generate encoded image data.

[0073] In embodiments according to this disclosure, the first parameter setter may set the first configuration parameter of the image signal processor in any of the following ways.

[0074] like Figure 2 In the illustrated embodiment, the image signal processor calculates the current complexity of the current image frame and provides the current complexity to the first parameter setter. The first parameter setter then sets a first configuration parameter for the image signal processor based on the current complexity, corresponding to at least one subsequent image frame.

[0075] like Figure 3 In the illustrated embodiment, the encoder calculates the current complexity of the currently processed image data corresponding to the current image frame and provides the current complexity to the first parameter setter. The first parameter setter is configured to set a first configuration parameter for the image signal processor corresponding to at least one subsequent image frame based on the current complexity.

[0076] like Figure 4 In the illustrated embodiment, the image signal preprocessor calculates the current complexity of the current image frame and provides the current complexity to the first parameter setter. The first parameter setter then sets a first configuration parameter for the image signal processor based on the current complexity, corresponding to the current image frame or the current image frame and at least one subsequent image frame.

[0077] like Figure 5In the illustrated embodiment, the image signal preprocessor calculates a first complexity of the current image frame and provides the first complexity to the first parameter setter. The image signal processor calculates a second complexity of the current image frame and provides the second complexity to the first parameter setter. The encoder calculates a third complexity of the currently processed image data corresponding to the current image frame and provides the third complexity to the first parameter setter. Then, the first parameter setter sets the first configuration parameter based on at least two of the first, second, and third complexities.

[0078] In some embodiments, the method further includes setting second configuration parameters for the encoder by a second parameter setter. In some embodiments, the encoder encodes the processed image data based on the second configuration parameters to generate the encoded image data.

[0079] In embodiments according to this disclosure, the second configuration parameter of the encoder may be set by the second parameter setter in any of the following ways.

[0080] like Figure 7 In the illustrated embodiment, the image signal processor calculates the current complexity of the current image frame and provides the current complexity to the second parameter setter. The second parameter setter then sets a second configuration parameter for the encoder based on the current complexity, corresponding to either the currently processed image data or the currently processed image data plus at least one subsequent processed image data.

[0081] like Figure 8 In the illustrated embodiment, the encoder calculates the current complexity of the currently processed image data corresponding to the current image frame and provides the current complexity to the second parameter setter. The second parameter setter then sets a second configuration parameter for the encoder based on the current complexity, corresponding to at least one subsequent processed image data.

[0082] like Figure 9 In the illustrated embodiment, the encoding preprocessor calculates the current complexity of the currently processed image data corresponding to the current image frame and provides the current complexity to the second parameter setter. The second parameter setter then sets a second configuration parameter for the encoder based on the current complexity, corresponding to either the currently processed image data or the currently processed image data plus at least one subsequent processed image data.

[0083] like Figure 10In the illustrated embodiment, the encoding preprocessor calculates a first complexity of the currently processed image data corresponding to the current image frame and provides the first complexity to the second parameter setter. The image signal processor calculates a second complexity of the currently processed image data corresponding to the current image frame and provides the second complexity to the second parameter setter. The encoder calculates a third complexity of the currently processed image data corresponding to the current image frame and provides the third complexity to the second parameter setter. Then, the second parameter setter sets the second configuration parameter for the encoder based on at least two of the first, second, and third complexities.

[0084] Figure 12 This is a schematic diagram illustrating the structure of an electronic device according to an embodiment of the present disclosure. (As shown...) Figure 12 As shown, the electronic device includes a memory 121 and a processor 122. The memory 121 is configured to store a computer program. In some embodiments, the memory 121 is further configured to store image data required to output an input image frame as encoded image data. The processor 122 is communicatively connected to the memory 121 and is configured to invoke the computer program to execute the method for outputting image data according to the above description.

[0085] In some embodiments, the memory 121 includes various media capable of storing program code, such as ROM, RAM, magnetic disk, USB flash drive, memory card, or optical disk. The processor 122 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0086] Furthermore, this disclosure provides a computer-readable storage medium storing a computer program and image data required to output an input image frame as encoded image data. The computer program is executed by a processor to implement the aforementioned method for outputting image data. The storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disk, USB flash drive, memory card, or optical disk. In some embodiments, the computer-readable storage medium is further configured to store the image data required to output an input image frame as encoded image data.

[0087] In summary, the apparatus, method, storage medium, and electronic device for outputting image data disclosed herein evaluate the encoding status by considering the complexity associated with the input image frame, and then adjust the parameters of the image signal processor based on the encoding status, thereby effectively improving the performance of image processing. Adjustments are made to the video source at the encoder input, achieving source-level adjustment and making the entire image processing system more efficient. It is applicable to various application scenarios, even those with low or limited target bitrates, or complex application scenarios. Therefore, this disclosure effectively overcomes the various shortcomings of the prior art and has high industrial applicability.

[0088] The above embodiments are merely illustrative of the principles and effects of this disclosure and are not intended to limit this disclosure. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this disclosure. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this disclosure should still be covered by the claims of this disclosure.

Claims

1. An apparatus for outputting image data, characterized in that, It includes a first parameter setter, an image signal processor, a second parameter setter, and an encoder, among which The first parameter setter is configured to set a first configuration parameter for the image signal processor based on the complexity associated with the input image frame, the complexity including intra-frame complexity and inter-frame complexity; The image signal processor is configured to perform image signal processing on at least one image frame based on the first configuration parameters to generate processed image data. The second parameter setter is configured to set a second configuration parameter for the encoder; The encoder is configured to encode the processed image data based on the second configuration parameters to generate encoded image data. The encoder is further configured to calculate the current complexity of the currently processed image data corresponding to the current image frame, and provide the current complexity to the first parameter setter; The first parameter setter is configured to set a first configuration parameter for the image signal processor corresponding to at least one subsequent image frame based on the current complexity.

2. The apparatus according to claim 1, characterized in that, The image signal processor is further configured to calculate the current complexity of the current image frame and provide the current complexity to the first parameter setter; The first parameter setter is configured to set a first configuration parameter for the image signal processor corresponding to at least one subsequent image frame based on the current complexity.

3. The apparatus according to claim 1, characterized in that, It also includes an image signal preprocessor configured to calculate the current complexity of the current image frame and provide the current complexity to the first parameter setter; The first parameter setter is configured to set a first configuration parameter for the image signal processor based on the current complexity, corresponding to the current image frame or the current image frame and at least one subsequent image frame.

4. The apparatus according to claim 1, characterized in that, It also includes an image signal preprocessor configured to calculate a first complexity of the current image frame and provide the first complexity to the first parameter setter; The image signal processor is further configured to calculate a second complexity of the current image frame and provide the second complexity to the first parameter setter; The encoder is further configured to calculate a third complexity of the currently processed image data corresponding to the current image frame, and to provide the third complexity to the first parameter setter; The first parameter setter is configured to set the first configuration parameter based on at least two of the first complexity, the second complexity, and the third complexity.

5. The apparatus according to claim 1, characterized in that, The image signal processor is further configured to calculate the current complexity of the current image frame and provide the current complexity to the second parameter setter; The second parameter setter is configured to set a second configuration parameter for the encoder based on the current complexity, corresponding to the currently processed image data or the currently processed image data and at least one subsequent processed image data.

6. The apparatus according to claim 1, characterized in that, The encoder is further configured to calculate the current complexity of the currently processed image data corresponding to the current image frame, and provide the current complexity to the second parameter setter; The second parameter setter is configured to set a second configuration parameter for the encoder corresponding to at least one subsequently processed image data, based on the current complexity.

7. The apparatus according to claim 1, characterized in that, It also includes an encoding preprocessor configured to calculate the current complexity of the currently processed image data corresponding to the current image frame and provide the current complexity to the second parameter setter; The second parameter setter is configured to set a second configuration parameter for the encoder based on the current complexity, corresponding to the currently processed image data or the currently processed image data and at least one subsequent processed image data.

8. The apparatus according to claim 1, characterized in that, It also includes an encoding preprocessor configured to calculate a first complexity of the currently processed image data corresponding to the current image frame and provide the first complexity to the second parameter setter; The image signal processor is further configured to calculate a second complexity of the currently processed image data corresponding to the current image frame, and to provide the second complexity to the second parameter setter; The encoder is further configured to calculate a third complexity of the currently processed image data corresponding to the current image frame, and to provide the third complexity to the second parameter setter; The second parameter setter is configured to set the second configuration parameter for the encoder based on at least two of the first complexity, the second complexity, and the third complexity.

9. A method for outputting image data, characterized in that, include: A first parameter setter sets a first configuration parameter for the image signal processor based on the complexity associated with the input image frame, the complexity including intra-frame complexity and inter-frame complexity; The image signal processor performs image signal processing on at least one image frame according to the first configuration parameters to generate processed image data; The second parameter setter sets the second configuration parameters for the encoder; and The encoder performs encoding processing on the processed image data based on the second configuration parameters to generate encoded image data. The first parameter setter sets the first configuration parameters for the image signal processor based on the complexity associated with the input image frame, including: the encoder calculates the current complexity of the currently processed image data corresponding to the current image frame and provides the current complexity to the first parameter setter; And the first parameter setter sets a first configuration parameter for the image signal processor based on the current complexity, corresponding to at least one subsequent image frame.

10. The method according to claim 9, characterized in that, The first configuration parameters for the image signal processor, set by the first parameter setter based on the complexity associated with the input image frame, include: The image signal processor calculates the current complexity of the current image frame and provides the current complexity to the first parameter setter. The first parameter setter sets a first configuration parameter for the image signal processor based on the current complexity, corresponding to at least one subsequent image frame.

11. The method according to claim 9, characterized in that, The first configuration parameters for the image signal processor, set by the first parameter setter based on the complexity associated with the input image frame, include: The image signal preprocessor calculates the current complexity of the current image frame and provides the current complexity to the first parameter setter; The first parameter setter sets a first configuration parameter for the image signal processor based on the current complexity, corresponding to the current image frame or the current frame and at least one subsequent image frame.

12. The method according to claim 9, characterized in that, The first configuration parameters for the image signal processor, set by the first parameter setter based on the complexity associated with the input image frame, include: The image signal preprocessor calculates the first complexity of the current image frame and provides the first complexity to the first parameter setter; The image signal processor calculates the second complexity of the current image frame and provides the second complexity to the first parameter setter; The encoder calculates the third complexity of the currently processed image data corresponding to the current image frame and provides the third complexity to the first parameter setter; The first parameter setter sets the first configuration parameter based on at least two of the first complexity, the second complexity, and the third complexity.

13. The method according to claim 9, characterized in that, The second configuration parameters for the encoder, set by the second parameter setter, include: The image signal processor calculates the current complexity of the current image frame and provides the current complexity to the second parameter setter. The second parameter setter sets a second configuration parameter for the encoder based on the current complexity, corresponding to the currently processed image data or the currently processed image data and at least one subsequent processed image data.

14. The method according to claim 9, characterized in that, The second configuration parameters for the encoder, set by the second parameter setter, include: The encoder calculates the current complexity of the currently processed image data corresponding to the current image frame and provides the current complexity to the second parameter setter; The second parameter setter sets a second configuration parameter for the encoder based on the current complexity, corresponding to at least one image data to be processed later.

15. The method according to claim 9, characterized in that, The second configuration parameters for the encoder, set by the second parameter setter, include: The encoding preprocessor calculates the current complexity of the currently processed image data corresponding to the current image frame and provides the current complexity to the second parameter setter; The second parameter setter sets a second configuration parameter for the encoder based on the current complexity, corresponding to the currently processed image data or the currently processed image data and at least one subsequent processed image data.

16. The method according to claim 9, characterized in that, The second configuration parameters for the encoder, set by the second parameter setter, include: The encoding preprocessor calculates a first complexity of the currently processed image data corresponding to the current image frame and provides the first complexity to the second parameter setter; The image signal processor calculates a second complexity of the currently processed image data corresponding to the current image frame, and provides the second complexity to the second parameter setter; The encoder calculates the third complexity of the currently processed image data corresponding to the current image frame and provides the third complexity to the second parameter setter; The second parameter setter sets the second configuration parameter for the encoder based on at least two of the first complexity, the second complexity, and the third complexity.

17. An electronic device, characterized in that, The electronic device includes: Memory, configured to store computer programs; and A processor, communicatively connected to the memory, is configured to invoke the computer program to perform the method according to any one of claims 9 to 16.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor to implement the method according to any one of claims 9 to 16.

Images