Image data processing method and device, computer equipment, chip and chip module

Abstract

The invention relates to an image data processing method and device, computer equipment, a chip and a chip module, relates to the technical field of image processing, and can control the bandwidth while giving consideration to image processing fluency. The method comprises the following steps: determining the current data storage amount of a data cache unit of an image signal processing pipeline; in response to the fact that the current data storage amount reaches the data overflow threshold value, executing a bandwidth control strategy; the bandwidth control strategy indicates to retain part of image data in image frame data acquired by the image signal processing assembly line, and / or reduce the speed of acquiring an image data frame by the image signal processing assembly line; the image frame data is image data associated with a target image frame, and the target image frame comprises a single image frame; and transmitting the target image data obtained after the bandwidth control strategy is executed to a data cache unit.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image data processing method, apparatus, computer equipment, chip, chip module, computer-readable storage medium, and computer program product. Background Technology

[0002] With the rapid development of terminal device technology, the image processing capabilities of terminal devices are becoming increasingly powerful. During the operation of an image processing system, the image signal processing pipeline receives and processes image data. In scenarios with high camera resolution or large images, this leads to increased power consumption and bandwidth. When the bandwidth usage is high, it usually directly causes the system to run slowly or even experience memory overflow, severely impacting the performance of the image processing system.

[0003] In related technologies, a common solution to alleviate bandwidth constraints is to drop all frames directly. That is, for a certain image frame, all image data associated with that image frame is discarded and not output to memory through the image processing pipeline, thereby reducing bandwidth usage during data transmission.

[0004] However, in the above method, all image data accompanying the image frame is discarded, which often leads to image display stuttering and affects the smoothness of image processing. Summary of the Invention

[0005] Therefore, it is necessary to provide an image data processing method, apparatus, computer equipment, chip, chip module, computer-readable storage medium, and computer program product to address the aforementioned technical problems.

[0006] In a first aspect, this application provides an image data processing method, including:

[0007] Determine the current data storage capacity of the data buffer unit in the image signal processing pipeline;

[0008] In response to the current data storage volume reaching a data overflow threshold, a bandwidth control strategy is executed; the bandwidth control strategy indicates that a portion of the image data in the image frame data acquired by the image signal processing pipeline is retained, and / or, the speed at which the image signal processing pipeline acquires the image data frames is reduced; the image frame data is image data associated with a target image frame, and the target image frame includes a single image frame;

[0009] The target image data obtained after executing the bandwidth control strategy is transmitted to the data cache unit.

[0010] In one embodiment, the step of executing a bandwidth control strategy in response to the current data storage volume reaching a data overflow threshold includes:

[0011] In response to the current data storage reaching a data overflow threshold, and the image frame data source of the image signal processing pipeline including an image sensor, multiple data processing units of the image signal processing pipeline are determined; the multiple data processing units are used to perform different types of processing on the data collected by the image sensor, and the image frame data of the target image frame is obtained according to the processing results of the multiple data processing units.

[0012] By closing some of the plurality of data processing units, and based on the processing result of the first data processing unit that is not closed among the plurality of data processing units, the retained target image data is obtained.

[0013] In one embodiment, the data overflow threshold includes multiple data overflow thresholds corresponding to different bandwidth levels;

[0014] The shutdown of a portion of the plurality of data processing units includes:

[0015] Obtain the first bandwidth level corresponding to the data overflow threshold reached by the data storage volume;

[0016] Under the first bandwidth level, identify the target data processing units to be shut down among the plurality of data processing units, and perform shutdown processing on a portion of the plurality of data processing units based on the target data processing units to be shut down.

[0017] In one embodiment, the target image frame includes an image frame that the application requests to process;

[0018] After obtaining the retained target image data based on the processing result of the first data processing unit that is not turned off among the plurality of data processing units, the method further includes:

[0019] For the second data processing unit that has been shut down among the plurality of data processing units, a data loss prompt is generated; the data loss prompt indicates that the processing result of the second data processing unit is missing.

[0020] Send the data missing notification to the application.

[0021] In one embodiment, the data overflow threshold includes multiple data overflow thresholds corresponding to different bandwidth levels; the step of executing a bandwidth control strategy in response to the current data storage volume reaching the data overflow threshold includes:

[0022] In response to the data storage amount reaching a data overflow threshold, and the image frame data source of the image signal processing pipeline includes memory, a second bandwidth level corresponding to the data overflow threshold reached by the data storage amount is determined;

[0023] Based on the second bandwidth level, the data capture parameters of the data capture module are determined, and the image frame data of the target image frame is obtained from the memory based on the data capture parameters and the data capture module.

[0024] In one embodiment, the target image frame includes an image frame that the application requests to process;

[0025] After transmitting the target image data obtained after executing the bandwidth control strategy to the data buffer unit, the method further includes:

[0026] Determine the bandwidth recovery mode configured for the application;

[0027] Based on the fact that the bandwidth recovery mode is the first recovery mode and the current data storage volume has not reached the data overflow threshold, the bandwidth control strategy is stopped during the processing of subsequent image frames of the current image frame.

[0028] Based on the fact that the bandwidth recovery mode is the second recovery mode, and a bandwidth control shutdown operation triggered for the application is detected, the bandwidth control strategy is stopped from being executed during the processing of subsequent image frames of the current image frame.

[0029] Secondly, this application also provides an image data processing apparatus, comprising:

[0030] The data storage monitoring module is used to determine the current data storage volume of the data buffer unit in the image signal processing pipeline;

[0031] A bandwidth control module is configured to execute a bandwidth control strategy in response to the current data storage amount reaching a data overflow threshold; the bandwidth control strategy instructs to retain a portion of the image data in the image frame data acquired by the image signal processing pipeline, and / or to reduce the speed at which the image signal processing pipeline acquires the image data frames; the image frame data is image data associated with a target image frame, and the target image frame includes a single image frame;

[0032] The data transmission module is used to transmit the target image data obtained after executing the bandwidth control strategy to the data cache unit.

[0033] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the image data processing method described above.

[0034] Fourthly, this application also provides a chip, including a processor and a communication interface, wherein the processor is configured to cause the chip to perform the steps of the image data processing method described above.

[0035] Fifthly, this application also provides a chip module, including a communication module, a power module, a storage module, and a chip, wherein:

[0036] The power module is used to provide power to the chip module;

[0037] The storage module is used to store data and instructions;

[0038] The communication module is used for internal communication within the chip module, or for communication between the chip module and external devices.

[0039] The chip is used to perform the steps of the image data processing method described above.

[0040] Sixthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image data processing method described above.

[0041] In a seventh aspect, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the image data processing method described above.

[0042] The aforementioned image data processing method, apparatus, computer equipment, chip, chip module, computer-readable storage medium, and computer program product can determine the current data storage volume of the data buffer unit in the image signal processing pipeline. Furthermore, in response to the current data storage volume reaching a data overflow threshold, a bandwidth control strategy can be executed. This strategy instructs the retention of a portion of the image data in the image frame data acquired by the image signal processing pipeline, and / or reduces the speed at which the image signal processing pipeline acquires image data frames. The image frame data is image data associated with a target image frame, which includes a single image frame. Then, the target image data acquired after executing the bandwidth control strategy is transmitted to the data buffer unit. In this embodiment, when bandwidth is tight, the amount of data written to memory by the data buffer unit can be reduced by executing the bandwidth control strategy, effectively preventing memory data overflow. Simultaneously, during the execution of the bandwidth control strategy, a portion of the image frame data (i.e., the target image data) can be retained and transmitted to the data buffer unit for subsequent processing. This alleviates bandwidth pressure while ensuring that image tasks can be processed normally. Therefore, it not only reduces bandwidth usage but also avoids excessive image data loss, ensuring the image task processing effect of the terminal device. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating an image data processing method in one embodiment;

[0045] Figure 2 This is a schematic diagram of the hardware framework of a shooting system in one embodiment;

[0046] Figure 3a This is a signal timing diagram of a first recovery mode in one embodiment;

[0047] Figure 3b This is a signal timing diagram of a second recovery mode in one embodiment;

[0048] Figure 4 This is a flowchart illustrating an image data processing method in another embodiment;

[0049] Figure 5 This is a structural block diagram of an image data processing device in one embodiment;

[0050] Figure 6 This is an internal structural diagram of a computer device in one embodiment;

[0051] Figure 7 This is an internal structure diagram of a chip module in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various objects, but these objects are not limited by these terms. These terms are only used to distinguish the first object from the second object. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the solutions, or any combination of multiple solutions.

[0054] With the rapid development of mobile device technology, the image processing capabilities of terminal devices are becoming increasingly powerful, including image processing systems such as camera systems. Simultaneously, the requirements for hardware-based image signal processing units (HW ISP IP) within terminal device chips are also increasing. For example, smartphones are equipped with more cameras, and mobile phone photography can perform a wider variety of functions. Correspondingly, in the process of image processing, for multi-camera, high-frame-rate, and high-resolution image processing scenarios, image processing systems often need to withstand greater data transmission pressure.

[0055] Bandwidth size and bandwidth usage are crucial metrics for evaluating system performance. During image processing, the image signal processing pipeline receives and processes image data. In scenarios with high camera resolution or large image sizes, this leads to increased power consumption and bandwidth usage. Excessive bandwidth usage typically results in slower system performance or even memory overflow, severely impacting image processing system performance. In the design of Image Signal Processor (ISP) IPs (IP cores, referring to independently functional circuit modules within a chip), taking image capture as an example, after receiving data from the image sensor, the image processing system typically processes it through a series of related modules. Some modules output statistical data and image data to memory. In-depth analysis reveals that for more complex shooting scenarios, the ISP often transmits large amounts of image data. In such scenarios, due to high bandwidth usage, memory overflow issues frequently occur, preventing the system from functioning properly.

[0056] To alleviate the bandwidth shortage, the main solution is to drop all frames directly. That is, for a certain image frame, all image data associated with that image frame is discarded. For example, the image data used for previewing, the image data used for taking pictures, and the statistical data of the related 3A (i.e., automatic exposure (AE), automatic white balance (AWB), and automatic focus (AF)) modules of a certain image frame are all discarded to alleviate the bandwidth shortage to the greatest extent.

[0057] However, in the above method, all image data accompanying the image frame is discarded, which often leads to image display stuttering and affects the smoothness of image function usage.

[0058] This application provides an image data processing method, apparatus, computer equipment, chip, chip module, computer-readable storage medium, and computer program product, which can alleviate the bandwidth constraints of the image signal processing pipeline while improving the smoothness of image processing.

[0059] In one embodiment, an image data processing method is provided. This embodiment illustrates the application of this method to an image processing chip as an example. The image processing chip can provide image processing functions and implement the image data processing method provided in this embodiment. It is understood that this method can also be applied to a terminal, which can be a terminal device having at least one of the following functions: shooting, image display, and image processing. Examples include personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc.; portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc.

[0060] like Figure 1 As shown, in this embodiment, the method may include the following steps:

[0061] S101, determine the current data storage amount of the data buffer unit of the image signal processing pipeline.

[0062] Among them, the image signal processing pipeline can refer to a hardware and / or software architecture consisting of multiple image processing modules (such as noise reduction module, white balance module, color correction module, etc.) that continuously processes image data in a preset order.

[0063] A data buffer unit can be a temporary data storage component in an image signal processing pipeline, used to temporarily store intermediate data output by each processing module or image data to be written to memory. The data buffer unit can be a hardware unit or a software unit; for example, it can be a data buffer or queue employing a First-In-First-Out (FIFO) mechanism. The current data storage capacity represents the total amount of data actually stored in the data buffer unit.

[0064] In some embodiments, the used storage resources of the data cache unit can be monitored in real time or periodically. For example, the data storage volume of the data cache unit can be monitored at preset time intervals, and the monitored data storage volume can be directly used as the current data storage volume; or, the data storage volume monitored within a preset time period can be statistically analyzed, and the current data storage volume of the data cache unit can be determined based on the statistical results.

[0065] S102, in response to the current data storage amount reaching the data overflow threshold, a bandwidth control strategy is executed. The bandwidth control strategy instructs to retain a portion of the image data in the image frame data acquired by the image signal processing pipeline, and / or to reduce the speed at which the image signal processing pipeline acquires image data frames. The image frame data is image data associated with a target image frame, and the target image frame includes a single image frame.

[0066] The data overflow threshold can be a preset storage threshold to prevent data overflow, such as when data in the data cache unit is not written to memory in time, causing a data overflow. The data overflow threshold can be preset based on one or more factors, including the actual storage capacity of the data cache unit, the data processing capability of the memory communicating with the data cache unit, image data processing requirements, and chip performance. Its specific value can be flexibly adjusted according to different application scenarios. For example, the appropriate data overflow threshold can be determined by combining the typical data volume of image frames. When the current data storage volume reaches the preset data overflow threshold, a preset bandwidth control strategy will be triggered to prevent data storage devices from overflowing due to excessive data, ensuring the stability of memory processing image data.

[0067] The bandwidth control strategy can control the growth rate of data storage in the data cache unit when the cache storage reaches the data overflow threshold by adjusting the bandwidth (i.e., the amount of data written per unit time) of the data written to the data cache unit in the image signal processing pipeline. Since the memory can perform image processing on the data provided by the data cache unit, the data write speed of the memory can be adjusted accordingly by controlling the data storage of the data cache unit to avoid data overflow.

[0068] Image frame data can be all image data associated with the target image frame. In some examples, image data that the image signal processing pipeline obtains for the target image frame and that is related to the generation / display of the target image frame itself can be identified as image frame data, such as image data used for previewing the target image frame, image data used for taking pictures, and statistical data from the 3A module.

[0069] In related technologies, when the bandwidth of the image signal processing pipeline is limited, the main approach is to discard the entire image data of the whole frame to achieve rapid relief. However, this process can easily lead to display stuttering. Therefore, this embodiment can implement a bandwidth control strategy when it determines that the current data storage volume has reached a data overflow threshold.

[0070] Specifically, in this embodiment, by implementing a bandwidth control strategy, for the image frame data of a single image frame acquired through the image signal processing pipeline, a portion of the image frame data can be discarded, while a portion of the image data of the target image frame can be retained. That is, in this embodiment, for the image frame data of the target image frame, a portion of the image data can be retained, while other image data besides the retained image data can be discarded.

[0071] Furthermore, the amount of data stored in the data buffer unit can also be controlled by reducing the speed at which the signal processing pipeline fetches image data frames from the data source. In some exemplary embodiments, such as... Figure 2 The diagram illustrates a hardware framework for a camera system, including an image sensor input module, a data fetch input module, an image front processing unit (IFPU), a bandwidth control module, a data buffer unit (FIFO), and related hardware (MO AXI / M1AXI). The image signal processing pipeline acquires image frame data of the target image frame through the input terminal. After processing, the obtained target image data is written to memory through the data buffer unit. In this embodiment, a bandwidth control strategy can be implemented to reduce the acquisition speed of the target image frame data, thereby avoiding data overflow caused by writing too much data to the data buffer unit at once. For example, writing the target image frame data to the data buffer unit all at once can be avoided.

[0072] In some optional embodiments, the data overflow threshold can be dynamically adjusted in conjunction with the processing rate of the image signal processing pipeline. For example, when the processing rate of the image signal processing pipeline increases, the data overflow threshold can be increased to improve the efficiency of data processing while ensuring that the system does not experience data overflow. Conversely, when the processing rate decreases, the data overflow threshold can be decreased to prioritize ensuring the stable operation of the system.

[0073] S103 transmits the target image data obtained after executing the bandwidth control strategy to the data buffer unit.

[0074] The target image data can be the image data obtained after executing the bandwidth control strategy in step S102, that is, the target image data can be a part of the complete image frame data of the target image frame.

[0075] In specific implementation, after executing the bandwidth control strategy, the target image data corresponding to the target image frame can be obtained. The data flow of the target image data decreases compared to before the bandwidth control strategy was executed, but not all image frame data of the target image frame is completely lost. Furthermore, the acquired target image data can be transmitted to the data buffer unit. By executing the bandwidth control strategy provided in this embodiment, while transmitting the image data of the target image frame normally, data overflow in the data buffer unit or the memory retrieving data from the data buffer unit can be avoided, effectively ensuring the continuity and smoothness of image task processing.

[0076] In some embodiments, the target image data can be further written into memory by the data caching unit. The memory can be a storage component for storing data related to the image task. In some possible embodiments, the memory can be Double Data Rate Synchronous Dynamic Random Access Memory (DDR) or Synchronous Dynamic Random-access Memory (SDRAM).

[0077] For example, after determining the target image data corresponding to the target image frame, the target image data can be transferred to the data cache unit. The data cache unit then writes this portion of the target image data into memory. Subsequently, image tasks related to the target image frame can be executed based on the target image data already written into memory. For instance, the image task for the target image frame can refer to a specific processing task performed on the target image frame, such as one or more tasks like image preview, image capture, image enhancement, and image recognition. The image task can be executed based on the target image data stored in memory.

[0078] In the aforementioned image data processing method, the current data storage volume of the data buffer unit of the image signal processing pipeline can be determined. Then, in response to the current data storage volume reaching a data overflow threshold, a bandwidth control strategy can be executed. This strategy instructs the retention of a portion of the image data in the image frame data acquired by the image signal processing pipeline, and / or reduces the speed at which the image signal processing pipeline acquires image data frames. The image frame data is image data associated with a target image frame, which includes a single image frame. Subsequently, the target image data acquired after executing the bandwidth control strategy is transmitted to the data buffer unit. In this embodiment, when bandwidth is tight, the amount of data written to memory by the data buffer unit can be reduced by executing the bandwidth control strategy, effectively preventing memory data overflow. Simultaneously, during the execution of the bandwidth control strategy, a portion of the image frame data (i.e., the target image data) can be retained and transmitted to the data buffer unit for subsequent processing. This alleviates bandwidth pressure while ensuring that image tasks can be processed normally. Therefore, it not only reduces bandwidth usage but also avoids excessive image data loss, ensuring the image task processing effect of the terminal device.

[0079] In one embodiment, step S102, in response to the current data storage amount reaching the data overflow threshold, executing a bandwidth control strategy may include the following steps:

[0080] In response to the current data storage reaching a data overflow threshold, and the image frame data source of the image signal processing pipeline including an image sensor, multiple data processing units of the image signal processing pipeline are determined; the multiple data processing units are used to perform different types of processing on the data acquired by the image sensor, and image frame data of the target image frame is obtained according to the processing results of the multiple data processing units; at least some of the multiple data processing units are turned off, and the retained target image data is obtained based on the processing result of the first data processing unit that is not turned off among the multiple data processing units.

[0081] Image sensors can include devices for converting optical signals into electrical signals (i.e., raw image data), such as CMOS sensors and CCD sensors.

[0082] A data processing unit can refer to a module in an image signal processing pipeline that performs specific image processing functions. Different units have different processing functions and can work together to complete the conversion from raw data to target image frame data. In some embodiments, each data processing unit can perform independent data processing functions and output processing results with corresponding effects. For example, multiple data processing units may include a data processing unit for acquiring preview image data, a data processing unit for acquiring captured image data, and a data processing unit for acquiring 3A module statistics. The data processing unit for acquiring preview image data is also called the preview data processing path, the data processing unit for acquiring captured image data is also called the capture data processing path, and the data processing unit for acquiring 3A module statistics is also called the 3A path. Each data processing unit performs different data processing functions and provides different types of image data processing results.

[0083] In a specific implementation, the source of image frame data for the image signal processing pipeline can be determined. In some optional embodiments, the source of image frame data can be determined based on configuration information. For example, the source of image frame data can be determined based on scene information configured by the application. When the application is configured for an online scene, the source of image frame data for the image signal processing pipeline can be determined to include the image sensor. When the application is configured for an offline scene, the source of image frame data for the image signal processing pipeline can be determined to include memory.

[0084] When the image frame data source of an image signal processing pipeline includes an image sensor, multiple data processing units can be defined within the pipeline. These units process the raw data acquired by the image sensor to obtain processing results. Specifically, after acquiring raw data, the image sensor in the pipeline can input the raw data into multiple data processing units, which then process the input data individually to obtain processing results. In other words, multiple data processing units connected in parallel can process the input raw data in parallel.

[0085] Furthermore, when the current data storage volume reaches the data overflow threshold, some of the multiple data processing units can be shut down. Shutting down can be understood as switching the data processing unit from an active state to a non-active state. This could be done by cutting off the power supply to the hardware unit, suspending task scheduling of the software unit, or disabling the functional interfaces of the processing unit. This temporarily stops the shut-down data processing units from participating in image data processing, thereby reducing the bandwidth consumption of transmitting the processing results. For ease of distinction, the part of the multiple data processing units that is not shut down can be referred to as the first data processing unit, and the part that is shut down can be referred to as the second data processing unit. Subsequently, the retained target image data can be obtained based on the processing results of the first data processing unit that is not shut down.

[0086] In this embodiment, in response to the current data storage reaching the data overflow threshold, and the image frame data source of the image signal processing pipeline includes the image sensor, by turning off some of the multiple data processing units that perform different types of processing on the data collected by the image sensor, and then obtaining the target image data based on the first data processing unit that is not turned off, it is possible to more accurately control whether the output results of each path data processing module in the image signal processing pipeline occupy bandwidth, and achieve precise discard control of image frame data.

[0087] In one embodiment, the data overflow threshold includes multiple data overflow thresholds corresponding to different bandwidth levels; correspondingly, shutting down a portion of the multiple data processing units may include the following steps:

[0088] Obtain the first bandwidth level corresponding to the data overflow threshold reached by the data storage volume; determine the target data processing unit to be shut down among multiple data processing units under the first bandwidth level; and perform shutdown processing on some of the multiple data processing units according to the target data processing unit to be shut down.

[0089] In this context, bandwidth level refers to the different data flow levels that an image processing chip or terminal device can handle during the operation of an image signal processing pipeline. Different bandwidth levels can correspond to different load states. In some examples, as the bandwidth level increases, the flow of image data continues to increase, and the possibility or severity of data overflow in the data cache unit or memory increases accordingly.

[0090] In practical applications, the current data storage volume of the data cache unit can be obtained. Taking the data cache unit as a FIFO as an example, in some embodiments, the amount of data in the FIFO can be calculated, and the current bandwidth level can be determined based on the comparison results of the amount of data in the FIFO and each preset data overflow threshold. For ease of distinction, the bandwidth level in this embodiment is referred to as the first bandwidth level. For example, the current data storage volume can be compared with multiple preset data overflow thresholds. Each data overflow threshold can correspond to a bandwidth level, and the setting of multiple data overflow thresholds can be determined based on the processing capability of the image processing chip under different bandwidth levels.

[0091] Furthermore, it is possible to determine the data processing units to be shut down under different bandwidth levels. The number of data processing units to be shut down varies under different bandwidth levels. In some embodiments, the shutdown priority of multiple data processing units can be determined based on their respective importance. Then, the data processing units to be shut down under different bandwidth levels can be determined based on the shutdown priority. For example, as the bandwidth level increases, the number of data processing units in the shutdown state increases accordingly. At the same time, each time a data processing unit is shut down, the shutdown priority of the data processing unit is determined based on the importance of the data processing unit in the image signal processing pipeline and / or the correlation between the image data processing unit and the current image task scenario. The data processing unit that needs to be shut down is determined based on the shutdown priority each time a data processing unit is shut down.

[0092] Therefore, after identifying the first bandwidth level, the target data processing unit to be shut down under the first bandwidth level can be determined, and the target data processing unit among the multiple data processing units can be shut down.

[0093] In a specific example, data processing unit a is used to acquire capture image data, data processing unit b is used to acquire preview image data, and data processing unit c is used to acquire 3A statistical data. Data processing units a, b, and c can be prioritized in advance. Under bandwidth constraints, less influential data can be discarded first, thus ensuring priority is met.

[0094] For example, when the data storage volume in the FIFO is lower than the preset data overflow threshold corresponding to level_1, it can be determined that the bandwidth usage is normal and the data output of each module and storage is normal. When the data storage volume in the FIFO reaches the preset data overflow threshold corresponding to level_1, data processing unit a can be shut down. When the data storage volume in the FIFO reaches the preset data overflow threshold corresponding to level_2, data processing unit c can be shut down. When the data storage volume in the FIFO reaches the preset data overflow threshold corresponding to level_3, the data processing unit used to acquire motion statistics can be shut down. When the data storage volume in the FIFO reaches the preset data overflow threshold corresponding to level_4, data processing unit b can be shut down. If only one bandwidth level is set, the current frame can be closed in response to the data storage volume reaching the data overflow threshold corresponding to a single bandwidth level.

[0095] In some exemplary embodiments, the 3A data-related modules and motion statistics-related data processing units may include one or more of the following: a Lens Shading Correction Monitor (LSCM) module, an Automatic Exposure Monitor (AEM) module, an Automatic Focus Monitor (AFM) module, a Phase Pixel Extraction (PPE) module, and a Local Motion Estimation (LME) module; the data processing units related to image preview data (preview path data) and image capture data (capture path) may include multiple output modules, such as Capture Output, preview Output, VCH Output, etc.

[0096] In this embodiment, by obtaining the first bandwidth level corresponding to the data overflow threshold reached by the data storage amount, and determining the target data processing unit to be shut down among multiple data processing units under the first bandwidth level, the number of data processing units to be shut down can be flexibly controlled according to different bandwidth constraints. This can improve the flexibility of bandwidth control in different image processing scenarios while maintaining normal processing of the target image frame.

[0097] In one embodiment, the target image frame includes an image frame requested for processing by the application; after obtaining the retained target image data based on the processing result of the first data processing unit (which is not closed among multiple data processing units), the method further includes:

[0098] For the second data processing unit that has been shut down among multiple data processing units, a data missing prompt is generated; a data discard prompt indicates that the processing result of the second data processing unit is missing; and a data missing prompt is sent to the application.

[0099] The data missing information can be used to inform the application's data processing unit of the data processing status. It can be used to inform the application that the processing result of the second data processing unit, which is in a closed state, is missing. For example, it can include the identifier of the second data processing unit, the identifier of the image frame corresponding to the missing data, or the range.

[0100] In a specific implementation, bandwidth control can be achieved by shutting down some of the data processing units in the image signal processing pipeline. For ease of distinction, the shut-down data processing units can be referred to as the second data processing units. Since the second data processing units are shut down, they temporarily do not output corresponding processing results, which may affect the subsequent execution flow of the application in some possible scenarios. Therefore, in this embodiment, after obtaining the processing results of each of the non-shut-down first data processing units, a data loss warning can be generated for the shut-down second data processing units and sent to the application, indicating that some image data in the target image frame is missing.

[0101] In this embodiment, on the one hand, the corresponding processing results can be obtained normally through the first data processing unit, so that the relevant image tasks for the target image frame can continue to be carried out, avoiding the loss of all image data of the target image frame and resulting in abnormal image display. On the other hand, by sending a data missing prompt to the second data processing unit to the application, the application can be informed of the data missing status in a timely manner, so as to take corresponding measures, thereby improving the reliability of image frame processing and the response efficiency of the application.

[0102] In one embodiment, the data overflow threshold includes multiple data overflow thresholds corresponding to different bandwidth levels. In step S102, in response to the current data storage amount reaching the data overflow threshold, a bandwidth control strategy is executed, which may include the following steps:

[0103] In response to the data storage reaching a data overflow threshold, and the image frame data source of the image signal processing pipeline includes memory, a second bandwidth level corresponding to the data overflow threshold is determined; based on the second bandwidth level, the data capture parameters of the data capture module are determined, and the image frame data of the target image frame is obtained from memory based on the data capture parameters and the data capture module.

[0104] The data crawling parameters can be parameters used to control the data crawling speed of the data crawling module. For example, the data crawling module can include one or more of the following functions: data crawling interval, number of data crawling modules, and data crawling speed of the data crawling module.

[0105] In a specific implementation, the image frame data source for the image signal processing pipeline can also include memory, for example, Figure 2 As shown, image frame data can be obtained through the data fetching module (Fetch Input). When the data storage volume is detected to have reached the data overflow threshold, and the image frame data source of the image signal processing pipeline includes memory, the bandwidth level corresponding to the current data storage volume reaching the data overflow threshold can be determined. The method for determining the bandwidth level can refer to the aforementioned embodiments, and will not be repeated here. For ease of distinction, the bandwidth level in this embodiment can be referred to as the second bandwidth level.

[0106] Furthermore, the data fetching parameters of the data fetching module used to fetch image frame data can be determined based on the second bandwidth level. In an exemplary embodiment, the data fetching parameters may include a gap number, which is a register of the data fetching module. The speed of fetching image frame data can be modified by changing the gap number. That is, by modifying the gap number, the speed of obtaining image frame data from memory in offline scenarios can be changed, thereby controlling the amount of data in the data cache unit.

[0107] In some embodiments, as the second bandwidth level increases, the bandwidth shortage increases accordingly. Therefore, the data capture speed of image frame data can be negatively correlated with the second bandwidth level, that is, as the second bandwidth level increases, the data capture speed of image frame data decreases accordingly.

[0108] Subsequently, image frame data of the target image frame can be obtained from memory according to the data capture parameters and the data capture module. For example, the matching data capture speed can be determined according to the second bandwidth level, and then the corresponding data capture parameters can be determined according to the data capture speed. After the data capture module is configured according to the data capture parameters, the data capture module can obtain image frame data from memory according to the executed data capture speed.

[0109] In this embodiment, by determining the data capture parameters of the data capture module according to the second bandwidth level, and by obtaining the image frame data of the target image frame from memory according to the data capture parameters and the data capture module, the data capture speed of the image frame data can be accurately controlled to match the current second bandwidth level when the image frame data source includes memory. This prevents data overflow and avoids discarding all image data of the target image frame, ensuring that the task of processing the target image frame can continue.

[0110] In one embodiment, the target image frame includes an image frame that the application requests to process; after step S103, the following may also be included:

[0111] Determine the bandwidth recovery mode configured for the application; if the bandwidth recovery mode is the first recovery mode and the current data storage volume has not reached the data overflow threshold, stop executing the bandwidth control policy during the processing of subsequent image frames of the current image frame; if the bandwidth recovery mode is the second recovery mode and a bandwidth control shutdown operation triggered for the application is detected, stop executing the bandwidth control policy during the processing of subsequent image frames of the current image frame.

[0112] The first recovery mode can be called automatic mode, and the second recovery mode can be called manual mode.

[0113] In practice, the bandwidth recovery mode configured for the application can be determined first. Different bandwidth recovery modes can correspond to different stop execution conditions. By pre-configuring the bandwidth recovery mode, the application can more flexibly adjust the bandwidth control strategy under different usage scenarios and data processing needs. When the bandwidth recovery mode is the first recovery mode, and the current data storage volume of the data cache unit has not reached the data overflow threshold, it can be determined that the risk of data overflow has decreased. At this time, it is not necessary to continue to limit the data transmission rate through the corresponding bandwidth control strategy.

[0114] In some embodiments, taking a FIFO as an example, in the first recovery mode, such as Figure 3a The timing diagram shows that when the data storage volume in the FIFO reaches the preset level_n, the corresponding level_n_drop_start signal is pulled high; when the data storage volume in the FIFO falls below the preset level_n, the level_n_drop_done signal is pulled high. In some examples, if the data storage volume drops from bandwidth level_3 to bandwidth level_2, the bandwidth control policy corresponding to bandwidth level_3 can be canceled, and the bandwidth control policy corresponding to bandwidth level_2 can continue to be executed.

[0115] In some alternative embodiments, when the determined bandwidth recovery mode is the second recovery mode, the triggering condition for stopping the execution of the bandwidth control policy can be related to user operations on the application. Specifically, it can detect whether a bandwidth control shutdown operation triggered for the application exists, which can be manually initiated by the user through the application interface. When this bandwidth control shutdown operation is detected, it can be determined that the application no longer needs to be constrained by the bandwidth control policy in the current scenario. In this case, the execution of the bandwidth control policy can be stopped during the processing of subsequent image frames of the current image frame.

[0116] In some embodiments, taking a FIFO as an example, in the second recovery mode, such as Figure 3b As shown in the timing diagram, when the data in the FIFO reaches the preset level_n, the corresponding level_n_drop_start signal will be pulled high; in response to the user's bandwidth control shutdown operation triggered by the application, the application can configure the level_n_drop register, and then the level_n_drop_close signal will be pulled high, the level_n_drop_done signal will also be pulled high accordingly, and the execution of the bandwidth control policy will stop.

[0117] In this embodiment, the bandwidth control strategy can be stopped according to the first recovery mode or the second bandwidth mode. This allows the bandwidth limit to be lifted in a timely manner after the bandwidth control strategy is executed and the bandwidth shortage is relieved, ensuring the efficiency and smoothness of subsequent image frame processing and avoiding the impact of unnecessary bandwidth restrictions on the normal display of image frames.

[0118] To enable those skilled in the art to better understand the embodiments of this application, the embodiments of this application are illustrated below by specific examples, but it should be understood that the embodiments of this application are not limited thereto.

[0119] During image data processing in the image signal processing pipeline, the image preprocessing unit receives the Bayer RAW image (a raw image acquired through a Bayer filter) from the image sensor and performs processing in the RAW domain (the raw data area of ​​the photosensitive chip), RGB domain (the color standard area for device display), and YUV domain (a highly efficient compression area for storage / transmission). During this process, data interaction with DDR data occurs through the data caching unit, and bandwidth may be limited in some scenarios.

[0120] To address bandwidth constraints, a common approach is to drop all frames. This means that 3A statistics, preview image data, and capture image data for a given image frame are all discarded and not output to DDR, reducing bandwidth usage. However, this process, involving the discarding of all data, can cause preview stuttering and other issues, impacting the user experience.

[0121] This embodiment provides a bandwidth control method in the data transmission process of an image signal processing pipeline, which can more precisely control all storage paths. The specific modules to be shut down can be set by the user, making it more flexible to use.

[0122] In specific implementation, such as Figure 4 As shown, the FIFO (ficus) data cache unit needed to view the image data of the current Nth frame can be determined based on whether the application is configured for an Online or Offline scene. Different priority data cache units can be set for different scenes, with higher-priority data cache units having priority access to the DDR (Data Retrieval Center). In terms of scene configuration, real-time image data acquired from sensors can be set to Online mode, while non-real-time image data acquired from the DDR via the fetch module (data capture module) can be set to Offline mode.

[0123] For online scenarios, the bandwidth control module can determine the current bandwidth level based on the amount of data stored in the FIFO. If the amount of data in the FIFO has not reached the preset data overflow threshold, no bandwidth adjustment is made. If the amount of data reaches the preset threshold, the corresponding functional modules that need to be shut down are closed according to the currently determined bandwidth level. In some application scenarios that require the key collection of data from a specific module, the data of that specific module can be retained first. At the same time, different types of data, such as capture image data, preview image data, and 3A (autofocus, auto exposure, auto white balance) statistics, are prioritized. When bandwidth is tight, data with a weaker impact on the overall processing effect can be discarded first to ensure the normal processing of key data. In some embodiments, when the next bandwidth level is reached (e.g., level 2), the new data processing unit can be shut down while keeping the data processing unit shut down in the previous level (e.g., level 1). Of course, in other embodiments, the data processing units shut down at different bandwidth levels can be partially different or completely different.

[0124] For offline scenarios, the bandwidth control module can also determine the current bandwidth level based on the amount of data stored in the FIFO. If the data value does not reach the preset data overflow threshold, no processing is performed. If the data value reaches the data overflow threshold, the gap number of the fetch module is adjusted according to the determined bandwidth level to adapt to the current bandwidth situation, thereby alleviating the bandwidth shortage problem.

[0125] During data processing, the application can determine whether a notification signal needs to be triggered when each data processing unit in the current frame completes processing (a virtual completion signal can be emitted by a closed second data processing unit to indicate that it has completed processing) based on the "done" mode configured in the application. This notification signal informs the application whether the processing result of the corresponding data processing unit is accurate. When the "done" mode is configured to 0, the "done" signal is not triggered, and the application does not need to be aware of this signal status. When the "done" mode is configured to 1, the "done" signal is triggered. If the current frame also receives a "level_n_drop_start" signal at this time, it indicates that the image frame data of the current frame is incomplete. A data loss notification can then be used to inform the application that the processing result of the second data processing unit is missing.

[0126] Furthermore, based on the recovery mode configured in the application, it can be determined whether a bandwidth recovery operation should be performed in the (N+1)th frame (the frame following the current frame). There are two recovery modes: automatic and manual. After performing the bandwidth recovery operation and stopping the execution of the bandwidth control policy, it can be confirmed that bandwidth control has ended.

[0127] The above processing not only reduces bandwidth usage but also avoids excessive image data loss, thus ensuring the quality of mobile phone images.

[0128] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0129] Based on the same inventive concept, this application also provides an image data processing apparatus for implementing the image data processing method described above. This apparatus can be applied to or integrated into a chip or chip module, for example. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more image data processing apparatus embodiments provided below can be found in the limitations of the image data processing method described above, and will not be repeated here.

[0130] In one exemplary embodiment, such as Figure 5 As shown, an image data processing apparatus is provided, comprising:

[0131] The data storage monitoring module 501 is used to determine the current data storage volume of the data buffer unit in the image signal processing pipeline.

[0132] The bandwidth control module 502 is configured to execute a bandwidth control strategy in response to the current data storage amount reaching a data overflow threshold; the bandwidth control strategy indicates that a portion of the image data in the image frame data acquired by the image signal processing pipeline is retained, and / or that the speed at which the image signal processing pipeline acquires the image data frames is reduced; the image frame data is image data associated with a target image frame, and the target image frame includes a single image frame;

[0133] The data transmission module 503 is used to transmit the target image data obtained after executing the bandwidth control strategy to the data cache unit.

[0134] In one embodiment, the bandwidth control module 502 is configured to:

[0135] In response to the current data storage reaching a data overflow threshold, and the image frame data source of the image signal processing pipeline including an image sensor, multiple data processing units of the image signal processing pipeline are determined; the multiple data processing units are used to perform different types of processing on the data collected by the image sensor, and the image frame data of the target image frame is obtained according to the processing results of the multiple data processing units.

[0136] By closing some of the plurality of data processing units, and based on the processing result of the first data processing unit that is not closed among the plurality of data processing units, the retained target image data is obtained.

[0137] In one embodiment, the data overflow threshold includes multiple data overflow thresholds corresponding to different bandwidth levels;

[0138] The bandwidth control module 502 is used for:

[0139] Obtain the first bandwidth level corresponding to the data overflow threshold reached by the data storage volume;

[0140] Under the first bandwidth level, identify the target data processing units to be shut down among the plurality of data processing units, and perform shutdown processing on a portion of the plurality of data processing units based on the target data processing units to be shut down.

[0141] In one embodiment, the target image frame includes an image frame that the application requests to process;

[0142] The device is also used for:

[0143] For the second data processing unit that has been shut down among the plurality of data processing units, a data loss prompt is generated; the data loss prompt indicates that the processing result of the second data processing unit is missing.

[0144] Send the data missing notification to the application.

[0145] In one embodiment, the data overflow threshold includes multiple data overflow thresholds corresponding to different bandwidth levels; the bandwidth control module 502 is used for:

[0146] In response to the data storage amount reaching a data overflow threshold, and the image frame data source of the image signal processing pipeline includes memory, a second bandwidth level corresponding to the data overflow threshold reached by the data storage amount is determined;

[0147] Based on the second bandwidth level, the data capture parameters of the data capture module are determined, and the image frame data of the target image frame is obtained from the memory based on the data capture parameters and the data capture module.

[0148] In one embodiment, the target image frame includes an image frame that the application requests to process;

[0149] The device is also used for:

[0150] Determine the bandwidth recovery mode configured for the application;

[0151] Based on the fact that the bandwidth recovery mode is the first recovery mode and the current data storage volume has not reached the data overflow threshold, the bandwidth control strategy is stopped during the processing of subsequent image frames of the current image frame.

[0152] Based on the fact that the bandwidth recovery mode is the second recovery mode, and a bandwidth control shutdown operation triggered for the application is detected, the bandwidth control strategy is stopped from being executed during the processing of subsequent image frames of the current image frame.

[0153] Regarding the modules / units included in the various devices and products described in the above embodiments, they can be software modules / units, hardware modules / units, or a combination of both. For example, for various devices and products applied to or integrated into a chip, all of their modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs that run on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits; for various devices and products applied to or integrated into a chip module, all of their modules / units can be implemented using hardware methods such as circuits, and different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The components can be implemented using software programs that run on the processor integrated within the chip module. The remaining (if any) modules / units can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into the terminal, each of its components / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or in different components within the terminal. Alternatively, at least some modules / units can be implemented using software programs that run on the processor integrated within the terminal, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits.

[0154] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements an image data processing method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0155] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0156] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0157] Based on the same inventive concept, this application also provides a chip, including a processor coupled to a memory, for executing a computer program or instructions stored in the memory, and implementing the steps in the above method embodiments when the processor executes the computer program or instructions.

[0158] It is understood that the chip involved in the embodiments of this application may be a field-programmable gate array (FPGA), may be an application-specific integrated circuit (ASIC), may be a system on chip (SoC), may be a central processor unit (CPU), may be a network processor (NP), may be a digital signal processor (DSP), may be a microcontroller unit (MCU), may be a programmable logic device (PLD), or other integrated chips, etc.

[0159] Based on the same inventive concept, this application also provides a chip module, such as... Figure 7 As shown, the chip module includes a communication module, a power module, a storage module, and a chip. Among them:

[0160] The power module is used to provide power to the chip module; the storage module is used to store data and instructions; the communication module is used for internal communication within the chip module, or for communication between the chip module and external devices; this chip corresponds to the chip in the above chip embodiment.

[0161] The implementation method of this chip module can be found in the relevant content of the above chip embodiment, and will not be repeated here.

[0162] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0163] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0164] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0165] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0166] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0167] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image data processing method, characterized in that, The method includes: Determine the current data storage capacity of the data buffer unit in the image signal processing pipeline; In response to the current data storage volume reaching a data overflow threshold, a bandwidth control strategy is executed; the bandwidth control strategy indicates that a portion of the image data in the image frame data acquired by the image signal processing pipeline is retained, and / or, the speed at which the image signal processing pipeline acquires the image data frames is reduced; the image frame data is image data associated with a target image frame, and the target image frame includes a single image frame; The target image data obtained after executing the bandwidth control strategy is transmitted to the data cache unit.

2. The method according to claim 1, characterized in that, The step of executing a bandwidth control strategy in response to the current data storage volume reaching a data overflow threshold includes: In response to the current data storage reaching a data overflow threshold, and the image frame data source of the image signal processing pipeline including an image sensor, multiple data processing units of the image signal processing pipeline are determined; the multiple data processing units are used to perform different types of processing on the data collected by the image sensor, and the image frame data of the target image frame is obtained according to the processing results of the multiple data processing units. By closing some of the plurality of data processing units, and based on the processing result of the first data processing unit that is not closed among the plurality of data processing units, the retained target image data is obtained.

3. The method according to claim 2, characterized in that, The data overflow threshold includes multiple data overflow thresholds corresponding to different bandwidth levels; The shutdown of a portion of the plurality of data processing units includes: Obtain the first bandwidth level corresponding to the data overflow threshold reached by the data storage volume; Under the first bandwidth level, identify the target data processing units to be shut down among the plurality of data processing units, and perform shutdown processing on a portion of the plurality of data processing units based on the target data processing units to be shut down.

4. The method according to claim 2, characterized in that, The target image frame includes the image frame that the application requests to process; After obtaining the retained target image data based on the processing result of the first data processing unit that is not turned off among the plurality of data processing units, the method further includes: For the second data processing unit that has been shut down among the plurality of data processing units, a data missing prompt is generated; The data discard message indicates that the processing result of the second data processing unit is missing; Send the data missing notification to the application.

5. The method according to claim 1, characterized in that, The data overflow threshold includes multiple data overflow thresholds corresponding to different bandwidth levels; the step of executing a bandwidth control strategy in response to the current data storage volume reaching the data overflow threshold includes: In response to the data storage amount reaching a data overflow threshold, and the image frame data source of the image signal processing pipeline includes memory, a second bandwidth level corresponding to the data overflow threshold reached by the data storage amount is determined; Based on the second bandwidth level, the data capture parameters of the data capture module are determined, and the image frame data of the target image frame is obtained from the memory based on the data capture parameters and the data capture module.

6. The method according to any one of claims 1 to 5, characterized in that, The target image frame includes the image frame that the application requests to process; After transmitting the target image data obtained after executing the bandwidth control strategy to the data buffer unit, the method further includes: Determine the bandwidth recovery mode configured for the application; Based on the fact that the bandwidth recovery mode is the first recovery mode and the current data storage volume has not reached the data overflow threshold, the bandwidth control strategy is stopped during the processing of subsequent image frames of the current image frame. Based on the fact that the bandwidth recovery mode is the second recovery mode, and a bandwidth control shutdown operation triggered for the application is detected, the bandwidth control strategy is stopped from being executed during the processing of subsequent image frames of the current image frame.

7. An image data processing apparatus, characterized in that, The device includes: The data storage monitoring module is used to determine the current data storage volume of the data buffer unit in the image signal processing pipeline; A bandwidth control module is configured to execute a bandwidth control strategy in response to the current data storage amount reaching a data overflow threshold; the bandwidth control strategy instructs to retain a portion of the image data in the image frame data acquired by the image signal processing pipeline, and / or to reduce the speed at which the image signal processing pipeline acquires the image data frames; the image frame data is image data associated with a target image frame, and the target image frame includes a single image frame; The data transmission module is used to transmit the target image data obtained after executing the bandwidth control strategy to the data cache unit.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A chip, characterized in that, The device includes a processor and a communication interface, wherein the processor is configured to cause the chip to perform the steps of the method described in any one of claims 1 to 6.

10. A chip module, characterized in that, This includes communication modules, power modules, storage modules, and chips, among which: The power module is used to provide power to the chip module; The storage module is used to store data and instructions; The communication module is used for internal communication within the chip module, or for communication between the chip module and external devices. The chip is used to perform the steps of the method according to any one of claims 1 to 6.

Images