Image processing method, apparatus and system

Abstract

Embodiments of the present application disclose an image processing method, device and system. Embodiments of the present application multiplex a same physical transmission channel to transmit an instruction packet and a pixel packet, and set a packet identification bit in the instruction packet and the pixel packet, so that the instruction packet and the pixel packet are distinguished based on the packet identification bit, thereby reducing a command loading time between different frames or different fragments, and supporting low-fine-grained image processing operations. Meanwhile, embodiments of the present application implement generation and retransmission of the data packet and the instruction packet in a pure hardware mode, thereby reducing a running burden of a central processing unit and improving parallel utilization efficiency of hardware.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically, to an image processing method, apparatus, and system. Background Technology

[0002] Currently, image processors (IPCs) process received image data in frames or slices. For each received frame or slice, the IPC first receives configuration data and then processes the image data based on this configuration data to ensure correct processing of each frame or slice. However, in existing IPCs, instruction data and image data are typically transmitted through different physical transmission channels, and time-sharing processing of instructions and data is performed via software interrupts. This increases the time delay in receiving configuration and image data, resulting in longer command loading times between different frames or slices, making it difficult to support fine-grained image processing operations. Furthermore, existing IPCs rely on software-based interrupt handling mechanisms to generate and resend image and configuration data, which also increases the workload on the central processing unit (CPU). Summary of the Invention

[0003] In view of this, embodiments of the present invention provide an image processing method, apparatus, and system to reduce command loading time between different frames or different slices, support fine-grained image processing operations, reduce the operating burden of the central processing unit, and improve the parallel utilization efficiency of hardware.

[0004] In a first aspect, embodiments of the present invention aim to provide an image processor, the image processor comprising: Multiple sub-processing modules, wherein each of the sub-processing modules is used to perform a corresponding image processing operation; Each of the sub-processing modules is configured to perform the following steps: Receive the current instruction packet; Parse and obtain the pixel packet attribute identifier field in the current instruction packet; The corresponding pixel packet sequence is received according to the pixel packet attribute identifier field, wherein the pixel packet sequence includes at least one pixel packet, each pixel packet and the current instruction packet are multiplexed and transmitted on the same physical transmission channel, each pixel packet is transmitted continuously after the current instruction packet, each pixel packet and the current instruction packet include a packet identifier bit, and each pixel packet and the current instruction packet are distinguished based on the packet identifier bit; Perform corresponding image processing operations on the pixel data to be processed in each pixel packet to obtain the target pixel data; Generate a target instruction packet and a corresponding pixel packet sequence based on the target pixel data; The target instruction packet and the corresponding pixel packet sequence are sent to the subsequent receiving module. Further, the sub-processing module is specifically configured as follows: Determine the image configuration specifications; The target pixel data is organized according to the image configuration specifications to generate a target instruction packet and a corresponding pixel packet sequence.

[0005] Furthermore, before performing the corresponding image processing operation on the pixel data to be processed in each of the pixel packets, the sub-processing module is also configured to: The configuration identifier field in the current instruction packet is parsed and obtained, wherein the configuration identifier field is at least used to characterize whether it is necessary to switch configuration parameters when processing the pixel data carried by the pixel packet sequence; In response to the detection that the configuration identifier field indicates that it is necessary to switch configuration parameters when processing the pixel data carried by the pixel packet sequence, the current instruction packet is parsed to obtain the local configuration parameters corresponding to the pixel packet sequence; Switch configuration parameters based on the local configuration parameters.

[0006] In other words, pixel data is transmitted continuously in a streaming manner. During transmission, each segment of pixel data transmitted continuously is called a pixel packet, and the data stream composed of multiple segments of pixel data is called a pixel packet sequence. Instruction packets are inserted at the boundaries of the data stream. Furthermore, for the sub-processing module, the image processing input of the sub-processing module will receive the instruction packet corresponding to the next data stream after receiving the last pixel packet of the current data stream. At the same time, before the arrival of a new data stream, the sub-processing module will parse the instruction packet and the internal image processing module controller will load the register configuration. If the new data stream arrives at the sub-processing module before the instruction packet is parsed and the register configuration is loaded, the new data stream will be buffered and backpressured by the sub-processing module until the instruction packet is parsed and the register configuration is loaded. Also, for the sub-processing module, the image processing module output of the sub-processing module will insert the instruction packet corresponding to the next data stream after sending the last pixel packet of the previous data stream, and ensure that the instruction packet is sent out before the first pixel packet of the next data stream is sent.

[0007] Secondly, embodiments of the present invention aim to provide an image processing method, which is applicable to a sub-processing module in an image processor, the method comprising: Receive the current instruction packet; Parse and obtain the pixel packet attribute identifier field in the current instruction packet; The corresponding pixel packet sequence is received according to the pixel packet attribute identifier field, wherein the pixel packet sequence includes at least one pixel packet, each pixel packet and the current instruction packet are multiplexed and transmitted on the same physical transmission channel, each pixel packet is transmitted continuously after the current instruction packet, each pixel packet and the current instruction packet include a packet identifier bit, and each pixel packet and the current instruction packet are distinguished based on the packet identifier bit; Perform corresponding image processing operations on the pixel data to be processed in each pixel packet to obtain the target pixel data; Generate a target instruction packet and a corresponding pixel packet sequence based on the target pixel data; The target instruction packet and the corresponding pixel packet sequence are sent to the subsequent receiving module.

[0008] Thirdly, embodiments of the present invention aim to provide an image processing system, the image processing system comprising: Multiple image sensors are used to acquire image data; The image processor as described in the first aspect.

[0009] This invention reuses the same physical transmission channel to transmit command packets and pixel packets, and sets packet identifier bits in both packets to distinguish them, thereby reducing command loading time between different frames or segments and supporting fine-grained image processing operations. Furthermore, this invention implements the generation and retransmission of data packets and command packets entirely in hardware, thus reducing the CPU's workload and improving the efficiency of hardware parallel utilization. Attached Figure Description

[0010] The above and other objects, features and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which: Figure 1 This is a schematic diagram of the image processing system according to an embodiment of the present invention; Figure 2 This is a schematic diagram of an image processor according to an embodiment of the present invention; Figure 3 This is a schematic diagram of another image processor according to an embodiment of the present invention; Figure 4 This is a flowchart of an image processing method according to an embodiment of the present invention; Figure 5 This is a flowchart of the configuration parameter switching method according to an embodiment of the present invention; Figure 6 This is a flowchart of an image processing method according to an embodiment of the present invention; Figure 7 This is a flowchart of a data processing mode adjustment method according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the target instruction packet and the corresponding pixel packet sequence in an embodiment of the present invention; Figure 9 This is a schematic diagram of the image processing process according to an embodiment of the present invention; Figure 10 This is a schematic diagram of an image processing apparatus according to an embodiment of the present invention. Detailed Implementation

[0011] The present application is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without these details. To avoid obscuring the substance of the present application, well-known methods, processes, flows, elements, and circuits are not described in detail.

[0012] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0013] Unless the context explicitly requires it, words such as "including" or "contains" throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, meaning "including but not limited to".

[0014] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0015] The solutions described in this specification and embodiments, if involving the processing of personal information, will be processed only on the premise of having a legal basis (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be processed within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

[0016] The image processing system in this embodiment of the invention can be applied to any scenario where image processing is required, such as security monitoring, intelligent terminal image acquisition, autonomous driving environmental perception, industrial visual inspection, medical image analysis, aerial surveying, etc. Furthermore, depending on the specific application scenario, the image processing system in this embodiment of the invention can be deployed in corresponding terminal devices or system platforms, such as webcams, smartphones, action cameras, edge AI inference boxes, and autonomous driving and advanced driver assistance systems, etc., and this application does not impose any limitations on this.

[0017] Figure 1 This is a schematic diagram of an image processing system according to an embodiment of the present invention. Figure 1 As shown, the image processing system 1 includes multiple image sensors 11 and an image processor 12.

[0018] Image sensor 11 is a sensor used to acquire image data. Optionally, image sensor 11 can be CMOS (Complementary Metal-Oxide Semiconductor), CIS (Contact Image Sensor), or CCD (Charge-Coupled Device); this application does not impose any limitations on this. Image processor 12 is a hardware processing module with image processing capabilities. Image processor 12 can be connected to each image sensor 11. In image processing system 1, multiple image sensors 11 can acquire image data. Image processor 12 can process the image data acquired by each image sensor 11 and finally output image data that meets the requirements.

[0019] It should be noted that, depending on the specific application scenario, multiple image sensors 11 and image processors 12 can be integrated uniformly or separately. Furthermore, the connection method between the multiple image sensors 11 and image processors 12 can be through an internal device bus (such as an on-chip interconnect bus and a high-speed serial expansion bus), an external wired connection (such as Ethernet and coaxial cable), and / or a wireless network connection (such as a wireless LAN or mobile network), etc., and this application does not impose any limitations on this.

[0020] Furthermore, as an implementation approach, the image processor can be configured as a pipelined image processor. Compared to single-stage image processors, pipelined image processors offer advantages such as high throughput and high resource parallelism. A pipelined image processor breaks down the complete image processing flow into multiple image processing operations, each implemented by a different internal sub-processing module. When processing image data, the pipelined image processor allows the image data to sequentially pass through each internal sub-processing module, with each sub-processing module performing its corresponding image processing operation, thereby achieving a complete image processing flow for the image data.

[0021] Figure 2 This is a schematic diagram of an image processor according to an embodiment of the present invention. Figure 2 As shown, the image processor 2 includes multiple sub-processing modules 21.

[0022] Each sub-processing module 21 is used to perform a corresponding image processing operation. After being input into the image processor 2, the image data to be processed passes through each sub-processing module 21 sequentially. Each sub-processing module 21 can perform its corresponding image processing operation as the image data passes through. Thus, through the sequential cooperation of the sub-processing modules, the image processor 2 can realize a complete image processing flow for the image data.

[0023] It should be noted that, in this embodiment of the invention, after being input into the image processor 2, the image data to be processed is carried and transmitted in the form of a pixel packet sequence. The pixel packet sequence includes at least one pixel packet. Each pixel packet carries a portion of the pixel data from the image data. Simultaneously, the pixel packet sequence has a corresponding instruction packet. This instruction packet is transmitted before the pixel packet sequence. The instruction packet is configured to include a pixel packet attribute identifier field. The pixel packet attribute identifier field is used to characterize the attributes of the pixel packet sequence transmitted subsequently. When an instruction packet is received, each sub-processing module 21 can parse and obtain the pixel packet attribute identifier field in the instruction packet, and receive and process the corresponding pixel packet sequence under the indication of the pixel packet attribute identifier field. Furthermore, to reduce the time delay in receiving the pixel packet sequence and the instruction packet, each pixel packet and instruction packet is multiplexed using the same physical transmission channel. Moreover, to distinguish between the two types of data packets transmitted through the same multiplexed physical transmission channel, each pixel packet and instruction packet is configured to include a packet identifier bit, allowing the pixel packet and instruction packet to be distinguished based on the packet identifier bit.

[0024] Specifically, taking any sub-processing module 21 as an example, sub-processing module 21 can receive the current instruction packet from the previous sending object. After receiving the current instruction packet, sub-processing module 21 can parse and obtain the pixel packet attribute identifier field in the current instruction packet, and receive the corresponding pixel packet sequence according to the pixel packet attribute identifier field. Then, it performs the corresponding image processing operation on the pixel data to be processed in each pixel packet to obtain the target pixel data. Furthermore, after obtaining the target pixel data, sub-processing module 21 can regenerate the target instruction packet and the corresponding pixel packet sequence according to the target pixel data, and send the target instruction packet and the corresponding pixel packet sequence to the subsequent receiving module.

[0025] Therefore, by reusing the same physical transmission channel to transmit command packets and pixel packets, and setting packet identifier bits in the command packets and pixel packets so that they can be distinguished based on the packet identifier bits, embodiments of the present invention can reduce command loading time between different frames or different segments and support low-granularity image processing operations. Simultaneously, by implementing the generation and retransmission of data packets and command packets in pure hardware, embodiments of the present invention can reduce the operating burden of the central processing unit and improve the parallel utilization efficiency of hardware.

[0026] Optionally, the sub-processing modules included in the image processor may differ depending on the actual application scenario, and this application does not impose any limitations on this. Illustratively, the sub-processing modules included in the image processor may include, but are not limited to, a level correction module for performing black level correction, a shadow correction module for performing lens shadow correction, a bad pixel correction module for performing bad pixel correction, a de-mosaic module for performing de-mosaic, a noise reduction module for performing noise reduction, a white balance module for performing automatic white balance, a color correction module for performing color correction, a gamma correction module for performing gamma correction, a color space conversion module for performing color space conversion, an edge enhancement module for performing edge enhancement, and / or a color adjustment module for performing color adjustment, etc., which will not be elaborated further here.

[0027] Optionally, the sub-processing modules included within the image processor can be fixed or pre-configured by relevant personnel according to actual needs; this application does not impose any limitations on this. As one configuration, multiple sub-processing modules within the image processor can be configured to be connected in the form of a processing link. It should be noted that in this configuration, the subsequent receiving module specifically refers to the next connected object of the current sub-processing module in the processing link. Furthermore, as an implementation method, the operating logic of each sub-processing module within the image processor can be described by relevant personnel using a hardware description language (such as VHDL or Verilog), and then the operating logic can be mapped onto a programmable integrated circuit (e.g., a Field Programmable Gate Array, FPGA) through programming. Alternatively, in some embodiments, the sub-processing modules within the image processor can also be implemented using an ASIC (Application-Specific Integrated Circuit); this application does not impose any limitations on this.

[0028] Optionally, in practical applications, to meet the processing requirements of image data, each sub-processing module needs to switch its own image processing configuration parameters according to the current processing object. Simultaneously, each sub-processing module also needs to store and retrieve context cache data based on the current processing object. To support these functions, the image processor can also be configured to include shared registers and a memory controller.

[0029] Figure 3 This is a schematic diagram of another image processor according to an embodiment of the present invention. (See diagram below.) Figure 3 As shown, the image processor 3 includes multiple sub-processing modules 31, a shared register 32, and a storage controller 33.

[0030] Among them, multiple sub-processing modules 31 and Figure 2 The multiple sub-processing modules 21 are identical and will not be described again here. A shared register 32 is connected to each sub-processing module 31. The shared register 32 can be a register shared by multiple sub-processing modules 31. The internal space of the shared register 32 can be divided into sub-storage spaces corresponding to each sub-processing module. Each sub-storage space can be used to store relevant data corresponding to each sub-processing module 31 (e.g., local configuration parameters and context cache data). Whenever there is a data retrieval requirement, the sub-processing module 31 can retrieve the required data from its corresponding sub-storage space. Furthermore, a storage controller 33 is connected to the shared register 32. In this embodiment of the invention, configuration parameters are typically issued by the software in the form of global configuration parameters. The storage controller 33 can be used to implement the splitting and separate storage of global configuration parameters. Specifically, whenever a global configuration parameter is received, the storage controller 33 can break down the global configuration parameter into local configuration parameters corresponding to each sub-processing module 31, and store the local configuration parameters corresponding to each sub-processing module 31 in the sub-storage space corresponding to each sub-processing module 31, so that each sub-processing module 31 can read them at any time as needed.

[0031] Figure 4 This is a flowchart of an image processing method according to an embodiment of the present invention. It should be noted that... Figure 4 The execution entity of the image processing method shown can be any sub-processing module in the above embodiments. By executing... Figure 4 The image processing method shown allows a sub-processing module to process image data upon its arrival and then send the processed image data to a subsequent receiving object. For example... Figure 4 As shown, the image processing method may specifically include the following steps: Step S100: Receive the current instruction packet.

[0032] Specifically, the sub-processing module can receive the current instruction packet from the previous sending object. Here, the current instruction packet can refer to the instruction packet currently received by the sub-processing module. It should be understood that, here, the previous sending object can refer to another sub-processing module in the image processor, or it can refer to the image sensor; this application does not limit this.

[0033] Step S200: Parse and obtain the pixel packet attribute identifier field in the current instruction packet.

[0034] Specifically, after receiving the current instruction packet, the sub-processing module can parse and obtain the pixel packet attribute identifier field from the current instruction packet. The pixel packet attribute identifier field can be used to characterize the attributes of the pixel packet sequence to be transmitted subsequently. The attributes of the pixel packet sequence can be used to support the sub-processing module in receiving and processing the pixel packet sequence.

[0035] Optionally, as a configuration method, the pixel packet attribute identifier field can be set as a single bit or bit sequence within the instruction packet and pixel packet. This single bit or bit sequence can characterize the attributes of the pixel packet sequence through internally recorded binary values. In step S200, the sub-processing module can parse the current instruction packet to obtain the pixel packet attribute identifier field from the corresponding position within the current instruction packet. It should be noted that the length and position of the packet identifier bit within the instruction packet and pixel packet can be set by relevant personnel according to actual needs, and this application does not impose any restrictions on this.

[0036] Step S300: Receive the corresponding pixel packet sequence according to the pixel packet attribute identifier field.

[0037] Specifically, after obtaining the pixel packet attribute identifier field, the sub-processing module can receive the corresponding pixel packet sequence based on the pixel packet attribute identifier field. The pixel packet sequence includes at least one pixel packet. It should be understood that the pixel packet sequence can be a carrier of the image data to be processed. Each pixel packet in the pixel packet sequence can carry a portion of the pixel data from the image data.

[0038] It should be noted that, in this embodiment of the invention, each pixel packet and the current command packet can be transmitted using the same physical transmission channel. Furthermore, each pixel packet and command packet includes a packet identifier bit, which can be used by the sub-processing module to distinguish between them. After multiplexing transmission using the same physical transmission channel, each pixel packet and the current command packet can share the same data bit width, clock domain, and handshake signal. Therefore, this embodiment of the invention can reduce command loading time between different frames or different slices, supporting fine-grained image processing operations.

[0039] Optionally, as a configuration method, the packet identifier can be set as a single bit or a sequence of bits within the instruction packet and pixel packet. This single bit or sequence of bits can represent the type of data packet through internally recorded binary values. It should be noted that the length and position of the packet identifier in the instruction packet and pixel packet can be set by relevant personnel according to actual needs, and this application does not impose any restrictions on this. Furthermore, to facilitate the sub-processing module in distinguishing between instruction packets and pixel packets based on the packet identifier, the length and position of the packet identifier in the instruction packet and pixel packet can be the same.

[0040] It should be noted that since each pixel packet and instruction packet is distinguished only by a packet identifier bit, for older sub-processing modules that lack instruction packet parsing capabilities, this embodiment of the invention can also configure them to skip the currently received data packet when the packet identifier bit of the currently received data packet is detected to be a preset value (a specific packet identifier bit value used to indicate that the current data packet is an instruction packet). Therefore, this embodiment of the invention can achieve compatibility with older sub-processing modules. Furthermore, it should also be noted that the current instruction packet and its corresponding pixel packet sequence come from the same sending object, which can be another sub-processing module in the image processor or an image sensor; this application does not impose any limitations on this.

[0041] It should be understood that, in the embodiments of the present invention, the data format of the pixel data carried by the pixel packet can be set by relevant personnel according to actual needs, and this application does not impose any limitations on it. Indicatively, as one setting method, the pixel data carried by the pixel packet can be set as raw Bayer data and / or YUV data, etc. Raw Bayer data is raw pixel data directly output by the image sensor without de-mosaic processing. Raw Bayer data uses any one of the color information (R, G, B) to represent the pixel value of a pixel. YUV data is pixel data that records color information and luminance information separately. YUV data uses the luminance component (Y, Luma) and chrominance component (U / V, Chroma) to represent the pixel value of a pixel.

[0042] Optionally, different frames or different segments may require different configuration parameters for processing. Therefore, before performing the corresponding image processing operation on the pixel data to be processed in each pixel packet, the sub-processing module can also switch the image processing configuration parameters used according to the current processing object.

[0043] Figure 5 This is a flowchart of a configuration parameter switching method according to an embodiment of the present invention. It should be noted that by executing... Figure 5 The configuration parameter switching method shown allows sub-processing modules to switch the image processing configuration parameters used based on the current processing object. For example... Figure 5 As shown, the configuration parameter switching method may specifically include the following steps: Step S310: Parse and obtain the configuration identifier field in the current instruction package.

[0044] Specifically, the sub-processing module can parse and obtain the configuration identifier field in the current instruction packet. This configuration identifier field can at least indicate whether configuration parameters need to be switched when processing the pixel data carried by the pixel packet sequence.

[0045] Optionally, as a configuration method, the configuration identifier field can also be set as a single bit or bit sequence within the instruction packet. This single bit or bit sequence can represent, through internally recorded binary values, whether configuration parameters need to be switched when processing the pixel data carried by the pixel packet sequence. In step S310, the sub-processing module can parse the current instruction packet to obtain the configuration identifier field from the corresponding position within the current instruction packet. It should be noted that the length and position of the configuration identifier field within the instruction packet and pixel packet can be set by relevant personnel according to actual needs, and this application does not impose any restrictions on this.

[0046] Step S320: In response to detecting that the configuration identifier field indicates that it is necessary to switch configuration parameters when processing the pixel data carried by the pixel packet sequence, obtain the local configuration parameters corresponding to the pixel packet sequence.

[0047] Specifically, after obtaining the configuration identifier field, if it is detected that the configuration identifier field indicates that the configuration parameters need to be switched when processing the pixel data carried by the pixel packet sequence, the sub-processing module can obtain the local configuration parameters corresponding to the pixel packet sequence.

[0048] It should be noted that, in this embodiment of the invention, for any pixel packet sequence, the pixel packet sequence has corresponding global configuration parameters. These global configuration parameters can be generated and distributed by software, representing the image processing configuration parameters required by the image processor when processing the pixel packet sequence. Further, depending on the configuration object, the global configuration parameters are subdivided into local configuration parameters corresponding to each sub-processing module. The local configuration parameters corresponding to each sub-processing module can refer to the image processing configuration parameters required by each sub-processing module when processing the pixel packet sequence. In step S320, the local configuration parameters obtained by the sub-processing module can specifically be the local configuration parameters corresponding to itself.

[0049] Optionally, as one implementation, in addition to the sub-processing modules, the image processor may also include shared registers. These shared registers may include sub-storage spaces corresponding to each sub-processing module. Each sub-storage space can be used to store local configuration parameters corresponding to each sub-processing module. In step S320, the sub-processing module can specifically obtain the local configuration parameters corresponding to the pixel packet sequence within its corresponding sub-storage space. Furthermore, to facilitate the sub-processing module in determining the local configuration parameters corresponding to the current pixel packet sequence, a configuration identifier field can also be used to characterize the index position of the local configuration parameters corresponding to the pixel packet sequence. In this implementation, the sub-processing module can obtain the local configuration parameters corresponding to the pixel packet sequence within its corresponding sub-storage space based on the configuration identifier field.

[0050] Optionally, as a method for updating local configuration parameters, the image processor may further include a storage controller. The local configuration parameters stored in each sub-storage space can be updated by this storage controller. Specifically, the storage controller can receive global configuration parameters corresponding to pixel packet sequences and break down the global configuration parameters into local configuration parameters corresponding to each sub-processing module. Then, for each sub-processing module, the storage controller can store the corresponding local configuration parameters in the corresponding sub-storage space. It should be understood that the global configuration parameters can be pre-distributed to the storage controller by software via a corresponding bus; this application does not limit the method of distributing global configuration parameters.

[0051] Step S330: Switch the configuration parameters according to the local configuration parameters.

[0052] Specifically, after obtaining the local configuration parameters corresponding to the pixel packet sequence, the sub-processing module can switch the configuration parameters according to the local configuration parameters.

[0053] It should be noted that, in the embodiments of the present invention, the content of the local configuration parameters obtained by different sub-processing modules may be different, and this application does not impose any limitations on this. Illustratively, for a noise reduction module used to perform noise reduction operations, the local configuration parameters obtained by the noise reduction module may include luminance / chrominance noise reduction intensity and spatial / temporal noise reduction weights, etc. For a bad pixel correction module used to perform bad pixel correction operations, the content of the local configuration parameters obtained by the bad pixel correction module may include a bad pixel detection threshold. For a demosaic module used to perform demosaic operations, the content of the local configuration parameters obtained by the demosaic module may include interpolation algorithm selection information and edge detection thresholds, etc., which will not be further elaborated here.

[0054] Therefore, by setting a configuration identifier field within the instruction packet and utilizing this field to support the switching of local configuration parameters used by sub-processing modules, this embodiment of the invention not only enables the image processor to set configuration parameters on a sub-processing module basis, but also allows each sub-processing module to quickly switch the configuration parameters it uses. This provides the necessary support for implementing streaming overlay processing (i.e., loading the configuration of the next segment of data and starting processing based on the attributes in the instruction packet before the previous frame or segment has been processed). Furthermore, due to its ability to quickly switch configuration parameters, this embodiment of the invention can also support the cross-processing of image data from multiple image sensors, or the cross-processing of segments from different frames.

[0055] Step S400: Perform corresponding image processing operations on the pixel data to be processed in each pixel packet to obtain target pixel data.

[0056] Specifically, after receiving the pixel packet sequence corresponding to the current instruction packet, the sub-processing module can perform corresponding image processing operations on the pixel data to be processed in each pixel packet to obtain the target pixel data.

[0057] Optionally, in step S400, depending on the specific processing requirements, the sub-processing module can process the pixel data to be processed in each pixel packet sequentially. Specifically, upon receiving each pixel packet in the pixel packet sequence, the sub-processing module can perform the corresponding image processing operation on the pixel data to be processed in a single pixel packet. Alternatively, the sub-processing module can also process the pixel data to be processed in multiple pixel packets simultaneously using a batch processing method. Specifically, after receiving all pixel packets in the pixel packet sequence, the sub-processing module can simultaneously perform the corresponding image processing operation on the pixel data to be processed in multiple pixel packets.

[0058] Optionally, for some sub-processing modules, the processing of slices by these sub-processing modules needs to rely on context data. For example, for a sub-processing module used to perform bad pixel repair operations, after detecting a defective pixel (a pixel whose pixel value is abnormal due to sensor manufacturing defects or damage), the sub-processing module needs to rely on context data (i.e., the pixel values ​​of normal pixels around the defective pixel) to estimate and replace the correct pixel value for the defective pixel. In step S400, to meet the processing requirements of this sub-processing module, the sub-processing module can also perform corresponding image processing operations on the pixel data to be processed in each pixel packet in a matching manner according to the data structure and structural position of the pixel data carried by the pixel packet sequence.

[0059] Figure 6 This is a flowchart of an image processing method according to an embodiment of the present invention. It should be noted that by executing... Figure 6 The image processing method shown allows sub-processing modules to perform corresponding image processing operations on the pixel data to be processed in each pixel packet by matching the data structure and structural position of the pixel data carried by the pixel packet sequence. For example... Figure 6 As shown, the image processing method may specifically include the following steps: Step S411: Determine whether the data structure of the pixel data carried by the pixel packet sequence is a slice.

[0060] Specifically, the pixel packet attribute identifier field may include a data structure identifier field. The data structure identifier field can be used to characterize the data structure of the pixel data carried by the pixel packet sequence. In this step, the sub-processing module can determine whether the data structure of the pixel data carried by the pixel packet sequence is a slice based on the data structure identifier field. If yes, the sub-processing module can execute step S412 to further determine whether the data structure of the pixel data carried by the pixel packet sequence is the first slice of the frame (i.e., the first slice in the frame). If no, the sub-processing module can execute step S414 to directly perform the corresponding image processing operation on the pixel data to be processed in each pixel packet.

[0061] Step S412: Determine whether the pixel data carried by the pixel packet sequence is the first slice of the frame.

[0062] Specifically, in addition to the data structure identifier field, the pixel packet attribute identifier field may also include a boundary identifier field. The boundary identifier field can be used to characterize whether the pixel data carried by the pixel packet sequence belongs to a boundary data structure. In this step, when it is determined that the data structure of the pixel data carried by the pixel packet sequence is a slice, the sub-processing module can further determine whether the pixel data carried by the pixel packet sequence is the first slice of the frame based on the boundary identifier field. If yes, the sub-processing module can execute step S413. If no, the sub-processing module can execute step S414 to directly perform the corresponding image processing operation on the pixel data to be processed in each pixel packet.

[0063] Step S413: Obtain target context cache data, and based on the target context cache data, perform corresponding image processing operations on the pixel data to be processed in each pixel packet.

[0064] Specifically, when it is detected that the pixel data carried by the pixel packet sequence is a slice in the data structure identifier field, and the pixel data carried by the pixel packet sequence is not the first slice of the frame in the boundary identifier field, the sub-processing module can obtain the target context cache data and perform corresponding image processing operations on the pixel data to be processed in each pixel packet based on the target context cache data. It should be understood that here, the target context cache data can refer to the relevant pixel data stored by the sub-processing module when processing other slices in the same frame as the current slice. Furthermore, as a method of acquisition, the target context cache data can be obtained by the sub-processing module from its own corresponding sub-storage space.

[0065] Step S414: Perform corresponding image processing operations on the pixel data to be processed in each pixel packet.

[0066] Specifically, when the data structure identifier field indicates that the pixel data carried by the pixel packet sequence is a frame, or when the data structure identifier field indicates that the pixel data carried by the pixel packet sequence is a slice, and the boundary identifier field indicates that the pixel data carried by the pixel packet sequence is the first slice of a frame, the sub-processing module can directly perform the corresponding image processing operation on the pixel data to be processed in each pixel packet.

[0067] Optionally, based on the data structure and structural position of the pixel data carried by the pixel packet sequence, the sub-processing module can also manage and maintain the context cache data in a matching manner. Specifically, when it is detected that the data structure identifier field indicates that the pixel data carried by the pixel packet sequence is a slice, and the boundary identifier field indicates that the pixel data carried by the pixel packet sequence is the first slice of the frame, the sub-processing module can determine and store the target context cache data based on the pixel data to be processed and / or the target pixel data. When it is detected that the data structure identifier field indicates that the pixel data carried by the pixel packet sequence is a slice, and the boundary identifier field indicates that the pixel data carried by the pixel packet sequence is not a boundary slice, the sub-processing module can update the target context cache data based on the pixel data to be processed and / or the target pixel data. And, when it is detected that the data structure identifier field indicates that the pixel data carried by the pixel packet sequence is a slice, and the boundary identifier field indicates that the pixel data carried by the pixel packet sequence is the last slice of the frame, the sub-processing module can delete the target context cache data.

[0068] It should be noted that related technologies typically reserve a portion of shared pixel areas between adjacent image slices when dividing the image into slices to support image processing operations based on context data. Compared to related technologies, by setting data structure identifier fields and boundary identifier fields within the instruction packet, and utilizing these fields to support sub-processing modules in storing and retrieving context data independently, the embodiments of the present invention can avoid redundant image data storage and computation, thereby improving image processing efficiency.

[0069] Optionally, after processing all slices of the current frame, the sub-processing module can also generate corresponding frame-level statistics so that relevant modules can use the frame-level statistics to determine the corresponding configuration parameters for the next frame. Specifically, when it is detected that the data structure identifier field indicates that the pixel data carried by the pixel packet sequence is a slice, and the boundary identifier field indicates that the pixel data carried by the pixel packet sequence is the last slice of the frame, the sub-processing module can determine the frame-level statistics. The frame-level statistics are used to support the determination of configuration parameters for subsequent frames.

[0070] Optionally, depending on the application scenario, the image data to be processed can be divided into offline data and online data. Offline data refers to image data stored in memory that is not acquired in real time. Online data refers to image data acquired in real time by the image sensor. Compared to online data, offline data does not have a real-time output requirement; therefore, processing offline data requires sensitivity to power consumption while having relatively relaxed latency constraints. Conversely, online data requires real-time output; therefore, processing online data requires sensitivity to latency while having relatively relaxed power consumption constraints. In this embodiment of the invention, to meet the processing needs of different types of data, the sub-processing module can also adjust the data processing mode before performing the corresponding image processing operation on the pixel data to be processed in each pixel packet.

[0071] Figure 7 This is a flowchart of a data processing mode adjustment method according to an embodiment of the present invention. It should be noted that by executing... Figure 7 The data processing mode adjustment method shown allows sub-processing modules to adjust their own data processing mode based on the data type of the pixel data carried by the pixel packet sequence. For example... Figure 7 As shown, the data processing mode adjustment method may specifically include the following steps: Step S421: Parse and obtain the data type identifier field in the current instruction package.

[0072] Specifically, the sub-processing module can parse and obtain the data type identifier field in the current instruction packet. The data type identifier field is used to characterize the data type of the pixel data carried by the pixel packet sequence.

[0073] Optionally, as a configuration method, the data type identifier field can also be set as a single bit or bit sequence within the instruction packet. This single bit or bit sequence can represent the data type of the pixel data carried by the pixel packet sequence through internally recorded binary values. In step S421, the sub-processing module can parse the current instruction packet to obtain the data type identifier field from the corresponding position within the current instruction packet. It should be noted that the length and position of the data type identifier field within the instruction packet and pixel packet can be set by relevant personnel according to actual needs, and this application does not impose any restrictions on this.

[0074] Step S422: Switch the processing mode according to the data type identifier field so that the switched processing mode matches the data type of the pixel data carried by the pixel packet sequence.

[0075] Specifically, after obtaining the data type identifier field, the sub-processing module can switch the processing mode according to the data type identifier field so that the switched processing mode matches the data type of the pixel data carried by the pixel packet sequence.

[0076] It should be noted that, in this embodiment of the invention, the processing mode of the sub-processing module can be set to include an online data processing mode and an offline data processing mode. The online data processing mode can be used to instruct the processing of pixel packet sequences at a high frequency and with strict timing constraints. In the online data processing mode, the sub-processing module can operate under a preset high-frequency clock signal and strictly receive and process the pixel packet sequences according to the timing sequence. The offline data processing mode can be used to instruct the processing of pixel packet sequences at a low frequency and with relaxed timing constraints. In the offline data processing mode, the sub-processing module can operate under a preset low-frequency clock signal and optimize the reception and processing time of the pixel packet sequences according to the actual situation (for example, for pixel packet sequences that require the same configuration parameters, the sub-processing module can receive and process them centrally, thereby saving configuration parameter switching time and improving image processing efficiency).

[0077] Therefore, by setting a data type identifier field in the instruction package and using a data structure identifier field and a boundary identifier field to support the sub-processing module in switching the processing mode adopted, the embodiments of the present invention can dynamically adapt to the image processing needs of different application scenarios, thereby improving the fit with the application scenarios.

[0078] Step S500: Generate a target instruction packet and a corresponding pixel packet sequence based on the target pixel data.

[0079] Specifically, after obtaining the target pixel data, the sub-processing module can generate a target instruction packet and a corresponding pixel packet sequence based on the target pixel data.

[0080] Optionally, in step S500, as one implementation, the sub-processing module can first determine the image configuration specifications, and then organize the target pixel data according to the image configuration specifications to generate a target instruction packet and a corresponding pixel packet sequence. The image configuration specifications can be determined by the sub-processing module based on the pixel packet attribute identifier field in the current instruction packet.

[0081] Optionally, in addition to the relevant fields and identifier bits described above, the pixel packet attribute identifier field can also be set to include a slice index identifier field, wherein the slice index identifier field is used to characterize the arrangement sequence number of the pixel data carried by the pixel packet sequence in the frame. When determining the image configuration specifications, the sub-processing module can specifically determine the arrangement sequence number of the pixel data carried by the current pixel packet sequence in the frame based on the slice index identifier field, thereby determining the position of the pixel data carried by the current pixel packet sequence in the frame.

[0082] It should be noted that the flag and field contents included in the instruction package given above are for illustrative purposes only. In actual applications, the flag and field contents included in the instruction package can be adjusted according to actual needs. Illustratively, the flag and field contents included in the instruction package may also include a region of interest (ROI) field (used to indicate whether the pixel data carried by the pixel packet sequence needs to be processed using a specific processing method adapted to the ROI) and an algorithm application field (used to indicate whether the pixel data carried by the pixel packet sequence needs to be processed using a specific processing algorithm), etc.

[0083] Figure 8 This is a schematic diagram of the target instruction packet and the corresponding pixel packet sequence according to an embodiment of the present invention. It should be noted that... Figure 8 A box filled with shaded text can represent a command packet, while a box without shaded text can represent a pixel packet. For example... Figure 8 As shown, the sub-processing module can generate a target instruction packet (as shown in target instruction packet 81) and a corresponding pixel packet sequence (including multiple pixel packets, each pixel packet is shown in pixel packets 821 and 822) based on the target pixel data.

[0084] Step S600: Send the target instruction packet and the corresponding pixel packet sequence to the subsequent receiving module.

[0085] Specifically, after regenerating the target instruction packet and the corresponding pixel packet sequence, the sub-processing module can send the target instruction packet and the corresponding pixel packet sequence to the subsequent receiving module. The subsequent receiving module can be another sub-processing module within the image processor. Alternatively, the subsequent receiving module can also be a back-end storage unit (e.g., system memory) or a control unit (e.g., a central processing unit or microcontroller) connected to the image processor; this application does not impose any limitations on this. It should be understood that after receiving the instruction packet and the corresponding pixel packet sequence, the back-end storage unit or control unit can perform image data storage or subsequent processing on the image data based on the instruction packet and the corresponding pixel packet sequence; this application does not impose any limitations on this either.

[0086] It should be noted that when sending the target command packet and the corresponding pixel packet sequence, the sub-processing module will first send the target command packet, and then continuously send each pixel packet in the pixel packet sequence corresponding to the target command packet. For example... Figure 8As shown, when sending the target instruction packet and the corresponding pixel packet sequence, the sub-processing module can first send the target instruction packet 81. Then, after sending the target instruction packet 81, the sub-processing module can send pixel packet 821, and send the pixel packets of each row sequentially from left to right until pixel packet 822 is sent. It should be understood that before pixel packet 822 is sent, even if the next instruction packet and its corresponding pixel packet sequence have been generated, the sub-processing module will not send them, but will wait for pixel packet 822 to be sent before sending the next instruction packet and its corresponding pixel packet sequence. Furthermore, when sending the next instruction packet and its corresponding pixel packet sequence, the sub-processing module will also first send the next instruction packet, and then continuously send each pixel packet in the pixel packet sequence corresponding to the next instruction packet.

[0087] This invention reuses the same physical transmission channel to transmit command packets and pixel packets, and sets packet identifier bits in both packets to distinguish them, thereby reducing command loading time between different frames or segments and supporting fine-grained image processing operations. Furthermore, this invention implements the generation and retransmission of data packets and command packets entirely in hardware, thus reducing the CPU's workload and improving the efficiency of hardware parallel utilization.

[0088] Figure 9 This is a schematic diagram of the image processing process according to an embodiment of the present invention. Figure 9 As shown, during image processing, image sensor 91 can first generate instruction packet A and corresponding pixel packet sequence based on the image data to be processed, and then send image data generation instruction packet A and corresponding pixel packet sequence a to image processor 92. It should be understood that the image data to be processed can be online data acquired in real time by image sensor 91, or offline data acquired in non-real-time. Furthermore, the image data to be processed can be a complete frame, or a slice of a segmented frame; this application does not impose any limitations on this.

[0089] Furthermore, within the image processor 92, the instruction packet A and the corresponding pixel packet sequence a can first pass through the sub-processing module 921. The sub-processing module 921 can receive and process the pixel packet sequence a according to the instruction packet A, and perform corresponding image processing operations on the pixel data carried by the pixel packet sequence a to obtain the target pixel data. After obtaining the target pixel data, the sub-processing module 921 can regenerate the instruction packet B and the corresponding pixel packet sequence b according to the target pixel data, and reuse the same physical transmission channel to continue sending the instruction packet B and the corresponding pixel packet sequence b.

[0090] Furthermore, instruction packet B and the corresponding pixel packet sequence b will pass through sub-processing module 922. Sub-processing module 922 can receive and process pixel packet sequence b according to instruction packet B, and perform corresponding image processing operations on the pixel data carried by pixel packet sequence b to obtain target pixel data. After obtaining the target pixel data, sub-processing module 922 can regenerate instruction packet C and the corresponding pixel packet sequence c according to the target pixel data, and continue to send instruction packet C and the corresponding pixel packet sequence c.

[0091] Furthermore, the instruction packet C and the corresponding pixel packet sequence c will pass through the sub-processing module 922. The sub-processing module 923 can receive and process the pixel packet sequence c according to the instruction packet C, and perform corresponding image processing operations on the pixel data carried by the pixel packet sequence c to obtain the target pixel data. After obtaining the target pixel data, the sub-processing module 923 can regenerate the instruction packet D and the corresponding pixel packet sequence d according to the target pixel data, and reuse the same physical transmission channel to continue sending the instruction packet D and the corresponding pixel packet sequence d.

[0092] Furthermore, the instruction packet D and the corresponding pixel packet sequence d are ultimately received by the subsequent receiving module 93. The subsequent receiving module 93 can be a post-storage unit or control unit connected to the image processor 92. After receiving the instruction packet D and the corresponding pixel packet sequence d, the subsequent receiving module 93 can perform image data storage or subsequent processing based on the instruction packet D and the corresponding pixel packet sequence d. It should be noted that the image processing process shown above is merely illustrative; in actual applications, the scale of the sub-processing modules within the image processor and the transmission paths between these sub-processing modules are not limited to this.

[0093] Figure 10 This is a schematic diagram of an image processing apparatus according to an embodiment of the present invention. Figure 10 As shown, the image processing apparatus of this embodiment includes an instruction packet receiving unit 101, an instruction packet parsing unit 102, a pixel packet receiving unit 103, an execution unit 104, a packet generation unit 105, and a packet sending unit 106.

[0094] Specifically, the instruction packet receiving unit 101 is used to receive the current instruction packet.

[0095] The instruction packet parsing unit 102 is used to parse and obtain the pixel packet attribute identifier field in the current instruction packet.

[0096] The pixel packet receiving unit 103 is used to receive a corresponding pixel packet sequence according to the pixel packet attribute identifier field. The pixel packet sequence includes at least one pixel packet. Each pixel packet and the current instruction packet are multiplexed and transmitted through the same physical transmission channel. Each pixel packet is transmitted continuously after the current instruction packet. Each pixel packet and the current instruction packet include a packet identifier bit. Each pixel packet and the current instruction packet are distinguished based on the packet identifier bit.

[0097] The execution unit 104 is used to perform corresponding image processing operations on the pixel data to be processed in each pixel packet to obtain target pixel data.

[0098] The packet generation unit 105 is used to generate a target instruction packet and a corresponding pixel packet sequence based on the target pixel data.

[0099] The packet sending unit 106 is used to send the target instruction packet and the corresponding pixel packet sequence to the subsequent receiving module.

[0100] This invention reuses the same physical transmission channel to transmit command packets and pixel packets, and sets packet identifier bits in both packets to distinguish them, thereby reducing command loading time between different frames or segments and supporting fine-grained image processing operations. Furthermore, this invention implements the generation and retransmission of data packets and command packets entirely in hardware, thus reducing the CPU's workload and improving the efficiency of hardware parallel utilization.

[0101] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image processor, characterized in that, The image processor includes: Multiple sub-processing modules, wherein each of the sub-processing modules is used to perform a corresponding image processing operation; Each of the sub-processing modules is configured to perform the following steps: Receive the current instruction packet; Parse and obtain the pixel packet attribute identifier field in the current instruction packet; The corresponding pixel packet sequence is received according to the pixel packet attribute identifier field, wherein the pixel packet sequence includes at least one pixel packet, each pixel packet and the current instruction packet are multiplexed and transmitted on the same physical transmission channel, each pixel packet is transmitted continuously after the current instruction packet, each pixel packet and the current instruction packet include a packet identifier bit, and each pixel packet and the current instruction packet are distinguished based on the packet identifier bit; Perform corresponding image processing operations on the pixel data to be processed in each pixel packet to obtain the target pixel data; Generate a target instruction packet and a corresponding pixel packet sequence based on the target pixel data; The target instruction packet and the corresponding pixel packet sequence are sent to the subsequent receiving module.

2. The image processor according to claim 1, characterized in that, The sub-processing module is specifically configured as follows: Determine the image configuration specifications; The target pixel data is organized according to the image configuration specifications to generate a target instruction packet and a corresponding pixel packet sequence.

3. The image processor according to claim 1, characterized in that, Before performing the corresponding image processing operation on the pixel data to be processed in each of the pixel packets, the sub-processing module is further configured to: The configuration identifier field in the current instruction packet is parsed and obtained, wherein the configuration identifier field is at least used to characterize whether it is necessary to switch configuration parameters when processing the pixel data carried by the pixel packet sequence; In response to the detection that the configuration identifier field indicates that it is necessary to switch configuration parameters when processing the pixel data carried by the pixel packet sequence, the local configuration parameters corresponding to the pixel packet sequence are obtained. Switch configuration parameters based on the local configuration parameters.

4. The image processor according to claim 3, characterized in that, The image processor further includes: The shared register includes a sub-storage space corresponding to each of the sub-processing modules, and each of the sub-storage spaces is used to store the local configuration parameters corresponding to each of the sub-processing modules. The sub-processing module is specifically configured as follows: Retrieve the local configuration parameters corresponding to the pixel packet sequence within the corresponding sub-storage space.

5. The image processor according to claim 4, characterized in that, The image processor further includes: The storage controller is configured to perform the following steps: Receive global configuration parameters corresponding to the pixel packet sequence; The global configuration parameters are broken down into local configuration parameters corresponding to each of the sub-processing modules; For each of the sub-processing modules, the corresponding local configuration parameters are stored in the corresponding sub-storage space.

6. The image processor according to claim 1, characterized in that, The pixel packet attribute identifier field includes a data structure identifier field and a boundary identifier field. The data structure identifier field is used to characterize the data structure of the pixel data carried by the pixel packet sequence, and the boundary identifier field is used to characterize whether the pixel data carried by the pixel packet sequence belongs to a boundary data structure. The sub-processing module is specifically configured as follows: In response to detecting that the data structure identifier field indicates that the pixel data carried by the pixel packet sequence is a slice, and the boundary identifier field indicates that the pixel data carried by the pixel packet sequence is the first slice of a frame, the corresponding image processing operation is performed on the pixel data to be processed in each pixel packet; In response to detecting that the data structure identifier field indicates that the pixel data carried by the pixel packet sequence is a slice, and the boundary identifier field indicates that the pixel data carried by the pixel packet sequence is not the first slice of the frame, target context cache data is obtained, and based on the target context cache data, corresponding image processing operations are performed on the pixel data to be processed in each pixel packet.

7. The image processor according to claim 6, characterized in that, Before receiving the next instruction packet, the sub-processing module is also configured to: In response to detecting that the data structure identifier field indicates that the pixel data carried by the pixel packet sequence is a slice, and the boundary identifier field indicates that the pixel data carried by the pixel packet sequence is the first slice of a frame, the target context cache data is determined and stored based on the pixel data to be processed and / or the target pixel data; In response to detecting that the data structure identifier field indicates that the pixel data carried by the pixel packet sequence is a slice, and the boundary identifier field indicates that the pixel data carried by the pixel packet sequence is not a boundary slice, the target context cache data is updated according to the pixel data to be processed and / or the target pixel data; And / or, In response to detecting that the data structure identifier field indicates that the pixel data carried by the pixel packet sequence is a slice, and the boundary identifier field indicates that the pixel data carried by the pixel packet sequence is the last slice of a frame, the target context cache data is deleted.

8. The image processor according to claim 6, characterized in that, After performing corresponding image processing operations on the pixel data to be processed in each of the pixel packets to obtain the target pixel data, the sub-processing module is further configured to: In response to detecting that the data structure identifier field indicates that the pixel data carried by the pixel packet sequence is a slice, and the boundary identifier field indicates that the pixel data carried by the pixel packet sequence is the last slice of a frame, frame-level statistics are determined, wherein the frame-level statistics are used to support the determination of configuration parameters for subsequent frames.

9. The image processor according to claim 6, characterized in that, The pixel packet attribute identifier field also includes a slice index identifier field, wherein the slice index identifier field is used to characterize the sequence number of the pixel data carried by the pixel packet sequence in the frame.

10. The image processor according to claim 1, characterized in that, Before performing corresponding image processing operations on the pixel data to be processed in each of the pixel packets to obtain the target pixel data, the sub-processing module is further configured to: Parse and obtain the data type identifier field in the current instruction packet, wherein the data type identifier field is used to characterize the data type of the pixel data carried by the pixel packet sequence; The processing mode is switched according to the data type identifier field so that the switched processing mode matches the data type of the pixel data carried by the pixel packet sequence. The processing mode includes an online data processing mode and an offline data processing mode. The online data processing mode is used to indicate that the pixel packet sequence is processed in a high-frequency and strictly time-constrained manner, and the offline data processing mode is used to indicate that the pixel packet sequence is processed in a low-frequency and relaxed time-constrained manner.

11. The image processor according to claim 1, characterized in that, The plurality of sub-processing modules are configured to be connected in the form of a processing link, and the subsequent receiving module is the next connection object of the sub-processing module in the processing link.

12. An image processing method, characterized in that, The image processing method is applicable to a sub-processing module in an image processor, and the method includes: Receive the current instruction packet; Parse and obtain the pixel packet attribute identifier field in the current instruction packet; The corresponding pixel packet sequence is received according to the pixel packet attribute identifier field, wherein the pixel packet sequence includes at least one pixel packet, each pixel packet and the current instruction packet are multiplexed and transmitted on the same physical transmission channel, each pixel packet is transmitted continuously after the current instruction packet, each pixel packet and the current instruction packet include a packet identifier bit, and each pixel packet and the current instruction packet are distinguished based on the packet identifier bit; Perform corresponding image processing operations on the pixel data to be processed in each pixel packet to obtain the target pixel data; Generate a target instruction packet and a corresponding pixel packet sequence based on the target pixel data; The target instruction packet and the corresponding pixel packet sequence are sent to the subsequent receiving module.

13. An image processing system, characterized in that, The image processing system includes: Multiple image sensors are used to acquire image data; The image processor as described in any one of claims 1-11.

Images