Image processing method and device, image processing system and storage medium
By independently implementing protocol conversion and AI algorithms in the image processing system, the problem of replacing AI algorithms in the prior art affecting other functions of ISPs is solved, and flexible image processing is realized and development difficulty is reduced.
Patent Information
- Application Number
- CN202411120925.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-12-27
AI Technical Summary
In the prior art, in image processing, protocol conversion and AI algorithm are integrated into a general ISP, resulting in the effect of other image processing functions of the ISP when replacing the AI algorithm, which increases the difficulty of development.
By independently implementing protocol conversion and AI algorithms in the image processing system, the protocol of the original image data is converted into the protocol required by the AI algorithm, and the processed data protocol is adjusted to the original protocol and sent to the ISP to avoid other effects on the ISP. Image processing function.
It reduces the difficulty of development when replacing AI algorithms and improves the flexibility and efficiency of image processing.
Smart Images
Figure CN119211749A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, device, image processing system and storage medium. Background Art
[0002] When a camera is shooting an image, it first needs to collect raw image data through an image sensor, then process the raw image data, and then display the processed image on a display. To this end, an image processing method needs to be provided. Summary of the invention
[0003] The purpose of the embodiments of the present invention is to provide an image processing method, device, image processing system and storage medium to process an image. The specific technical solution is as follows:
[0004] In a first aspect of an embodiment of the present invention, an image processing method is first provided, which is applied to an image processing system. The method includes:
[0005] Receiving raw image data collected by an image sensor;
[0006] Based on the protocol conversion, the protocol of the original image data is converted into a preset protocol to obtain first image data, and the first image data is input into an artificial intelligence AI algorithm so that the AI algorithm performs image processing on the first image data to obtain second image data, and then the protocol of the second image data is adjusted to the same protocol as the original image data to obtain third image data, and the third image data is sent to an image signal processor ISP;
[0007] Among them, the preset protocol is the protocol required by the AI algorithm; the ISP has other image processing functions in addition to protocol conversion and AI algorithm image processing functions.
[0008] In a possible embodiment, the receiving raw image data collected by the image sensor includes:
[0009] Receive valid image data in the raw image data input by the target interface; wherein the valid image data is obtained after the target interface performs synchronous parsing and protocol parsing on the raw image data collected by the image sensor.
[0010] In a possible embodiment, if the image processing system is applied to an application-specific integrated circuit ASIC, the valid image data is in an intelligent peripheral interface IPI format.
[0011] In a possible embodiment, if the image processing system is applied to a field programmable gate array (FPGA), the valid image data is in an advanced extensible interface (AXI) format.
[0012] In a possible embodiment, before inputting the first image data into an artificial intelligence (AI) algorithm, the method further includes:
[0013] Performing data preprocessing on the first image data; wherein the data preprocessing includes at least one of the following data processing: image data format cropping, black level correction, data row and column transposition, multiple exposure analysis, and bad pixel correction;
[0014] The step of inputting the first image data into an AI algorithm comprises:
[0015] The preprocessed image data is input into the AI algorithm.
[0016] In a second aspect of the embodiments of the present invention, an image processing device is provided, which is applied to an image processing system. The device includes:
[0017] A data receiving module, used for receiving raw image data collected by the image sensor;
[0018] a protocol conversion module, configured to convert the protocol of the original image data into a preset protocol based on protocol conversion to obtain first image data, and input the first image data into an artificial intelligence (AI) algorithm so that the AI algorithm performs image processing on the first image data to obtain second image data, then adjust the protocol of the second image data to the same protocol as the original image data to obtain third image data, and send the third image data to an image signal processor (ISP);
[0019] Among them, the preset protocol is the protocol required by the AI algorithm; the ISP has other image processing functions in addition to protocol conversion and AI algorithm image processing functions.
[0020] In a possible embodiment, the data receiving module is specifically used to:
[0021] Receive valid image data in the raw image data input by the target interface; wherein the valid image data is obtained after the target interface performs synchronous parsing and protocol parsing on the raw image data collected by the image sensor.
[0022] In a possible embodiment, if the image processing system is applied to an application-specific integrated circuit ASIC, the valid image data is in an intelligent peripheral interface IPI format.
[0023] In a possible embodiment, if the image processing system is applied to a field programmable gate array (FPGA), the valid image data is in an advanced extensible interface (AXI) format.
[0024] In a possible embodiment, before the protocol conversion module inputs the first image data into the artificial intelligence AI algorithm, it also includes:
[0025] A preprocessing module, used for performing data preprocessing on the first image data; wherein the data preprocessing includes at least one of the following data processing: image data format cropping, black level correction, data row and column transposition, multiple exposure analysis, and bad pixel correction;
[0026] The step of inputting the first image data into an AI algorithm comprises:
[0027] The preprocessed image data is input into the AI algorithm.
[0028] In a third aspect of the embodiments of the present invention, there is also provided an image processing system, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
[0029] Memory, used to store computer programs;
[0030] The processor is used to implement any method step described in the first aspect when executing the program stored in the memory.
[0031] In another aspect of the embodiments of the present application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any method step described in the first aspect is implemented.
[0032] Beneficial effects of the embodiments of the present invention:
[0033] In the image processing method provided by the embodiment of the present invention, when the image processing system receives the original image data collected by the image sensor, the protocol of the original image data can be converted into the protocol required by the AI algorithm based on protocol conversion to obtain the first image data, and the first image data is input into the AI algorithm. The AI algorithm performs image processing on the first image data to obtain the second image data, and then adjusts the protocol of the second image data to the same protocol as the original image data to obtain the third image data, and sends the third image data to the ISP, thereby completing the image processing.
[0034] Furthermore, since the functions of protocol conversion and AI algorithm are independent of ISP, the ISP in the embodiment of the present invention only has other image processing functions except protocol conversion and AI algorithm image processing functions. Therefore, compared with the prior art of integrating protocol conversion and AI algorithm into a general ISP, in the embodiment of the present invention, when switching different AI algorithms, only the parts related to the AI algorithm can be replaced, avoiding the situation where other image processing functions in the ISP are affected by replacing the AI algorithm part, thereby reducing the development difficulty of replacing the AI algorithm.
[0035] Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.
[0037] Figure 1 A schematic diagram of a flow chart of a first image processing method provided by an embodiment of the present invention;
[0038] Figure 2 A schematic diagram of the structure of a protocol conversion (receiving) unit provided in an embodiment of the present invention;
[0039] Figure 3 A schematic diagram of a flow chart of a second image processing method provided by an embodiment of the present invention;
[0040] Figure 4 A schematic diagram of the structure of a processor interface-data receiving interface for the mobile industry provided by an embodiment of the present invention;
[0041] Figure 5 A schematic diagram of the structure of a mobile industry processor interface-data transmission interface provided by an embodiment of the present invention;
[0042] Figure 6 A schematic diagram of the structure of a low voltage differential signal-data receiving interface provided by an embodiment of the present invention;
[0043] Figure 7 A schematic diagram of a flow chart of a third image processing method provided by an embodiment of the present invention;
[0044] Figure 8 A schematic diagram of an image data format clipping method provided by an embodiment of the present invention;
[0045] Fig. 9A topological diagram of a bad pixel correction method provided by an embodiment of the present invention;
[0046] Fig.10 A schematic diagram of a first data row-column transposition method provided by an embodiment of the present invention;
[0047] Fig.11 A schematic diagram of a second data row-column transposition method provided by an embodiment of the present invention;
[0048] Fig.12 A schematic diagram of a first type of HDR data separation provided by an embodiment of the present invention;
[0049] Fig.13 A schematic diagram of a second HDR data separation provided by an embodiment of the present invention;
[0050] Fig.14 A schematic diagram of constructing a Laplace pyramid provided by an embodiment of the present invention;
[0051] Fig.15 A schematic diagram of the fusion of a base layer provided by an embodiment of the present invention;
[0052] Fig.16 A schematic diagram of detail layer fusion provided by an embodiment of the present invention;
[0053] Fig.17 A schematic diagram of the fusion of a Laplacian pyramid provided by an embodiment of the present invention;
[0054] Fig.18 A schematic diagram of the structure of an image enhancement imaging system provided by an embodiment of the present invention;
[0055] Fig.19 A logical schematic diagram of an image processing system provided by an embodiment of the present invention;
[0056] Fig. 20 A schematic diagram of the structure of an image processing device provided by an embodiment of the present invention;
[0057] Fig.21 A schematic diagram of the structure of an image processing system provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field based on the present invention belong to the scope of protection of the present invention.
[0059] The embodiment of the present invention provides an image processing method, which is applied to an image processing system. Figure 1 , is a flow chart of a first image processing method provided in an embodiment of the present invention, the method includes S101-S102.
[0060] S101, receiving raw image data collected by an image sensor.
[0061] In S101, the raw image data refers to the raw (unprocessed) data collected by the image sensor. The resolution, interface rate, data transmission mode, etc. of the image sensor will affect the format of the raw data. The format of the raw data may be different for different image sensor models. For example, the format of the raw data collected by the image sensor may be 1920*1080 pixels, 1280*720 pixels, or other formats.
[0062] The original image data can be input to the image processing system through the MIPI (Mobile Industry Processor Interface) interface, or through the LVDS (Low-Voltage Diffferential Signaling) interface. The relevant contents of the MIPI interface and the LVDS interface are Figure 3 The embodiments shown are described in detail.
[0063] S102, based on protocol conversion, convert the protocol of the original image data into a preset protocol to obtain first image data, and input the first image data into an artificial intelligence algorithm so that the artificial intelligence algorithm performs image processing on the first image data to obtain second image data, then adjust the protocol of the second image data to the same protocol as the original image data to obtain third image data, and send the third image data to an image signal processor.
[0064] In S102, the preset protocol is the protocol required by the AI (Artificial Intelligence) algorithm; the ISP (Image Signal Processing) has other image processing functions in addition to the protocol conversion and AI algorithm image processing functions.
[0065] Specifically, protocol conversion refers to converting data from external hardware or specific interfaces into the standard format or timing required by AI algorithms. For example, converting IPI (Intelligent Peripheral Interface) or AXI (Advanced eXtensible Interface) interface timing data into the internal standard timing data required by AI algorithms.
[0066] The protocol conversion is implemented by the protocol conversion (receiving) unit in the image processing system. The protocol conversion (receiving) unit may also be referred to as a protocol conversion (RX, Receive) unit. Exemplarily, after receiving the IPI and AXI interface timing data, the protocol conversion (RX) unit performs timing conversion on the data and converts the data into the internal standard timing data required by the AI algorithm.
[0067] See also Figure 2 , is a structural diagram of a protocol conversion (receiving) unit provided in an embodiment of the present invention.
[0068] from Figure 2 It can be seen that the input data of the protocol conversion (receiving) unit is the IPI and AXI interface timing data, and the protocol conversion unit converts the IPI and AXI interface timing data into the internal standard timing data required by the AI algorithm.
[0069] After the first image data is obtained through protocol conversion, the first image data is input into the AI algorithm, and the first image data is processed by the AI algorithm. Different AI algorithms have different image processing functions. For example, the AI image enhancement algorithm can enhance the image brightness, enhance the image quality, generate dynamic images based on static images, etc.
[0070] After the first image data is processed by the AI algorithm to obtain the second image data, it is necessary to adjust the protocol of the second image data to the same protocol as the original image data through a protocol conversion (sending) unit to obtain the third image data, and send the third image data to the ISP.
[0071] The protocol conversion (transmit) unit may also be referred to as a protocol conversion (TX, Transmit) unit. Exemplarily, the protocol conversion (TX) unit converts the internal standard timing data required by the AI algorithm into IPI and AXI interface timing data.
[0072] In a possible embodiment, the AXI interface timing data may be AXI_stream (Advanced Extensible Interface Stream) interface timing data.
[0073] In a possible embodiment, if the first image data is separated into long and short frames before being processed by the AI algorithm, the protocol conversion (TX) unit needs to assemble the separated long and short frames in addition to converting the internal standard timing data required by the AI algorithm into IPI and AXI interface timing data.
[0074] If the original long and short frames adopt the inter-row long and short frame interleaving method, the separated long and short frames will be assembled in the inter-row long and short frame interleaving method; if the original long and short frames adopt the inter-frame long and short frame interleaving method, the separated long and short frames will be assembled in the inter-frame long and short frame interleaving method.
[0075] In addition, ISP has other image processing functions besides protocol conversion and AI algorithm image processing functions, such as noise reduction, color correction, sharpening, white balance adjustment, image compression, etc.
[0076] By selecting the above embodiment, when the image processing system receives the raw image data collected by the image sensor, the protocol of the raw image data can be converted into the protocol required by the AI algorithm based on the protocol conversion to obtain the first image data, and the first image data is input into the AI algorithm. The AI algorithm performs image processing on the first image data to obtain the second image data, and then adjusts the protocol of the second image data to the same protocol as the raw image data to obtain the third image data, and sends the third image data to the ISP, thereby completing the image processing.
[0077] Furthermore, since the functions of protocol conversion and AI algorithm are independent of ISP, the ISP in the embodiment of the present invention only has other image processing functions except protocol conversion and AI algorithm image processing functions. Therefore, compared with the prior art of integrating protocol conversion and AI algorithm into a general ISP, in the embodiment of the present invention, when switching different AI algorithms, only the parts related to the AI algorithm can be replaced, avoiding the situation where other image processing functions in the ISP are affected by replacing the AI algorithm part, thereby reducing the development difficulty of replacing the AI algorithm.
[0078] In one possible embodiment, see Figure 3 , is a flow chart of a second image processing method provided by an embodiment of the present invention, which is similar to the above Figure 1 Compared with the embodiment shown, the above S101 can be implemented by S101A.
[0079] S101A, receiving valid image data from the original image data input from the target interface.
[0080] In S101A, the effective image data is obtained after the target interface performs synchronous parsing and protocol parsing on the raw image data collected by the image sensor. In the process of sending the raw image data through the target interface, in addition to the effective image data, the raw image data sent also has additional packaging information interspersed in the effective image data to ensure the accuracy of data transmission. Therefore, it is necessary to perform synchronous parsing and protocol parsing on the raw image data to extract the effective image data.
[0081] The target interface may be a MIPI interface, or an LVDS interface, or an HDMI (High Definition Multimedia Interface) interface, a Dp (Display Port), or the like.
[0082] In a possible embodiment, the MIPI interface is a MIPI_CSI (Mobile Industry Processor Interface Camera Serial Interface) interface. The MIPI_CSI interface includes a MIPI_CSI data receiving interface and a MIPI_CSI data sending interface. The MIPI_CSI data receiving interface is used to receive the original image data collected by the image sensor. The MIPI_CSI data sending interface is used to send the data processed by the AI algorithm to the ISP.
[0083] See also Figure 4 , which is a structural diagram of a mobile industry processor interface-data receiving interface provided by an embodiment of the present invention.
[0084] from Figure 4 It can be seen that the mobile industry processor interface-data receiving interface is also the MIPI_CSI data receiving interface. The MIPI_CSI data receiving interface includes the mobile industry processor interface-physical layer (Mobile Industry Processor Interface Digital Physical Layer, MIPI_DPHY), the mobile industry processor interface-protocol layer (Mobile Industry Processor Interface Protocol, MIPI_PROTOCOL) and the mobile industry processor interface-control layer (Mobile Industry Processor Interface Controller, MIPI_CONTROLLER).
[0085] The input data of the mobile industry processor interface-data receiving interface is collected by the image sensor, and the output data is in the format of the intelligent peripheral interface / high-performance expansion bus interface.
[0086] The physical layer is used to receive the differential clock and differential signal input by the image sensor, convert the differential clock and differential signal into serial data through synchronous analysis, and then send the serial data to the protocol layer.
[0087] The protocol layer parses the MIPI protocol, extracts valid image data from the serial data, and sends the valid image data to the control layer.
[0088] By selecting the above embodiment, the accuracy of data transmission can be improved by performing synchronous analysis and protocol analysis on the original image data collected by the image sensor to obtain valid image data.
[0089] In a possible embodiment, if the image processing system is applied to an ASIC (Application Specific Integrated Circuit), the valid image data is in an IPI format.
[0090] When the image processing system is applied to ASIC, the control layer parses the valid image data into IPI format and then performs protocol conversion.
[0091] By using the above embodiment, if the image processing system is applied to ASIC, the format of the effective image data is parsed into the IPI format, and the effective image data can be transmitted through the IPI interface.
[0092] In a possible embodiment, if the image processing system is applied to FPGA (Field-Programmable Gate Array), the valid image data is in AXI format.
[0093] When the image processing system is applied to FPGA, the control layer parses the valid image data into the AXI format and then performs protocol conversion. For example, the control layer parses the valid image data into the AXI_Stream format.
[0094] By using the above embodiment, if the image processing system is applied to FPGA, the format of the effective image data is parsed into the AXI format, and the effective image data can be transmitted through the AXI interface.
[0095] See also Figure 5 , which is a structural diagram of a mobile industry processor interface-data sending interface provided by an embodiment of the present invention.
[0096] from Figure 5It can be seen that the mobile industry processor interface-data transmission interface is also the MIPI_CSI data transmission interface. The MIPI_CSI data transmission interface includes the mobile industry processor interface-physical layer, the mobile industry processor interface-protocol layer and the mobile industry processor interface-control layer.
[0097] The input data of the mobile industry processor interface-data transmission interface is in the format of intelligent peripheral interface / high-performance expansion bus interface, and the output data of the mobile industry processor interface-data transmission interface is subsequently processed by the image signal processor.
[0098] The control layer is used to parse the data in the IPI format or the AXI format into valid image data and send it to the protocol layer.
[0099] The protocol layer converts the valid image data into serial data through the MIPI protocol and sends it to the physical layer.
[0100] The physical layer is used to convert serial data into differential clock and differential signal, and then output to the image signal processor.
[0101] See also Figure 6 , is a structural diagram of a low voltage differential signal-data receiving interface provided in an embodiment of the present invention.
[0102] from Figure 6 It can be seen that the low voltage differential signal-data receiving interface is also the LVDS data receiving interface, and the LVDS data receiving interface includes a low voltage differential signal-physical layer (Low-Voltage Diffferential Signaling Digital Physical Layer, LVDS_DPHY) and a low voltage differential signal-protocol layer (Low-Voltage Diffferential Signaling Protocol, LVDS_PROTOCOL).
[0103] The input data of the low voltage differential signal-data receiving interface is collected by the image sensor, and the output data is data in the intelligent peripheral interface format.
[0104] Among them, the physical layer is used to receive the differential clock and differential signal input by the image sensor, and convert the differential clock and differential signal into serial data through synchronous analysis, and then send the serial data to the protocol layer.
[0105] The protocol layer is used to parse the serial data into IPI format and then perform protocol conversion.
[0106] In one possible embodiment, see Figure 7, is a flow chart of a third image processing method provided by an embodiment of the present invention, which is similar to the aforementioned Figure 1 Compared with the illustrated embodiment, the above S102 can be implemented by S102A.
[0107] S102A, based on protocol conversion, converts the protocol of the original image data into a preset protocol to obtain first image data, performs data preprocessing on the first image data, and inputs the image data obtained after the preprocessing into an artificial intelligence algorithm so that the artificial intelligence algorithm performs image processing on the first image data to obtain second image data, adjusts the protocol of the second image data to the same protocol as the original image data, obtains third image data, and sends the third image data to an image signal processor.
[0108] The data preprocessing includes at least one of the following data processing: image data format cropping, black level correction, data row and column transposition, multiple exposure analysis, and bad pixel correction.
[0109] By selecting the above embodiment, the first image data is preprocessed before being input into the AI algorithm, so that the data format after the preprocessing can meet the requirements of the AI algorithm, thereby improving the processing efficiency of the AI algorithm.
[0110] Specifically, image data format cropping is used to crop image data to a specified resolution. Cropping the resolution of input image data according to the requirements of the AI algorithm can ensure the correctness of the AI algorithm calculation.
[0111] See also Figure 8 , is a schematic diagram of an image data format cropping method provided by an embodiment of the present invention.
[0112] from Figure 8 It can be seen that before the image data format is cropped, the width of the image is a1 and the height of the image is b1. After the image data format is cropped, the width of the image is a2 and the height of the image is b2.
[0113] In addition, in the process of image sensors converting analog signals into digital signals, due to the existence of dark current, the accuracy of digital signals is often affected by noise. Image sensors generally add a fixed value to reduce the impact of noise on low-noise areas. Black level correction is to subtract this fixed value in the digital domain. Black level correction is achieved through the BLC (Black Level Correction) unit.
[0114] Black level correction can be achieved in two ways:
[0115] Method 1: In black level acquisition mode, the BLC unit receives n frames of dark condition data, and uses the average values of the four channels R, Gr, Gb, and B as the black level values Ir, Igr, Igb, and Ib, respectively. Among them, n frames of dark condition data refer to n frames of image data collected by the image sensor under no light conditions.
[0116] Subtract the black level value corresponding to the channel from the signal strength corresponding to the channel to obtain the corrected image data. For example, taking the R channel as an example, the signal strength calculation formula of the R channel after black level correction is as follows:
[0117] R ′ =RI r
[0118] Among them, R ′ is the signal strength of the R channel after black level correction, and R is the signal strength of the R channel before black level correction.
[0119] Method 2: In black frame acquisition mode, the BLC unit receives n frames of dark condition data and sends it to DDR (Double Data Rate, double-rate synchronous dynamic random access memory) through the internal bus. The AI algorithm calculates the average value of each pixel in the n frames of dark condition data and sends it to DDR.
[0120] For example, the BLC unit receives 50 frames of dark condition data, each frame of dark condition data includes 100 pixels, and the black frame pixel value of each pixel is equal to the average value of the pixels corresponding to the 50 frames of dark condition data.
[0121] The pixel value of each pixel is subtracted from the corresponding black frame pixel value to obtain the corrected image data.
[0122] In addition, when the image sensor collects raw image data, due to process defects in the light collection array or errors in the photoelectric signal conversion process, the image pixel values in the digital signal will be inaccurate, forming bad pixels. Bad pixels can be found and corrected through bad pixel correction. Bad pixel correction is achieved through the DPC (Defective Pixel Correction) unit.
[0123] Specifically, when the difference between the pixel value of a certain pixel and the pixel values of eight adjacent pixels is greater than a threshold, the pixel is considered to be a bad pixel. The following takes the R channel as an example to describe the bad pixel correction method in detail.
[0124] See also Fig. 9 , which is a topological diagram of a bad pixel correction method provided in an embodiment of the present invention.
[0125] from Fig. 9It can be seen that the eight pixel points adjacent to the pixel point R0 are respectively the pixel point R1, the pixel point R2, the pixel point R3, the pixel point R4, the pixel point R5, the pixel point R6, the pixel point R7, and the pixel point R8.
[0126] In addition, the six pixel points Gr are the pixel points of the Gr channel, the six pixel points Gb are the pixel points of the Gb channel, and the four pixel points B are the pixel points of the B channel.
[0127] The bad pixel correction method specifically includes the following three steps:
[0128] Step 1, compare pixel R0 with pixel R1-pixel R8. If the pixel value of pixel R0 is the maximum value or the minimum value, pixel R0 may be a bad pixel.
[0129] Step 2, calculate the absolute value of the difference between pixel R0 and pixel R1-pixel R8, and compare the calculated 8 absolute values of the difference with the threshold respectively. If the 8 absolute values of the difference are all greater than the threshold, pixel R0 is a bad pixel.
[0130] Step 3, compare the pixel changes of the R channel pixels in the horizontal, vertical, and two oblique directions, take the direction with the smallest pixel value change as the bad pixel correction direction, and take the average value of the R channel pixels in this direction as the pixel value of pixel R0 after correction.
[0131] For example, the vertical pixel value changes of pixel points R2, R0, and R7 are the smallest, and the pixel value of pixel point R0 after correction is equal to the average of the pixel value of pixel point R2, the pixel value of pixel point R0 before correction, and the pixel value of pixel point R7.
[0132] See also Fig.10 , which is a schematic diagram of a first data row-column transposition method provided in an embodiment of the present invention.
[0133] from Fig.10 It can be seen that before the data rows and columns are transposed, the width of the image data is A1, and the height of the image data is B1. After the data rows and columns are transposed, the width of the image data is A2, and the height of the image data is B2.
[0134] The data row and column transposition is realized by the data row and column transposition (receiving) unit, which is the data row and column transposition (RX_TRANS, Receiver T ransposition) unit. The data row and column transposition (RX_TRANS) unit is used to transpose the rows and columns of the input data. For example, a small number of rows and a large number of columns are transposed to increase the row cycle, so as to reduce the pressure of AI algorithm processing data.
[0135] The formula for transposing data rows and columns is:
[0136] I ′ (y,x)=I(x,y)
[0137] Among them, x represents the row, y represents the column, and I ′ is the image data after data row and column transposition, and I is the image data before data row and column transposition.
[0138] In addition, after the image data is processed by the data row-column transposition (RX_TRANS) and the AI algorithm, the image data after the transposition transformation needs to be converted into the image data before the transposition transformation by the data row-column transposition (transmission) unit. Among them, the data row-column transposition (transmission) unit is the data row-column transposition (TX_TRANS, TransmitTransposition) unit.
[0139] See also Fig.11 , which is a schematic diagram of a second data row and column transposition method provided in an embodiment of the present invention.
[0140] from Fig.11 It can be seen that before the data rows and columns are transposed, the width of the image data is A2, and the height of the image data is B2. After the data rows and columns are transposed, the width of the image data is A1, and the height of the image data is B1.
[0141] In order to combine two or more frames of images with the same scene but different exposure times into a high dynamic range image, it is necessary to perform HDR (High Dynamic Range, multiple exposure processing) on the image data.
[0142] There are two main ways to input raw data to image sensors: inter-row long and short frame interleaving and inter-frame long and short frame interleaving.
[0143] See also Fig.12 , which is a schematic diagram of the first type of HDR data separation provided in an embodiment of the present invention.
[0144] from Fig.12 It can be seen that a white frame with a black border represents a long frame, and a gray frame with a black border represents a short frame. The long and short frames are interlaced between rows.
[0145] See also Fig.13 , which is a schematic diagram of the second HDR data separation provided in an embodiment of the present invention.
[0146] from Fig.13 It can be seen that a white frame with a black border represents a long frame, and a gray frame with a black border represents a short frame. The long frames and short frames are interlaced between the long and short frames.
[0147] Specifically, the process of combining long frames and short frames into one frame by Laplacian pyramid fusion is as follows: Figure 14-16 To reflect.
[0148] See also Fig.14 , which is a schematic diagram of constructing a Laplace pyramid provided by an embodiment of the present invention.
[0149] from Fig.14 It can be seen that after Gaussian filtering the long frame original image data, the base layer LG0 is obtained; after downsampling and Gaussian filtering the base layer LG0, the base layer LG1 is obtained, and after upsampling the base layer LG1, the detail layer LL0 is obtained by subtracting it from the base layer LG0; after downsampling and Gaussian filtering the base layer LG1, the base layer LG2 is obtained, and after upsampling the base layer LG2, the detail layer LL1 is obtained by subtracting it from the base layer LG1; after downsampling and Gaussian filtering the base layer LG2, the base layer LG3 is obtained, and after upsampling the base layer LG3, the detail layer LL2 is obtained by subtracting it from the base layer LG2; after downsampling and Gaussian filtering the base layer LG3, the base layer LG4 is obtained, and after upsampling the base layer LG4, the detail layer LL3 is obtained by subtracting it from the base layer LG3. So far, the base layers LG0, LG1, LG2, LG3, LG4 and the detail layers LL0, LL1, LL2, LL3 can be obtained.
[0150] The short frame original image data is processed in the same manner to obtain base layers SG0, SG1, SG2, SG3, SG4 and detail layers SL0, SL1, SL2, SL3.
[0151] See also Fig.15 , which is a schematic diagram of the fusion of a base layer provided in an embodiment of the present invention.
[0152] from Fig.15 It can be seen that the base layer LGn corresponding to the long frame and the base layer SGn corresponding to the short frame are fused in a weighted manner to obtain the base layer FGn, where n represents the number of base layers corresponding to the long frame and the short frame.
[0153] For example, the base layer LG0 corresponding to the long frame is fused with the base layer SG0 corresponding to the short frame in a weighted manner to obtain the base layer FG0. The base layer LG1 corresponding to the long frame is fused with the base layer SG1 corresponding to the short frame in a weighted manner to obtain the base layer FG1. The base layer LG2 corresponding to the long frame is fused with the base layer SG2 corresponding to the short frame in a weighted manner to obtain the base layer FG2. The base layer LG3 corresponding to the long frame is fused with the base layer SG3 corresponding to the short frame in a weighted manner to obtain the base layer FG3. The base layer LG4 corresponding to the long frame is fused with the base layer SG4 corresponding to the short frame in a weighted manner to obtain the base layer FG4.
[0154] See also Fig.16, which is a schematic diagram of detail layer fusion provided in an embodiment of the present invention.
[0155] from Fig.16 It can be seen that the detail layer LLn corresponding to the long frame and the detail layer SLn corresponding to the short frame are fused in a weighted manner to obtain the detail layer FLn, where n represents the number of base layers corresponding to the long frame and the short frame.
[0156] For example, the detail layer LL0 corresponding to the long frame is fused with the detail layer SL0 corresponding to the short frame in a weighted manner to obtain the detail layer FL0. The detail layer LL1 corresponding to the long frame is fused with the detail layer SL1 corresponding to the short frame in a weighted manner to obtain the detail layer FL1. The detail layer LL2 corresponding to the long frame is fused with the detail layer SL2 corresponding to the short frame in a weighted manner to obtain the detail layer FL2. The detail layer LL3 corresponding to the long frame is fused with the detail layer SL3 corresponding to the short frame in a weighted manner to obtain the detail layer FL3. At this point, the base layers FG0, FG1, FG2, FG3, FG4 and the detail layers FL0, FL1, FL2, FL3 can be obtained.
[0157] Among them, the weight coefficient during weight fusion is related to the contrast, saturation and brightness of the image data.
[0158] See also Fig.17 , which is a schematic diagram of the fusion of a Laplacian pyramid provided in an embodiment of the present invention.
[0159] from Fig.17 It can be seen that after upsampling the base layer and fusing it with the corresponding detail layer, the final fused image FG0 is obtained.
[0160] For example, after upsampling the base layer FG4 and fusing it with the detail layer FL3, the base layer FG3 is obtained. After upsampling the base layer FG3 and fusing it with the detail layer FL2, the base layer FG2 is obtained. After upsampling the base layer FG2 and fusing it with the detail layer FL1, the base layer FG1 is obtained. After upsampling the base layer FG1 and fusing it with the detail layer FL0, the base layer FG0 is obtained.
[0161] Combine the following Fig.18 , the above image processing method is described in detail. Fig.18 , is a structural schematic diagram of an image enhancement imaging system provided in an embodiment of the present invention.
[0162] from Fig.18 It can be seen that the image enhancement imaging system includes a system initialization unit, an image data interface, a data pre-processing unit, a data post-processing unit, a data cache unit, a data transport unit, an internal bus, an image sensor, an image signal processor, an artificial intelligence algorithm unit, a double-rate synchronous dynamic random access memory, and a flash memory.
[0163] The artificial intelligence algorithm unit contains various AI algorithms, the double rate synchronous dynamic random access memory is DDR, and the flash memory is FLASH.
[0164] The data pre-processing unit includes a protocol conversion (receiving) unit, a data format cutting unit, a black level correction unit, a bad pixel correction unit, a multiple exposure analysis unit, and a data row and column transposition (receiving) unit. The data pre-processing unit is used to perform data pre-processing.
[0165] The data post-processing unit includes a data row and column transposition (sending) unit and a protocol conversion (sending) unit.
[0166] The system initialization unit includes a serial peripheral interface / advanced peripheral bus, a parameter initialization and configuration unit. The serial peripheral interface is SPI (Serial Peripheral Interface), and the advanced peripheral bus is APB (Advanced Peripheral Bus).
[0167] The image data interface includes a mobile industry processor interface and a low voltage differential signal interface, the mobile industry processor interface is a MIPI interface, and the low voltage differential signal interface is a LVDS interface. The MIPI interface may be a MIPI_CSI interface.
[0168] Specifically, the image processing process includes the following steps:
[0169] Step 1: The system initialization unit performs system initialization, and the system parameters are configured to the image processing system through the SPI or APB interface, and the black frame data collected in advance is stored in the DDR for subsequent black level correction.
[0170] Step 2: The image sensor collects and inputs raw data.
[0171] Step 3: Receive raw data through the MIPI interface or the LVDS interface, and send the received raw data to a protocol conversion (RX) unit.
[0172] Step 4: The protocol conversion (RX) unit converts the MIPI or LVDS protocol data into the internal standard timing data required by the AI algorithm.
[0173] Step 5: After the data pre-processing unit performs data format cropping, black level correction, bad pixel correction, multiple exposure analysis, and data row and column transposition (RX_TRANS), it obtains data that conforms to the AI algorithm data processing format, sends the data to the data cache unit through the internal bus, and sends data preparation completion information to the data handling unit.
[0174] Step 6: After receiving the information sent by the data pre-processing unit, the data transport unit reads data from the data storage unit according to the line cycle and frame cycle of the AI algorithm.
[0175] Step 7: The AI algorithm performs image processing on the data and returns the processed data to the data handling unit.
[0176] Step 8: The data handling unit sends the data to the data post-processing unit through the internal bus.
[0177] Step 9: The data post-processing unit performs data row-column transposition (TX_TRANS) and protocol conversion (TX) on the data processed by the AI algorithm, and then outputs it to the image data interface.
[0178] Step 10: The image data interface sends the data to the ISP via the MIPI interface or the LVDS interface.
[0179] In addition, the data handling unit has the following functions during image data processing:
[0180] Function 1: After the parameter configuration is completed, determine the frame period, line period and data arrangement.
[0181] Function 2, receiving the frame processing completion information sent by the data pre-processing unit and starting data transfer.
[0182] Function 3, transfers data from the data cache unit through the internal bus, converts the data into a specified data arrangement, and sends the data according to the specified frame period and line period.
[0183] Function 4, receives the data processed by the AI algorithm and sends it to the data cache unit through the internal bus.
[0184] See also Fig.19 , which is a logical schematic diagram of an image processing system provided by an embodiment of the present invention.
[0185] from Fig.19 It can be seen that the image processing logic of the image processing system is:
[0186] 1. After the image sensor collects image data, the image data is input into the image processing system through the target interface; wherein the target interface is a MIPI interface or a LVDS interface.
[0187] 2. After the image processing system performs protocol conversion and data preprocessing on the input image data, the processed data is sent to the artificial intelligence algorithm unit.
[0188] 3. After the artificial intelligence algorithm unit processes the data, it sends the data to the image processing system.
[0189] 4. After the image processing system performs protocol conversion on the data, the data is sent to the image signal processor through the target interface; wherein the target interface is a MIPI interface or a LVDS interface.
[0190] Corresponding to the above-mentioned image processing method, the embodiment of the present invention further provides an image processing device, see Fig. 20 , is a schematic diagram of the structure of an image processing device provided by an embodiment of the present invention, the device comprising:
[0191] The data receiving module 2001 is used to receive the original image data collected by the image sensor;
[0192] The protocol conversion module 2002 is used to convert the protocol of the original image data into a preset protocol based on protocol conversion to obtain first image data, and input the first image data into the AI algorithm so that the AI algorithm performs image processing on the first image data to obtain second image data, then adjust the protocol of the second image data to the same protocol as the original image data to obtain third image data, and send the third image data to the ISP;
[0193] Among them, the preset protocol is the protocol required by the AI algorithm; ISP has other image processing functions in addition to protocol conversion and AI algorithm image processing functions.
[0194] By selecting the above embodiment, when the image processing system receives the raw image data collected by the image sensor, the protocol of the raw image data can be converted into the protocol required by the AI algorithm based on the protocol conversion to obtain the first image data, and the first image data is input into the AI algorithm. The AI algorithm performs image processing on the first image data to obtain the second image data, and then adjusts the protocol of the second image data to the same protocol as the raw image data to obtain the third image data, and sends the third image data to the ISP, thereby completing the image processing.
[0195] Furthermore, since the functions of protocol conversion and AI algorithm are independent of ISP, the ISP in the embodiment of the present invention only has other image processing functions except protocol conversion and AI algorithm image processing functions. Therefore, compared with the prior art of integrating protocol conversion and AI algorithm into a general ISP, in the embodiment of the present invention, when switching different AI algorithms, only the parts related to the AI algorithm can be replaced, avoiding the situation where other image processing functions in the ISP are affected by replacing the AI algorithm part, thereby reducing the development difficulty of replacing the AI algorithm.
[0196] In a possible embodiment, the data receiving module 2001 is specifically used for:
[0197] Receive valid image data from the raw image data input by the target interface; wherein the valid image data is obtained after the target interface performs synchronous parsing and protocol parsing on the raw image data collected by the image sensor.
[0198] By selecting the above embodiment, the accuracy of data transmission can be improved by performing synchronous analysis and protocol analysis on the original image data collected by the image sensor to obtain valid image data.
[0199] In a possible embodiment, if the image processing system is applied to ASIC, the valid image data is in IPI format.
[0200] By using the above embodiment, if the image processing system is applied to ASIC, the format of the effective image data is parsed into the IPI format, and the effective image data can be transmitted through the IPI interface.
[0201] In a possible embodiment, if the image processing system is applied to FPGA, the valid image data is in AXI format.
[0202] By using the above embodiment, if the image processing system is applied to FPGA, the format of the effective image data is parsed into the AXI format, and the effective image data can be transmitted through the AXI interface.
[0203] In a possible embodiment, before the protocol conversion module 2002 inputs the first image data into the AI algorithm, the method further includes:
[0204] A preprocessing module, used for performing data preprocessing on the first image data; wherein the data preprocessing includes at least one of the following data processing: image data format cropping, black level correction, data row and column transposition, multiple exposure analysis, and bad pixel correction;
[0205] Inputting the first image data into the AI algorithm includes:
[0206] The preprocessed image data is input into the AI algorithm.
[0207] By selecting the above embodiment, the first image data is preprocessed before being input into the AI algorithm, so that the data format after the preprocessing can meet the requirements of the AI algorithm, thereby improving the processing efficiency of the AI algorithm.
[0208] The embodiment of the present invention also provides an image processing system, such as Fig.21 As shown, it includes a processor 2101 , a communication interface 2102 , a memory 2103 and a communication bus 2104 , wherein the processor 2101 , the communication interface 2102 , and the memory 2103 communicate with each other via the communication bus 2104 .
[0209] Memory 2103, used for storing computer programs;
[0210] The processor 2101 is used to implement any method step of the above-mentioned image processing method when executing the program stored in the memory 2103.
[0211] By selecting the above embodiment, when the image processing system receives the raw image data collected by the image sensor, the protocol of the raw image data can be converted into the protocol required by the AI algorithm based on the protocol conversion to obtain the first image data, and the first image data is input into the AI algorithm. The AI algorithm performs image processing on the first image data to obtain the second image data, and then adjusts the protocol of the second image data to the same protocol as the raw image data to obtain the third image data, and sends the third image data to the ISP, thereby completing the image processing.
[0212] Furthermore, since the functions of protocol conversion and AI algorithm are independent of ISP, the ISP in the embodiment of the present invention only has other image processing functions except protocol conversion and AI algorithm image processing functions. Therefore, compared with the prior art of integrating protocol conversion and AI algorithm into a general ISP, in the embodiment of the present invention, when switching different AI algorithms, only the parts related to the AI algorithm can be replaced, avoiding the situation where other image processing functions in the ISP are affected by replacing the AI algorithm part, thereby reducing the development difficulty of replacing the AI algorithm.
[0213] The communication bus mentioned in the above image processing system can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0214] The communication interface is used for communication between the above image processing system and other devices.
[0215] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0216] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0217] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned image processing methods are implemented.
[0218] By selecting the above embodiment, when the image processing system receives the raw image data collected by the image sensor, the protocol of the raw image data can be converted into the protocol required by the AI algorithm based on the protocol conversion to obtain the first image data, and the first image data is input into the AI algorithm. The AI algorithm performs image processing on the first image data to obtain the second image data, and then adjusts the protocol of the second image data to the same protocol as the raw image data to obtain the third image data, and sends the third image data to the ISP, thereby completing the image processing.
[0219] Furthermore, since the functions of protocol conversion and AI algorithm are independent of ISP, the ISP in the embodiment of the present invention only has other image processing functions except protocol conversion and AI algorithm image processing functions. Therefore, compared with the prior art of integrating protocol conversion and AI algorithm into a general ISP, in the embodiment of the present invention, when switching different AI algorithms, only the parts related to the AI algorithm can be replaced, avoiding the situation where other image processing functions in the ISP are affected by replacing the AI algorithm part, thereby reducing the development difficulty of replacing the AI algorithm.
[0220] In another embodiment of the present invention, a computer program product including instructions is provided, which, when executed on a computer, enables the computer to execute any image processing method in the above embodiments.
[0221] By selecting the above embodiment, when the image processing system receives the raw image data collected by the image sensor, the protocol of the raw image data can be converted into the protocol required by the AI algorithm based on the protocol conversion to obtain the first image data, and the first image data is input into the AI algorithm. The AI algorithm performs image processing on the first image data to obtain the second image data, and then adjusts the protocol of the second image data to the same protocol as the raw image data to obtain the third image data, and sends the third image data to the ISP, thereby completing the image processing.
[0222] Furthermore, since the functions of protocol conversion and AI algorithm are independent of ISP, the ISP in the embodiment of the present invention only has other image processing functions except protocol conversion and AI algorithm image processing functions. Therefore, compared with the prior art of integrating protocol conversion and AI algorithm into a general ISP, in the embodiment of the present invention, when switching different AI algorithms, only the parts related to the AI algorithm can be replaced, avoiding the situation where other image processing functions in the ISP are affected by replacing the AI algorithm part, thereby reducing the development difficulty of replacing the AI algorithm.
[0223] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.
[0224] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0225] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, image processing system, and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.
[0226] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims
1. An image processing method, characterized in that: Applied to an image processing system, the method comprises: Receiving raw image data collected by an image sensor; Based on the protocol conversion, the protocol of the original image data is converted into a preset protocol to obtain first image data, and the first image data is input into an artificial intelligence AI algorithm so that the AI algorithm performs image processing on the first image data to obtain second image data, and then the protocol of the second image data is adjusted to the same protocol as the original image data to obtain third image data, and the third image data is sent to an image signal processor ISP; Among them, the preset protocol is the protocol required by the AI algorithm; the ISP has other image processing functions in addition to protocol conversion and AI algorithm image processing functions.
2. The method according to claim 1, characterized in that The receiving of raw image data collected by the image sensor comprises: Receive valid image data in the raw image data input by the target interface; wherein the valid image data is obtained after the target interface performs synchronous parsing and protocol parsing on the raw image data collected by the image sensor.
3. The method according to claim 2, characterized in that If the image processing system is applied to an application-specific integrated circuit ASIC, the valid image data is in an intelligent peripheral interface IPI format.
4. The method according to claim 2, characterized in that: If the image processing system is applied to a field programmable gate array (FPGA), the valid image data is in an advanced extensible interface (AXI) format.
5. The method according to any one of claims 1 to 4, characterized in that Before inputting the first image data into the artificial intelligence AI algorithm, the method further includes: Performing data preprocessing on the first image data; wherein the data preprocessing includes at least one of the following data processing: image data format cropping, black level correction, data row and column transposition, multiple exposure analysis, and bad pixel correction; The step of inputting the first image data into an AI algorithm comprises: The preprocessed image data is input into the AI algorithm.
6. An image processing device, characterized in that: Applied to an image processing system, the device comprises: A data receiving module, used for receiving raw image data collected by the image sensor; a protocol conversion module, configured to convert the protocol of the original image data into a preset protocol based on protocol conversion to obtain first image data, and input the first image data into an artificial intelligence (AI) algorithm so that the AI algorithm performs image processing on the first image data to obtain second image data, then adjust the protocol of the second image data to the same protocol as the original image data to obtain third image data, and send the third image data to an image signal processor (ISP); Among them, the preset protocol is the protocol required by the AI algorithm; the ISP has other image processing functions in addition to protocol conversion and AI algorithm image processing functions.
7. The device according to claim 6, characterized in that The data receiving module is specifically used for: Receive valid image data in the raw image data input by the target interface; wherein the valid image data is obtained after the target interface performs synchronous parsing and protocol parsing on the raw image data collected by the image sensor.
8. The device according to claim 7, characterized in that If the image processing system is applied to an application-specific integrated circuit ASIC, the valid image data is in an intelligent peripheral interface IPI format.
9. The device according to claim 7, characterized in that If the image processing system is applied to a field programmable gate array (FPGA), the valid image data is in an advanced extensible interface (AXI) format.
10. The device according to any one of claims 6 to 9, characterized in that: Before the protocol conversion module inputs the first image data into the artificial intelligence AI algorithm, it also includes: A preprocessing module, used for performing data preprocessing on the first image data; wherein the data preprocessing includes at least one of the following data processing: image data format cropping, black level correction, data row and column transposition, multiple exposure analysis, and bad pixel correction; The step of inputting the first image data into an AI algorithm comprises: The preprocessed image data is input into the AI algorithm.
11. An image processing system, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 5 when executing a program stored in a memory.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 5 are implemented.