Data transmission, receiving method and device, chip, equipment, storage medium

Abstract

This application discloses a data transmission and reception method and apparatus, chip, device, and storage medium; wherein, the data transmission method includes: converting a first dataset to be transmitted from M channels into a second dataset to be transmitted from N channels; wherein, M is greater than 1, N is greater than 0 and less than M; packaging the N second datasets to be transmitted according to the MIPI protocol to obtain a first batch of one or more MIPI data packets; and transmitting the first batch of one or more MIPI data packets to an application processor through N data channels.

Description

Technical Field

[0001] The present application relates to electronic technology, and is related to but not limited to data transmission, receiving methods and devices, chips, equipment, and storage media. Background Art

[0002] The Mobile Industry Processor Interface (MIPI) has become the main standard for image sensor data transmission. The original intention was to standardize the interface between the camera, display, RF baseband and processor inside the mobile phone to meet the needs of large-capacity image transmission. However, the increase in data channels of MIPI hardware means an increase in hardware costs. Summary of the invention

[0003] In view of this, the data transmission and reception method and apparatus, chip, device, and storage medium provided in the embodiments of the present application can reduce the hardware requirements for MIPI, and even if there are more data sets to be transmitted, fewer data channels can be used to complete the data transmission. The data transmission and reception method and apparatus, chip, device, and storage medium provided in the embodiments of the present application are implemented as follows:

[0004] The data transmission method provided by the embodiment of the present application includes: converting a first data set to be transmitted from M channels into a second data set to be transmitted from N channels; wherein M is greater than 1, and N is greater than 0 and less than M; packaging the N channels of the second data set to be transmitted according to the MIPI protocol to obtain a first batch of one or more MIPI data packets; and transmitting the first batch of one or more MIPI data packets to an application processor through N data channels.

[0005] The data receiving method provided in the embodiment of the present application includes: determining whether a whole line of data is currently received based on a specific line data length; if a whole line of data is currently received, generating a line interrupt; in response to the line interrupt, parsing a line of data currently received to obtain parsed data; and sending the parsed data to a corresponding first image processing chip.

[0006] The image processing chip provided by the embodiment of the present application includes: a conversion circuit, used to convert a first data set to be transmitted from M channels into an N-channel second data set to be transmitted; wherein M is greater than 1, and N is greater than 0 and less than M; a MIPI-TX circuit, used to package the N-channel second data sets to be transmitted according to the MIPI protocol to obtain a first batch of one or more MIPI data packets; and transmit the first batch of one or more MIPI data packets to an application processor through N-channel data channels.

[0007] The application processor provided in the embodiment of the present application includes: a MIPI-RX circuit, which is used to determine whether a whole line of data is currently received based on a specific line data length; if a whole line of data is currently received, sending a line interrupt to a data parsing module; the data parsing module is used to parse a line of data currently received in response to the line interrupt, obtain parsed data, and send the parsed data to a corresponding first image processing chip; the first image processing chip is used to execute a specific image processing algorithm according to each parsed data received.

[0008] An electronic device provided by an embodiment of the present application includes: a second image processing chip, at least N data channels and an application processor, N being greater than 0; wherein a conversion circuit of the second image processing chip converts a first data set to be transmitted from M channels into N second data sets to be transmitted; wherein M is greater than 1, and N is greater than 0 and less than M; a MIPI-TX circuit of the second image processing chip packages the N second data sets to be transmitted according to the MIPI protocol to obtain a first batch of one or more MIPI data packets; and transmits the first batch of one or more MIPI data packets to a MIPI-RX circuit of an application processor through N data channels; the MIPI-RX circuit of the application processor determines whether a whole row of data is currently received according to a specific row data length; if a whole row of data is currently received, a row interrupt is sent to a data parsing module; the data parsing module of the application processor, in response to the row interrupt, parses a row of data currently received to obtain parsed data, and sends the parsed data to a corresponding first image processing chip; the first image processing chip of the application processor is used to execute a specific image processing algorithm according to each parsed data received.

[0009] The data transmission device provided in the embodiment of the present application includes: a conversion module, which is used to convert a first data set to be transmitted from M channels into an N-channel second data set to be transmitted; wherein M is greater than 1, and N is greater than 0 and less than M; a packing module, which is used to pack the N-channel second data set to be transmitted according to the MIPI protocol to obtain a first batch of one or more MIPI data packets; and a transmission module, which is used to transmit the first batch of one or more MIPI data packets to an application processor through N-channel data channels.

[0010] The data receiving device provided in the embodiment of the present application includes: a receiving module, which is used to determine whether a whole line of data is currently received based on a specific line data length; and if a whole line of data is currently received, send a line interrupt to a data parsing module; a data parsing module, which is used to parse a line of data currently received to obtain parsed data; and send the parsed data to a corresponding first image processing chip.

[0011] An electronic device provided in an embodiment of the present application includes a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the method on the image processing chip side described in the embodiment of the present application is implemented, or when the processor executes the program, the method on the application processor side described in the embodiment of the present application is implemented.

[0012] The computer-readable storage medium provided in the embodiment of the present application has a computer program stored thereon, characterized in that when the computer program is executed by a processor, the method on the image processing chip side described in the embodiment of the present application is implemented, or when the computer program is executed by a processor, the method on the application processor side described in the embodiment of the present application is implemented.

[0013] In the embodiment of the present application, when receiving the first data set to be transmitted from M channels, these data sets are not directly packaged according to the MIPI protocol, but the M channels of the first data set to be transmitted are first organized into N channels of the second data set to be transmitted which is smaller than M channels; then, the N channels of the second data set to be transmitted are packaged according to the MIPI protocol; thus, compared with the solution of directly packaging and transmitting the M channels of the first data set to be transmitted, the second image processing chip can complete the data transmission through fewer data channels, that is, transmit the data to the application processor through N channels instead of M channels. It can be seen that the solution provided by the embodiment of the present application reduces the hardware requirements for MIPI, and even if there are more channels of data sets to be transmitted, fewer data channels can be used to complete the data transmission, so that MIPI with fewer hardware data channels can also meet the data transmission requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0015] Figure 1 A schematic diagram of the basic architecture of data transmission for image processing;

[0016] Figure 2 A schematic diagram of the implementation flow of the data transmission method provided in the embodiment of the present application;

[0017] Figure 3 A schematic diagram of the implementation flow of the data transmission and data receiving method according to an embodiment of the present application;

[0018] Figure 4 A schematic diagram of the structure of an electronic device according to an embodiment of the present application;

[0019] Figure 5 A schematic diagram of the position of 3A data output in the ISP pipeline (i.e., the ISP signal processing flow);

[0020] Figure 6 A schematic diagram of the position of 3A data output in the ISP pipeline (i.e., the ISP signal processing flow);

[0021] Figure 7 This is a schematic diagram of the data transmission process in the PreISP chip solution;

[0022] Figure 8 This is a schematic diagram of the architecture of the pre-processing ISP chip of the PreISP chip;

[0023] Fig. 9 A schematic diagram of the software framework of the AP side platform of the embodiment of the present application;

[0024] Fig.10 A schematic diagram of the implementation flow of the data transmission and receiving method according to an embodiment of the present application;

[0025] Fig.11 This is a schematic diagram comparing the results before and after the rearrangement of the embodiment of the present application;

[0026] Fig.12 This is a schematic diagram of the structure of the data transmission device according to an embodiment of the present application;

[0027] Fig.13 This is a structural diagram of a data receiving device according to an embodiment of the present application;

[0028] Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the present application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0031] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0032] It should be pointed out that the terms "first\second\third" etc. involved in the embodiments of the present application do not represent a specific ordering of objects. It can be understood that "first\second\third" etc. can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0033] In order to facilitate the understanding of the embodiments of the present application, a basic architecture of data transmission for image processing is first provided. Figure 1 This is a schematic diagram of the architecture, which only shows some key modules, such as Figure 1 As shown, the architecture 1 includes a second image processing chip 10 and an application processor 11; wherein the second image processing chip 10 includes an ISP chip 101, a MIPI-TX module 102, an ISP chip 103 and a MIPI-TX module 104; the application processor 11 includes a MIPI-RX module 111, an ISP chip 112, a 3A algorithm module 113, a MIPI-RX module 114, an ISP chip 115 and a 3A algorithm module 116; wherein the 3A algorithm is an automatic focus (Automatic Focus, AF) algorithm, an automatic exposure (Automatic Exposure, AE) algorithm and an automatic white balance (Automatic White Balance, AWB) algorithm;

[0034] The image sensor 20 transmits the sensing data to the ISP chip 101, which processes the data and transmits the obtained statistical data (hereinafter referred to as 3A data), raw image data and PD data required for the 3A algorithm to the MIPI-TX module 102. The MIPI-TX module 102 transmits the received data sets from various channels to the MIPI-RX module 111 on the application processor side through the MIPI data channel. The MIPI-RX module 111 transmits the 3A data and PD data to the 3A algorithm module 113, and transmits the raw image data to the ISP chip 112; wherein the PD data is phase information, which is used for focusing.

[0035] The transmission path of the sensing data of the image sensor 21 and the type of the image sensor 20 are not described in detail here.

[0036] It should be noted that the above architecture 1 is only an example given for the convenience of understanding the embodiments of the present application, and cannot limit the scope of the embodiments of the present application. There is no limitation on the structure of the second image processing chip and the structure of the application processor applicable to the embodiments of the present application, and there is no limitation on the data content transmitted by the data channel, which is not limited to 3A data, PD data, and raw image data.

[0037] The present application provides a data transmission method for transmitting image data between an application processor and an image sensor. Figure 2 A schematic diagram of the implementation flow of the data transmission method provided in the embodiment of the present application is shown in FIG. Figure 2 As shown, the method may include the following steps 201 to 203:

[0038] Step 201 : The second image processing chip converts M first data sets to be transmitted into N second data sets to be transmitted; wherein M is greater than 1, and N is greater than 0 and less than M.

[0039] It is understandable that in the image processing solution, a lot of data needs to be transmitted through MIPI, such as 3A data, raw image data, and PD data, etc. And these data sets to be transmitted, the same type of data sets to be transmitted, may come from different sources, and accordingly, the transmission paths are also different. For example, the type of the first data set to be transmitted is 3A data, wherein the statistical data required for the AWB algorithm comes from the LSC module, the statistical data required for the AE algorithm comes from the WB module, and the statistical data required for the AF algorithm comes from the Demosaic module. For another example, the type of the first data sets to be transmitted is all raw image data, wherein one path of raw image data comes from the first image sensor, and the other path of raw image data comes from the second image sensor; for another example, the type of the first data sets to be transmitted is all raw image data, but one path is short exposure data, one path is medium exposure data, and one path is long exposure data.

[0040] It should be noted that the first data sets to be transmitted on the M channels can be data sets of different types or data sets of the same type. In the case of the same type, for example, in some embodiments, the first data set to be transmitted includes at least one of the following: image statistical data (referred to as 3A data), PD data and original image data required to execute the 3A algorithm, that is, the first data sets to be transmitted on the M channels are all 3A data, or, are all PD data, or are all original image data. In the case of different types, for example, the first data sets to be transmitted on the M channels include 3A data, PD data and original image data, etc.

[0041] In some embodiments, the M-channel first data sets to be transmitted are data sets generated at the same time or in the same time period. M-channels can be understood as M transmission paths or M different modules. The meaning of N-channels is similar to that of M-channels. For the second image processing chip, the N-channel second data sets to be transmitted may come from N-channel transmission paths or may be output by N different modules.

[0042] Step 202: The second image processing chip packages the N second data sets to be transmitted according to the MIPI protocol to obtain the first batch of one or more MIPI data packets.

[0043] There may be a situation where the length of a second data set to be transmitted exceeds the upper limit of the specified MIPI data packet length, so it may be necessary to split the data set into two MIPI data packets.

[0044] Step 203: The second image processing chip transmits the first batch of one or more MIPI data packets to the application processor through N data channels.

[0045] Understandably, in the MIPI protocol, if several data channels are generated at the same time or in the same time period, they are usually transmitted using the same number of data channels. For example, if 3 channels of raw image data are generated at the same time, the MIPI-TX module that complies with the MIPI protocol will transmit through 3 data channels. This places higher requirements on the number of data channels of MIPI hardware so that it can meet the data transmission needs in various scenarios. For example, the number of data channels is required to be able to cope with not only 3 data sets generated at the same time, but also 5, 6 or even 8 data sets generated at the same time. However, the increase in hardware data channels brings with it hardware costs and design space costs. How to cope with data transmission needs in different scenarios at a lower hardware cost within a limited device space has become a technical problem to be solved.

[0046] In an embodiment of the present application, when a first data set to be transmitted is received from M channels, these data sets are not packaged directly according to the MIPI protocol, but are first organized into N channels of second data sets to be transmitted that are smaller than M channels; then, these N channels of second data sets to be transmitted are packaged according to the MIPI protocol; in this way, compared with the solution of directly packaging and transmitting the M channels of first data sets to be transmitted, the second image processing chip can complete data transmission through fewer data channels, that is, transmit the data to the application processor through N channels instead of M channels.

[0047] The present application provides a method for data transmission and data reception. Figure 3 Schematic diagram of the implementation flow of the data transmission and data receiving method of the embodiment of the present application, such as Figure 3 As shown, the method may include the following steps 301 to 308:

[0048] Step 301: The second image processing chip decomposes the data to be transmitted from the first data set to be transmitted in M ​​channels into at least one row of data according to a specific row data length.

[0049] In some embodiments, the length of each row of data after decomposition is fixed, and is a specific row data length. Further, in some embodiments, there is a certain interval between adjacent rows, that is, blank data, so that a certain amount of time can be reserved for the receiving end to process the received row data.

[0050] There is no limit on the length of a specific row of data. For example, the length of each row of data is 1024 bytes.

[0051] Step 302: The second image processing chip rearranges the at least one row of data to obtain N second data sets to be transmitted; wherein M is greater than 1 and N is greater than 0.

[0052] In some embodiments, the second data set to be transmitted includes both the content of the data itself and metadata of the data.

[0053] In the embodiment of the present application, the value of N is not limited. It can be set according to the number of MIPI data channels, and N is less than or equal to the number of MIPI data channels. For example, the first data set to be transmitted from 3 channels is organized as the second data set to be transmitted from 1 channel. For another example, the first data set to be transmitted from 6 channels is organized as the second data set to be transmitted from 2 channels. M and N may be in a multiple relationship or not.

[0054] In some embodiments, the second image processing chip can implement step 302 as follows: determine the first type of each row of data; update the label used to characterize the first type of the row of data into the row of data; and rearrange the updated at least one row of data to obtain the N second data sets to be transmitted.

[0055] For example, the first type of label is used to identify whether the corresponding row data is the content of the data itself or metadata. Each row of data is marked with a corresponding first type, so that for the application processor, when parsing the received MIPI data packet, there is no need to analyze and identify the type of the row of data, thereby improving the overall data processing efficiency of the application processor.

[0056] Further, in some embodiments, the at least one row of data after update can be rearranged as follows: determine the second type of each row of data after update; the second type belongs to the subtype of the first type; rearrange the at least one row of data after update according to the mapping relationship between the second type and the row number to obtain the N-way second data set to be transmitted; in this way, it is convenient for the application processor to quickly parse. For example, the application processor can directly determine the second type of the row of data according to the row number of the currently received data.

[0057] Furthermore, the second image processing chip can know the second type that should be stored in each row according to the preset mapping table, that is, the mapping relationship between the second type and the row number is recorded in the mapping table. For example, as shown in Table 1, the first row is used to store the raw image data of long exposure, the second row is used to store the raw image data of short exposure, the third row is used to store the raw image data of medium exposure, the fourth row is used to store the statistical data required by the AWB algorithm in the 3A data (hereinafter referred to as AWB data), the fifth row is used to store the statistical data required by the AE algorithm in the 3A data (hereinafter referred to as AE data), the sixth row is used to store the statistical data required by the AF algorithm (hereinafter referred to as AF data), and so on.

[0058] Table 1

[0059] First row Long exposure raw image data Second row Short exposure raw image data Third row Raw image data exposed in Fourth row AWB data Fifth row AE Data Line 6 AF data

[0060] The data length of each row is fixed, and the number of rows can be adjusted according to the size of the total data set received. The corresponding mapping tables are different for total data sets of different sizes. For example, if the first M-channel data set to be transmitted is raw image data, its total data size is 3K bytes, and the specific row data length is 1024 bytes, the content of the corresponding mapping table is shown in Table 2, wherein the first row is agreed to store long exposure raw image data, the second row is agreed to store medium exposure raw image data, and the third row is agreed to store short exposure raw image data. For another example, if the first M-channel data set to be transmitted is raw image data, its total data size is 6K bytes, and the specific row data length is 1024 bytes, the content of the corresponding mapping table is shown in Table 3, wherein the first and second rows are agreed to store long exposure raw image data, the third and fourth rows are agreed to store medium exposure raw image data, and the fifth and sixth rows are agreed to store short exposure raw image data.

[0061] Table 2

[0062] First row Long exposure raw image data Second row Long exposure raw image data Third row Raw image data exposed in Fourth row Raw image data exposed in Fifth row Short exposure raw image data Line 6 Short exposure raw image data

[0063] Table 3

[0064]

[0065]

[0066] When the second image processing chip re-arranges the at least one row of data after update, it can call the corresponding mapping table according to the M-way first data set to be transmitted or the total data size of the at least one row of data after update, and then re-arrange the at least one row of data after update according to the second type corresponding to each row as agreed in the mapping table.

[0067] Step 303: The second image processing chip packages the N second data sets to be transmitted according to the MIPI protocol to obtain the first batch of one or more MIPI data packets.

[0068] It can be understood that packaging is to encapsulate data, for example, adding header information at the beginning of the second data set to be transmitted, adding trailer information at the end of the second data set to be transmitted, etc. It can be seen that the encapsulated second data set to be transmitted, that is, the data length of each row in the MIPI data packet is a specific row data length. In some embodiments, one second data set to be transmitted is encapsulated into one MIPI data packet.

[0069] Step 304: The second image processing chip evenly distributes the first batch of one or more MIPI data packets to the N data channels and transmits them to the application processor through the N data channels.

[0070] For example, N data lanes can transmit MIPI data packets line by line to the application processor.

[0071] The second image processing chip evenly or unevenly distributes the first batch of one or more MIPI data packets to N data channels. Different data channels are responsible for transmitting the allocated MIPI data packets. In some embodiments, one data channel can transmit the MIPI data packets to the application processor in serial line by line; wherein there is a fixed size of blank data between adjacent lines of data, that is, a blank interval, the purpose of which is to reserve a certain amount of time for the MIPI-RX module of the application processor to parse the received line data.

[0072] Step 305, the application processor determines whether a whole row of data is currently received according to the specific row data length; if yes, execute step 306; otherwise, continue to execute step 305;

[0073] Step 306, the application processor generates a line interrupt;

[0074] Step 307: In response to the line interrupt, the application processor parses a line of data currently received to obtain parsed data.

[0075] It can be understood that in the embodiment of the present application, each time the MIPI-RX module of the application processor receives a line of data, a line interrupt is generated, thereby triggering the data parsing module to parse the currently received line of data; in this way, the line interrupt is used to achieve parsing while transmitting, without waiting for all line data (i.e., cached data for image processing) to be transmitted before parsing, so that data transmission can achieve zero-delay transmission, thereby improving the overall data processing efficiency of the application processor. For example, for the AF algorithm, quickly obtaining parsed data can increase the autofocus speed, thereby improving the user experience.

[0076] Step 308: The application processor sends the parsed data to the corresponding first image processing chip.

[0077] In some embodiments, the application processor may implement step 307 as follows: determine the row number of the parsed data; determine the second type of the parsed data according to the agreed second type stored in each row; and send the parsed data to the corresponding first image processing chip according to the second type of the parsed data.

[0078] For example, if the second type of row data obtained by parsing is 3A data or PD data, the data is transmitted to the 3A algorithm module, which is an image processing chip. For another example, if the second type of parsed data is raw image data, the data is transmitted to the ISP chip.

[0079] An embodiment of the present application provides an electronic device, Figure 4 is a schematic diagram of the structure of an electronic device according to an embodiment of the present application, such as Figure 4 As shown, the electronic device 4 includes a second image processing chip 40, an application processor 41 and at least N data channels 42; wherein the second image processing chip 40 includes a conversion circuit 401 and a MIPI-TX circuit 402; the application processor 41 includes a MIPI-RX circuit 411, a data parsing module 412 and a first image processing chip 413; wherein,

[0080] The conversion circuit 401 is used to convert the first data set to be transmitted from M channels into the second data set to be transmitted from N channels; wherein M is greater than 1, and N is greater than 0 and less than M;

[0081] The MIPI-TX circuit 402 is used to package the N second data sets to be transmitted according to the MIPI protocol to obtain the first batch of one or more MIPI data packets; and

[0082] The first batch of one or more MIPI data packets are transmitted to the MIPI-RX circuit 411 of the application processor through N data channels.

[0083] The MIPI-RX circuit 411 determines whether a whole row of data is currently received according to a specific row data length; if a whole row of data is currently received, a row interrupt is sent to the data parsing module 412; otherwise, the receiving continues and determines whether a whole row of data is received according to the length of the currently received data;

[0084] The data parsing module 412 , in response to the line interrupt, parses a line of data currently received to obtain parsed data, and sends the parsed data to the corresponding first image processing chip 413 .

[0085] In some embodiments, the functions of the data analysis module 412 may be implemented by software, for example, by running relevant programs through a CPU to implement the functions of the module.

[0086] The first image processing chip 413 is used to execute a specific image processing algorithm according to each received parsed data.

[0087] For example, the first image processing chip 413 is a chip for implementing the 3A algorithm. In another example, the first image processing chip 413 is an ISP chip.

[0088] In some embodiments, the conversion circuit 401 can decompose the data to be transmitted in the M-way first data set to be transmitted into at least one row of data according to a specific row data length; determine the first type of each row of data; update the label of the first type used to characterize the row data into the row data; determine the second type of each row of data after the update; the second type is a subtype of the first type; according to the mapping relationship between the second type and the row number, the at least one row of data after the update is rearranged to obtain the N-way second data set to be transmitted; the MIPI-TX circuit 402 is used to package the N-way second data set to be transmitted according to the MIPI protocol to obtain the first batch of one or more MIPI data packets; and transmit the first batch of one or more MIPI data packets to the MIPI-RX circuit 411 of the application processor through N data channels.

[0089] Accordingly, the MIPI-RX circuit 411 can determine whether a whole row of data is currently received based on a specific row of data length; if a whole row of data is currently received, a row interrupt is sent to the data parsing module 412; the data parsing module 412, in response to the row interrupt, parses a row of data currently received to obtain parsed data; determines the row number of the parsed data; determines the second type of the parsed data based on the agreed second type stored in each row; and sends the parsed data to the corresponding first image processing chip based on the second type of the parsed data.

[0090] It should be noted that the above description of the electronic device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the electronic device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0091] In the PreISP chip (ie, an example of the second image processing chip) solution, a lot of data needs to be transmitted through the Mobile Industry Processor Interface (MIPI), for example, 3A data, raw image data, and PD data all need to be transmitted through MIPI.

[0092] The relevant introduction of 3A data is as follows:

[0093] The 3A algorithms, namely the automatic exposure (AE) algorithm, the automatic white balance (AWB) algorithm and the autofocus (AF) algorithm, require statistical data that are generated in the image signal processing (ISP) chip. The ISP chip counts the brightness and color of each pixel in the received image data, and the statistical results will be mainly used for the 3A algorithm.

[0094] The location where 3A data is output in the ISP pipeline (i.e., the ISP signal processing flow) is as follows Figure 5 As shown, the statistical data required by the AF algorithm include: statistics on the image output by the image sensor, statistics on the image output by the Degamma module, statistics on the image output by the Demosaic module, and statistics on the image output by the YUV NR / Sharpening module; the statistical data required by the AE algorithm include: statistics on the image output by the LSC module, statistics on the image output by the WB module, statistics on the image output by the CCM module, and statistics on the image output by the CSM module; the statistical data required by the AWB algorithm include: statistics on the image output by the LSC module, statistics on the image output by the Demosaic module, and statistics on the image output by the CCM module. It can be seen that there are many statistical locations for AF.

[0095] It should be noted that the BLC module is used for black level correction; the Degamma module is used to adjust the contrast of the picture; the NR module is used for noise reduction; the LSC (Lens Shade Correction) module is used for lens shading correction; the WB module is used for white balance; the Demosaic module is used for demosaicing; the CCM module is used for color correction; the Gamma module is used to adjust the contrast of the picture; the Sharpening module is used for sharpening; the CSM module is used for color space conversion; and the YUV NR / Sharpening module is used for noise reduction / sharpening.

[0096] In some embodiments, the output location of 3A data is typically configured as follows: Figure 6 As shown, the statistical data required by the AWB algorithm is mainly obtained by counting the images output by the LSC module, the statistical data required by the AE algorithm is mainly obtained by counting the images output by the WB module, and the statistical data required by the AF algorithm is mainly obtained by counting the images output by the Demosaic module. Figure 6In the algorithm, the AE module is mainly used to count the statistical data required by the AE algorithm, the AF module is mainly used to count the statistical data required by the AF algorithm, and the AWB module is mainly used to count the statistical data required by the AWB algorithm.

[0097] Among them, the statistical data required by the AE algorithm mainly include R, G, B histograms and brightness information, and the statistical data is usually the region of interest (ROI) of the image.

[0098] The statistical data required by the AWB algorithm mainly include the statistics of the number of R, G, and B pixels in the ROI area, the ratio of red to green, and the ratio of blue to green.

[0099] The statistical data required by the AF algorithm include the sharpness values ​​of each small block in the ROI area.

[0100] From the current ISP structure, the ISP will generate a lot of AE statistics, AWB statistics and AF statistics, which need to be sent to the 3A algorithm module for processing. When the system includes multiple cameras, the statistics of each camera will be counted separately and then sent to the 3A algorithm module on the AP side.

[0101] The DOL sensor can generate three channels of raw image data and PD data simultaneously; the PD data is phase information used for focusing.

[0102] Figure 7 The data transmission process in the PreISP chip solution is shown. From the figure, it can be seen that due to hardware limitations, MIPI will transmit this data in multiple channels.

[0103] In the case of smooth zoom and other modes, when multiple cameras need to be turned on at the same time, the 3A data of each image sensor needs to be transmitted back, and each sensor needs to be counted, which doubles the data volume. In addition, the 3DOL (snoy multi-exposure) sensor will output long, medium, and short raw image data. 3A data needs to be counted for each data channel. If there are 2 3DOL sensors in the system, 3 channels × 2 sensors × 3A data need to be counted. It can be seen that at least 18 types of statistical data need to be used and transmitted. Add (3 channels of Raw image data + 3 channels of PD data) × 2 sensors, and there are a total of 30 channels of data.

[0104] However, due to hardware limitations, the above PreISP chip solution encounters the following problems:

[0105] (1) The number of data channels of MIPI hardware is small and cannot meet the demand;

[0106] (2) After the data is arranged in random order, the parsing performance is poor, which becomes a bottleneck affecting performance.

[0107] Based on this, an exemplary application of an embodiment of the present application in a practical application scenario will be described below.

[0108] In the embodiment of the present application, the MIPI data of the PreISP chip is organized into 1024 bytes per line through buffer hardware, different data form a frame, and different types of data are placed in fixed lines by convention. When the platform end (i.e., the AP end) receives the MIPI data line by line, and generates a line interrupt. The platform end counts the line number and parses the received data according to the line number.

[0109] like Figure 8 As shown in the figure, only the key modules are listed. The architecture of the pre-processing ISP chip of the PreISP chip, the hardware mainly includes: CPU, MIPI, NPU and conversion circuit, etc.; Among them,

[0110] CPU: Responsible for interacting with the AP. Receives control commands and data from the AP and sends information back to the AP. Also responsible for controlling modules such as the ISP chip and NPU.

[0111] ISP chip: responsible for basic image processing. Similar to conventional ISP functions, it includes modules such as bad pixel correction, dark current correction, lens shading correction, digital gain, white balance, demosaic, color correction, gamma and noise reduction;

[0112] Conversion circuit: responsible for organizing multiple channels of MIPI data into one channel of 3A, one channel of raw image data and one channel of PD data.

[0113] NPU: runs AI algorithms to process preview data, such as noise reduction, HDR, and / or super-resolution.

[0114] MIPI-TX module: packages and encapsulates the organized data (such as 3A, raw image data, and PD data), and then sends it to the ISP chip on the AP side through the MIPI protocol. The ISP chip on the AP side further processes the received data.

[0115] AP module:

[0116] Data parsing (Unpack) module: It is a software module used to unpack the packaged data of the PreISP chip. After unpacking, the 3A data is divided into the statistical data required by the AE, AF, and AWB algorithms, and the raw data is divided into three channels of long, medium, and short data. After unpacking, the corresponding algorithm can be used directly.

[0117] ISP chip: responsible for basic image processing, mainly used for photo processing in the embodiment of this application. Similar to the functions of conventional ISP chips, it is more powerful than the ISP chip in the PreISP chip, but it cannot be customized and its functions are relatively fixed. An ISP chip contains ISP 0 and ISP 1, and the ISP chip can receive data. In addition, the 3A algorithm is executed on the CPU.

[0118] CPU: responsible for the control of the PreISP chip, such as power on, power off, firmware loading and runtime control.

[0119] 3A algorithm module: used to process 3A data and realize 3A functions.

[0120] In the embodiment of the present application, the software framework of the AP side platform is as follows: Fig. 9 As shown,

[0121] The kernel layer mainly includes device module drivers, such as the image sensor driver, I2C driver, SDIO driver, and the PreISP chip driver on the platform. The driver can be used to configure device initialization and state switching.

[0122] The HAL layer mainly controls the platform side to receive the image data, statistical data and ambient light information output by the PreISP chip through SDIO, and then executes the 3A algorithm; Zero Shutter Lang (ZSL) is responsible for receiving and parsing raw image data, as well as further post-processing algorithms for raw image data and ISP pipeline control for photography.

[0123] Framework layer: Android's original framework, which is the middleware between HAL and APP.

[0124] App layer: mainly uses the camera through Google's standard Camera API2, the system camera or third-party camera applications;

[0125] The operation flow chart of the embodiment of the present application is as follows Fig.10 As shown, the steps include steps 101 to 106:

[0126] Step 101, turn on the dual cameras and control the pre-processing PreISP chip to start working;

[0127] Step 102, the PreISP chip starts to process the image data and generates 3A data (stats), and sends these statistical data to the conversion circuit.

[0128] Step 103: The conversion circuit organizes the multiple channels of raw image data, PD data and 3A data into one channel of 3A data, one channel of raw image data and one channel of PD data.

[0129] Step 104, packaging the data into a MIPI data packet and sending it to the AP through the MIPI-TX module;

[0130] Step 105: The AP receives the MIPI data packet and sends the MIPI data packet to the data parsing module for depacketization.

[0131] Step 106, the unpacked raw image data is sent to the ISP chip, and the ISP chip processes it based on the pipeline. The unpacked 3A data and PD data are sent to the 3A algorithm module.

[0132] The conversion circuit of the PreISP chip is a hardware-implemented module that mainly implements the following functions:

[0133] The PreISP chip receives the raw image data, PD data and generated 3A data from the image sensor; then, the conversion circuit rearranges these data. Fig.11 As shown in the figure, before rearrangement, the data length of each row is different, and after rearrangement, each row has a fixed data length, that is, 1024 bytes, and there is a data interval of a certain length (that is, blank) between adjacent rows. After rearrangement, the raw image data is organized into one data set, the PD data is organized into one data set, and the 3A data is organized into one data set.

[0134] After rearranging the data, each row of data corresponds to a fixed length of data, and a fast parsing protocol has been reached. The packaging protocol is as follows:

[0135] (1) After packing, the packed row width is fixed at 1024 bytes.

[0136] (2) Each packed row corresponds to a mapping table, which is used to allocate fixed data by agreement. For example, the first row is the raw image data of long exposure, and the second row is the raw image data of short exposure, etc.

[0137] The data parsing module on the AP side is a software module. Using the same mapping table as the PreISP chip, the received data blocks can be quickly parsed by software. The pseudo code is as follows:

[0138]

[0139] Among them, i represents the line number, line_count_of_pack represents the number of lines in the MIPI data packet, the length of each line is 1024, the value of line_count_of_pack is an integer, which can be adjusted according to the total data size. Each cycle will determine whether the line is raw image data, PD data or 3A data according to the mapping table; one cycle can parse all packets, and after parsing, they can be sent to the 3A algorithm module or ISP chip for processing.

[0140] The triggering mode of the data parsing module on the AP side can be triggered by line interrupt. In the MIPI protocol on the AP side, it can be set to generate a line interrupt for each line of data received. That is, the parsing is triggered while transmitting. There is no need to wait for all buffer data (buffer) to be transmitted before parsing.

[0141] In the embodiments of the present application, (1) in the PreISP chip solution, transmission is performed according to a fixed line spacing and a fixed number of lines, and the parsing speed is fast; (2) line interruption is used to parse while transmitting, so that the transmission can achieve zero-delay transmission; (3) 1024 bytes are transmitted per line, so that the parsing speed on the AP side is faster;

[0142] In an embodiment of the present application, the raw image data and meta data are packaged in a regular arrangement so that the MIPI receiving end can use less VC_DT and can receive quickly; for the raw image data and meta data, they are decomposed into several lines, each of which has a fixed length. A tag is added in front of each line, and the tag is used to indicate the data type (the types of raw image data and meta image data are inconsistent). Then, the data is packaged into a MIPI data packet of the same VC_DT. It is sent to the AP through the same VC_DT of the MIPI hardware.

[0143] Also, the line interrupt method is used to trigger the data analysis module to analyze line by line, thereby achieving quasi-zero delay analysis; the line interrupt generated when receiving the data packet is enabled on the AP side, so that when the line interrupt is received, it can receive and analyze line by line in real time. In this way, when processing AF autofocus data, data can be received and responded to faster.

[0144] The above solution of the embodiment of the present application can be extended to the transmission of other data, not limited to raw image data, 3A data and PD data. In the future, other newly added data types may also use this method.

[0145] Based on the foregoing embodiments, an embodiment of the present application provides a data transmission device, which includes the modules included and the units included in the modules, and can be implemented by a processor; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be an image processing chip, a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0146] Fig.12 Schematic diagram of the structure of the data transmission device according to the embodiment of the present application. Fig.12 As shown, the data transmission device 12 includes:

[0147] The conversion module 121 is used to convert the first data set to be transmitted from M channels into the second data set to be transmitted from N channels; wherein M is greater than 1, and N is greater than 0 and less than M;

[0148] A packing module 122 is used to pack the N second data sets to be transmitted according to the MIPI protocol to obtain a first batch of one or more MIPI data packets;

[0149] The transmission module 123 is used to transmit the first batch of one or more MIPI data packets to the application processor through N data channels.

[0150] In some embodiments, the conversion module 121 is used to: decompose the data to be transmitted in the M-way first data set to be transmitted into at least one row of data according to a specific row data length; rearrange the at least one row of data to obtain the N-way second data set to be transmitted; accordingly, the transmission module 123 is used to: evenly distribute the first batch of one or more MIPI data packets to the N-way data channels and transmit them to the application processor through the N-way data channels.

[0151] In some embodiments, the conversion module 121 is used to: determine the first type of each row of data; update the label used to characterize the first type of the row of data into the row of data; and rearrange the updated at least one row of data to obtain the N second data sets to be transmitted.

[0152] In some embodiments, the conversion module 121 is used to: determine the second type of each row of data after update; the second type is a subtype of the first type; and rearrange the at least one row of data after update according to the mapping relationship between the second type and the row number to obtain the N second data sets to be transmitted.

[0153] In some embodiments, the specific row data length corresponds to a maximum amount of data read from the memory within a unit time.

[0154] In some embodiments, the first data set to be transmitted includes at least one of the following: statistical data, PD data, and original image data required to execute the 3A algorithm.

[0155] The present application also provides a data receiving device. Fig.13 Schematic diagram of the structure of the data receiving device according to the embodiment of the present application. Fig.13 As shown, the data receiving device 13 includes:

[0156] The receiving module 131 is used to determine whether a whole line of data is currently received according to a specific line data length; and if a whole line of data is currently received, send a line interrupt to the data parsing module;

[0157] The data analysis module 132 is used to analyze a row of data currently received to obtain analyzed data; and send the analyzed data to the corresponding first image processing chip.

[0158] In some embodiments, the data parsing module 132 is used to: determine the row number of the parsed data; determine the second type of the parsed data according to the agreed second type stored in each row; and send the parsed data to the corresponding first image processing chip according to the second type of the parsed data.

[0159] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0160] It should be noted that the division of modules in the above device is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or may exist physically alone, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software functional unit. It may also be implemented in the form of a combination of software and hardware.

[0161] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0162] An embodiment of the present application provides an electronic device, Fig.14 Schematic diagram of the hardware entity of the electronic device of the embodiment of the present application, such as Fig.14 As shown, the electronic device 14 includes a memory 141 and a processor 142. The memory 141 stores a computer program that can be run on the processor 142. When the processor 142 executes the program, the steps in the method provided in the above embodiment are implemented.

[0163] It should be noted that the memory 141 is configured to store instructions and applications executable by the processor 142, and can also cache data to be processed or processed by the processor 142 and various modules in the electronic device 14 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0164] An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the method provided in the above embodiment are implemented.

[0165] An embodiment of the present application provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the steps of the method provided in the above method embodiment.

[0166] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium, storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0167] It should be understood that "one embodiment" or "an embodiment" or "some embodiments" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in one embodiment" or "in some embodiments" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. The above description of each embodiment tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. For the sake of brevity, this article will not repeat them.

[0168] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist at the same time, and object B exists alone.

[0169] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0170] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or modules can be electrical, mechanical or other forms.

[0171] The modules described above as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules; they may be located in one place or distributed on multiple network units; some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0172] In addition, all functional modules in the embodiments of the present application may be integrated into one processing unit, or each module may be a separate unit, or two or more modules may be integrated into one unit; the above-mentioned integrated modules may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0173] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0174] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0175] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0176] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0177] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0178] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data transmission method for transmitting image data between an application processor and an image sensor, characterized in that: The method is applied to a second image processing chip, and the method includes: Converting a first data set to be transmitted from M channels into a second data set to be transmitted from N channels; wherein M is greater than 1, and N is greater than 0 and less than M; According to the MIPI protocol, the N second data sets to be transmitted are packaged to obtain a first batch of one or more MIPI data packets; Transmitting the first batch of one or more MIPI data packets to the application processor through N data channels; The step of converting the first data set to be transmitted from M channels into the second data set to be transmitted from N channels includes: Decomposing the data to be transmitted in the M first data sets to be transmitted into at least one row of data according to a specific row data length; Rearranging the at least one row of data to obtain the N second data sets to be transmitted; The transmitting the first batch of one or more MIPI data packets to the application processor through N data channels includes: evenly distributing the first batch of one or more MIPI data packets to the N data channels and transmitting the first batch of one or more MIPI data packets to the application processor through the N data channels.

2. The method according to claim 1, characterized in that The step of rearranging the at least one row of data to obtain the N second data sets to be transmitted includes: Determine the first type of each row of data; updating a first type of label used to characterize the row data into the row data; The updated at least one row of data is rearranged to obtain the N second data sets to be transmitted.

3. The method according to claim 2, characterized in that The step of rearranging the updated at least one row of data to obtain the N second data sets to be transmitted includes: Determine a second type of each row of data after update; wherein the second type is a subtype of the first type; According to the mapping relationship between the second type and the row number, the at least one row of updated data is rearranged to obtain the N channels of second data sets to be transmitted. 4 . The method according to claim 1 , wherein the specific row data length corresponds to a maximum amount of data read from a memory in a unit time.

5. The method according to claim 1, characterized in that The first data set to be transmitted includes at least one of the following: image statistical data, PD data and original image data required to execute the 3A algorithm.

6. A data receiving method, characterized in that: The method is applied to an application processor, and the method includes: Determine whether a whole row of data is currently received according to a specific row of data length; the data includes one or more MIPI data packets sent by the second image processing chip; the MIPI data packets are obtained by the method described in any one of claims 1 to 5; If a whole line of data is currently received, a line interrupt is generated; In response to the line interruption, a line of data currently received is parsed to obtain parsed data; The parsed data is sent to the corresponding first image processing chip.

7. The method according to claim 6, characterized in that The step of sending the parsed data to a corresponding image processing module comprises: Determining the row number of the parsed data; Determine the second type of the parsed data according to the agreed second type stored in each row; According to the second type of the parsed data, the parsed data is sent to the corresponding first image processing chip.

8. A second image processing chip, characterized in that: The chip comprises: A conversion circuit, used for converting a first data set to be transmitted from M channels into a second data set to be transmitted from N channels; wherein M is greater than 1, and N is greater than 0 and less than M; The MIPI-TX circuit is used to package the N second data sets to be transmitted according to the MIPI protocol to obtain the first batch of one or more MIPI data packets; and Transmitting the first batch of one or more MIPI data packets to the application processor through N data channels; The step of converting the first data set to be transmitted from M channels into the second data set to be transmitted from N channels includes: Decomposing the data to be transmitted in the M first data sets to be transmitted into at least one row of data according to a specific row data length; Rearranging the at least one row of data to obtain the N second data sets to be transmitted; The transmitting the first batch of one or more MIPI data packets to the application processor through N data channels includes: evenly distributing the first batch of one or more MIPI data packets to the N data channels and transmitting the first batch of one or more MIPI data packets to the application processor through the N data channels.

9. An application processor, characterized in that: The application processor comprises: A MIPI-RX circuit, used for determining whether a whole line of data is currently received according to a specific line data length; if a whole line of data is currently received, sending a line interrupt to a data parsing module; the data includes one or more MIPI data packets sent by a second image processing chip; the MIPI data packets are obtained by the method described in any one of claims 1 to 5; a data parsing module, configured to parse a row of data currently received in response to the row interruption, obtain parsed data, and send the parsed data to the corresponding first image processing chip; The first image processing chip is used to execute a specific image processing algorithm according to each received parsed data.

10. An electronic device, characterized in that: The device comprises: a second image processing chip, at least N data channels and an application processor, where N is greater than 0; wherein: The conversion circuit of the second image processing chip is used to convert the first data sets to be transmitted from M channels into N channels of second data sets to be transmitted; wherein M is greater than 1, and N is greater than 0 and less than M; The MIPI-TX circuit of the second image processing chip is used to package the N-channel second data sets to be transmitted according to the MIPI protocol to obtain the first batch of one or more MIPI data packets; and transmit the first batch of one or more MIPI data packets to the MIPI-RX circuit of the application processor through the N-channel data channels; wherein, the conversion of the first data set to be transmitted from the M-channel to be transmitted into the N-channel second data set to be transmitted includes: according to a specific row data length, decomposing the data to be transmitted in the M-channel first data set to be transmitted into at least one row of data; rearranging the at least one row of data to obtain the N-channel second data set to be transmitted; wherein, the transmission of the first batch of one or more MIPI data packets to the application processor through the N-channel data channels includes: evenly distributing the first batch of one or more MIPI data packets to the N-channel data channels and transmitting them to the application processor through the N-channel data channels; The MIPI-RX circuit of the application processor is used to determine whether a whole line of data is currently received according to a specific line data length; if a whole line of data is currently received, send a line interrupt to the data parsing module; the data includes one or more MIPI data packets sent by the second image processing chip; the MIPI data packet is obtained by the method described in any one of claims 1 to 5; The data parsing module of the application processor is used to parse a row of data currently received in response to the row interrupt to obtain parsed data, and send the parsed data to the corresponding first image processing chip; The first image processing chip of the application processor is used to execute a specific image processing algorithm according to each received parsed data.

11. A data transmission device, characterized in that: The device comprises: A conversion module, used for converting a first data set to be transmitted from M channels into a second data set to be transmitted from N channels; wherein M is greater than 1, and N is greater than 0 and less than M; A packing module, used for packing the N second data sets to be transmitted according to the MIPI protocol to obtain the first batch of one or more MIPI data packets; A transmission module, used for transmitting the first batch of one or more MIPI data packets to the application processor through N data channels; The step of converting the first data set to be transmitted from M channels into the second data set to be transmitted from N channels includes: Decomposing the data to be transmitted in the M first data sets to be transmitted into at least one row of data according to a specific row data length; Rearranging the at least one row of data to obtain the N second data sets to be transmitted; The transmitting the first batch of one or more MIPI data packets to the application processor through N data channels includes: evenly distributing the first batch of one or more MIPI data packets to the N data channels and transmitting the first batch of one or more MIPI data packets to the application processor through the N data channels.

12. A data receiving device, characterized in that: The device comprises: A receiving module, configured to determine whether a whole line of data is currently received according to a specific line data length; and if a whole line of data is currently received, send a line interrupt to a data parsing module; the data includes one or more MIPI data packets sent by a second image processing chip; the MIPI data packets are obtained by the method described in any one of claims 1 to 5; The data analysis module is used to analyze a line of data currently received to obtain analyzed data; and send the analyzed data to the corresponding first image processing chip.

13. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 5 is implemented; or, when the processor executes the program, the method according to claim 6 or 7 is implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented; or, when the computer program is executed by a processor, the method according to claim 6 or 7 is implemented.

Images