Deinterleaving method and deinterleaving device

Abstract

The invention discloses a de-interleaving method and a de-interleaving device, the method is applied to a multi-core heterogeneous chip, and the multi-core heterogeneous chip at least comprises an interface module and a processing module; wherein the interface module is used for receiving target interleaving data for a target image; the method comprises the following steps: acquiring target interleaving data of an interface module by utilizing at least two different channels of a processing module, wherein the at least two different channels comprise a first channel and a second channel; adopting a first processing mode to process the target interleaved data of the first channel to obtain first data; adopting a second processing mode to process the target interleaved data of the second channel to obtain second data; and based on the first data and the second data, de-interleaving the target interleaved data to obtain target de-interleaved data, the de-interleaved data being used for restoring the target image.

Description

Technical Field

[0001] The present application relates to the field of interleaving technology, and in particular to a deinterleaving method and a deinterleaving device. Background Art

[0002] Given the widespread and compatible nature of Composite Video Broadcast Signal (CVBS), most image acquisition devices, such as cameras, support CVBS interfaces. CVBS signals can be processed by an analog video decoder to produce a video decoder output signal. The video decoder output signal can be considered an interlaced signal obtained by subjecting the CVBS signal to certain processing steps, such as interleaving. However, achieving efficient deinterleaving of interlaced signals has become a pressing technical challenge. Summary of the Invention

[0003] The present application provides a deinterleaving method and a deinterleaving device to at least solve the above technical problems existing in the prior art.

[0004] According to a first aspect of the present application, a deinterleaving method is provided, which is applied to a multi-core heterogeneous chip, wherein the multi-core heterogeneous chip includes at least an interface module and a processing module; wherein the interface module is configured to receive target interleaved data for a target image; and the method includes:

[0005] Obtaining target interleaved data of the interface module using at least two different channels of the processing module, wherein the at least two different channels include a first channel and a second channel;

[0006] Using a first processing method, processing the target interleaved data of the first channel to obtain first data;

[0007] Using a second processing method, processing the target interleaved data of the second channel to obtain second data;

[0008] The target interleaved data is deinterleaved based on the first data and the second data to obtain target deinterleaved data, where the deinterleaved data is used to restore the target image.

[0009] In one possible implementation manner, the first processing method is used to process the target interleaved data of the first channel to obtain the first data, including:

[0010] Obtaining data indicated by the first indication information from the target interleaved data of the first channel;

[0011] using the data indicated by the first indication information as the first data;

[0012] The first indication information comes from the first processing method.

[0013] In one possible implementation manner, the adopting the second processing mode to process the target interleaved data of the second channel to obtain the second data includes:

[0014] obtaining data indicated by the second indication information from the target interleaved data of the second channel;

[0015] using the data indicated by the second indication information as the second data;

[0016] The second indication information comes from the second processing method.

[0017] In one embodiment, the method further comprises:

[0018] The first indication information is used to indicate a first cropping position of the target interleaved data of the first channel;

[0019] Data corresponding to the first cropping position is cropped out from the target interleaved data of the first channel to obtain data indicated by the first indication information.

[0020] In one embodiment, the method further comprises:

[0021] The second indication information is used to indicate a second cropping position of the target interleaved data of the second channel;

[0022] Data corresponding to the second cropping position is cropped out from the target interleaved data of the second channel to obtain data indicated by the second indication information.

[0023] In one possible implementation manner, deinterleaving target interleaved data based on the first data and the second data to obtain target deinterleaved data includes:

[0024] The first data includes a plurality of sub-data, and the second data includes a plurality of sub-data;

[0025] Interleaving is performed on the plurality of sub-data of the first data and the plurality of sub-data of the second data to deinterleave target interleaved data to obtain target deinterleaved data.

[0026] In one embodiment, the method further comprises:

[0027] Grouping the multiple sub-data of the first data to obtain L groups;

[0028] Grouping the plurality of sub-data of the second data to obtain P groups;

[0029] Interleaving the L groups and the P groups to obtain data consisting of the L groups and the P groups interleaved;

[0030] The data consisting of L groups and P groups interleaved is used as target deinterleaved data;

[0031] Wherein, L and P are positive integers greater than or equal to 2.

[0032] In one embodiment, the method further comprises:

[0033] According to the group identifiers of the L groups, the L groups are sequentially written into the odd-numbered areas of the Q storage areas; according to the group identifiers of the P groups, the P groups are sequentially written into the even-numbered areas of the Q storage areas;

[0034] or,

[0035] According to the group identifiers of the L groups, the L groups are sequentially written into the even-numbered areas of the Q storage areas; according to the group identifiers of the P groups, the P groups are sequentially written into the odd-numbered areas of the Q storage areas;

[0036] The data written into the Q storage areas are used as target deinterleaved data;

[0037] Among them, Q storage areas are areas opened in the memory, Q=L+P.

[0038] In one embodiment, the interface module includes a first channel and a second channel; wherein the target interleaved data is received through the first channel; and the second channel is used to receive desired interleaved data for a desired image; wherein the target image and the desired image are images captured by different cameras;

[0039] The method further comprises:

[0040] obtaining desired interleaved data from the second channel of the interface module using at least two different channels of the processing module, wherein the at least two different channels include a third channel and a fourth channel;

[0041] Processing the expected interleaved data of the third channel using a third processing method to obtain third data;

[0042] Processing the expected interleaved data of the fourth channel using a fourth processing manner to obtain fourth data;

[0043] The expected interleaved data is deinterleaved based on the third data and the fourth data to obtain expected deinterleaved data, where the expected deinterleaved data is used to restore the expected image.

[0044] According to a second aspect of the present application, a deinterleaving device is provided, which is applied to a multi-core heterogeneous chip. The device includes:

[0045] An interface module, configured to receive target interleaved data for a target image;

[0046] a processing module, configured to obtain target interleaved data of the interface module through at least two different channels, wherein the at least two different channels include a first channel and a second channel;

[0047] Using a first processing method, processing the target interleaved data of the first channel to obtain first data;

[0048] Using a second processing method, processing the target interleaved data of the second channel to obtain second data;

[0049] The target interleaved data is deinterleaved based on the first data and the second data to obtain target deinterleaved data, where the deinterleaved data is used to restore the target image.

[0050] According to a third aspect of the present application, an electronic device is provided, including:

[0051] at least one processor; and

[0052] a memory communicatively connected to the at least one processor; wherein,

[0053] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in this application.

[0054] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the present application.

[0055] According to a fifth aspect of the present application, a computer program product is provided, comprising a computer program or instructions, which implement the method described in the present application when executed by a processor.

[0056] The technical solution of the present application utilizes at least two different channels of a processing module to obtain data to be deinterleaved (target interleaved data), and adopts respective processing methods to perform targeted processing on the interleaved data of respective channels, which can effectively reduce the deinterleaving time, avoid the resource consumption problem caused by the need to use more resources such as processing resources and storage resources for deinterleaving in related technologies, and achieve efficient deinterleaving.

[0057] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The above and other objects, features and advantages of the exemplary embodiments of the present application will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present application are shown in an illustrative and non-limiting manner, in which:

[0059] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

[0060] Figure 1 A schematic diagram of a multi-core heterogeneous system in an embodiment of the present application is shown. Figure 1 ;

[0061] Figure 2 A schematic diagram of a multi-core heterogeneous system in an embodiment of the present application is shown. Figure 2 ;

[0062] Figure 3 A schematic diagram of a multi-core heterogeneous system in an embodiment of the present application is shown. Figure 3 ;

[0063] Figure 4 The schematic diagram of the implementation process of the deinterleaving method in the embodiment of the present application is shown Figure 1 ;

[0064] Figure 5 The schematic diagram of the implementation process of the deinterleaving method in the embodiment of the present application is shown Figure 2 ;

[0065] Figure 6 The implementation block diagram of the deinterleaving method in the embodiment of the present application is shown. Figure 1 ;

[0066] Figure 7 The implementation block diagram of the deinterleaving method in the embodiment of the present application is shown. Figure 2 ;

[0067] Figure 8 The implementation block diagram of the deinterleaving method in the embodiment of the present application is shown. Figure 3 ;

[0068] Figure 9 The following is a schematic diagram showing the structure of a deinterleaving device in an embodiment of the present application;

[0069] Figure 10 A schematic diagram of the structure of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0070] In order to make the purpose, features, and advantages of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0071] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0072] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0073] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0074] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0075] It should be understood that in the various embodiments of the present application, the size of the serial number of each implementation process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0076] The deinterleaving device of the present application is located in a multi-core heterogeneous system, and the processing logic of the deinterleaving method of the present application is deployed in the multi-core heterogeneous system. In the embodiment of the present application, the multi-core heterogeneous system is a multi-core heterogeneous chip. A multi-core heterogeneous chip refers to a chip that integrates two or more processor cores in a single chip. For example, a single SOC (system on chip) chip that integrates two or more processor cores. Each processor core in the multi-core heterogeneous chip can act as an independent processor, and can independently run the instructions that each processor core needs to run to achieve the tasks that each processor core needs to achieve. It can be understood that a multi-core heterogeneous chip is a chip with a multi-core processor. Compared with a single-core processor chip, the independent operation of each core task can speed up the operation speed and improve the multi-task execution capability, thereby bringing the advantage of high performance. And the multi-core processor is set on the same chip, which has the advantage of low cost.

[0077] like Figure 1 As shown, the multi-core heterogeneous chip of the present application includes multiple processor cores. The multiple processor cores include a first processor core, a second processor core...an Lth processor core. L is a positive integer greater than or equal to 2, and is flexibly set according to actual conditions. Among the multiple processor cores, each processor core is equivalent to a computing engine, and its type and / or quantity may be different. Among them, the types of processor cores include cores with strong computing power and cores with strong real-time performance (fast computing). In actual applications, most of the multiple processor cores are processor cores of different types, and a few cores are processor cores of the same type. Alternatively, the multiple processor cores may be processor cores of different types, and thus the multi-core heterogeneous chip is composed of two or more processor cores with different architectures. The difference in the type and / or number of processor cores can achieve different architectures between the processor cores to a certain extent.

[0078] In practical applications, among all processor cores, as long as there are two or more processor cores of different types, such processor cores can be called multi-core heterogeneous, and the chip including these processor cores can be regarded as a multi-core heterogeneous chip.

[0079] For example, because embedded processors (ARM) have the advantages of low cost and low power consumption, digital signal processors (DSPs) have the advantage of specialized digital processing, and field-programmable gate arrays (FPGAs) have the advantage of high-speed processing, each type of processor is used as a processor core. These types of processors are designed on the same SOC chip, resulting in a multi-core heterogeneous SOC chip.

[0080] like Figure 2As shown, each processor core and the hardware resources connected to each processor core, such as a clock controller, an interrupt controller, and a memory space, constitute each hardware domain. That is, a multi-core heterogeneous chip includes multiple hardware domains. In a multi-core heterogeneous chip, each hardware domain is a collection of a group of hardware resources. Different hardware domains are isolated from each other, and this isolation can be regarded as a physical isolation, such as the hardware design within the same hardware domain is at a similar position in the multi-core heterogeneous chip, and the hardware design within different hardware domains is at different positions in the multi-core heterogeneous chip to achieve isolation in physical position. Of course, the mutual isolation between different hardware domains in the embodiment of the present application may not be physical isolation, but logical isolation. This logical isolation can be reflected in: hardware resources within the same hardware domain need to use the same communication identifier to access within the hardware domain. That is, hardware resources within the same hardware domain can access each other based on the communication identifier within the hardware domain. Hardware resources in different hardware domains are accessed using different communication identifiers.

[0081] In practical applications, it is preferred that the mutual isolation between different hardware domains is a logical isolation, which can at least save chip space.

[0082] like Figure 3 As shown, in a multi-core heterogeneous chip, an operating system can be configured for each hardware domain. For example, a first operating system is configured for the first hardware domain, a second operating system is configured for the second hardware domain, and so on. The operating systems configured for different hardware domains can be the same, can be different, and are preferably different. For example, the first operating system configured for the first hardware domain is a Linux system, and the second operating system configured for the second hardware domain is an Android system. Among them, since the Linux system has the characteristics of high security and the Android system has the characteristics of lightness, tasks with high security requirements in the multi-core heterogeneous system can be handed over to Linux for execution, and tasks that need to run lightly in the multi-core heterogeneous system can be handed over to the Android system for execution. Therefore, different operating systems on different hardware domains can be used in the multi-core heterogeneous system to achieve efficient execution of each task.

[0083] In a multi-core heterogeneous chip, an operating system may be configured for each hardware domain in most hardware domains according to actual needs, while an operating system may not be configured for a small number of hardware domains, depending on specific usage.

[0084] In the embodiments of the present application, there is also a need for communication between hardware domains. When communication needs exist between different hardware domains, an inter-core communication mechanism can be used to achieve communication between hardware domains. The inter-core communication mechanism in a multi-core heterogeneous system includes a mailbox mechanism suitable for instruction transmission and a memory sharing mechanism suitable for data sharing. Inter-core communication within a single SOC chip ensures that data is transmitted within the same chip, ensuring data security and transmission speed.

[0085] Typically, hardware resources within different hardware domains vary. These differences may manifest themselves in hardware types, models, and quantities. This diversity, to a certain extent, reflects the heterogeneity of multi-core heterogeneous systems. As discussed previously, the concept of multi-core heterogeneity in this application is a hardware-level concept and has nothing to do with the software level.

[0086] like Figure 1 As shown, the multi-core heterogeneous chip in the embodiment of the present application also includes various types of control units. Various types of control units include but are not limited to: a power control unit, a non-volatile storage control unit, a volatile storage control unit, etc. Among them, the power control unit is used to control the power supply unit to realize the supply of power to the multi-core heterogeneous chip. The non-volatile storage control unit is used to control the access of at least one processor core in the multi-core heterogeneous chip to the non-volatile storage unit. The volatile storage control unit is used to control the access of at least one processor core in the multi-core heterogeneous chip to the volatile storage unit.

[0087] Among them, the power supply unit, non-volatile storage unit, and volatile storage unit are hardware resources outside the multi-core heterogeneous chip and can be called upon when the multi-core heterogeneous chip needs them. In addition, hardware such as the audio output unit (such as a speaker), audio acquisition unit (such as a microphone), and video output unit (such as a display) are hardware resources outside the multi-core heterogeneous chip and can also be called upon when the multi-core heterogeneous chip needs them to achieve normal audio and video output.

[0088] The deinterleaving method of the embodiment of the present application is implemented on a multi-core heterogeneous chip. The multi-core heterogeneous chip includes an interface module and a processing module. The interface module is used to obtain interleaved data from the outside that needs to be deinterleaved. The processing module obtains the interleaved data that needs to be deinterleaved from the interface module through two different channels of the processing module. By using respective processing methods to perform targeted processing on the interleaved data of respective channels, the deinterleaving time can be effectively reduced, and the resource consumption problem caused by the need to use more resources such as processing resources and storage resources for deinterleaving in related technologies can be avoided, thereby achieving efficient deinterleaving.

[0089] The deinterleaving scheme in this application is suitable for any deinterleaving scenario that can be achieved through the cooperation between the interface module and the processing module in a multi-core heterogeneous chip. Among them, the following scenarios are included in the suitable deinterleaving scenarios:

[0090] It can be understood that an analog video decoder has two output modes: frame mode and field mode. The signals output in the two output modes can be regarded as interlaced signals obtained by interlacing CVBS in their respective modes. The interlaced signal output by the video decoder needs to be deinterlaced to restore the original signal. Compared with the field mode of the video decoder, the frame mode has better stability and does not have the aliasing problem, so the frame mode is widely used in the industry. Based on this, the technical solution of the present application can achieve efficient deinterlacing of the interlaced signal output by the video decoder in the frame mode.

[0091] Figure 4 The schematic diagram of the implementation process of the deinterleaving method in the embodiment of the present application is shown Figure 1 . The deinterleaving method in this application is applied to a multi-core heterogeneous chip. The multi-core heterogeneous chip includes at least an interface module and a processing module; wherein the interface module is used to receive target interleaved data for a target image. The target interleaved data may be data originating from outside the multi-core heterogeneous chip. The multi-core heterogeneous chip receives interleaved data from the outside through the interface module.

[0092] like Figure 4 As shown, the deinterleaving method in the embodiment of the present application includes the following steps.

[0093] S401: Obtain target interleaved data of an interface module by utilizing at least two different channels of a processing module, wherein the at least two different channels include a first channel and a second channel.

[0094] In this application, the target interleaved data is data obtained by interleaving the target image. The target interleaved data is data to be deinterleaved, which in this application is data that can be deinterleaved using the scheme shown in S401 to S404.

[0095] The processing module includes two or more channel interfaces, each corresponding to a channel. Thus, the processing module includes two or more channels. During implementation, the interface module can send target interleaved data to the first and second channel interfaces of the processing module, respectively. The first channel interface receives the target interleaved data to obtain the target interleaved data using the first channel. The second channel interface receives the target interleaved data to obtain the target interleaved data using the second channel.

[0096] Using two channels to obtain the data to be deinterleaved (target interleaved data) can greatly facilitate the subsequent independent processing of the data to be deinterleaved (target interleaved data) of each channel.

[0097] S402: Process the target interleaved data of the first channel using a first processing method to obtain first data.

[0098] S403: Process the target interleaved data of the second channel using a second processing method to obtain second data.

[0099] S402 and S403 can be performed sequentially or simultaneously, depending on the specific circumstances. The first processing mode and the second processing mode can be the same processing mode or different processing modes. Preferably, they are different processing modes. Using respective processing modes, the target interleaved data of different channels are processed to obtain respective processed data (first data and second data). This achieves targeted processing of the interleaved data of different channels, ensuring the independent processing and accuracy of the data of different channels.

[0100] S404: Deinterleave the target interleaved data based on the first data and the second data to obtain target deinterleaved data, where the deinterleaved data is used to restore the target image.

[0101] In this step, the target interleaved data is deinterleaved based on two data (first data and second data) obtained by processing the data to be deinterleaved of the two channels, providing a new technical solution for data deinterleaving.

[0102] S401-S404 utilize at least two different channels of the processing module to obtain the target interleaved data (target interleaved data), and then employ respective processing methods to perform targeted processing on the interleaved data of the different channels, thereby achieving deinterleaving of the target interleaved data. This effectively reduces deinterleaving time, avoids the resource consumption problem caused by the high number of processing and storage resources required for deinterleaving in related technologies, and achieves efficient deinterleaving.

[0103] In this application, the data of different channels are processed separately, which ensures the independence and accuracy of data processing, thereby improving the accuracy of deinterleaving, and is an efficient deinterleaving solution. The deinterleaving solution of this application is easy to implement and promote in engineering.

[0104] In the present application, the first processing method and the second processing method include respective indication information. For example, the first processing method includes first indication information, and the second processing method includes second indication information. The function of the indication information is to indicate the data that can be obtained from the target interleaved data. Based on this, the scheme of using the first processing method in the present application to process the target interleaved data of the first channel to obtain the first data may be: obtaining the data indicated by the first indication information from the target interleaved data of the first channel; using the data indicated by the first indication information as the first data; wherein the first indication information is derived from the first processing method. The scheme of using the second processing method in the present application to process the target interleaved data of the second channel to obtain the second data may be: obtaining the data indicated by the second indication information from the target interleaved data of the second channel; using the data indicated by the second indication information as the second data; wherein the second indication information is derived from the second processing method.

[0105] In this application, by using indication information under different processing modes, that is, using different indication information to achieve targeted processing of target interleaved data of different channels, the accuracy of processing of the to-be-deinterleaved data of different channels can be achieved, the deinterleaving time can be effectively reduced, and the efficiency of deinterleaving can be ensured.

[0106] In the present application, the first indication information and the second indication information can be used to indicate the cropping position. Furthermore, the first indication information is used to indicate the first cropping position of the target interleaved data of the first channel; the data corresponding to the first cropping position in the target interleaved data of the first channel is cropped to obtain the data indicated by the first indication information. That is, the data corresponding to the first cropping position in the cropped target interleaved data is the first data obtained by processing the target interleaved data of the first channel using the first processing method. The second indication information is used to indicate the second cropping position of the target interleaved data of the second channel; the data corresponding to the second cropping position in the target interleaved data of the second channel is cropped to obtain the data indicated by the second indication information. That is, the data corresponding to the second cropping position in the cropped target interleaved data is the second data obtained by processing the target interleaved data of the second channel using the second processing method.

[0107] In the present application, the first indication information and the second indication information can be used to indicate the filtering data. Furthermore, the first indication information is used to indicate which data / data in the target interleaved data of the first channel are filtered (or extracted). For the target interleaved data of the first channel, the data indicated by the first indication information is filtered (or extracted), and the filtered (or extracted) data is used as the first data. Similarly, the second indication information is used to indicate which data / data in the target interleaved data of the second channel are filtered (or extracted). For the target interleaved data of the second channel, the data indicated by the second indication information is filtered (or extracted), and the filtered (or extracted) data is used as the second data.

[0108] Typically, the first indication information and the second indication information indicate different cropping positions, that is, the first cropping position and the second cropping position are different. The first indication information and the second indication information indicate different filtered data. The first indication information and the second indication information can both indicate cropping positions, or both indicate filtered (or extracted) data. Of course, it is also possible that one of the first indication information and the second indication information indicates a cropping position, and the other indicates filtered (or extracted) data. This is not specifically limited.

[0109] In the above solution, the processing of the to-be-deinterleaved data of different channels is achieved through the two types of indication information (first indication information and second indication information) given in the two processing modes, which is easy to implement and promote in engineering.

[0110] In the present application, the data obtained by processing target interleaved data of different channels using different processing methods includes multiple sub-data. For example, the first data includes multiple sub-data, and the second data includes multiple sub-data. Thus, in the present application, a scheme for deinterleaving the target interleaved data based on the first data and the second data to obtain target deinterleaved data can be: interleaving the multiple sub-data of the first data and the multiple sub-data of the second data to deinterleave the target interleaved data to obtain the target deinterleaved data.

[0111] In the present application, multiple sub-data of the first data and multiple sub-data of the second data can be deinterleaved in the form of equal data interleaving. For example, the interleaving processing of the two data (the first data and the second data) is achieved by interleaving the 1st to 5th sub-data of the first data, the 1st to 5th sub-data of the second data, the 6th to 10th sub-data of the first data, the 6th to 10th sub-data of the second data... etc., so as to achieve deinterleaving of the target sub-data. Equal data interleaving in the present application means that both data are interleaved with the same number of sub-data, such as, in the aforementioned example, the first data and the second data are interleaved with 5 sub-data.

[0112] The multiple sub-data of the first data and the multiple sub-data of the second data can be deinterleaved in the form of unequal data interleaving. In this application, unequal data interleaving refers to interleaving two data with at least partially different numbers of sub-data.

[0113] For example, interleaving is performed by interleaving the first to fifth sub-data of the first data, the first to third sub-data of the second data, the sixth to tenth sub-data of the first data, the fourth to sixth sub-data of the second data, the eleventh to fifteenth sub-data of the first data, and the seventh to ninth sub-data of the second data. In this example, interleaving is performed using five sub-data of the first data and three sub-data of the second data, respectively, with different numbers of sub-data.

[0114] As another example, interleaving is performed by interleaving the first to fifth sub-data of the first data, the first to third sub-data of the second data, the sixth to ninth sub-data of the first data, the fourth to seventh sub-data of the second data, the tenth to fifteenth sub-data of the first data, and the eighth to fourteenth sub-data of the second data. In this example, the interleaving is performed on both the first and second data using a random number of sub-data.

[0115] The target deinterleaved data may be the result of interleaving the multiple sub-data of the first data and the multiple sub-data of the second data. In the interleaving processing scheme of the present application, the interleaving processing of the multiple sub-data of the first data and the multiple sub-data of the second data also includes any other reasonable situations, which are not described in detail in this application.

[0116] By interleaving multiple sub-data included in two types of data (first data and second data), deinterleaving of the data to be deinterleaved can be achieved. Compared with the complex and cumbersome deinterleaving schemes in related technologies, the deinterleaving scheme in this application is simple and easy to implement. It can effectively reduce the deinterleaving time and improve the efficiency of the deinterleaving scheme.

[0117] In the present application, at least one of the first data and the second data can be grouped, and based on the grouping, deinterleaving of the target interleaved data can be achieved. Taking grouping of both the first data and the second data as an example, the scheme for deinterleaving the target interleaved data can be: grouping multiple sub-data of the first data to obtain L groups; grouping multiple sub-data of the second data to obtain P groups; interleaving the L groups and the P groups to obtain data consisting of the L and P groups interleaved; and using the data consisting of the L and P groups interleaved as the target deinterleaved data; wherein L and P are positive integers greater than or equal to 2.

[0118] In the present application, the multiple sub-data of the first data can be grouped into a random number of sub-data. For example, in L groups, the number of sub-data included in each group is different, or the number of sub-data included in at least some groups in each group is different. Alternatively, the multiple sub-data of the first data can be grouped into the same number of sub-data. That is, the number of sub-data included in each group is the same. Similarly, the multiple sub-data of the second data can be grouped into a random number of sub-data, or the same number of sub-data. The first data and the second data are interleaved in the form of the first group of the first data, the first group of the second data, the second group of the first data, the second group of the second data, etc., to achieve interleaving processing of the target interleaved data, thereby obtaining the target deinterleaved data.

[0119] Compared with the complex and tedious deinterleaving schemes in related technologies, this deinterleaving scheme implemented based on the grouped data of the first data and the second data can greatly reduce the deinterleaving time and ensure the high efficiency of deinterleaving.

[0120] When multiple sub-data of the first data and multiple sub-data of the second data are grouped, the multiple groups of the first data and the multiple groups of the second data can be grouped and identified so as to uniquely identify each group of the first data and each group of the second data. In addition, an area for storing each group is opened up in the memory. For example, Q (Q=L+P) storage areas are opened up, and the Q storage areas are divided into odd areas and even areas. Based on this, in the present application, the group identifications of the multiple groups of the first data and the multiple groups of the second data, as well as the odd areas and even areas of the Q storage areas, can be used to achieve deinterleaving of the target interleaved data. The specific scheme is as follows:

[0121] According to the group identifiers of the L groups, the L groups are sequentially written into the odd-numbered areas of the Q storage areas; according to the group identifiers of the P groups, the P groups are sequentially written into the even-numbered areas of the Q storage areas; or, according to the group identifiers of the L groups, the L groups are sequentially written into the even-numbered areas of the Q storage areas; according to the group identifiers of the P groups, the P groups are sequentially written into the odd-numbered areas of the Q storage areas; the data written into the Q storage areas is used as the target deinterleaved data.

[0122] Exemplarily, the first group in L groups is written to area 1 of Q storage areas, the second group in L groups is written to area 3 of Q storage areas, the third group in L groups is written to area 5 of Q storage areas, the fifth group in L groups is written to area 7 of Q storage areas, and so on, until all groups in L groups are written sequentially to odd-numbered areas of Q storage areas.

[0123] Write the first group in the P group to area 2 of the Q storage areas, write the second group in the P group to area 4 of the Q storage areas, write the third group in the P group to area 6 of the Q storage areas, write the fifth group in the P group to area 8 of the Q storage areas, and so on, until all the groups in the P groups are written sequentially to the odd-numbered areas of the Q storage areas.

[0124] In simple terms, the aforementioned solution deinterleaves the data to be deinterleaved by grouping the first and second data and writing the grouped data to memory. This one-step grouping and one-step writing operation deinterleaves the data to be deinterleaved, making it an efficient deinterleaving solution. This provides a feasible solution for deinterleaving the data to be deinterleaved.

[0125] In the present application, the interface module includes two or more interfaces, and each interface corresponds to a channel. That is, the interface module includes a first channel and a second channel. The target interleaved data obtained by interleaving the target image can be received through the first channel of the interface module. The second channel corresponding to the second interface of the interface module can be used to receive the expected interleaved data for the expected image. The target image and the expected image are images captured by different cameras. For example, the target image is an image captured by a camera that can capture the driver's face or full body image. The expected image is an image captured by a camera that can capture the passenger's face or full body image. The target interleaved data is the data obtained by interleaving the expected image, which can be used Figure 5 The scheme shown is used for deinterleaving. Figure 5 As shown, the de-interleaving scheme in this application also includes:

[0126] S501: Obtain desired interleaved data from a second channel of an interface module using at least two different channels of a processing module, wherein the at least two different channels include a third channel and a fourth channel.

[0127] In the present application, the desired interleaved data received by the second channel of the interface module from outside the multi-core heterogeneous chip can be sent by the second channel of the interface module to the third channel interface and the fourth channel interface of the processing module, respectively. The third channel interface of the processing module receives the desired interleaved data, so as to obtain the desired interleaved data using the third channel of the processing module. The third channel interface of the processing module receives the desired interleaved data, so as to obtain the desired interleaved data using the fourth channel of the processing module.

[0128] In the present application, when the first channel and the second channel of the processing module are used to receive the target interleaved data from the first channel of the interface module, the third channel and the fourth channel of the processing module can be used to receive the expected interleaved data from the second channel of the interface module. For each interleaved data from outside the multi-core heterogeneous chip, in the multi-core heterogeneous chip of the present application, two channels of the processing module can be used for reception. Using two channels to receive the data to be deinterleaved (target interleaved data or expected interleaved data) can facilitate the subsequent independent processing of the data to be deinterleaved from different channels.

[0129] S502: Using a third processing method, process the expected interleaved data of the third channel to obtain third data.

[0130] S503: Using a fourth processing method, process the expected interleaved data of the fourth channel to obtain fourth data.

[0131] S502 and S503 can be executed successively or simultaneously, depending on the specific circumstances. The third processing method and the fourth processing method can be the same processing method or different processing methods. Preferably, different processing methods. The third processing method and the fourth processing method can be different processing methods from the first processing method and the second processing method, or can be the same processing methods as the first processing method and the second processing method. In some embodiments, the third processing method can be the same processing method as one of the first processing method and the second processing method, and the fourth processing method can be the same processing method as the other processing method in the first processing method and the second processing method. For example, the third processing method is the same processing method as the first processing method, and the fourth processing method is the same processing method as the second processing method.

[0132] For understanding of the implementation process of S502 and S503, reference may be made to the aforementioned understanding of the implementation process of S402 and S403, and similarities are not repeated here.

[0133] The desired interleaved data of different channels are processed in respective processing modes to obtain respective processed data (third data and fourth data), thereby achieving targeted processing of the to-be-deinterleaved data of different channels and ensuring the independence and accuracy of processing of the data of different channels.

[0134] S504: Deinterleave the expected interleaved data based on the third data and the fourth data to obtain expected deinterleaved data, where the expected deinterleaved data is used to restore the expected image.

[0135] In this step, based on the two data (third data and fourth data) obtained by processing the to-be-deinterleaved data (desired interleaved data) of the two channels (the third channel and the fourth channel), the desired interleaved data is deinterleaved, providing a new technical solution for data deinterleaving. For an understanding of the implementation process of S504, please refer to the aforementioned implementation solution of S404 and will not be repeated here.

[0136] In steps S501 to S504, the third and fourth channels of the processing module are used to obtain the target interleaved data (target interleaved data). The interleaved data of the third and fourth channels are then processed in their respective processing modes, thereby achieving deinterleaving of the target interleaved data. This effectively reduces deinterleaving time, avoids the resource consumption problem caused by the high number of processing and storage resources required for deinterleaving in related technologies, and achieves efficient deinterleaving.

[0137] If the target image is an image captured by a camera capable of capturing the driver's face or full body image, and the desired image is an image captured by a camera capable of capturing the passenger's face or full body image, then the technical solution of the present application can restore the driver's face or full body image by deinterleaving the interleaved data for the target image. Furthermore, the passenger's face or full body image can also be restored by deinterleaving the interleaved data for the desired image, thereby enabling independent monitoring of the driver's and passenger's mental states, or whether they are wearing seat belts, etc., and is therefore suitable for such applications.

[0138] This application can also be considered as utilizing multiple interfaces of the interface module and multiple channels of the processing module to achieve deinterleaving of the interleaved data to restore the desired image and / or target image. This multi-interface and multi-channel solution ensures independent and accurate processing of the interleaved data, providing new technical support for deinterleaving solutions.

[0139] The following combination Figure 6 、 Figure 7 and Figure 8 The technical solution of this application is further explained.

[0140] exist Figure 6-Figure 8 In the embodiment, the interface module is MIPI-CSI, the processing module is a pixel processing module (CSI, Camera Sensor Interface) module, and the multi-core heterogeneous chip SOC includes a mobile industry processor interface camera serial interface (MIPI-CSI, Mobile Industry Processor Interface Camera Serial Interface), a CSI processing module, a memory (Memory) and a mobile industry processor interface display serial interface (MIPI-DSI, Mobile Industry Processor Interface Display Serial Interface) as an example, MIPI-CSI is the video acquisition interface device inside the SOC, and MIPI-DSI is the display output device interface inside the SOC. If the target interleaved data deinterleaved by the technical solution of the present application is regarded as the target image that has been restored, it can be output to the display screen (LCD) outside the SOC through the MIPI-DSI of the SOC for displaying the original image deinterleaved by the present application.

[0141] by Figure 6-Figure 8 For example, when applied to vehicles such as private cars and buses, the vehicle includes multiple cameras. Figure 6 and Figure 8 In the example, Camera is a camera used to capture the environment of the vehicle from all angles. Taking Camera 1 as a camera used to capture the driver's face or full-body image, and using the face or full-body image as the target image, the application scenario of the technical solution of the present application in a vehicle is explained.

[0142] exist Figure 6 and Figure 8 In the embodiment, the camera is connected to the video decoder through the CVBS interface. The video decoder can communicate with the SOC to execute the deinterlacing scheme of the present application. Furthermore, the video decoder can communicate with the SOC through the MIPI-CSI located in the SOC. CSI uses different channels to receive images from outside the SOC through the MIPI-CSI interface to implement the deinterlacing scheme of the present application. The deinterlaced image (which can be used as the target deinterlaced data), that is, the restored image is output to the LCD through MIPI-DSI.

[0143] Assume that the size of the driver's facial image captured by camera 1 is M rows * N columns, and both M and N are positive integers greater than or equal to 2. Camera 1 captures the driver's facial image and sends the captured driver's facial image to the video decoder through the cvbs interface. Taking into account that the output signal of the video decoder has good stability in the frame mode and does not have a jagged phenomenon, the video decoder uses the frame mode to perform interleaving processing on the driver's facial image. The interleaving processing of the video decoder using the frame mode can be: the odd rows in the driver's facial image are used as the top field (top field) of the image, and the even rows in the driver's facial image are used as the bottom field (bottom field) of the image. The image composed of the top field and the bottom field can be regarded as an interleaved image obtained by interleaving the driver's facial image in the frame mode. The interleaved image can be used as target interleaved data, waiting to be deinterleaved. As Figure 7 As shown, the size of the interlaced image is 960 (byte) × 480 rows as an example, wherein the upper half 960 × 240 is the top field, and the lower half 960 × 240 is the bottom field.

[0144] It can be understood that the odd and even lines of an image can be considered as interlaced lines of the image. Camera 1 can collect interlaced data (odd and even lines) of the driver's facial image and transmit the collected interlaced data to the video decoder via the CVBS interface. The video decoder integrates the two interlaced data of odd and even lines into a frame mode image with the upper half being the top field and the lower half being the bottom field. The frame mode image is an image obtained by interlacing processing in the frame mode of the video decoder. The reason why it has good stability is that a frame mode image includes all the information of the original image (driver's facial or full-body image). The information of the driver's facial or full-body image is not separated or divided and is all included in the frame mode image. The frame mode image is the aforementioned interlaced image with the upper half being the top field and the lower half being the bottom field.

[0145] In this application, the frame mode image obtained by the camera 1 and the video decoder outside the SOC is used as the target interleaved data for interleaving the driver's face or full body image. For the convenience of description, this image can be called an interleaved image.

[0146] like Figure 7As shown in the figure, at the software level, MIPI-CSI has at least four channels, VC0 to VC3. CSI also has at least four channels, IP10 to IP13, with corresponding outputs, CH0 to CH3. The video decoder transmits interlaced images to the SOC. Within the SOC, MIPI-CSI's first channel, VC0, receives the interlaced image. This interlaced image is then transmitted or mapped to CSI's first and second channels, IP10 and IP11. This allows CSI to use these channels to obtain the deinterlaced image from outside the SOC.

[0147] The CSI in this application has a cropping function (crop function) for cropping the top field of the interlaced image in the first channel IP10 and the bottom field in the second channel IP11. Considering that the size of the interlaced image is 960*480, the top half is the top field and the bottom half is the bottom field. If the top and bottom fields are to be cropped, the cropping positions for cropping the top field of the interlaced image and the cropping positions for cropping the bottom field of the interlaced image need to be specified.

[0148] In this application scenario, taking the coordinates of the upper left pixel of the interlaced image as (0, 0), the coordinates of the lower right pixel as (960, 480), the width of the image as 960 bytes, and the height as 480 lines as an example, the interlaced image in the first channel IP10 is cropped with a width of 960 bytes and a height of 240 lines according to the starting position (0, 0). The top field of the interlaced image can be cropped out and used as the first data.

[0149] The interlaced image in the second channel IP11 is cropped with a width of 960 bytes and a height of 240 lines according to the starting position (0, 240). The bottom field of the interlaced image can be cropped and used as the second data.

[0150] The cropping position may be (starting position abscissa, starting position ordinate, cropping width, cropping height). For the interlaced image in the first channel IP10, the cropping position (first cropping position) is (0, 0, 960, 240), and for the interlaced image in the second channel IP11, the cropping position (second cropping position) is (0, 240, 960, 240).

[0151] like Figure 7As shown, CSI outputs the top field of the interlaced image through ch0 and outputs the bottom field of the interlaced image through ch 1. Among them, the top field of the interlaced image output through ch0 can be considered as the data output through the first channel IP10 of CSI. The bottom field of the interlaced image output through ch1 can be considered as the data output through the second channel IP11 of CSI. Using the cropping function of CSI, ch0 only outputs the top field and ch1 only outputs the bottom field. Using the top field output by ch0 and the bottom field output by ch1, the interlaced image is deinterlaced to restore the driver's facial or full-body image. The process can be:

[0152] The top field output by ch0 includes multiple lines of data. The bottom field output by ch1 includes multiple lines of data. A buffer 1 area is allocated in the memory and used as a storage area. The size of the buffer 1 area is 960 bytes × 480 lines, which is equivalent to the sum of the top and bottom fields. The width of the buffer 1 area is 960 bytes and the height is 480 lines. Yes, the number of rows in the buffer 1 area is the same as the sum of the number of rows in the top and bottom fields, and the number of columns in the buffer 1 area is the same as the sum of the number of columns in the top and bottom fields. Simply put, the buffer 1 area can be considered to include odd and even rows.

[0153] CSI can sequentially write the data of each row of the top field to each odd-numbered row of the buffer 1 area, and sequentially write the data of each row of the bottom field to each even-numbered row of the buffer 1 area. For example, the first row of the top field is written to the first row of the buffer 1 area, the second row of the top field is written to the third row of the buffer 1 area, the third row of the top field is written to the fifth row of the buffer 1 area, and so on. Furthermore, the first row of the bottom field is written to the second row of the buffer 1 area, the second row of the bottom field is written to the fourth row of the buffer 1 area, the third row of the bottom field is written to the sixth row of the buffer 1 area, and so on, until all rows of buffer 1 are filled.

[0154] In this application, there are registers base and stride inside CSI. Each channel corresponds to a register base and stride. Among them, the base of each channel represents the base address of the memory where the data output by the channel needs to be written. The stride of each channel represents the offset address that needs to be generated when CSI wants to write the next row of data output by the channel after writing one row of data. By modifying the values ​​of the register base and stride of each channel, the data output by ch0 can be written to the odd rows such as 1, 3, 5, 7 in buffer 1, and the output data of ch1 can be written to the even rows such as 2, 4, 6, 8 in buffer 1. In this way, the deinterlacing operation of the interlaced image is completed, and the deinterlacing operation of the interlaced image is realized. Among them, the data in buffer 1 can be regarded as: the data obtained after the deinterlacing of the interlaced image from outside the SOC is realized through the cooperation between the MIPI-CSI and CSI of the SOC, and the processing of the CSI. This data can be used as the target deinterlaced data.

[0155] Each row of the top field can be considered as the result of grouping the multiple sub-data included in the top field, with each grouping forming a row, and each row including one or more sub-data of the top field. Each row of the bottom field can be considered as the result of grouping the multiple sub-data included in the bottom field, with each grouping forming a row, and each row including one or more sub-data of the top field. In the aforementioned scenario, the top field includes L rows, the bottom field includes P rows, and the number of rows in buffer 1 is P + L rows.

[0156] The image formed by the data in buffer 1 can be used as the driver's facial or full-body image restored by deinterlacing the interlaced image. The image size is 960 bytes * 480 lines. This image can be output to the LCD display located outside the SOC through the MIPI-DSI interface of the SOC for display.

[0157] It can be seen from the above scheme that for the interlaced image from the video decoder (located outside the SOC), the interlaced image is mapped to two different channels of the CSI through the cooperation of the MIPI-CSI interface and the CSI inside the SOC, and the corresponding fields in the two different channels are cropped using the cropping function of the CSI. The field data output by the two different channels are written to buffer 1 in an interlaced manner using an interlaced write operation, so as to deinterlace the interlaced data from the video decoder and restore the image captured by camera 1.

[0158] Simply put, the above technical solution can be regarded as a deinterlacing or de-interleaving solution that is achieved by cropping the corresponding fields in different channels and writing the corresponding field data into the buffer in an interlaced manner using an interlaced write operation. This deinterlacing solution is achieved through a one-step cropping and one-step interlaced write operation. This solution is simple and easy to implement, can effectively reduce the deinterlacing time, and avoids the resource consumption problem caused by the need to use more resources such as processing resources (CPU) and storage resources (requiring more buffers) for deinterlacing in related technologies, thereby achieving efficient deinterlacing.

[0159] In layman's terms, this application can directly restore the deinterlaced complete frame image through the cooperation between MIPI-CSI and CSI in the SOC, as well as the processing of CSI, shorten the deinterlacing time and improve the deinterlacing efficiency.

[0160] In addition, considering that the interlaced image is a signal obtained by interlacing the CVBS signal in the frame mode by the video decoder, the technical solution of the present application is a solution for deinterlacing the interlaced signal output by the video decoder in the frame mode. This solution can be regarded as a new deinterlacing solution proposed for the interlaced signal output by the video decoder in the frame mode. Because the frame mode has good stability, when the deinterlacing solution of the present application is used to deinterlace the interlaced image, the image captured by the camera can also be accurately and stably restored without the occurrence of aliasing caused by deinterlacing errors.

[0161] While camera 1 captures the driver's facial or full-body image and sends it to the video decoder for frame-mode interleaving so that the SOC can achieve deinterleaving, the vehicle's camera 2 can capture the passenger's facial or full-body image (which can be used as the desired image). The image size is also 960 bytes * 480 lines. The image is transmitted to the video decoder through the CVBS interface. The video decoder interleaves the image in frame mode to obtain an interleaved image (which can be used as the desired interleaved image). The video decoder transmits the interleaved image to the SOC. The second channel VC1 of the mipi-csi in the SOC receives the interleaved image. The interleaved image is transmitted or mapped to the third channel IP12 and the fourth channel IP13 of the CSI, so that the CSI uses the third channel IP12 and the fourth channel IP13 to obtain the image to be deinterleaved from outside the SOC.

[0162] The interlaced image in the third channel IP12 is cropped with a width of 960 bytes and a height of 240 lines according to the starting position (0, 0). The top field of the interlaced image can be cropped and used as the third data. The interlaced image in the fourth channel IP13 is cropped with a width of 960 bytes and a height of 240 lines according to the starting position (0, 240). The bottom field of the interlaced image can be cropped and used as the fourth data.

[0163] The third channel IP12 corresponds to ch2, and the fourth channel IP13 corresponds to ch3. Using the cropping function of CSI, ch2 only outputs the top field of the interlaced image, and ch3 only outputs the bottom field of the interlaced image. A buffer 2 area is opened in the memory and used as a storage area. For an understanding of the buffer 2 area, please refer to the above description of the buffer 1 area. CSI can write the data of each row of the top field to the odd rows of the buffer 2 area in sequence, and write the data of each row of the bottom field to the even rows of the buffer 2 area in sequence, until the rows of buffer 2 are full. Alternatively, CSI can write the data of each row of the top field to the even rows of the buffer 2 area in sequence, and write the data of each row of the bottom field to the odd rows of the buffer 2 area in sequence.

[0164] The image formed by the data in buffer 2 can be used as the passenger's facial or full-body image restored by deinterlacing the interlaced image. The image size is 960 bytes * 480 lines. This image can be output to the LCD display located outside the SOC through the MIPI-DSI interface of the SOC for display.

[0165] As can be seen from the aforementioned solution, for interlaced images from the video decoder (located outside the SOC), the interlaced images are mapped to two different CSI channels within the SOC through the cooperation of the MIPI-CSI interface and the CSI. The CSI's cropping function is used to crop the corresponding fields in the two different channels, and an interlaced write operation is used to write the field data output by the two different channels to buffer 2 in an interlaced manner. This deinterlaced data from the video decoder is then deinterlaced, restoring the image captured by camera 2. This solution, which achieves deinterlacing or de-interlacing (de-interlacing) through a one-step cropping and one-step interlaced write operation, is simple and easy to implement, effectively reducing deinterlacing time and improving deinterlacing efficiency.

[0166] In this application, the restored driver's facial or full-body image, in addition to being displayed, can also be used to analyze the driver's mental state based on the image to determine whether the driver has violated regulations such as smoking or drowsiness. The restored passenger's facial or full-body image, in addition to being displayed, can also be used to analyze the passenger's behavior based on the image to determine whether the passenger has violated regulations such as not wearing a seat belt.

[0167] In this application, by leveraging the coordination between MIPI-CSI and CSI, as well as CSI processing, it is possible to simultaneously deinterleave interleaved images captured by two or more cameras. This solution can be considered as utilizing the multiple interfaces of MIPI-CSI and the multiple channels of CSI to achieve image restoration. This multi-interface and multi-channel solution ensures the processing independence and accuracy of interleaved data, shortens deinterleaving time, and improves deinterleaving accuracy, providing new technical support for deinterleaving solutions.

[0168] If the interleaved image received from the video decoder through VC0 of mipi-csi is regarded as a data packet, the data packet can be received by the SOC through the VC 0 channel at the software level. At the hardware level, Figure 8 As shown, the hardware interfaces Clane 0 to Clane 3 correspond to the VC 0 channel in the software. The data packets received by the VC 0 channel in the software can be distributed to the four hardware interfaces Clane 0 to Clane 3. The four hardware interfaces are used to transmit the interleaved image from the Videodecoder to the SOC. At the hardware level, in addition to the four hardware interfaces Clane 0 to Clane 3, the video decoder and the SOC also include a clock signal Clane for indicating data packet transmission, an I2C signal for indicating the control signal for transmission, and an I2C clk signal for indicating the clock signal for transmitting the control signal. The deinterleaving scheme in this application can be deinterleaving implemented at the hardware level or at the software level.

[0169] In simple terms, the deinterlacing or de-interlacing implemented in this application is achieved in the following manner: MIPI-CSI distributes interlaced images from outside the SOC to two different channels of the CSI; the CSI uses the crop function to ensure that the output of the first channel, channel 0, retains only the top field of the interlaced image, and the output of the second channel, channel 1, retains only the bottom field of the interlaced image; by configuring the values ​​of the registers base addr and stride of each CSI channel, the two channels are controlled to write the top and bottom field data into buffer1 in an interlaced manner to complete the deinterlacing operation. The deinterlacing solution achieved through the one-step cropping and one-step writing operation is simple and easy to implement, does not increase excessive bandwidth and memory usage, can effectively shorten the deinterlacing time, and improve the efficiency of deinterlacing.

[0170] exist Figure 6-Figure 8 The description is based on the example of a memory located inside the SOC. In addition, the memory can also be located outside the SOC. The deinterleaving scheme when the memory is located outside the SOC is similar to the deinterleaving scheme when the memory is located inside the SOC, and is not described again.

[0171] The embodiment of the present application provides a deinterleaving device, which is applied to a multi-core heterogeneous chip. Figure 9 As shown, the device includes:

[0172] Interface module 901, configured to receive target interleaved data for a target image;

[0173] A processing module 902 is configured to obtain target interleaved data of an interface module through at least two different channels, wherein the at least two different channels include a first channel and a second channel;

[0174] Using a first processing method, processing the target interleaved data of the first channel to obtain first data;

[0175] Using a second processing method, processing the target interleaved data of the second channel to obtain second data;

[0176] The target interleaved data is deinterleaved based on the first data and the second data to obtain target deinterleaved data, where the deinterleaved data is used to restore the target image.

[0177] In some embodiments, the processing module 902 is configured to:

[0178] Obtaining data indicated by the first indication information from the target interleaved data of the first channel;

[0179] using the data indicated by the first indication information as the first data;

[0180] The first indication information comes from the first processing method.

[0181] In some embodiments, the processing module 902 is configured to:

[0182] obtaining data indicated by the second indication information from the target interleaved data of the second channel;

[0183] using the data indicated by the second indication information as the second data;

[0184] The second indication information comes from the second processing method.

[0185] In some embodiments, the processing module 902 is configured to:

[0186] The first indication information is used to indicate a first cropping position of the target interleaved data of the first channel;

[0187] Data corresponding to the first cropping position is cropped out from the target interleaved data of the first channel to obtain data indicated by the first indication information.

[0188] In some embodiments, the processing module 902 is configured to:

[0189] The second indication information is used to indicate a second cropping position of the target interleaved data of the second channel;

[0190] Data corresponding to the second cropping position is cropped out from the target interleaved data of the second channel to obtain data indicated by the second indication information.

[0191] In some embodiments, the processing module 902 is configured to:

[0192] The first data includes a plurality of sub-data, and the second data includes a plurality of sub-data;

[0193] Interleaving processing is performed on the multiple sub-data of the first data and the multiple sub-data of the second data to perform deinterleaving, thereby obtaining target deinterleaved data.

[0194] In some embodiments, the processing module 902 is configured to:

[0195] Grouping the multiple sub-data of the first data to obtain L groups;

[0196] Grouping the plurality of sub-data of the second data to obtain P groups;

[0197] Interleaving the L groups and the P groups to obtain data consisting of the L groups and the P groups interleaved;

[0198] The data consisting of L groups and P groups interleaved is used as target deinterleaved data;

[0199] Wherein, L and P are positive integers greater than or equal to 2.

[0200] In some embodiments, the processing module 902 is configured to:

[0201] According to the group identifiers of the L groups, the L groups are sequentially written into the odd-numbered areas of the Q storage areas; according to the group identifiers of the P groups, the P groups are sequentially written into the even-numbered areas of the Q storage areas;

[0202] or,

[0203] According to the group identifiers of the L groups, the L groups are sequentially written into the even-numbered areas of the Q storage areas; according to the group identifiers of the P groups, the P groups are sequentially written into the odd-numbered areas of the Q storage areas;

[0204] The data written into the Q storage areas are used as target deinterleaved data;

[0205] Among them, Q storage areas are areas opened in the memory, Q=L+P.

[0206] In some embodiments,

[0207] The interface module 901 includes a first channel and a second channel; wherein the target interleaved data is received through the first channel; the second channel is used to receive the desired interleaved data for the desired image; wherein the target image and the desired image are images captured by different cameras;

[0208] The processing module 902 is configured to:

[0209] Obtaining desired interleaved data for the second channel from the interface module using at least two different channels, wherein the at least two different channels include a third channel and a fourth channel;

[0210] Processing the expected interleaved data of the third channel using a third processing method to obtain third data;

[0211] Processing the expected interleaved data of the fourth channel using a fourth processing manner to obtain fourth data;

[0212] The expected interleaved data is deinterleaved based on the third data and the fourth data to obtain expected deinterleaved data, where the expected deinterleaved data is used to restore the expected image.

[0213] It should be noted that the deinterleaving device in the embodiment of the present application solves the problem based on a principle similar to that of the aforementioned deinterleaving method. Therefore, the implementation process and implementation principle of the deinterleaving device can refer to the description of the implementation process and implementation principle of the aforementioned method, and the repeated parts will not be repeated.

[0214] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0215] The electronic device includes at least one processor; and a memory in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the deinterleaving method of the present application. The computer instructions are used to enable the computer to perform the deinterleaving method of the present application.

[0216] The present application also provides a computer program product, comprising a computer program / instruction, which implements the deinterleaving method of the present application when executed by a processor.

[0217] Figure 10 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0218] like Figure 10 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0219] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0220] The computing unit 801 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the deinterleaving method. For example, in some embodiments, the deinterleaving method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the deinterleaving method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the deinterleaving method by any other appropriate means (e.g., by means of firmware).

[0221] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0222] The program code for implementing the methods of the present application can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow charts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0223] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0224] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0225] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0226] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0227] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A deinterleaving method, characterized in that: Applied to a multi-core heterogeneous chip, the multi-core heterogeneous chip at least includes an interface module and a processing module; wherein the interface module is used to receive target interleaved data for a target image; the method includes: Obtaining target interleaved data of the interface module using at least two different channels of the processing module, wherein the at least two different channels include a first channel and a second channel; Using a first processing method, processing the target interleaved data of the first channel to obtain first data; Using a second processing method, processing the target interleaved data of the second channel to obtain second data; Based on the first data and the second data, the target interleaved data is deinterleaved to obtain target deinterleaved data, and the deinterleaved data is used to restore the target image.

2. The method according to claim 1, characterized in that The method of adopting a first processing mode to process the target interleaved data of the first channel to obtain first data includes: Obtaining data indicated by the first indication information from the target interleaved data of the first channel; Using the data indicated by the first indication information as the first data; The first indication information comes from the first processing method.

3. The method according to claim 1, characterized in that The adopting the second processing mode to process the target interleaved data of the second channel to obtain the second data includes: obtaining data indicated by the second indication information from the target interleaved data of the second channel; using the data indicated by the second indication information as the second data; The second indication information comes from the second processing method.

4. The method according to claim 2, characterized in that: Also includes: The first indication information is used to indicate a first cropping position of target interleaved data of a first channel; Data corresponding to the first cropping position in the target interleaved data of the first channel is cropped to obtain data indicated by the first indication information.

5. The method according to claim 3, characterized in that: Also includes: The second indication information is used to indicate a second cropping position of the target interleaved data of the second channel; Data corresponding to the second cropping position in the target interleaved data of the second channel is cropped to obtain data indicated by the second indication information.

6. The method according to any one of claims 1 to 5, characterized in that: The step of deinterleaving the target interleaved data based on the first data and the second data to obtain the target deinterleaved data includes: The first data includes a plurality of sub-data, and the second data includes a plurality of sub-data; Interleaving processing is performed on a plurality of sub-data of the first data and a plurality of sub-data of the second data to deinterleave target interleaved data to obtain target deinterleaved data.

7. The method according to claim 6, characterized in that Also includes: Grouping the multiple sub-data of the first data to obtain L groups; Grouping the plurality of sub-data of the second data to obtain P groups; Interleaving the L groups and the P groups to obtain data consisting of the L groups and the P groups interleaved; Using data interleaved by L groups and P groups as target deinterleaved data; Wherein, L and P are positive integers greater than or equal to 2.

8. The method according to claim 7, characterized in that Also includes: According to the group identifiers of the L groups, the L groups are sequentially written into the odd-numbered areas of the Q storage areas; according to the group identifiers of the P groups, the P groups are sequentially written into the even-numbered areas of the Q storage areas; or, According to the group identifiers of the L groups, the L groups are sequentially written into the even-numbered areas of the Q storage areas; according to the group identifiers of the P groups, the P groups are sequentially written into the odd-numbered areas of the Q storage areas; The data written into the Q storage areas are used as target deinterleaved data; Among them, Q storage areas are areas opened in the memory, Q=L+P.

9. The method according to claim 1, characterized in that: The interface module includes a first channel and a second channel; wherein the target interleaved data is received through the first channel; the second channel is used to receive the expected interleaved data for the expected image; wherein the target image and the expected image are images captured by different cameras; The method further comprises: obtaining the desired interleaved data from the second channel of the interface module using at least two different channels of the processing module, wherein the at least two different channels include a third channel and a fourth channel; Using a third processing method, processing the expected interleaved data of the third channel to obtain third data; Using a fourth processing method, processing the expected interleaved data of the fourth channel to obtain fourth data; Based on the third data and the fourth data, the expected interleaved data is deinterleaved to obtain expected deinterleaved data, where the expected deinterleaved data is used to restore the expected image.

10. A deinterleaving device, characterized in that: Applied to a multi-core heterogeneous chip, the device comprises: An interface module, used for receiving target interleaved data for a target image; A processing module, configured to obtain target interleaved data of the interface module through at least two different channels, wherein the at least two different channels include a first channel and a second channel; Using a first processing method, processing the target interleaved data of the first channel to obtain first data; Using a second processing method, processing the target interleaved data of the second channel to obtain second data; Based on the first data and the second data, the target interleaved data is deinterleaved to obtain target deinterleaved data, and the deinterleaved data is used to restore the target image.

Images