Data interpolation processing method and device of CIS sensor and electronic equipment

Abstract

The invention discloses a data interpolation processing method and device of a CIS sensor and electronic equipment. The method comprises the steps that original data streams collected by a CIS sensor are sequentially stored in a multi-level cache, the CIS sensor is formed by splicing a plurality of IC arrays, and the original data streams are a set of data collected by the IC arrays; in the sequential storage process, under the condition that the interpolation position is detected, determining interpolation data of the interpolation position based on the data stored in the multi-level cache; and inserting the interpolation data into the original data stream to obtain a target data stream. The technical problems of limited interpolation position and low interpolation accuracy of the data interpolation processing method of the CIS sensor in the related technology are solved.

Description

Technical Field

[0001] The present invention relates to the field of sensors, and in particular to a data interpolation processing method, device and electronic equipment for a CIS sensor. Background Art

[0002] As the application range of complementary metal-oxide-semiconductor image sensor (CIS) becomes wider and wider, the requirements for scanning quality are also getting higher and higher. However, since the CIS sensor itself is composed of multiple integrated circuit arrays (ICs), there is an inherent problem of data loss at the IC seam. To improve image quality, it is necessary to fill in the seams.

[0003] In the interpolation scheme of the related art, the interpolation operation is completed during the data acquisition process, and the interpolation point is calculated based on the end data of the current channel and the start data of the next channel (such as Figure 1 As shown in the figure, there are problems such as limited interpolation position and low interpolation accuracy, which can easily lead to poor image data effect.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a data interpolation processing method, device and electronic device for a CIS sensor, so as to at least solve the technical problems of limited interpolation position and low interpolation accuracy existing in the data interpolation processing method for a CIS sensor in the related art.

[0006] According to one aspect of an embodiment of the present invention, a data interpolation processing method for a CIS sensor is provided, comprising: sequentially storing raw data streams collected by the CIS sensor in a multi-level cache, wherein the CIS sensor is composed of a plurality of IC arrays, and the raw data stream is a collection of data collected by the plurality of IC arrays; in the process of sequential storage, when an interpolation position is detected, determining interpolation data of the interpolation position based on data stored in the multi-level cache; and inserting the interpolation data into the raw data stream to obtain a target data stream.

[0007] Optionally, the raw data stream collected by the CIS sensor is stored in a multi-level cache in sequence, including: increasing the bit width of the front data corresponding to the multiple IC arrays in the raw data stream to obtain a processed data stream, wherein the front data is used to indicate the data collected from the starting position of the data collection of the previous IC array of the corresponding IC array to the corresponding IC splicing seam position, and the IC splicing seam position is used to indicate the splicing position between the corresponding IC array and the previous IC array; storing the processed data stream in the multi-level cache in sequence.

[0008] Optionally, storing the raw data stream collected by the CIS sensor in a multi-level cache in sequence includes: obtaining a data output order in which the CIS sensor outputs the raw data stream; and storing the raw data stream in the multi-level cache in sequence according to the data output order.

[0009] Optionally, in the process of sequential storage, when an interpolation position is detected, before determining the interpolation data of the interpolation position based on the data stored in the multi-level cache, the method also includes: in the process of sequential storage, detecting whether an IC splicing seam position is passed, wherein the IC splicing seam position is used to indicate the splicing position between the corresponding IC array and the previous IC array; in the case of detecting that the IC splicing seam position is passed, determining the passed IC splicing seam position as the interpolation position.

[0010] Optionally, determining the interpolation data of the interpolation position based on the data stored in the multi-level cache includes: obtaining neighbor data within a predetermined neighborhood range of the interpolation position from the data stored in the multi-level cache, wherein the predetermined neighborhood range includes a first neighborhood range before the interpolation position and a second neighborhood range after the interpolation position; and determining the interpolation data based on the neighbor data.

[0011] Optionally, after inserting the interpolation data into the original data stream to obtain the target data stream, the method further includes: determining the number of interpolation points of the interpolation position; determining the target distance based on the number of interpolation points; and determining the output data of the multi-level cache based on the target distance.

[0012] Optionally, the number of interpolation points at the interpolation position is one or more.

[0013] According to another aspect of an embodiment of the present invention, a data interpolation processing device for a CIS sensor is provided, comprising: a storage module, used to sequentially store the original data stream collected by the CIS sensor in a multi-level cache, wherein the CIS sensor is spliced ​​by multiple IC arrays, and the original data stream is a collection of data collected by the multiple IC arrays; a determination module, used to determine the interpolation data of the interpolation position based on the data stored in the multi-level cache when an interpolation position is detected during the sequential storage process; and an interpolation module, used to insert the interpolation data into the original data stream to obtain a target data stream.

[0014] According to another aspect of an embodiment of the present invention, a non-volatile storage medium is provided. The non-volatile storage medium stores a plurality of instructions. The instructions are suitable for being loaded by a processor and executing any one of the data interpolation processing methods for the CIS sensor.

[0015] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-mentioned data interpolation processing methods for the CIS sensor.

[0016] In an embodiment of the present invention, the original data stream collected by the CIS sensor is sequentially stored in a multi-level cache, wherein the CIS sensor is composed of a plurality of IC arrays, and the original data stream is a collection of data collected by the plurality of IC arrays; in the process of sequential storage, when an interpolation position is detected, the interpolation data of the interpolation position is determined based on the data stored in the multi-level cache; the interpolation data is inserted into the original data stream to obtain the target data stream, thereby achieving the goal of placing the multi-level cache before interpolation, providing temporary storage space for data through the use of the multi-level cache, and facilitating the calling of the previous and next data for interpolation calculation when needed. In this way, the purpose of finding a suitable interpolation position in the data stream without interrupting the continuity of data transmission is achieved, thereby achieving the technical effect of improving the data interpolation flexibility and interpolation accuracy of the CIS sensor, and further solving the technical problems of limited interpolation position and low interpolation accuracy in the data interpolation processing method of the CIS sensor in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0018] Figure 1is a schematic diagram of a data interpolation processing method of a CIS sensor according to the related art;

[0019] Figure 2 is a flow chart of a data interpolation processing method of a CIS sensor according to an embodiment of the present invention;

[0020] Figure 3 is a schematic diagram of an optional data interpolation processing method for a CIS sensor according to an embodiment of the present invention;

[0021] Figure 4 is an optional schematic diagram of multi-level cache output without interpolation according to an embodiment of the present invention;

[0022] Figure 5 is a schematic diagram of an optional interpolation of a point according to an embodiment of the present invention;

[0023] Figure 6 is a schematic diagram of an optional interpolation of multiple points according to an embodiment of the present invention;

[0024] Figure 7 is a schematic diagram of an optional post-interpolation shift according to an embodiment of the present invention;

[0025] Figure 8 is a schematic diagram of another optional interpolation of a point according to an embodiment of the present invention;

[0026] Fig. 9 is another optional schematic diagram of post-interpolation shifting according to an embodiment of the present invention;

[0027] Fig.10 4 is a schematic diagram of a data interpolation processing device for a CIS sensor according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] First, to facilitate understanding of the embodiments of the present invention, some terms or nouns involved in the present invention are explained below:

[0031] Complementary Metal-Oxide-Semiconductor Image Sensor (CIS) sensor is an integrated circuit chip used to convert light signals into electrical signals. CIS sensors are widely used in digital cameras, mobile phone cameras, security monitoring and other fields.

[0032] An integrated circuit array (IC) is an array structure composed of multiple integrated circuits. An IC array can contain multiple functional circuits, such as memory, processor, sensor, etc., and can implement multiple functions on the same chip, improving integration and performance. IC arrays are widely used in electronic products, such as microprocessors, memory chips, sensor arrays, etc.

[0033] Channel, in the data acquisition and processing of CIS sensors, a channel refers to an independent data acquisition or processing path in a CIS sensor. Each channel often corresponds to one or more IC (integrated circuit) arrays, which are responsible for converting optical signals into electrical signals and performing preliminary processing. When a CIS sensor is composed of multiple IC arrays, each IC array can be regarded as a "segment" of processed data, and these segments are arranged continuously to form the data acquisition area of ​​the entire sensor.

[0034] Pixels are the most basic photosensitive units on CIS sensors, which are responsible for capturing light intensity information in images. In an IC array or a channel, pixels are arranged in rows and columns to form a matrix. When reading data, the signals of these pixels are read out row by row or column by column.

[0035] In the interpolation scheme of the related art, the interpolation operation is completed during the data acquisition process, and the interpolation point is calculated based on the end data of the current channel and the start data of the next channel. Figure 1 is a schematic diagram of a data interpolation processing method of a CIS sensor according to the related art, such as Figure 1 As shown in the figure, this method has the following problems: 1) It can only interpolate at the end of the channel. If there are more than one IC pixel in each channel, all of them cannot be interpolated. 2) The number of IC pixels of all types plus 1 is a prime number, and the clock operating frequency cannot be reduced by increasing the bit width. This poses a great challenge to timing convergence in large-scale designs.

[0036] In summary, the data interpolation processing method of the CIS sensor in the related art has problems such as limited interpolation position, limited number of interpolation points, and low interpolation accuracy.

[0037] In response to the above problems, an embodiment of the present invention provides a method embodiment of data interpolation processing of a CIS sensor. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0038] Figure 2 is a flow chart of a data interpolation processing method for a CIS sensor according to an embodiment of the present invention. Figure 2 As shown, the method comprises the following steps:

[0039] Step S102, storing the raw data stream collected by the CIS sensor in a multi-level cache in sequence, wherein the CIS sensor is composed of multiple IC arrays, and the raw data stream is a collection of data collected by the multiple IC arrays;

[0040] The execution body of step S102 to step S106 may be an interpolation module, which is integrated into the CIS sensor in the form of hardware or software, or is arranged outside the CIS sensor.

[0041] Optionally, the multi-level cache is used to temporarily store the raw data stream. The CIS sensor can be used to collect image data for the target object, obtain the raw data stream, and send the raw data stream to the multi-level cache in sequence; the raw data stream is a collection of data collected by multiple IC arrays, that is, multiple IC arrays collect data for the target object respectively, and the obtained data is used as the raw data stream.

[0042] In step S102, the use of multi-level cache can provide temporary storage space for data, so that the previous and next data can be called for interpolation calculation when needed. This cache mechanism also allows the algorithm to find a suitable interpolation position in the data stream without interrupting the continuity of data transmission. Through multi-level cache, the data stream will not be directly affected by the interpolation operation during the transmission process, reducing the impact of the interpolation operation on data transmission delay, while providing the required data for interpolation calculation.

[0043] In an optional embodiment, the raw data stream collected by the CIS sensor is stored in a multi-level cache in sequence, including: increasing the bit width of the front data corresponding to the multiple IC arrays in the raw data stream to obtain a processed data stream, wherein the front data is used to indicate the data collected from the starting position of the data collection of the previous IC array of the corresponding IC array to the corresponding IC splicing seam position, and the IC splicing seam position is used to indicate the splicing position between the corresponding IC array and the previous IC array; the processed data stream is stored in the multi-level cache in sequence.

[0044] Optionally, the raw data stream can be stored in a multi-level cache in the order of data output of the CIS sensor (usually row by row or column by column). The data of each row or column will pass through the cache in turn until the storage is completed. In the above method, by preprocessing the front data in the raw data stream to increase the bit width, additional bits can be added to each sampling point of the data to reduce the dependence on the high-speed clock in subsequent processing. The processed data stream is then stored in a multi-level cache to slow down the output speed of the data so that interpolation processing can be performed at an appropriate time point. The design of the multi-level cache can provide sufficient data delay to ensure that there is enough time to calculate the interpolation points. In addition, by performing interpolation while increasing the bit width, the interpolation speed can also be increased.

[0045] Optionally, each sampling point data in the original data stream (such as the grayscale or color information of each pixel) can be in a data format represented by an 8-bit binary number (bit). For 8-bit data, it can represent 256 different numerical levels from 0 to 255 (i.e. (2^8-1)). Through data bit width processing, the sampling point data in the 8-bit data format can be increased in bit width (for example, from 8 bits to M bits, where M is a positive integer multiple of 8, which can be but is not limited to 32, 64, 80, etc.) to improve the accuracy of data processing, to meet the needs of the interpolation algorithm, and to reduce the clock frequency.

[0046] In an optional embodiment, the raw data stream collected by the CIS sensor is stored in a multi-level cache in sequence, including: obtaining a data output order of the raw data stream output by the CIS sensor; and storing the raw data stream in the multi-level cache in sequence according to the data output order.

[0047] It can be understood that by sequentially caching the original data stream in the multi-level cache according to the data output order of the CIS sensor, it is ensured that the original data stream will not be out of order when stored in the multi-level cache. Even if anomalies or losses occur in the data stream, effective interpolation recovery can be performed based on the stored ordered data, thereby improving the image quality. Clear storage order requirements enable the multi-level cache to process data in a more efficient manner, reduce the waiting time for data processing, and thus improve the overall efficiency of the interpolation algorithm, which is particularly important for processing high-speed data streams. In addition, since the original data stream has been stored in the output order, the calculation process can be simplified when performing interpolation point calculations later. No additional sorting or search operations are required, and interpolation calculations are performed directly based on the data in the cache, thereby improving data processing speed and accuracy.

[0048] Step S104, in the process of sequentially storing, when an interpolation position is detected, interpolation data of the interpolation position is determined based on the data stored in the multi-level cache;

[0049] In step S104, when the data stream is transmitted to a specific point (i.e., the interpolation position), the interpolation calculation is performed using the previous and next data stored in the multi-level cache to generate data of the interpolation point. Interpolation can be performed at any position of the data stream as needed to make up for the missing IC stitching data and improve the continuity and quality of the image.

[0050] In an optional embodiment, during the sequential storage process, when an interpolation position is detected, before determining the interpolation data of the interpolation position based on the data stored in the multi-level cache, the method also includes: during the sequential storage process, detecting whether an IC splicing seam position is passed, wherein the IC splicing seam position is used to indicate the splicing position between the corresponding IC array and the previous IC array; when it is detected that the IC splicing seam position is passed, the passed IC splicing seam position is determined as the interpolation position.

[0051] In the above method, by detecting the position of the IC stitching seam, it can be ensured that the interpolation operation can be performed at the correct position. The IC stitching seam is the key point that causes data loss, so interpolation at these positions can effectively make up for the lack of image information at the stitching seam and improve the continuity and quality of the image. Automatic detection of the stitching seam position enables the interpolation algorithm to automatically determine when to start interpolation according to the actual needs in the data stream without the need for additional control signals or human intervention, thereby improving the automation and adaptability of the interpolation process. After the IC stitching seam position is detected, interpolation is performed directly here, thereby avoiding unnecessary data storage and reading steps, thereby reducing the overall data processing delay. Compared with interpolation only at the end of the channel, the above method can complete the interpolation calculation earlier in the data stream and reduce the waiting time for subsequent processing. In addition, since the interpolation operation can be performed across the IC stitching seam, it is no longer subject to the limitation that the number of pixels of each IC is a prime number after +1, so the data bit width can be adjusted more flexibly, thereby reducing the clock frequency, which is a major improvement for the timing convergence of large designs. Reducing the clock frequency can reduce power consumption and improve the stability and reliability of the system.

[0052] Optionally, it is possible to detect whether the IC seam position has been passed based on the position counter. Specifically, in the process of storing the original data stream in a multi-level cache in sequence, the position counter can be used to record which pixel position of which IC array is currently being processed. The number of pixels of each IC array is known. When the value of the counter reaches the number of pixels of an IC array, it means that the data processing has passed the IC seam position, and the interpolation operation can be triggered at this time. In specific implementation, a flag bit can be set at the end of each IC array or at the beginning of the next IC array. When this flag bit is read, the counter is reset or increased to the starting position of the next IC array, and the interpolation algorithm is triggered to calculate the interpolation point. The position of the IC seam can also be identified by analyzing a specific pattern in the original data stream. For example, if there is a fixed pattern or a specific value between two IC arrays, this pattern or value can be detected to determine whether the seam position has been passed.

[0053] In an optional embodiment, interpolation data of an interpolation position is determined based on data stored in a multi-level cache, including: obtaining neighbor data within a predetermined neighborhood range of the interpolation position from the data stored in the multi-level cache, wherein the predetermined neighborhood range includes a first neighborhood range before the interpolation position and a second neighborhood range after the interpolation position; and determining the interpolation data based on the neighbor data.

[0054] Optionally, by considering the neighbor data (i.e., the previous and next data) around the interpolation position, the pixel information at the position can be more comprehensively evaluated, thereby obtaining a more accurate interpolation result. This interpolation method based on local information can more realistically reflect the continuity of the image and reduce the generation of edge effects and artifacts. By setting the first neighborhood range and the second neighborhood range, the algorithm can adjust the calculation method of the interpolation data according to different scenarios and requirements. For example, the size of the neighborhood range can be dynamically adjusted according to the complexity of the image content to optimize the interpolation effect. Since the interpolation calculation is based on the data in the cache, this means that the interpolation operation can be completed with a looser timing. By increasing the bit width to reduce the clock requirement, the challenge of timing convergence in large designs can be effectively solved, and the overall performance and stability of the system can be improved. And the above method is not only applicable to single-point interpolation, but also supports multi-point interpolation at any position in the channel. This provides more possibilities for improving image quality, especially when processing CIS sensor data spliced ​​by multiple ICs, which can more comprehensively compensate for data loss and improve image continuity and clarity. In addition, by defining how the interpolation data is determined, an interface is reserved for possible future algorithm improvements and sophistication. This means that without changing the overall architecture, the interpolation effect can be further optimized by adjusting the way the neighbor data is used or introducing a more complex mathematical model.

[0055] Optionally, interpolation data can be determined based on, but not limited to, neighbor data in the following manner: First, extract neighbor data within a predetermined neighborhood range of the interpolation position from a multi-level cache. This neighborhood range should be large enough to include valid information before and after the interpolation position, but not too large to avoid adding unnecessary computational complexity. For example, if linear interpolation or bilinear interpolation is used, it may only be necessary to extract a few adjacent pixels before and after; if a more complex interpolation algorithm is used, such as cubic spline interpolation or adaptive interpolation, a larger neighborhood range may be required. According to the application requirements and image characteristics, a suitable interpolation algorithm is selected, wherein the interpolation algorithm may be, but is not limited to, linear interpolation, bilinear interpolation, cubic spline interpolation, and adaptive interpolation, wherein linear interpolation can be used to calculate the value of the interpolation point based on the straight line relationship between two adjacent points; bilinear interpolation can be used to interpolate in a two-dimensional image by considering the influence of four nearest neighboring points; cubic spline interpolation can be used to smoothly interpolate by fitting a cubic polynomial, which is suitable for scenes requiring smooth transitions; adaptive interpolation can be used to dynamically adjust the interpolation strategy according to the characteristics of the data in the neighborhood (such as edges, textures, etc.) to obtain better image quality. The selected interpolation algorithm is further used to process and calculate the neighbor data to determine the value of the interpolation point. For example, in linear interpolation, the value of the interpolation point can be calculated by the values ​​of the two pixels before and after and their relative positions through a formula; in cubic spline interpolation, a set of equations containing four pixels needs to be solved to find a suitable cubic polynomial to determine the value of the interpolation point.

[0056] Step S106: insert the interpolation data into the original data stream to obtain the target data stream.

[0057] In step S106, after the interpolation calculation is completed, the interpolation data is added to the original data stream, thereby generating a target data stream after interpolation processing. This operation achieves data integrity and image quality improvement. The target data stream contains the original data and interpolation data, the data points are denser, the image details are richer, and the overall quality is improved. At the same time, in this way, the interference of the interpolation operation on the overall data transmission is minimized, ensuring the efficiency of data processing and the stability of the system.

[0058] In an optional embodiment, after inserting the interpolation data into the original data stream to obtain the target data stream, the method further includes: determining the number of interpolation points of the interpolation position; determining the target distance based on the number of interpolation points; and obtaining the output data of the multi-level cache based on the target distance.

[0059] Optionally, the interpolation position is the number of interpolation points, that is, the total number of insertion points for data insertion at the interpolation position. The above method covers two key aspects of data processing after interpolation: First, determine the number of interpolation points: When performing an interpolation operation, first determine the number of interpolation points that need to be inserted at a specific interpolation position. This step is determined based on the analysis of missing data in the original data stream or the requirements of the interpolation algorithm. The determination of the number of interpolation points directly affects the subsequent displacement operation and adjustment of data transmission, ensuring that the data stream after interpolation and the data stream before interpolation maintain continuity in data points. Second, determine the target distance based on the interpolation position and determine the multi-level cache output data: After interpolation, in order to maintain the continuity and alignment of the data stream, it is necessary to determine the target distance of data movement based on the number of interpolation points, that is, the distance of the roller displacement. Roller displacement is a figurative description, which actually refers to the rearrangement of data in the cache. For example, if 3 points are inserted at the first interpolation position, the target distance of the output data adjustment is also 3 points to ensure that subsequent data can be correctly filled into the interpolated position to maintain the continuity of the data stream. In the above process, determining the number of interpolation points at the interpolation position means that the amount of data for inserting data points can be dynamically adjusted according to actual needs, thereby improving the flexibility of data insertion; determining the target distance based on the interpolation points can ensure that the interpolated data can be accurately output according to the new position, avoiding misalignment or overlap, thereby ensuring the integrity and clarity of the output image; by adjusting the output data of the multi-level cache based on the target distance, the data flow can be effectively managed to ensure that even after adding data points, the overall data transmission efficiency remains at a high level.

[0060] In an optional embodiment, the number of interpolation points at the interpolation position is one or more. In the above manner, different image processing requirements and application scenarios can be adapted. For example, for occasions with high image quality requirements, it may be necessary to insert multiple points at each interpolation position to improve the smoothness and continuity of details; while for environments with limited processing speed or hardware resources, inserting a single point may be sufficient. This flexibility allows the algorithm to be more widely used to meet diverse user needs. The specific number of interpolation points is determined according to the physical limitations of the actual sensor. For example, one point is inserted into monochrome data with 600 dots per inch (Dots Per Inch, DPI), and three points are inserted into color data, thereby reducing the impact of data loss and making the image more complete and natural.

[0061] Through the above steps S102 to S106, it is possible to achieve the goal of placing the multi-level cache before interpolation, and to provide temporary storage space for data through the use of the multi-level cache, so as to facilitate the call of the previous and next data for interpolation calculation when needed. In this way, a suitable interpolation position can be found in the data stream without interrupting the continuity of data transmission, thereby achieving the technical effect of improving the data interpolation flexibility and interpolation accuracy of the CIS sensor, and further solving the technical problems of limited interpolation position and low interpolation accuracy in the data interpolation processing method of the CIS sensor in the related art.

[0062] Based on the above embodiments and optional embodiments, the present invention proposes an optional implementation mode: Figure 3 is a flow chart of an optional data interpolation processing method for a CIS sensor according to an embodiment of the present invention, such as Figure 3 As shown, the method includes:

[0063] S1, using a CIS sensor to collect image data for a target object to obtain a raw data stream, wherein the multiple CIS arrays included in the CIS sensor will respectively collect data for the target object, and transmit the multiple (N) CIS arrays to the corresponding random access memory RAM via corresponding channels, wherein the data collected by each CIS array includes the grayscale or color information of multiple pixel points corresponding to the CIS array.

[0064] S2, increasing the bit width of the front data of each corresponding CIS array in the original data stream, expanding the front data from 8 bits to 64 bits, and then sequentially storing them in a multi-level cache.

[0065] S3, the original data stream is output in sequence through multi-level caches. When it is not interpolated, that is, when it does not pass through the IC splicing seam position, the first-level cache data is output, thereby ensuring the continuity and originality of the data. Figure 4 is an optional schematic diagram of multi-level cache output without interpolation according to an embodiment of the present invention, such as Figure 4 As shown, when not interpolated, the output data of the multi-level cache is consistent with the original input data of the first-level cache.

[0066] S4, in the process of sequential storage, when the interpolation position is reached, the value of the interpolation point Tap_Inter is calculated according to the data before and after the interpolation position and the data is output, and the remaining data is shifted backward according to the number of interpolation points, wherein the number of interpolation points can be one or more. Figure 5 is a schematic diagram of an optional interpolation of a point according to an embodiment of the present invention, Figure 6is a schematic diagram of an optional interpolation of multiple points according to an embodiment of the present invention. When the interpolation position is after Tap(0)[0], an interpolation point Tap_Inter1 is correspondingly inserted. The output data of the multi-level cache after interpolation is as follows: Figure 5 When the interpolation position is after Tap(0)[0], three interpolation points Tap_Inter11 to Tap_Inter13 are inserted. The output data of the multi-level cache after interpolation is as follows Figure 6 as shown on the right.

[0067] S5, after interpolation, the multi-level cache outputs data according to the position after the roller displacement and enters the next level cache (i.e., the second level cache). The distance of roller displacement is equal to the number of interpolation points last time. Figure 7 is an optional schematic diagram of post-interpolation shifting according to an embodiment of the present invention, such as Figure 7 As shown in the figure, to store the next set of data, the data originally stored in the first-level cache will be shifted to the second-level cache accordingly, and the data will be migrated from the Tap(0)[0] to Tap(0)[7] positions to the Tap(1)[0] to Tap(1)[7] positions. However, due to the existence of interpolation, the data at the Tap(1)[7] position will be moved backward, and the output data of the corresponding multi-level cache will be the data at the Tap(1)[7] to Tap(0)[6] positions. Figure 7 as shown on the right.

[0068] S6, after reaching the next interpolation position, the data corresponding to the previous interpolation position will enter the next level cache (i.e., the second level cache), and continue to calculate the value of the new interpolation point based on the previous and next data of the corresponding interpolation position and output the data. Figure 8 is a schematic diagram of another optional interpolation point according to an embodiment of the present invention, where a new interpolation point Tap_Inter2 is added, and the output data of the multi-level cache after interpolation is as follows Figure 8 As shown on the right. After interpolation, the data is output again according to the position after the roller displacement. Fig. 9 is another optional post-interpolation shift schematic diagram according to an embodiment of the present invention. The newly added interpolation will continue to cause the rolling shift of the data. The output data of the multi-level cache after the shift is as follows Fig. 9 as shown on the right side of the diagram.

[0069] S7, looping through the operations of step S5 to step S6 to perform data interpolation until all data are interpolated.

[0070] This embodiment can achieve at least one of the following effects: 1) the clock requirement for the front data can be reduced by increasing the bit width; 2) interpolation can be performed at any position; 3) the entire row of images can be interpolated without affecting the overall data transmission, and only a few clock delays are added; 4) space is reserved for more complex interpolation algorithms.

[0071] In this embodiment, a data interpolation processing device for a CIS sensor is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the terms "module" and "device" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0072] According to an embodiment of the present invention, there is also provided an apparatus embodiment for implementing the data interpolation processing method of the CIS sensor. Fig.10 is a schematic diagram of the structure of a data interpolation processing device for a CIS sensor according to an embodiment of the present invention. Fig.10 As shown, the data interpolation processing device of the CIS sensor includes: a storage module 1000, a determination module 1002, and an interpolation module 1004, wherein:

[0073] The storage module 1000 is used to sequentially store the raw data stream collected by the CIS sensor in a multi-level cache, wherein the CIS sensor is composed of multiple IC arrays, and the raw data stream is a collection of data collected by the multiple IC arrays;

[0074] The determination module 1002 is connected to the storage module 1000 and is used to determine the interpolation data of the interpolation position based on the data stored in the multi-level cache when the interpolation position is detected during the sequential storage process;

[0075] The interpolation module 1004 is connected to the determination module 1002 and is used to insert the interpolation data into the original data stream to obtain the target data stream.

[0076] In an embodiment of the present invention, a storage module 1000 is provided to sequentially store the raw data stream collected by the CIS sensor in a multi-level cache, wherein the CIS sensor is composed of a plurality of IC arrays, and the raw data stream is a collection of data collected by the plurality of IC arrays; a determination module 1002 is connected to the storage module 1000, and is used to determine the interpolation data of the interpolation position based on the data stored in the multi-level cache when the interpolation position is detected during the sequential storage process; an interpolation module 1004 is connected to the determination module 1002, and is used to insert the interpolation data into the raw data stream to obtain the target data stream, so as to achieve the goal of placing the multi-level cache before the interpolation, and providing temporary storage space for the data through the use of the multi-level cache, so as to facilitate the calling of the previous and next data for interpolation calculation when needed. Thus, a suitable interpolation position is found in the data stream without interrupting the continuity of data transmission, thereby achieving the technical effect of improving the data interpolation flexibility and interpolation accuracy of the CIS sensor, and further solving the technical problems of the limited interpolation position and low interpolation accuracy in the data interpolation processing method of the CIS sensor in the related art.

[0077] It should be noted that the above modules can be implemented by software or hardware. For example, for the latter, it can be implemented in the following ways: the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0078] It should be noted that the storage module 1000, the determination module 1002, and the interpolation module 1004 correspond to steps S102 to S106 in the embodiment, and the examples and application scenarios implemented by the modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above embodiments. It should be noted that the modules as part of the device can be run in a computer terminal.

[0079] It should be noted that the optional or preferred implementation of this embodiment can refer to the relevant description in the embodiment, which will not be repeated here.

[0080] The data interpolation processing device of the CIS sensor may further include a processor and a memory. The storage module 1000, determination module 1002, interpolation module 1004, etc. are all stored in the memory as program modules, and the processor executes the program modules stored in the memory to implement corresponding functions.

[0081] The processor includes a kernel, which retrieves the corresponding program module from the memory. The kernel may be one or more. The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. The memory includes at least one memory chip.

[0082] According to an embodiment of the present application, an embodiment of a non-volatile storage medium is also provided. Optionally, in this embodiment, the non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute any one of the data interpolation processing methods for the CIS sensor.

[0083] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group, and the non-volatile storage medium includes a stored program.

[0084] Optionally, when the program is running, the device where the non-volatile storage medium is located is controlled to perform the following functions: the original data stream collected by the CIS sensor is stored in a multi-level cache in sequence, wherein the CIS sensor is composed of multiple IC arrays, and the original data stream is a collection of data collected by the multiple IC arrays; in the process of sequential storage, when an interpolation position is detected, the interpolation data of the interpolation position is determined based on the data stored in the multi-level cache; the interpolation data is inserted into the original data stream to obtain the target data stream.

[0085] According to an embodiment of the present application, an embodiment of a processor is also provided. Optionally, in this embodiment, the processor is used to run a program, wherein when the program is run, any one of the data interpolation processing methods for a CIS sensor is executed.

[0086] According to an embodiment of the present application, an embodiment of a computer program product is also provided. When executed on a data processing device, the program is suitable for executing the steps of initializing the data interpolation processing method of any one of the above-mentioned CIS sensors.

[0087] Optionally, the above-mentioned computer program product, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: storing the original data stream collected by the CIS sensor in a multi-level cache in sequence, wherein the CIS sensor is composed of multiple IC arrays, and the original data stream is a collection of data collected by the multiple IC arrays; in the process of sequential storage, when an interpolation position is detected, determining the interpolation data of the interpolation position based on the data stored in the multi-level cache; inserting the interpolation data into the original data stream to obtain the target data stream.

[0088] An embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the following steps are implemented: the original data stream collected by the CIS sensor is stored in a multi-level cache in sequence, wherein the CIS sensor is composed of a plurality of IC arrays, and the original data stream is a collection of data collected by the plurality of IC arrays; in the process of sequential storage, when an interpolation position is detected, interpolation data of the interpolation position is determined based on the data stored in the multi-level cache; and the interpolation data is inserted into the original data stream to obtain a target data stream.

[0089] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0090] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0091] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the above modules can be a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, modules or indirect coupling or communication connection of modules, which can be electrical or other forms.

[0092] The modules described above as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0093] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0094] If the above-mentioned integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable non-volatile storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a non-volatile storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present invention. The aforementioned non-volatile storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0095] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A data interpolation processing method for a CIS sensor, characterized in that: include: The raw data stream collected by the CIS sensor is stored in a multi-level cache in sequence, wherein the CIS sensor is composed of a plurality of IC arrays, and the raw data stream is a collection of data collected by the plurality of IC arrays; In the process of sequential storage, when an interpolation position is detected, interpolation data of the interpolation position is determined based on the data stored in the multi-level cache; The interpolation data is inserted into the original data stream to obtain a target data stream.

2. The method according to claim 1, characterized in that The storing of the raw data stream collected by the CIS sensor in the multi-level cache in sequence includes: Performing bit width increasing processing on the front data corresponding to the multiple IC arrays in the original data stream to obtain a processed data stream, wherein the front data is used to indicate data collected from the starting position of data collection of the previous IC array of the corresponding IC array to the corresponding IC splicing seam position, and the IC splicing seam position is used to indicate the splicing position between the corresponding IC array and the previous IC array; The processed data streams are sequentially stored in the multi-level cache.

3. The method according to claim 1, characterized in that The storing of the raw data stream collected by the CIS sensor in the multi-level cache in sequence includes: Obtaining a data output order of the raw data stream output by the CIS sensor; According to the data output order, the original data stream is stored in the multi-level cache in sequence.

4. The method according to claim 1, characterized in that In the process of sequential storage, when an interpolation position is detected, before determining the interpolation data of the interpolation position based on the data stored in the multi-level cache, the method further includes: In the process of sequential storage, detecting whether an IC splicing seam position is passed, wherein the IC splicing seam position is used to indicate a splicing position between a corresponding IC array and a previous IC array; In the case where the IC stitching seam position is detected, the IC stitching seam position passed is determined as the interpolation position.

5. The method according to claim 1, characterized in that: The step of determining the interpolation data of the interpolation position based on the data stored in the multi-level cache comprises: Acquire neighbor data within a predetermined neighborhood range of the interpolation position from the data stored in the multi-level cache, wherein the predetermined neighborhood range includes a first neighborhood range before the interpolation position and a second neighborhood range after the interpolation position; Based on the neighbor data, the interpolation data is determined.

6. The method according to claim 1, characterized in that After inserting the interpolation data into the original data stream to obtain the target data stream, the method further includes: Determining the number of interpolation points of the interpolation position; Determining a target distance based on the number of interpolation points; Based on the target distance, output data of the multi-level cache is obtained.

7. The method according to any one of claims 1 to 6, characterized in that The number of interpolation points at the interpolation position is one or more.

8. A data interpolation processing device for a CIS sensor, characterized in that: include: A storage module, used to store the raw data stream collected by the CIS sensor in a multi-level cache in sequence, wherein the CIS sensor is composed of a plurality of IC arrays, and the raw data stream is a collection of data collected by the plurality of IC arrays; A determination module, configured to determine, when an interpolation position is detected during the sequential storage process, interpolation data at the interpolation position based on data stored in the multi-level cache; The interpolation module is used to insert the interpolation data into the original data stream to obtain a target data stream.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the data interpolation processing method for a CIS sensor according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the data interpolation processing method of the CIS sensor according to any one of claims 1 to 7.

Images