Multifunctional image input and capture device and method

Abstract

The invention provides a multifunctional image input capturing device and method, and the method comprises the steps: obtaining corresponding image information through a plurality of detection modules, transmitting the image information to a corresponding receiving module, carrying out the digital-to-analog conversion of the image information through the receiving module, obtaining a corresponding interface signal, and transmitting the interface signal to an image capturing module, the image capture module decodes all the interface signals and performs data analysis processing to obtain corresponding interface time sequence data, and sends the interface time sequence data to the image processing module, and the image processing module performs image data processing on the interface time sequence data; corresponding analysis processing and format conversion are carried out on the interface through the image capturing module, the interface compatibility of the receiving module and the image processing module is improved, and the condition that the model selection of the module is limited due to interface incompatibility is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission, and in particular to a multifunctional image input capture device and method. Background Art

[0002] With the rapid development of mobile devices and multimedia applications, people have put forward higher requirements for data transmission speed and bandwidth, and the application of Mobile Industry Processor Interface (MIPI) is becoming more and more extensive. In the integrated circuit (IC) industry, audio and video system-on-chips (SoC) are basically equipped with MIPI controllers. The interface protocols and data formats supported by MIPI controllers from different manufacturers are different. The mainstream MIPI input controllers generally support image data interface (IDI) or digital video port (DVP) interface. In addition to MIPI, the on-chip output usually has high-definition multimedia interface (HDMI), display interface (DP), video graphics array (VGA), which generally only support DVP interface. Usually there is an image signal processing (ISP) module between the input and output ends, so interface protocol and data format conversion is required between the transmitter and the ISP. However, general conversion modules cannot support multiple video stream scenarios in large-scale SoC chips.

[0003] In the IC industry, large-scale video SoC chips are extremely complex and have a relatively long development cycle. The chip cannot do without the MIPI input and output controllers. The ISP module can perform a series of processing on image information and is also an indispensable part of the video chip. However, the output interface of most existing MIPI output controllers is the IDI protocol, while the input interface of the ISP module is mostly the DVP protocol. The interface protocols between the two are incompatible. To ensure the compatibility of the interface protocols between the two, a MIPI output controller with the DVP protocol must be used, or a camera with the DVP protocol must be used to ensure the compatibility of the interface protocols between the two, which limits the selection of modules.

[0004] In view of this, overcoming the defects of the prior art is an urgent problem to be solved in the field of this technology. Summary of the invention

[0005] The technical problem to be solved by the present invention is how to improve the interface compatibility between the MIPI input terminal and the image processing module within a larger module selection range.

[0006] The present invention adopts the following technical solution: In a first aspect, a multifunctional image input capture device is provided, comprising: a plurality of detection modules 1, a plurality of receiving modules 2, an image capture module 3 and an image processing module 5, wherein: The output end of each detection module 1 is connected to the input end of the corresponding single receiving module 2, the output ends of all receiving modules 2 are connected to the input end of the image capturing module 3, and the output end of the image capturing module 3 is connected to the image processing module 5; The detection module 1 is used to acquire corresponding image information and send it to the corresponding receiving module 2; the receiving module 2 is used to perform digital-to-analog conversion on the image information to obtain corresponding interface signals, and send the interface signals to the image capture module 3; the image capture module 3 is used to decode all the interface signals and perform data analysis to obtain corresponding interface timing data, and send the interface timing data to the image processing module 5, and the image processing module 5 is used to perform image data processing on the interface timing data.

[0007] Preferably, the image capture module 3 comprises: a data decoding module 31, a data processing module 32, a data buffer module 33 and a data conversion module 34, wherein: The input end of the data decoding module 31 is connected to the output ends of all the receiving modules 2, the output end of the data decoding module 31 is connected to the input end of the data processing module 32, the output end of the data processing module 32 is connected to the input end of the data buffer module 33, the output end of the data buffer module 33 is connected to the input end of the data conversion module 34, and the output end of the data conversion module 34 is connected to the input end of the image processing module 5; The data decoding module 31 is used to decode all interface signals from the receiving module 2, and send all decoded interface signals to the data processing module 32; the data processing module 32 is used to parse all the decoded interface signals to obtain parsed signals, and send the parsed signals to the data cache module 33; the data cache module 33 is used to cache the received parsed signals, and send the parsed signals to the data conversion module 34, and the data conversion module 34 is used to convert the received parsed signals into DVP format to obtain interface timing data, and send the interface timing data to the image processing module 5.

[0008] Preferably, the multifunctional image input capture device further comprises a control module 6, and the image capture module 3 further comprises a register module 35, wherein: The control module 6 is connected to the register module 35, and the register module 35 is connected to all modules in the image capture module 3; The control module 6 is used to send bus configuration to the register module 35; the register module 35 is used to send corresponding configuration information to all modules in the image capture module 3 according to the bus configuration, thereby defining and controlling all modules in the image capture module 3.

[0009] Preferably, the image capturing module 3 further comprises a signal selecting module 36, wherein: The signal selection module 36 is arranged on the transmission path between the data processing module 32 and the data cache module 33, the input end of the signal selection module 36 is connected to the output end of the data processing module 32, and the output end of the signal selection module 36 is connected to the input end of the data cache module 33; The signal selection module 36 is used to select one of all parsing signals according to the configuration information sent by the register module 35 , and send the selected parsing signal to the data cache module 33 .

[0010] Preferably, the multifunctional image input capture device further comprises a DDR module 7, and the image capture module 3 further comprises a MUX module 37, wherein: The MUX module 37 is arranged on the transmission path between the signal selection module 36 and the data cache module 33, the input end of the MUX module 37 is connected to the output end of the signal selection module 36, and the output end of the MUX module 37 is connected to the input end of the data cache module 33; The input end of the DDR module 7 is connected to the output end of the data processing module 32, and the output end of the DDR module 7 is connected to the input end of the MUX module 37; The DDR module 7 is used to receive the parsing signal from the data processing module 32 and cache the parsing signal. The DDR module 7 is also used to send the parsing signal to the MUX module 37; the MUX module 37 is used to select the parsing signal from the signal selection module 36 or the parsing signal from the DDR module 7 according to the configuration information sent by the register module 35, and send the selected parsing signal to the data cache module 33.

[0011] Preferably, the image capturing module 3 further comprises a writing module 38, wherein: The writing module 38 is arranged on the transmission path between the data processing module 32 and the DDR module 7, the input end of the writing module 38 is connected to the output end of the data processing module 32, and the output end of the writing module 38 is connected to the input end of the DDR module 7; The writing module 38 includes a first integration unit 381 and a plurality of sub-writing units 382, ​​the input ends of all the sub-writing units 382 are connected to the output end of the data processing module 32, the output ends of all the sub-writing units 382 are connected to the input end of the first integration unit 381, and the output end of the first integration unit 381 is connected to the input end of the DDR module 7; Each of the sub-write units 382 is used to receive the corresponding parsing signal in the data processing module 32, and the first integration unit 381 is used to arbitrate all the parsing signals received from the sub-write units 382 according to the configuration issued by the register module 35, and send all the parsing signals from the sub-write units 382 to the DDR module 7 in the configured order.

[0012] Preferably, the image capturing module 3 further comprises a reading module 39, wherein: The reading module 39 is arranged on the transmission path between the MUX module 37 and the DDR module 7. The input end of the reading module 39 is connected to the output end of the DDR module 7, and the output end of the reading module 39 is connected to the input end of the MUX module 37. The reading module 39 is used to read the parsed signal in the DDR module 7 and send it to the MUX module 37. The reading module 39 comprises an interface conversion unit 391, a second integration unit 392 and a plurality of sub-reading units 393, the input ends of all the sub-reading units 393 are connected to the output ends of the second integration unit 392, the input end of the second integration unit 392 is connected to the output end of the DDR module 7, the output ends of all the sub-reading units 393 are connected to the interface conversion unit 391, and the output end of the interface conversion unit 391 is connected to the input end of the MUX module 37; The second integration unit 392 is used to send the parsed signal in the DDR module 7 to the corresponding sub-reading unit 393 in sequence according to the configuration issued by the register module 35, and each sub-reading unit 393 then sends the parsed signal to the interface conversion unit 391. The interface conversion unit 391 is used to perform corresponding protocol conversion on the received parsed signal according to the configuration issued by the register module 35, and send it to the MUX module 37 in the configuration order.

[0013] Preferably, the image capture module 3 further includes a signal detection module 401 and an interruption module 402, wherein: The input end of the signal detection module 401 is connected to the data conversion module 34, the output end of the signal detection module 401 is connected to the input end of the interruption module 402, and the output end of the interruption module 402 is connected to the control module 6; The signal detection module 401 is used to compare the parsed signal input to the data conversion module 34 with the interface timing data after format conversion to obtain a comparison result, and determine whether the comparison result complies with the corresponding protocol. If it does not comply, the signal detection module 401 is used to send a corresponding trigger signal to the interrupt module 402; The interrupt module 402 is used to send a corresponding interrupt signal to the control module 6 according to the trigger signal sent by the signal detection module 401, and the control module 6 controls the data conversion module 34 to interrupt and perform corresponding correction according to the interrupt signal.

[0014] Preferably, the output end of the write module 38 is also connected to the input end of the interrupt module 402; the write module 38 is also used to send a corresponding trigger signal to the interrupt module 402 after writing a preset amount of parsed signal to the DDR module 7, and the interrupt module 402 is used to send a corresponding interrupt signal to the control module 6 according to the trigger signal sent by the write module 38, and the control module 6 controls the write module 38 to interrupt according to the interrupt signal.

[0015] In a second aspect, a multifunctional image input capture method is provided, which is applied to the multifunctional image input capture device, comprising: The corresponding image information is acquired through the detection module 1 and sent to the corresponding receiving module 2; The receiving module 2 performs digital-to-analog conversion on the image information to obtain a corresponding interface signal, and sends the interface signal to the image capturing module 3; The image capture module 3 decodes all the interface signals and performs corresponding data analysis to obtain corresponding interface timing data, and sends the interface timing data to the image processing module 5; The interface timing data is processed into image data by the image processing module 5 .

[0016] The present invention provides a multifunctional image input capture device and method, which obtains corresponding image information through multiple detection modules 1 and sends it to the corresponding receiving module 2. The receiving module 2 performs digital-to-analog conversion on the image information to obtain corresponding interface signals, and sends the interface signals to the image capture module 3. The image capture module 3 decodes all the interface signals and performs data analysis processing to obtain corresponding interface timing data, and sends the interface timing data to the image processing module 5. The image processing module 5 performs image data processing on the interface timing data; the image capture module 3 performs corresponding analysis processing and format conversion on the interface.

[0017] Furthermore, the image capture module 3 converts the interface signal into interface timing data in the DVP format, thereby ensuring that the protocol format of the signal finally output by the image capture module 3 is in the DVP format, which is compatible with most of the existing image processing modules 5 with DVP protocol interfaces, thereby realizing the docking and transmission between the receiving module 2 of the IDI protocol interface and the image processing module 5 of the DVP protocol interface, improving the interface compatibility of the receiving module 2 and the image processing module 5, increasing the selection range of the module, and avoiding limiting the selection of the module due to interface incompatibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 is a schematic diagram of a multifunctional image input and capture device provided by an embodiment of the present invention; Figure 2 is a schematic diagram of another multifunctional image input and capture device provided by an embodiment of the present invention; Figure 3 is a schematic diagram of another multifunctional image input and capture device provided by an embodiment of the present invention; Figure 4 is a schematic diagram of a multifunctional image input and capture device provided by an embodiment of the present invention; Figure 5 is a method flow chart of a multifunctional image input capturing method provided by an embodiment of the present invention; The diagram numbers are as follows: Detection module 1; receiving module 2; image capture module 3; data decoding module 31; sub-decoding unit 311; data processing module 32; data cache module 33; data conversion module 34; data controller module 341; timing control module 342; register module 35; signal selection module 36; MUX module 37; write module 38; first integration unit 381; sub-write unit 382; read module 39; interface conversion unit 391; second integration unit 392; sub-read unit 393; signal detection module 401; interrupt module 402; bandwidth statistics module 403; image processing module 5; control module 6; DDR module 7; output module 8; display module 9; interface protocol conversion module 10. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0021] In the description of the present invention, the terms "inside", "outside", "longitudinal", "lateral", "upper", "lower", "top", "bottom" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and do not require that the present invention must be constructed and operated in a specific orientation. Therefore, they should not be understood as limitations on the present invention.

[0022] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0023] The embodiment of the present invention provides a multifunctional image input capture device, such as Figure 1 As shown, it includes: multiple detection modules 1, multiple receiving modules 2, an image capturing module 3 and an image processing module 5, wherein: The output end of each detection module 1 is connected to the input end of the corresponding receiving module 2 , the output ends of all receiving modules 2 are connected to the input end of the image capturing module 3 , and the output end of the image capturing module 3 is connected to the image processing module 5 .

[0024] The detection module 1 is used to acquire image information and send it to the receiving module 2; the receiving module 2 is used to perform digital-to-analog conversion on the image information to obtain corresponding interface signals, and send the interface signals to the image capture module 3; the image capture module 3 is used to decode all the interface signals and perform data analysis processing to obtain corresponding interface timing data, and send the interface timing data to the image processing module 5, and the image processing module 5 is used to perform image data processing on the interface timing data.

[0025] The image processing module 5 may specifically be an ISP module; the image capturing module 3 may be a multifunctional video input capture (VICAP for short) module.

[0026] In this embodiment, the detection module 1 can be a camera, and the image capture module 3 can support 4 detection modules 1 to transmit corresponding image information at the same time. The output interface of the detection module 1 can be a point-to-point protocol (Point to Point Interface, abbreviated as: PPI) interface; each detection module 1 is connected to a corresponding receiving module 2; the receiving module 2 includes a digital-to-analog conversion module and a controller, and the PPI protocol interface is connected to the input end of the corresponding digital-to-analog conversion module, and the image information detected by the detection module 1 is sent to the digital-to-analog conversion module to convert the analog signal into a digital signal; the output end of the digital-to-analog conversion module is connected to the input end of the controller, and the output end of the controller is an IDI interface, and the IDI interface is connected to the image capture module 3, and the corresponding array signal is converted into an interface signal corresponding to the MIPI protocol through the controller and sent to the image capture module 3; the image capture module 3 can receive four-way interface signals.

[0027] The image capture module 3 performs corresponding decoding on the received interface signal, thereby parsing the input MIPI protocol interface signal into a user-defined mode that can be recognized by each module inside the image capture module 3, which is convenient for the development and use of each module inside the image capture module 3. At the same time, the image capture module 3 is also used to perform corresponding data parsing processing on the decoded interface signal: including time-division multiplexing of multiple interface signals, that is, only one of the interface signals is selected for transmission at the same time; also including the implementation of the deliver function to improve data processing efficiency; also including the implementation of the corresponding format conversion function, that is, rearranging the pixels in the interface signal and converting it into the corresponding required format. The image capture module 3 converts the interface signal into interface timing data in the DVP format, thereby ensuring that the protocol format of the signal finally output by the image capture module 3 is in the DVP format, which is compatible with most of the existing image processing modules 5 with DVP protocol interfaces, and realizes the connection and transmission between the receiving module 2 with the IDI protocol interface and the image processing module 5 with the DVP protocol interface; there is no need to limit the interface protocol of the receiving module 2 to the DVP protocol, and there is no need to limit the camera with the interface protocol as the DVP protocol, thereby improving the interface compatibility of the receiving module 2 and the image processing module 5, increasing the selection range of the module, and avoiding limiting the selection of the module due to interface incompatibility.

[0028] It should be noted that, in the present embodiment, the subsequent output end of the image processing module 5 is also connected to the input end of the output module 8, and the output end of the output module 8 is connected to the display module 9; the received related signals are displayed on the display screen through the display module 9; wherein, the output module 8 can be a High Definition Multimedia Interface (HDMI) module.

[0029] In this embodiment, after the image capture module 3 receives the corresponding interface signal, multiple internal modules are required to perform corresponding analysis and processing on the interface signal, so as to realize the corresponding functions and format conversion. Therefore, in this embodiment, Figure 2 As shown, the various modules inside the image capture module 3 involve the following designs: Figure 2 As shown, the image capturing module 3 includes: a data decoding module 31 , a data processing module 32 , a data buffering module 33 and a data conversion module 34 .

[0030] The input end of the data decoding module 31 is connected to the output ends of all the receiving modules 2, the output end of the data decoding module 31 is connected to the input end of the data processing module 32, the output end of the data processing module 32 is connected to the input end of the data cache module 33, the output end of the data cache module 33 is connected to the input end of the data conversion module 34, and the output end of the data conversion module 34 is connected to the input end of the image processing module 5.

[0031] The data decoding module 31 is used to decode all interface signals from the receiving module 2, and send all decoded interface signals to the data processing module 32; the data processing module 32 is used to parse all the decoded interface signals to obtain parsed signals, and send the parsed signals to the data cache module 33; the data cache module 33 is used to cache the received parsed signals, and send the parsed signals to the data conversion module 34, and the data conversion module 34 is used to convert the received parsed signals into DVP format to obtain interface timing data, and send the interface timing data to the image processing module 5.

[0032] In this embodiment, the data decoding module 31 includes multiple sub-decoding units 311, each sub-decoding unit 311 is connected to a corresponding one of the receiving modules 2, and is used to receive one of the interface signals and perform corresponding decoding processing. All sub-decoding units 311 are connected to the data processing module 32 and send the decoded interface signal of the corresponding one to the data processing module 32.

[0033] The data processing module 32 may be integrated with a deliver function for realizing parallel processing and caching of two lines of data, so as to improve data processing efficiency; the data processing module 32 may also be integrated with a corresponding Bayer format conversion function, which includes: original image format conversion, red green blue (RGB) format conversion and color coding format conversion, etc. The protocol format of the interface signal is converted according to the subsequently configured transmission requirements to obtain a parsed signal, thereby ensuring the compatibility of the parsed signal with the subsequent corresponding functions.

[0034] The data cache module 33 is used to temporarily cache the online input signal when the image capture module 3 performs online input (that is, after the detection module 1 receives the image information, it directly performs relevant analysis and processing and sends it to the image processing module 5, and then displays it through the display module 9), so as to avoid the signal transmission speed being too fast, resulting in the subsequent data conversion module 34 The analysis speed cannot keep up with the transmission speed, resulting in the loss of the final output image information. The capacity of the data cache module 33 needs to be set according to parameters such as the resolution and size of the currently acquired image information, and meet the first-in-first-out conditions. In this embodiment, the data cache module 33 can use 4 256X256 asynchronous first-in-first-out queues (First Input First Output, abbreviated as: FIFO), corresponding to a performance of 4k resolution and 60 frames per second.

[0035] The data conversion module 34 includes a data controller module 341 and a timing control module 342, wherein the input end of the data controller module 341 is connected to the output end of the data cache module 33, the output end of the data controller module 341 is connected to the input end of the timing control module 342, and the output end of the timing control module 342 is connected to the image processing module 5. The data controller module 341 is used to parse and process the data read from the data cache module 33, and encapsulate it into a corresponding format and output it to the timing control module 342; the timing control module 342 is used to generate the interface timing of the DVP protocol, and is used to parse the data from the data controller module 341 again, and convert it into configurable timing information compatible with the DVP interface, that is, the interface timing data. In this process, the host can provide the line / field blanking of the DVP timing of the interface timing data to the image processing module 5 more flexibly through the corresponding configuration information, so as to ensure the compatibility of the data transmitted to the image processing module 5.

[0036] In this embodiment, in order to realize the corresponding configuration control of the image capture module 3 and other modules, and complete the definition adjustment of each functional module during the transmission process, this embodiment also involves the following designs: Figure 3 As shown, the multifunctional image input capture device further includes a control module 6 , and the image capture module 3 further includes a register module 35 .

[0037] The control module 6 is connected to the register module 35, and the register module 35 is connected to all modules in the image capture module 3. The control module 6 is used to send bus configuration to the register module 35; the register module 35 is used to send corresponding configuration information to all modules in the image capture module 3 according to the bus configuration, so as to define and control all modules in the image capture module 3.

[0038] It should be noted that in order to avoid the connection relationship shown in the figure being too complicated, the connection lines between the register module 35 and most of the modules in the image capture module 3 are hidden and not drawn.

[0039] In this embodiment, the control module 6 can be a CPU processor, through which the software and hardware work in parallel, and the user can write a software driver so that the control module 6 sends read and write processing instructions to configure the relevant read and write data of other modules; in this embodiment, the control module 6 can use an open source RISC-V model CPU processor, and the corresponding data interface is two groups of 256-bit bus protocol (Advanced eXtensible Interface, abbreviated as: AXI) interfaces, one group of AXI interfaces can be Cfg_Bus (i.e., configuration bus), which is used to configure the registers of each module, and the other group of AXI interfaces can be Data_Bus (i.e., data bus), which is dedicated to read and write access to the data of various modules. The control module 6 can also receive corresponding interrupt signals, enter the interrupt processing program after receiving the abnormal interrupt issued by the module, obtain the interrupt flag bit after completing the interrupt processing, and realize the complete interrupt processing process.

[0040] In this embodiment, for the register module 35, the user can write corresponding software to drive the control module 6 to configure the register module 35, and send the bus configuration to the register module 35 through the control module 6, such as the resolution size, functional parameters and mode selection required to be configured for each module. The register module 35 is connected to all modules in the image capture module 3, and the configuration information corresponding to each module is sent accordingly according to the received bus configuration, thereby realizing the definition control of all modules in the image capture module 3; at the same time, the register module 35 can also accept the output results of other modules and report them to the control module 6.

[0041] In this embodiment, since the output interface of the control module 6 is an AXI interface and the register module 35 adopts an APB interface, in order to ensure the docking compatibility between the interfaces, an interface protocol conversion module 10 is also provided on the transmission path between the register module 35 and the control module 6. The input end of the interface protocol conversion module 10 is connected to the control module 6, and the output end of the interface protocol conversion module 10 is connected to the input port of the register module 35. The interface protocol conversion module 10 is used to convert the 256-bit AXI master signal from the control module 6 into a corresponding 32-bit APB slave signal to realize signal protocol conversion.

[0042] In this embodiment, the image capture module 3 receives 4 channels of image information at the same time. Since only one channel of image information needs to be displayed at the same time, after receiving multiple channels of analysis signals at the same time, the image capture module 3 needs to be configured according to user needs and select the required channel of analysis signal for subsequent transmission processing. Therefore, this embodiment also involves the following designs: Figure 3 As shown, the image capture module 3 also includes a signal selection module 36, wherein: the signal selection module 36 is arranged on the transmission path between the data processing module 32 and the data cache module 33, the input end of the signal selection module 36 is connected to the output end of the data processing module 32, and the output end of the signal selection module 36 is connected to the input end of the data cache module 33; the signal selection module 36 is used to select one of all the parsing signals according to the configuration information sent by the register module 35, and send the selected parsing signal to the data cache module 33.

[0043] In this embodiment, in addition to realizing the online input function of the received image information, the corresponding offline input function can also be realized, that is, after the received image information is analyzed and format converted, it is cached in the corresponding cache unit, and read from the cache unit when needed later, and output in the display module 9 to realize the corresponding offline input function. Therefore, this embodiment also involves the following designs: like Figure 3 As shown, the multifunctional image input capture device also includes a double data rate synchronous dynamic random access memory (DDR) module 7, and the image capture module 3 also includes a MUX module 37. The multiplexer (MUX) module 37 is arranged on the transmission path between the signal selection module 36 and the data cache module 33. The input end of the MUX module 37 is connected to the output end of the signal selection module 36, and the output end of the MUX module 37 is connected to the input end of the data cache module 33.

[0044] The input end of the DDR module 7 is connected to the output end of the data processing module 32 , and the output end of the DDR module 7 is connected to the input end of the MUX module 37 .

[0045] The DDR module 7 is used to receive the parsing signal from the data processing module 32 and cache the parsing signal. The DDR module 7 is also used to send the parsing signal to the MUX module 37; the MUX module 37 is used to select the parsing signal from the signal selection module 36 or the parsing signal from the DDR module 7 according to the configuration information sent by the register module 35, and send the selected parsing signal to the data cache module 33.

[0046] In this embodiment, the DDR module 7 serves as the main cache module of the entire transmission architecture. The information in the cache may include image data from each frame, firmware information of each module, and other large amounts of memory data. In this embodiment, the capacity of the DDR module 7 is at least 2G, and the interface corresponding to the DDR module 7 can be a set of 256-bit AXI buses. All modules with direct memory access (DMA) function or AXI bus interface can access the DDR module 7.

[0047] The MUX module 37 decides whether to receive the parsing signal from the signal selection module 36 or the parsing signal from the DDR module 7 according to the configuration information sent by the register module 35. When the function type sent by the register module 35 is online input, the MUX module 37 chooses to receive the parsing signal from the signal selection module 36, and continues to transmit the corresponding parsing signal, thereby outputting the image information currently acquired by the detection module 1. When the function type sent by the register module 35 is offline input, the MUX module 37 chooses to receive the parsing signal from the DDR module 7, and continues to transmit the corresponding parsing signal, thereby outputting the signal detected by the detection module 1 in the previous time and cached in the DDR module 7.

[0048] It is worth mentioning that, in this embodiment, the control module 6 is also directly connected to the DDR module 7 . On the one hand, the control module 6 can cache part of its own data in the DDR module 7 , and on the other hand, it can also directly read the data information in the DDR module 7 .

[0049] In this embodiment, when offline input is selected, the analysis signal obtained in the data processing module 32 needs to be written into the DDR module 7 for buffering, and the data processing module 32 includes multiple analysis signals, so it is necessary to perform corresponding processing on each analysis signal and write it into the DDR module 7. Therefore, this embodiment also designs the following design: Figure 3As shown, the image capture module 3 also includes a writing module 38, wherein: the writing module 38 is arranged on the transmission path between the data processing module 32 and the DDR module 7, the input end of the writing module 38 is connected to the output end of the data processing module 32, and the output end of the writing module 38 is connected to the input end of the DDR module 7.

[0050] The writing module 38 includes a first integration unit 381 and multiple sub-writing units 382, ​​the input ends of all the sub-writing units 382 are connected to the output end of the data processing module 32, the output ends of all the sub-writing units 382 are connected to the input end of the first integration unit 381, and the output end of the first integration unit 381 is connected to the input end of the DDR module 7.

[0051] Each of the sub-write units 382 is used to receive the corresponding parsing signal in the data processing module 32, and the first integration unit 381 is used to arbitrate all the parsing signals received from the sub-write units 382 according to the configuration issued by the register module 35, and send all the parsing signals from the sub-write units 382 to the DDR module 7 in the configured order.

[0052] like Figure 3 As shown, due to Figure 3 There are 4 analytical signals in Figure 3 The write module 38 in the embodiment includes four sub-write units 382, ​​each of which is used to write the corresponding parsed signal into the DDR module 7; in this embodiment, each sub-write unit 382 can be a DMA write module. At the same time, since the output interface of each sub-write unit 382 is an AXI interface, if the AXI interface of each sub-write unit 382 is connected to the DDR module 7 separately, the number of interfaces of the DDR module 7 is required to be relatively high. Therefore, this embodiment also integrates the AXI interfaces of all sub-write units 382 into a single interface connected to the DDR module 7 through the first integration unit 381, which reduces the interface requirements and the occupation of the docking route. The above process of writing to the DDR module 7 can be regarded as an online cache function.

[0053] Similarly, when the parsing signal cached in the DDR module 7 needs to be read and output, it is also necessary to read the parsing signals of different paths separately. Therefore, this embodiment also involves the following design: Figure 3As shown, the image capture module 3 also includes a reading module 39, wherein: the reading module 39 is arranged on the transmission path between the MUX module 37 and the DDR module 7, the input end of the reading module 39 is connected to the output end of the DDR module 7, and the output end of the reading module 39 is connected to the input end of the MUX module 37; the reading module 39 is used to read the parsed signal in the DDR module 7 and send it to the MUX module 37.

[0054] The reading module 39 includes an interface conversion unit 391, a second integration unit 392 and multiple sub-reading units 393, the input ends of all the sub-reading units 393 are connected to the output ends of the second integration unit 392, the input end of the second integration unit 392 is connected to the output end of the DDR module 7, the output ends of all the sub-reading units 393 are connected to the interface conversion unit 391, and the output end of the interface conversion unit 391 is connected to the input end of the MUX module 37.

[0055] The second integration unit 392 is used to send the parsed signal in the DDR module 7 to the corresponding sub-reading unit 393 in sequence according to the configuration issued by the register module 35, and each sub-reading unit 393 then sends the parsed signal to the interface conversion unit 391. The interface conversion unit 391 is used to perform corresponding protocol conversion on the received parsed signal according to the configuration issued by the register module 35, and send it to the MUX module 37 in the configuration order.

[0056] like Figure 3 As shown, due to Figure 3 There are 4 analytical signals in Figure 3 The reading module 39 in the embodiment includes four sub-reading units 393, each of which is used to read the corresponding parsing signal in the DDR module 7 and send it to the MUX module 37, that is, it is mainly used to read the frames of four different exposure formats in the DDR module 7; in this embodiment, each sub-reading module 39 can be a DMA reading module 39. At the same time, since the receiving interface of each sub-reading module 39 is an AXI interface, similarly, the four sub-reading modules 39 integrate the four AXI interfaces into one AXI interface and connect to the DDR module 7 through the second integration unit 392. The above process of reading the DDR module 7 can be regarded as an offline input function.

[0057] Correspondingly, the working mode of the image capture module 3 is shown in the following table:

[0058] In this embodiment, after processing the parsed signal, the data controller module 341 and the timing control module 342 in the data conversion module 34 need to detect whether the data output by the two modules conform to the corresponding protocol, and also need to detect whether the corresponding data volume conforms to the standard. Therefore, this embodiment also involves the following design: Figure 4 As shown, the image capture module 3 also includes a signal detection module 401 and an interrupt module 402, wherein: the input end of the signal detection module 401 is connected to the data conversion module 34, the output end of the signal detection module 401 is connected to the input end of the interrupt module 402, and the output end of the interrupt module 402 is connected to the control module 6.

[0059] The signal detection module 401 is used to compare the parsed signal input to the data conversion module 34 with the interface timing data after format conversion to obtain a comparison result, and to determine whether the comparison result complies with the corresponding protocol. If it does not comply, the signal detection module 401 is used to send a corresponding trigger signal to the interrupt module 402.

[0060] The interrupt module 402 is used to send a corresponding interrupt signal to the control module 6 according to the trigger signal sent by the signal detection module 401, and the control module 6 controls the data conversion module 34 to interrupt and perform corresponding correction according to the interrupt signal.

[0061] In this embodiment, the output end of the data controller module 341 and the output end of the timing control module 342 are both connected to the signal detection module 401. The signal detection module 401 is used to compare the data output by the data controller module 341 with the analysis signal received by the data conversion module 34 to determine whether the analysis processing of the data controller module 341 meets the requirements; it is also necessary to compare the data output by the timing control module 342 with the analysis signal received by the data conversion module 34 to determine whether the analysis processing of the timing control module 342 meets the requirements; when it does not meet the requirements, the signal detection module 401 sends the corresponding judgment result to the interrupt module 402 in an interrupt manner, and the interrupt module 402 sends an interrupt signal to the control module 6 to trigger the interrupt operation of the corresponding module.

[0062] On the other hand, Figure 4As shown, the output end of the write module 38 is also connected to the input end of the interrupt module 402; the write module 38 is also used to send a corresponding trigger signal to the interrupt module 402 after writing a preset amount of parsed signal to the DDR module 7, and the interrupt module 402 is used to send a corresponding interrupt signal to the control module 6 according to the trigger signal sent by the write module 38, and the control module 6 controls the write module 38 to interrupt according to the interrupt signal.

[0063] Furthermore, the image capture module 3 also includes a bandwidth statistics module 403, and the bandwidth statistics module 403 is connected to the register module 35, the write module 38 and the interrupt module 402 respectively. The bandwidth statistics module 403 is used to calculate the data bandwidth of the AXI bus according to the write module 38, and then transmit the result to the register module 35. The user can read the register module 35 through the control module 6 to obtain the bandwidth used by the current module.

[0064] In summary, the multifunctional image input capture device can achieve the following beneficial effects: (1) Improve the interface compatibility between the receiving module 2 and the image processing module 5. No matter which common interface the receiving module 2 uses, the image capture module 3 can support it and output the interface format required by the next level. When formulating the chip solution, the module selection is not restricted due to interface incompatibility, thus meeting the needs of users.

[0065] (2) Supporting multiple data transmission modes, the image capture module 3 provides three working modes, including: online caching, online input and offline input, which enriches the actual application scenarios of the SoC chip and enables the chip product to be applied to more different fields.

[0066] (3) Supporting time-division multiplexing of multiple cameras. When the SoC chip is connected to multiple cameras, there are usually multiple receiving modules 2. The image capture module 3 supports sending the data of the multiple receiving modules 2 to the ISP or output module 8. This improves image throughput and chip performance.

[0067] The embodiment of the present invention further provides a multifunctional image input capture method, which is used in the multifunctional image input capture device. Figure 5 As shown, the method flow includes: In step 101 , corresponding image information is acquired by the detection module 1 and sent to the corresponding receiving module 2 .

[0068] In step 102 , the image information is converted into digital-to-analog format by the receiving module 2 to obtain a corresponding interface signal, and the interface signal is sent to the image capturing module 3 .

[0069] In step 103 , the image capture module 3 decodes all the interface signals and performs data analysis to obtain corresponding interface timing data, and sends the interface timing data to the image processing module 5 .

[0070] In step 104 , the image processing module 5 performs image data processing on the interface timing data.

[0071] In this embodiment, corresponding image information is acquired through multiple detection modules 1 and sent to the corresponding receiving module 2. The image information is converted into digital form by the receiving module 2 to obtain a corresponding interface signal, and the interface signal is sent to the image capture module 3. The image capture module 3 decodes all the interface signals and performs data analysis to obtain corresponding interface timing data, and sends the interface timing data to the image processing module 5. The image processing module 5 performs image data processing on the interface timing data. The image capture module 3 performs corresponding analysis and format conversion on the interface to improve the interface compatibility of the receiving module 2 and the image processing module 5. No matter which common interface the receiving module 2 uses, the image capture module 3 can support it and output it to the interface format required by the next level to avoid limiting the selection of modules due to interface incompatibility.

[0072] For more detailed processes of each step and the specific structure of the image capture module 3, please refer to the aforementioned embodiments, which will not be described again here.

[0073] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multifunctional image input capture device, characterized in that: include: A plurality of detection modules (1), a plurality of receiving modules (2), an image capturing module (3) and an image processing module (5), wherein: The output end of each detection module (1) is connected to the input end of a corresponding single receiving module (2), the output ends of all receiving modules (2) are connected to the input end of the image capturing module (3), and the output end of the image capturing module (3) is connected to the image processing module (5); The detection module (1) is used to obtain corresponding image information and send it to the corresponding receiving module (2); the receiving module (2) is used to perform digital-to-analog conversion on the image information to obtain corresponding interface signals, and send the interface signals to the image capture module (3); the image capture module (3) is used to decode all the interface signals and perform data analysis processing to obtain corresponding interface timing data, and send the interface timing data to the image processing module (5); the image processing module (5) is used to perform image data processing on the interface timing data.

2. The multifunctional image input capture device according to claim 1, characterized in that: The image capture module (3) comprises: a data decoding module (31), a data processing module (32), a data buffer module (33) and a data conversion module (34), wherein: The input end of the data decoding module (31) is connected to the output ends of all the receiving modules (2), the output end of the data decoding module (31) is connected to the input end of the data processing module (32), the output end of the data processing module (32) is connected to the input end of the data buffer module (33), the output end of the data buffer module (33) is connected to the input end of the data conversion module (34), and the output end of the data conversion module (34) is connected to the input end of the image processing module (5); The data decoding module (31) is used to decode all interface signals from the receiving module (2) and send all decoded interface signals to the data processing module (32); the data processing module (32) is used to parse all the decoded interface signals to obtain parsed signals and send the parsed signals to the data buffer module (33); the data buffer module (33) is used to cache the received parsed signals and send the parsed signals to the data conversion module (34); the data conversion module (34) is used to convert the received parsed signals into a DVP format to obtain interface timing data and send the interface timing data to the image processing module (5).

3. The multifunctional image input capture device according to claim 2, characterized in that: The multifunctional image input capture device further comprises a control module (6), and the image capture module (3) further comprises a register module (35), wherein: The control module (6) is connected to the register module (35), and the register module (35) is connected to all modules in the image capture module (3); The control module (6) is used to send bus configuration to the register module (35); the register module (35) is used to send corresponding configuration information to all modules in the image capture module (3) according to the bus configuration, thereby defining and controlling all modules in the image capture module (3).

4. The multifunctional image input capture device according to claim 3, characterized in that: The image capture module (3) further comprises a signal selection module (36), wherein: The signal selection module (36) is arranged on a transmission path between the data processing module (32) and the data cache module (33), the input end of the signal selection module (36) is connected to the output end of the data processing module (32), and the output end of the signal selection module (36) is connected to the input end of the data cache module (33); The signal selection module (36) is used to select one of all parsing signals according to the configuration information sent by the register module (35), and send the selected parsing signal to the data cache module (33).

5. The multifunctional image input capture device according to claim 4, characterized in that: The multifunctional image input capture device further comprises a DDR module (7), and the image capture module (3) further comprises a MUX module (37), wherein: The MUX module (37) is arranged on a transmission path between the signal selection module (36) and the data cache module (33), the input end of the MUX module (37) is connected to the output end of the signal selection module (36), and the output end of the MUX module (37) is connected to the input end of the data cache module (33); The input end of the DDR module (7) is connected to the output end of the data processing module (32), and the output end of the DDR module (7) is connected to the input end of the MUX module (37); The DDR module (7) is used to receive the parsing signal from the data processing module (32) and cache the parsing signal. The DDR module (7) is also used to send the parsing signal to the MUX module (37); the MUX module (37) is used to select the parsing signal from the signal selection module (36) or the parsing signal from the DDR module (7) according to the configuration information sent by the register module (35), and send the selected parsing signal to the data cache module (33).

6. The multifunctional image input capture device according to claim 5, characterized in that: The image capture module (3) further comprises a writing module (38), wherein: The writing module (38) is arranged on a transmission path between the data processing module (32) and the DDR module (7), the input end of the writing module (38) is connected to the output end of the data processing module (32), and the output end of the writing module (38) is connected to the input end of the DDR module (7); The writing module (38) comprises a first integration unit (381) and a plurality of sub-writing units (382), the input ends of all the sub-writing units (382) are connected to the output end of the data processing module (32), the output ends of all the sub-writing units (382) are connected to the input end of the first integration unit (381), and the output end of the first integration unit (381) is connected to the input end of the DDR module (7); Each of the sub-write units (382) is used to receive a corresponding parsing signal in the data processing module (32), and the first integration unit (381) is used to arbitrate all parsing signals received from the sub-write units (382) according to the configuration sent by the register module (35), and send all parsing signals from the sub-write units (382) to the DDR module (7) in the configured order.

7. The multifunctional image input capture device according to claim 5, characterized in that: The image capturing module (3) further comprises a reading module (39), wherein: The reading module (39) is arranged on a transmission path between the MUX module (37) and the DDR module (7); an input end of the reading module (39) is connected to an output end of the DDR module (7), and an output end of the reading module (39) is connected to an input end of the MUX module (37); the reading module (39) is used to read a resolution signal in the DDR module (7) and send the resolution signal to the MUX module (37); The reading module (39) comprises an interface conversion unit (391), a second integration unit (392) and a plurality of sub-reading units (393); the input ends of all the sub-reading units (393) are connected to the output end of the second integration unit (392); the input end of the second integration unit (392) is connected to the output end of the DDR module (7); the output ends of all the sub-reading units (393) are connected to the interface conversion unit (391); and the output end of the interface conversion unit (391) is connected to the input end of the MUX module (37); The second integration unit (392) is used to send the parsed signal in the DDR module (7) to the corresponding sub-reading unit (393) in sequence according to the configuration sent by the register module (35), and each sub-reading unit (393) then sends the parsed signal to the interface conversion unit (391). The interface conversion unit (391) is used to perform corresponding protocol conversion on the received parsed signal according to the configuration sent by the register module (35), and send it to the MUX module (37) in the sequence of configuration.

8. The multifunctional image input capture device according to claim 6, characterized in that: The image capture module (3) further comprises a signal detection module (401) and an interruption module (402), wherein: The input end of the signal detection module (401) is connected to the data conversion module (34), the output end of the signal detection module (401) is connected to the input end of the interruption module (402), and the output end of the interruption module (402) is connected to the control module (6); The signal detection module (401) is used to compare the parsed signal input to the data conversion module (34) with the interface timing data after format conversion to obtain a comparison result, and to determine whether the comparison result complies with the corresponding protocol. If it does not comply, the signal detection module (401) is used to send a corresponding trigger signal to the interrupt module (402); The interrupt module (402) is used to send a corresponding interrupt signal to the control module (6) according to the trigger signal sent by the signal detection module (401), and the control module (6) controls the data conversion module (34) to interrupt and perform corresponding correction according to the interrupt signal.

9. The multifunctional image input capture device according to claim 8, characterized in that: The output end of the write module (38) is also connected to the input end of the interrupt module (402); the write module (38) is also used to send a corresponding trigger signal to the interrupt module (402) after writing a preset amount of parsed signal to the DDR module (7); the interrupt module (402) is used to send a corresponding interrupt signal to the control module (6) according to the trigger signal sent by the write module (38); the control module (6) controls the write module (38) to interrupt according to the interrupt signal.

10. A multifunctional image input capture method, used in the multifunctional image input capture device according to any one of claims 1 to 9, characterized in that: include: Acquiring corresponding image information through the detection module (1) and sending it to the corresponding receiving module (2); Performing digital-to-analog conversion on the image information through the receiving module (2) to obtain a corresponding interface signal, and sending the interface signal to the image capturing module (3); Decoding all the interface signals and performing corresponding data analysis processing through the image capture module (3) to obtain corresponding interface timing data, and sending the interface timing data to the image processing module (5); The interface timing data is processed into image data by the image processing module (5).

Images