Flexible hub for handling multi-sensor data

Abstract

The hub 702 receives the sensor data streams and distributes them to various systems that use the sensor data. A demultiplexer 804 receives the streams, filters out undesired streams, and provides the desired streams to the appropriate multiplexer or series of muxes 808A-808D. Each mux combines the received streams and provides an output stream to a respective formatter or output block 816A-816D. The formatter or output blocks 816A-816D are configured based on the destination of the mux output stream, such as the image signal processor 614, the processor 616, memory 704, or external transmission 706. The output blocks 816A-816D reformat e the received streams into a format appropriate for the recipient and provide the reformatted stream to the recipient.

Description

[Technical Field]

[0001] This field relates to processing sensor input data in system-on-chip (SoC) devices.

[0002] The use of electronic devices in automobiles is increasing every day. In addition to traditional engine controllers, transmission controllers, infotainment units, and body controllers, the emergence of numerous safety and autonomous systems has significantly increased the processing required within the automobile. For example, adaptive cruise control may involve intercommunication between a radar system, engine controller, and transmission controller. As another example, in a bird's-eye view display, the output of multiple different cameras located at various locations is provided to a processor, which processes the received video and generates a resulting bird's-eye view image, which is provided to the infotainment system for display to the driver. This increase in the number and variety of input sensors places a significant burden on the SoC device that receives the sensor data. Furthermore, sensor data is often used by multiple processes, placing increased demands on the SoC device. This burden is further complicated by reliability requirements for safety systems that use sensor data. Summary of the Invention

[0003] In one example, a hub is provided to receive sensor data and distribute it to various systems that use the sensor data. A demultiplexer (demux) receives the streams, filters out undesired streams, and provides the desired streams to an appropriate multiplexer (mux) or series of multiplexers. Each multiplexer combines the received streams and provides an output stream to a respective formatter or output block. The formatter or output blocks are configured based on the destination of the multiplexer output streams, such as an image signal processor, a processor, memory, or external transmission. The output block reformats the received streams into a format appropriate for the recipient and provides the reformatted stream to the recipient.

[0004] By providing the stream directly to one or more recipients in a format appropriate for each recipient, rather than feeding the stream into DRAM and having each recipient retrieve its own stream or streams and perform its own reformatting operation, delays in operating on the stream are reduced and DRAM bandwidth issues are reduced.

[0005] For a detailed description of various examples, reference will now be made to the accompanying drawings. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a diagram of a vehicle and the fields of view of various optical and radar sensors.

[0007] [Figure 2] 2 is a block diagram of the vehicle of FIG. 1 including optical and radar sensors, according to some examples.

[0008] [Figure 3] FIG. 3 is a block diagram of the optical sensor system of FIG. 2.

[0009] [Figure 4]FIG. 3 is a block diagram of the radar sensor system of FIG. 2.

[0010] [Figure 5] FIG. 5 is a block diagram of an SoC such as that used in the sensor system of FIGS. 3 and 4.

[0011] [Figure 6] 1 is a block diagram and a sensor data flow in an SoC according to the prior art;

[0012] [Figure 7] 1 is a block diagram and sensor data flow within an SoC including a hub for distributing sensor data.

[0013] [Figure 8] 8 is a block diagram of the SoC of FIG. 7 and a sensor data flow.

[0014] [Figure 9] FIG. 1 is a block diagram of a system that reformats sensor data for various destinations.

[0015] [Figure 10] FIG. 8 illustrates software and hardware operation and sensor data flow for a functional safety application of the SoC of FIG.

[0016] [Figure 11] FIG. 8 illustrates software and hardware operation and sensor data flow for an elastic processing application of the SoC of FIG. 7.

[0017] [Figure 12] FIG. 8 illustrates the general software and hardware operation of the SoC of FIG. 7. DETAILED DESCRIPTION OF THE INVENTION

[0018] Referring now to FIG. 1 , a vehicle 100 is shown. The vehicle 100 includes a set of cameras, other optical sensors, radar sensors, ultrasonic sensors, and / or other suitable sensors. In various examples, the optical sensors include a left camera 102 and a right camera 104 that provide stereo images from the front of the vehicle 100 for lane departure warning, traffic sign recognition, and collision warning. A rear camera 106 provides a rear view for parking assistance. A left camera 108 provides a left side view, and a right camera 110 provides a right side view. The images captured by the cameras 102, 104, 106, 108, and 110 are combined to provide a bird's-eye view.

[0019] Examples of radar sensors include a long-range radar 112 mounted at the front of the vehicle 100 for adaptive cruise control. Three shorter-range radar units 114L, 114C, and 114R are mounted at the front of the vehicle 100 to enable cross-traffic and forward collision warnings. Radar units 116L and 116R are mounted on the left and right sides of the vehicle 100 for blind spot detection. Radar units 118L and 118R are mounted at the rear of the vehicle 100 for rear-end collision warning. Exemplary ultrasonic units 120F and 120R are mounted at the front and rear of the vehicle 100 for parking assistance. These camera, radar, and ultrasonic sensors provide input to various advanced driver assistance systems (ADAS). These sensors are merely examples, and many other sensors, such as LIDAR (light detection and ranging) sensors, could be used as well.

[0020] Referring now to FIG. 2, the systems and sensors of vehicle 100 are shown. In this example, cameras 102, 104, and 106 are connected to a front and rear camera module 202. Cameras 108 and 110 are connected to a side camera module 204. Long-range radar 112 and front-end radar units 114L, 114C, and 114R are connected to a first radar module 206. Ultrasonic detectors 120R and 120L are also connected to the first radar module 206. Radar units 116L, 116R, 118L, and 118R are connected to a second radar module 208. The front and rear camera module 202, the side camera module 204, the first radar module 206, and the second radar module 208 are connected to a sensor fusion module 210, which integrates the various sensor outputs produced by the other modules. An infotainment module 212 is connected to the sensor fusion module 210 so that the vehicle driver can view generated images such as a 360° bird's eye view or receive warnings when an obstacle is nearby. More or fewer sensors can be connected to a given module, and multiple sensor types can be provided in a single module.

[0021] FIG. 3 illustrates a front and rear camera module 202, according to some examples. An SoC 300 forms the digital processing element of the front and rear camera module 202. The SoC 300 includes processing resources 302. The processing resources 302 may include any number of interconnected processing devices, ranging from cores to discrete processors, and may include general-purpose processing devices, digital signal processors, application-specific processors, controllers, and / or other suitable processing devices. The SoC 300 may also include a safety microcontroller (MCU) 304. A processor power supply 306 provides power to the SoC 300 and receives power from a system power supply 308. The system power supply 308 receives power from a battery 310 via a battery input protection circuit 312. A memory 314, including a flash memory 316 and a DRAM 318, is connected to the SoC 300. The flash memory 316 is a non-volatile memory used to store program instructions for operating the module, including instructions for implementing functional safety and performance monitoring performed by the processor, as described below. Optical systems 321, 323, and 325 in each camera 102, 104, and 106 provide light to respective CMOS sensors 322, 324, and 326 in imager block 329. The outputs of CMOS sensors 322, 324, and 326 are provided to SoC 300 to provide optical input. In one example, the outputs are provided as Mobile Industry Processor Interface (MIPI) Alliance Camera Serial Interface 2 (MIPI CSI-2) compliant data streams. In one example, the individual sensor CSI-2 data streams are provided to a CSI-2 aggregator, which combines the individual streams into a single multiplexed output stream, with the individual streams becoming different virtual channels on the CSI-2 output.A vehicle interface block 328 includes a CAN bus PHY 330 for connection to a controller area network (CAN) bus network within vehicle 100, an Ethernet PHY 332 for connections as needed within vehicle 100, and a SerDes 334 used to connect to various other elements within vehicle 100, such as sensor fusion module 210. Vehicle interface block 328 is connected to SoC 300.

[0022] FIG. 4 illustrates radar module 208 according to some examples. SoC 400 includes an associated processor 402 and safety MCU 404. A processor power supply 406 is connected to SoC 400 and receives its power from a system power supply 408. A battery 310 is connected to a battery input protection module 412, which provides its output to the system power supply 408. An RF power module 440 is connected to the system power supply 408. Memory 414, including flash memory 416 and DRAM 418, is connected to SoC 400. Flash memory 416 is a non-volatile memory used to store program instructions for operating the module, including instructions for implementing functional safety and performance monitoring performed by the processor, as described below. Similarly, a vehicle interface block 428, including a CAN bus PHY 430, an Ethernet PHY 432, and a SerDes 434, is connected to SoC 400 to provide connectivity for module 206. A radar front-end module 436 includes individual radar transceivers 438A-D associated with the individual radar units 116L, 116R, 118L, and 118R. The radar front-end module 436 is connected to transmit 442 and receive 444 antennas. The radar front-end module 436 is also connected to the SoC 400 and, in one example, provides a CSI-2 data stream to the SoC 400, with each radar transceiver 438A-D on a different virtual channel.

[0023] FIG. 5 is a block diagram of an example SoC 500, such as SoC 300 or SoC 400. A series of more powerful microprocessors 502, such as an ARM® A72 or A53 core, form the main general-purpose processing block of SoC 500, while a digital signal processor (DSP) 504 provides specialized computing capabilities. These microprocessors 502 and DSP 504 are processor 302 and processor 402. A simpler microprocessor 506, such as an ARM R5F core, provides general control capabilities in SoC 500. A high-speed interconnect 508 connects microprocessor 502, DSP 504, and microprocessor 506 to various other components within SoC 500. For example, a shared memory controller 510, including on-board RAM 512, is connected to high-speed interconnect 508 and serves as the on-board RAM for SoC 500. A DDR memory controller system 514 is connected to high-speed interconnect 508 and serves as an external interface to external DRAM. A video acceleration module 516 and a radar processing accelerator (PAC) module 518 are also connected to the high-speed interconnect 508. A vision processing accelerator module 520 is connected to the high-speed interconnect 508, as is a depth and motion PAC module 522. A graphics acceleration module 524 is connected to the high-speed interconnect 508. A display subsystem 526 is connected to the high-speed interconnect 508 and includes conversion logic 528 and output logic 530 to enable operation and connection with various video monitors. A system services block 532, including items such as a DMA controller, memory management unit, general-purpose I / O, and mailboxes, is provided for normal SoC 500 operation. A serial connectivity module 534 is connected to the high-speed interconnect 508 and includes SoC modules. A vehicle connectivity module 536 provides interconnects such as a PCIe block 538, a USB block 540, and an Ethernet switch 542. Acquisition / MIPI module 544 includes a four-lane CSI-2 compliant transmit block 546 and a four-lane CSI-2 receive module and hub 548. Further details regarding CSI-2 receive module and hub 548 are provided below.

[0024] An MCU island 560 is provided as a secondary subsystem, handling operation of the integrated SoC 500 when other components are powered down to conserve energy. The MCU island 560 also operates as the safety MCU 304, 404. An MCU ARM processor 562 acts as a master and is coupled to the high-speed interconnect 508 via an isolation interface 561. An MCU general-purpose I / O (GPIO) block 564 acts as a slave. An MCU RAM 566 is provided to serve as local memory for the MCU ARM processor 562. A CAN bus block 568 is connected to enable operation in a conventional CAN bus environment within the vehicle 100. An Ethernet MAC (medium access control) block 570 is provided for further connectivity in the vehicle 100. A non-volatile memory (NVM), such as flash memory 316 or 416, is connected to the MCU ARM processor 562.

[0025] This is one example SoC provided for illustrative purposes; many other SoC designs are possible, with varying numbers of processors, DSPs, accelerators, etc.

[0026] 6 illustrates the operation of a sensor module according to the prior art. A camera 602 provides an output sensor data stream to a CSI-2 receiver (RX) module 604, which provides the output sensor data stream to an SoC 606, and more specifically to a CSI-2 parser 608 within the SoC 606. The CSI-2 parser 608 inspects the input sensor data stream, validates packets, and provides signals to separate virtual channels. The output sensor data stream of the CSI-2 parser 608 is provided to a pixel packing and memory interface module 610 so that data from the camera 602 can be stored directly in a DRAM 612. An image system processing (ISP) block 614 is connected to the DRAM 612 to receive the sensor data stream from the camera 602, which is stored in the DRAM 612. Similarly, a processor 616 is connected to the DRAM 612 to receive data from the camera 602. Various users of the data, such as the ISP block 614 and the processor 616, must connect separately to the DRAM 612 and retrieve the desired sensor data or use a DMA channel to retrieve the sensor data. This introduces both operational delays as the sensor data stream must be stored, retrieved, and reformatted in the DRAM 612, and a potential bottleneck if the number of systems requesting data exceeds the available bandwidth of the DRAM 612. The restrictive and inflexible nature of this configuration limits the number of sensors that can be connected to a particular SoC 606 and, due to the convoluted data path, restricts the sensor performance of the SoC 606 to a lesser extent.

[0027] In FIG. 7 , a CSI-2 RX module 604, which receives sensor data from modules such as the imager block 329 or the radar front-end module 436, is connected to the SoC 700, and more particularly to a hub subsystem 702, such as a CSI-2 receive module and hub 548 within the SoC 700. The hub subsystem 702 distributes the data received from the CSI-2 RX module 604 to any or all of the DRAM 612, the ISP block 614, the processor 616, or the SRAM 704. The output of the hub subsystem 702 may also be provided to a CSI-2 transmit (TX) module 706, allowing the sensor data to be provided to a different SoC for use and processing. The hub subsystem 702 provides notifications to processor X 708. Processor Y 616 and processor X 708 can be any processor within the SoC 700. For example, in one example, processor Y 616 is microprocessor 502, and processor X 708 is microprocessor 506.

[0028] FIG. 8 is a block diagram of a hub subsystem 702, such as the CSI-2 receive module and hub 548. The CSI-2 RX module 604 provides its output sensor data stream to a CSI-2 parser 802, which analyzes the received input CSI-2 sensor data stream and determines the virtual channels present in the input sensor data stream. This allows the CSI-2 parser 802 to provide signals, such as shims for each packet, to split a single physical stream into a series of physical streams based on the logical streams or individual device streams present in the received sensor data stream, as defined by the virtual channel values. The output sensor data stream of the CSI-2 parser 802 is provided to a demultiplexer or demux 804 as a demultiplexer input sensor data stream. The demultiplexer 804 operates to provide the identified logical output sensor data stream of the CSI-2 parser 802 to various format chains based on the signals from the CSI-2 parser 802. For example, demultiplexer 804 may include a table 806 indicating the distribution of particular received virtual channels within the data stream. For example, virtual channel 0 input sensor data stream may be routed to multiplexer or mux 0, input 0. Similarly, virtual channel 2 may be provided to multiplexer 0, input 1. Virtual channel 3 may be provided to both mux 1, input 0 and mux 2, input 3, indicating that a single input logical sensor data stream may be distributed to multiple multiplexers for operation. Demultiplexer 804 performs stream duplication tasks to simultaneously provide individual sensor data streams to each multiplexer. Virtual channel 1 may be provided to mux 2, input 3, and demultiplexer 804 provides virtual channels 1 and 3 as a single physical sensor data stream. Demultiplexer 804 filters out virtual channels not programmed into table 806. Although table 806 is shown for illustrative purposes, individual registers may be provided to control the stream splitting and filtering operations of demultiplexer 804 .

[0029] Although a single stream that includes a virtual channel and is therefore considered to be the incoming sensor data stream is shown received by demultiplexer 804, in some examples, multiple sensor data streams or channels, each with a virtual channel, may be provided to demultiplexer 804. In such examples, table 806 may include additional columns to identify particular sensor data streams or channels.

[0030] The demultiplexer 804 is connected to a set of multiplexers 808A, 808B, 808C, and 808D. The multiplexers 808A-D operate to redistribute or rename the input sensor data streams to the desired virtual channels. As shown in table 810, the input sensor data stream received at input 0 of multiplexer 808A is output as virtual channel 0. The input sensor data stream received at input 1 is provided as an output sensor data stream on virtual channel 1. The input sensor data stream from input 2 is provided to virtual channel 7. If multiple virtual channels are provided to a single multiplexer input, the input number column is replaced by the input virtual channel value to identify the mapping between the input virtual channel and the output virtual channel. The output sensor data streams of the multiplexers 808A-D proceed to respective QoS or quality of service modules 812A-D. The QoS modules 812A-D each include a control bit 814 that indicates whether a particular sensor data stream should be operating in real-time or non-real-time mode. When operating in real-time mode, the QoS modules 812A-D provide buffering for real-time operation using included FIFOs. The FIFOs buffer real-time operation with priority escalation of the selected logical stream when the FIFO level becomes full.

[0031] The output sensor data streams of QoS modules 812A-D proceed to respective output blocks 814A, 814B, 816C, and 816D. Each output block 816A-D includes a register, as shown in table 818, to indicate the format of the particular output block. Thus, by selecting the CSI-2 TX option in table 818, output block 816A is configured as a CSI-2 formatter to provide the output sensor data stream to CSI-2 TX module 706. Note that although the output sensor data stream of output block 816A is shown connected to CSI-2 TX module 706, this is not a direct connection, but rather is via high-speed interconnect 508, so that the output sensor data stream of output block 816A can be directed to any of the different elements within SoC 700, particularly any potential recipient of the sensor data stream. The illustrated output block 816B is configured to operate as a video interface formatter and is connected to ISP block 614. The illustrated output block 816C is configured as a memory interface and is connected to the on-board SRAM 704. The output block 816D is configured to operate with the processor and / or RAM and is connected to the processor 616 and the DRAM 612.

[0032] Each output block 816A-D further includes data options associated with the selected format, such as data width, stride, packing, etc. Thus, table 818 can be viewed as having additional entries for specific format options. In one example, table 818 is a register in each output block 816A-D, and the format options are additional registers. In this example, each output block 816A-D can perform any of the available format operations, increasing the flexibility of hub subsystem 702.

[0033] Great flexibility is provided for different sensors, such as video or radar, because the output sensor data streams of output blocks 816A-D are provided over high-speed interconnect 508. For example, two different video sensor data streams can be formatted separately and provided to separate processors.

[0034] This configuration, including demultiplexer 804 and multiplexers 808A-D, provides the flexibility to split and combine logical sensor data streams from various sensors as needed. Demultiplexer 804 allows the input sensor data stream to be immediately split into respective threads for operation without first being stored in DRAM 612. In such a manner, ISP block 614 and processor 616 do not access DRAM 612 to obtain the data, so the data provided to them is more tiled and DRAM 612 bandwidth is not used. Output blocks 816A-D perform sensor data stream reformatting appropriate for selected recipients, further reducing processing time and therefore data processing timelines.

[0035] A safety monitor 820, such as the safety MCU 304, 404, or MCU island 560, and any additional specialized hardware, is connected to each of the demultiplexer 804, multiplexers 808A-D, QoS modules 812A-D, and output blocks 816A-D, and monitors each item's output sensor data stream for potential errors, such as packet or stream loss or errors. If an error is detected, an interrupt is provided and the MCU 304, 404, or MCU island 560 performs safety action, examples of which are shown below.

[0036] As shown in Figure 8, the output control block may have a series of different options for handling the sensor data stream, options such as CSI-2 TX, video or image format, memory format, and processor format. Referring to Figure 9, if the control block is configured in CSI-2 transmit format, the output sensor data stream from the QoS block 812 is provided to an extraction module 902, which extracts the desired virtual channels from the received input sensor data stream and provides the output sensor data stream to a re-formatter 904 to repackage them in the appropriate CSI-2 format along with the desired output virtual channels for provision to the CSI-2 TX module 706.

[0037] If the control block is configured for video or image operation, the output sensor data stream of the QoS block 812 is provided to a reformatter 906, which reformats the input sensor data stream from that provided according to the CSI-2 standard into the respective protocol for the video or ISP subsystem. The reformatted data is provided to a video interface logic block 908, which is connected to, for example, the ISP block 614.

[0038] If the data is to be provided to memory, the output block is configured to include a reformatter / pack module 910 to reformat the desired virtual channel and then pack or compress the data for storage in memory. The output sensor data stream of the reformatter / pack module 910 is provided to a memory interface logic block 912, which provides the output sensor data stream to DRAM 612 for storing the data.

[0039] If the data is to be directly manipulated by a processor, the data is provided to a reformatting block 914 to create the desired virtual channel sensor data stream for processing. After reformatting block 914, the data is provided to a processor interface logic block 916 for provision to the desired processor.

[0040] FIG. 10 illustrates some example operations of SoC 700. This example is for functional safety, where action is taken to mitigate risk and damage during a failure. In the example, two optical input sensor data streams SA and SB are provided, and if one of the input sensor data streams fails, the other stereo sensor data stream is provided to replace the failed data. The basic algorithm is shown in block 1002, which illustrates the basic algorithm for the example. As shown in the application section, in a first step, demultiplexer 804 is configured to assign input sensor data stream SA to virtual channel 0 and input sensor data stream SB to virtual channel 1, both of which are provided to two multiplexers 808. Next, as shown in the functional safety monitor section, the operation of the two sensor channels is monitored, and functional safety monitor 820 is configured to assign virtual channel 0 to sensor data stream SA and virtual channel 1 to sensor data stream SB. The default setting of the first multiplexer 808 is to select sensor data stream SA (i.e., virtual channel 0), and the default setting of the second multiplexer 808 is to select sensor data stream SB (i.e., virtual channel 1). If sensor data stream SA subsequently fails, the first multiplexer 808 is reconfigured to select virtual channel 1, resulting in sensor data stream SB being provided on the output. Similarly, if sensor data stream SB fails, the second multiplexer 808 is reconfigured to provide sensor data stream SA as an output. In this functional safety example, the performance monitor section is not used.

[0041] Alternatively, in another example, sensor data streams SA and SB are provided to a single multiplexer 808, which provides sensor data stream SA on output virtual channel 0 and sensor data stream SB on output virtual channel 1. If sensor data stream SA fails, sensor data stream SB is duplicated and the duplicate sensor data stream is provided as output virtual channel 0 to replace the normal sensor data stream SA. Similarly, if sensor data stream SB fails, sensor data stream SA is duplicated and the duplicate is provided as output virtual channel 1.

[0042] This algorithm is reflected in hardware block 1004. The demultiplexer 804 is configured to output the SA stereo sensor data stream as virtual channel 0 and the SB stereo sensor data stream as virtual channel 1 to two multiplexers 808. Each multiplexer 808 is configured based on the state of a particular virtual channel. For normal operation, the first multiplexer 808 simply passes the SA sensor data stream as virtual channel 0, and the second multiplexer 808 passes the SB sensor data stream as virtual channel 1. If a particular virtual channel or sensor data stream fails, the demultiplexer 804 is reprogrammed by the functional safety monitor 820 to duplicate the remaining sensor data stream and provide the duplicated sensor data stream in place of the original stream. The associated multiplexer 808 is reprogrammed, and the multiplexer 808 then reformats the duplicated sensor data stream to have the virtual channel of the failed sensor data stream. The operation of a single multiplexer 808, dual virtual channel input example, is similar. Hardware block diagrams are provided to illustrate these flows and operations in block diagram format.

[0043] In this way, functional safety is provided for the stereo sensor data streams SA and SB, so that there are always two sensor data streams provided from the hub subsystem 702 to the CSI-2 TX module 706, and as a result, the downstream SoC 1006 always receives two sensor data streams, which may be identical.

[0044] Depending on the vehicle's particular operation or location, when multiple systems residing within a given SoC are operating, the SoC may not have the processing power to properly process all of the incoming sensor data streams. In that case, some of the processing can be transferred to a different SoC with less overhead. This is known as elastic processing and can be easily developed using the hub subsystem 702. Referring to FIG. 11 , a basic algorithm is shown in block 1102, in which an application assigns various input channels to various virtual channels for passing through the system. In one example, the performance monitor section, which is the microprocessor 506 running appropriate software, determines that if performance on a particular SoC is above a given threshold, the sensor data streams for virtual channels 10-15 are to be transferred to use the SoC 1006 for processing. The transfer of the data streams for virtual channels 10-15 is performed in hardware components, as shown in block 1104, with the demultiplexer 804 configured to appropriately demultiplex the input channels into the appropriate multiplexes 808. If the SoC has the capability to process all channels, multiplexer 808 is configured to multiplex the various sensor data streams it receives and provide each multiplexed sensor data stream to an appropriately configured output block for formatting and delivery to a recipient. However, if performance monitors indicate that a particular channel or sensor data stream cannot be processed, demultiplexer 804 is reprogrammed to provide those sensor data streams to multiplexer 808, which then combines the sensor data streams as virtual channels 10-15 and provides an output configured for CSI-2 operation to CSI-2 reformatter 816, which provides the output sensor data stream to CSI-2 TX module 706 for delivery to SoC 1006.

[0045] With this highly flexible configuration provided by the demultiplexer, multiplexer, and output blocks, the SoC can be dynamically configured and reconfigured for various operations or functions as desired for a particular vehicle.

[0046] Figure 12 is a more general format of Figures 10 and 11. Application software 1202 determines the various channel or sensor data streams being received and the desired output in step 1204. In step 1206, failure actions for each associated stream are determined and provided to safety monitor 820. In step 1208, performance parameters are determined for the particular SoC and provided to the performance monitor for application.

[0047] Block 1210 is a control algorithm, which can be implemented in hardware, software, or a combination, that applies safety and performance characteristics or rules. In step 1212, the particular selected channel is cycled through and the demultiplexer 804 is programmed for appropriate filtering and demultiplexing operation. In step 1214, the particular safety monitoring condition is determined and pass criteria and failure actions are programmed into the safety monitor 820. In step 1216, the performance of the particular channel is monitored and in step 1218 a determination is made whether a threshold has been exceeded. If so, in step 1220, the multiplexer and potentially the demultiplexer are reconfigured to provide the associated sensor data stream to the external SoC. Thus, in the hub subsystem hardware 1224, the demultiplexer 804 is appropriately configured to filter the incoming sensor data stream and route the incoming sensor data stream to the appropriate multiplexer 808. The safety monitor and performance monitor operations are performed as desired, and the multiplexer 808 operates according to its programmed operations. When a safety or performance event occurs, the safety monitor 820 and performance monitor appropriately reconfigure the demultiplexer 804 and multiplexer 808 according to the provided rules.

[0048] The above description is illustrative and not limiting. For example, the above examples may be combined. Many other examples are possible upon review of the above description. The scope should, therefore, be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. In the appended claims, the terms "comprising" and "wherein" are used as the plain-English equivalents of the respective terms "including" and "wherein."

Claims

1. 1. A circuit for distributing a data stream to multiple recipients, comprising:

1. A demultiplexer comprising: an input for receiving a demultiplexer input data stream; a set of outputs, each selectively providing a subset of the demultiplexer input data streams based on a distribution of the subset of the demultiplexer input data streams; the demultiplexer, a plurality of multiplexers coupled to the demultiplexer, each of the plurality of multiplexers comprising: an input coupled to each of the set of outputs of the demultiplexer, the input receiving a respective subset of the demultiplexer input data stream; an output providing a multiplexer output data stream; the plurality of multiplexers, a plurality of output blocks, each of which: an input coupled to an output of each multiplexer of the plurality of multiplexers, the input receiving the multiplexer output data stream from the respective multiplexer as an output block input data stream; an output that provides a formatted output block output data stream for a recipient; the output block including: The circuit includes:

2. 2. The circuit of claim 1, the demultiplexer is configurable to provide a plurality of demultiplexer input data streams to a multiplexer of the plurality of multiplexers as a multiplexer input data stream; A circuit wherein a multiplexer of the plurality of multiplexers is configurable to combine the received multiplexer input data streams into a multiplexer output data stream.

3. 2. The circuit of claim 1, A circuit wherein the demultiplexer is configurable to provide one of the demultiplexer input data streams to two different multiplexers simultaneously.

4. 2. The circuit of claim 1, A circuit wherein the demultiplexer input data stream received at the demultiplexer comprises a plurality of individual data streams assigned to different virtual channels.

5. 5. The circuit of claim 4, a multiplexer of the plurality of multiplexers is configurable to provide a plurality of individual data streams that are assigned to different virtual channels on the multiplexer output data stream; The circuitry, wherein the different virtual channels assigned by the multiplexer are different virtual channels for the individual data streams than those received by the demultiplexer for those individual data streams.

6. 2. The circuit of claim 1, The circuit, wherein the plurality of recipients are at least two of an image signal processor, a processor, and a memory.

7. 7. The circuit of claim 6, the demultiplexer input data stream conforms to the MIPI CSI-2 (Mobile Industry Processor Interface Alliance Camera Serial Interface 2) industry standard; The circuitry wherein the plurality of recipients includes a transmit module for providing a transmitter output data stream in accordance with the industry standard.

8. 2. The circuit of claim 1, A circuit wherein the output of a multiplexer of the plurality of multiplexers is configurable to be directed to any of the plurality of recipients.

9. 2. The circuit of claim 1, a circuit, wherein each multiplexer of the plurality of multiplexers is configurable to operate with each of the plurality of recipients.

10. 2. The circuit of claim 1, a parser coupled to the demultiplexer to receive an input data stream and provide a parser output data stream to the demultiplexer as the demultiplexer input data stream, the parser validating the received parser input data stream.

11. 2. The circuit of claim 1, the circuit further comprising: a plurality of quality of service modules coupled to the plurality of multiplexers and the plurality of output blocks, one quality of service module for each multiplexer and each output block, each quality of service module receiving the multiplexer output data stream of the respective multiplexer as a quality of service module input data stream and providing a quality of service module output data stream to the respective output block as the output block input data stream.

12. 1. A system comprising:

1. A system on chip (SoC), comprising: a processor; a digital signal processor; Memory and an external memory interface; an interconnect coupled to and interconnecting said processor, said digital signal processor, said memory, and an external memory interface; a receiver module that receives a receiver module input data stream and provides a receiver module output data stream; a circuit coupled to the receiver module for distributing the data stream to a plurality of receivers, the circuit comprising: a demultiplexer that receives the receiver module output data stream from the receiver module as a demultiplexer input data stream, and selects and provides the demultiplexer input data stream as the plurality of demultiplexer output data streams based on a distribution of a plurality of demultiplexer output data streams; a plurality of multiplexers coupled to the demultiplexer, each multiplexer receiving a respective subset of the demultiplexer output data stream from the demultiplexer as a multiplexer input data stream and providing a multiplexer output data stream; a plurality of output blocks, one output block for each multiplexer, each of the plurality of output blocks coupled to a respective multiplexer of the plurality of multiplexers, each output block receiving the multiplexer output data stream from its respective multiplexer as an output block input data stream and providing an output block output data stream formatted for a selected recipient; a circuit including: A system including the system-on-chip.

13. 13. The system of claim 12, the demultiplexer is configurable to provide a plurality of demultiplexer output data streams as multiplexer input data streams to a multiplexer of the plurality of multiplexers; A system wherein a multiplexer of the plurality of multiplexers is configurable to combine the multiplexer input data streams into a multiplexer output data stream.

14. 13. The system of claim 12, A system wherein the demultiplexer is configurable to provide one of the demultiplexer output data streams to two different multiplexers simultaneously.

15. 13. The system of claim 12, A system wherein the demultiplexer input data stream received at the demultiplexer comprises a plurality of individual data streams assigned to different virtual channels.

16. 16. The system of claim 15, a multiplexer of the plurality of multiplexers is configurable to provide a plurality of individual data streams that are assigned to different virtual channels on the multiplexer output data stream; The system wherein the different virtual channels assigned by the multiplexer are different virtual channels for the individual data streams than those received by the demultiplexer for those individual data streams.

17. 13. The system of claim 12, the SoC further includes an image signal processor; at least two of the image signal processor, the processor, and the memory are each coupled to the interconnect; the plurality of output blocks are coupled to the interconnect; A system wherein an image signal processor and at least two of said processor and said memory are said plurality of recipients of said circuitry.

18. 13. The system of claim 12, The SoC, a transmit module coupled to the interconnect and providing a transmit module output data stream in accordance with the MIPI CSI-2 (Mobile Industry Processor Interface Alliance Camera Serial Interface 2) industry standard; said receiver module input data stream conforming to said industry standard; The system wherein the transmitting module is a receiver of the circuit.

19. 20. The system of claim 18, a non-volatile memory external to the SoC, the non-volatile memory being connected to the processor; monitoring the performance of the SoC; providing at least one demultiplexer input data stream received at the demultiplexer to the transmit module without being operated by the SoC other than the circuit; The system further comprises the non-volatile memory storing instructions for performing the above.

20. 13. The system of claim 12, The system, wherein the output of a multiplexer of the plurality of multiplexers is configurable to be directed to any of the plurality of recipients.

21. 13. The system of claim 12, A system wherein each multiplexer of the plurality of multiplexers is configurable to operate with each of the plurality of recipients.

22. 13. The system of claim 12, The SoC, The system further includes a parser coupled to the demultiplexer for receiving a receiver output data stream as a parser input data stream and for providing a parser output data stream to the demultiplexer, the parser validating the parser input data stream.

23. 13. The system of claim 12, The system, wherein the SoC further includes a plurality of quality of service modules coupled to the plurality of multiplexers and the plurality of output blocks, one quality of service module for each multiplexer and each output block, each quality of service module receiving the multiplexer output data stream of its respective multiplexer as a quality of service module input data stream and providing a quality of service module output data stream to its respective output block.

24. 13. The system of claim 12, a non-volatile memory external to the SoC, the non-volatile memory being connected to the processor; monitoring operation of said circuit for defects in the data stream; configuring the demultiplexer and one of the plurality of multiplexers to, upon determining a defect in a data stream, replicate another data stream to be provided as a replacement for the defect in the data stream; The system further comprises the non-volatile memory storing instructions for performing the above.

Images