Local generation of commands for vehicle sensors

Abstract

An apparatus for controlling an imaging sensor (28) in a vehicle (24) includes an Ethernet transceiver (72), a sensor interface (80), and a local processor (76). The Ethernet transceiver is configured to communicate with the remote processor over an in-vehicle Ethernet network. The sensor interface is configured to communicate with the imaging sensor. The local processor, local to the apparatus and remotely located from the remote processor, is configured to receive image data and auxiliary data related to the image data from the imaging sensor via the sensor interface, transmit at least the image data to the remote processor via the Ethernet transceiver, generate control commands locally and independently of the remote processor based on the auxiliary data for controlling operational aspects of the imaging sensor, and transmit the control commands to the imaging sensor via the sensor interface.

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 310,188, filed February 15, 2022, the disclosure of which is incorporated herein by reference.

[0002] The present disclosure relates generally to communication networks, and more specifically to a method and system for local control of sensors in a vehicle via a sensor bridge. [Background technology]

[0003] Various systems and applications, such as automotive systems and industrial control systems, employ sensors that are connected to Ethernet networks to transmit acquired data and receive control commands over the network.

[0004] The above description is provided as a summary of the related art in this field and should not be construed as an admission that any of the information it contains constitutes prior art to the present patent application. Summary of the Invention

[0005]

[0005] Embodiments described herein provide an apparatus for controlling an imaging sensor in a vehicle, the apparatus including an Ethernet transceiver, a sensor interface, and a local processor. The Ethernet transceiver is configured to communicate with a remote processor over an in-vehicle Ethernet network. The sensor interface is configured to communicate with the imaging sensor. The local processor, local to the apparatus and remotely located from the remote processor, is configured to receive image data and auxiliary data related to the image data from the imaging sensor via the sensor interface, generate control commands locally and independently of the remote processor based on the auxiliary data for controlling operational aspects of the imaging sensor by transmitting at least the image data to the remote processor via the Ethernet transceiver, and transmit the control commands to the imaging sensor via the sensor interface.

[0006] In some embodiments, the local processor is configured to locally generate at least some of the control commands on behalf of the remote processor, thereby reducing traffic load on the Ethernet network generated by the remote processor sending control commands over the Ethernet network to control the sensor. In other embodiments, the transceiver includes a transmitter and a receiver, and the local processor is configured to power down the receiver in response to detecting that no commands are received from the remote processor for a predetermined period of time. In yet other embodiments, the imaging sensor includes a camera configured to generate image data by capturing individual image frames and to provide auxiliary data for the individual image frames, and the local processor is configured to locally generate the control commands by processing the auxiliary data for the individual image frames.

[0007] In an embodiment, the individual image frames include a plurality of image pixels, and the auxiliary data includes a histogram of brightness levels associated with the image pixels of the given image frame, and the local processor is configured to locally generate the control instructions by processing the histogram of the given image frame. In another embodiment, the local processor is configured to generate the control instructions for controlling a quality level of the individual image frames by controlling one or more operational aspects of the camera selected from a list including exposure time, contrast, color balance, aperture, and sensor sensitivity. In yet another embodiment, the local processor is configured to locally provide the given control instructions to the sensor with a first latency that is less than a second latency incurred when providing the given control instructions remotely by the remote processor.

[0008] In some embodiments, the local processor is configured to selectively communicate control commands generated by the local processor to the remote processor. In other embodiments, the local processor is configured to receive from the remote processor a given control command generated by the remote processor based on one or both of the image data and the auxiliary data, and to transmit the received given control command to the imaging sensor. In yet other embodiments, the sensor includes a camera that generates the image data by capturing a plurality of image frames, and the given control command includes a focus command generated by the remote processor by analyzing two or more of the image frames for controlling a focus setting of the camera.

[0009] According to embodiments described herein, there is additionally provided a method for controlling an imaging sensor in a vehicle, the method comprising, in an apparatus communicating locally with the imaging sensor via a sensor interface and with a remote processor over an in-vehicle Ethernet network using an Ethernet transceiver, receiving image data and auxiliary data related to the image data from the imaging sensor via the sensor interface, and transmitting at least the image data over the Ethernet network to the remote processor using the Ethernet transceiver. Control commands are generated locally in the apparatus based on the auxiliary data and independent of the remote processor to control operational aspects of the imaging sensor. The control commands are transmitted to the imaging sensor via the sensor interface.

[0010] The present disclosure will be more fully understood from the following detailed description of the embodiments thereof in conjunction with the drawings, in which: [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram that illustrates generally an automotive Ethernet communication system with local sensor control, according to embodiments described herein.

[0012] [Diagram 2] FIG. 2 is a block diagram that generally illustrates a communication system, showing implementation details of a central processor and a sensor bridge, according to embodiments described herein.

[0013] [Diagram 3] FIG. 2 is a flow diagram that generally illustrates a method for local generation of commands for controlling a camera in a vehicle, according to embodiments described herein.

[0014] [Figure 4]1 is a flow diagram that generally illustrates a method for relaying commands generated by a remote controller for controlling a camera in a vehicle, according to embodiments described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] In a system involving communication with sensors over a network, it is sometimes necessary to control the sensors.

[0016] The embodiments described herein provide an improved system and method for controlling sensor operation by locally generating selected instructions for controlling various aspects of the sensor operation in a local processor separate from a remotely located central processor. The embodiments described herein relate primarily to automotive systems in which multiple sensors are connected to an Ethernet network. However, the disclosed techniques are suitable for use in any other suitable systems, for example in industrial control systems having various types of sensors. Non-limiting examples of sensors used in automotive systems include video cameras, speed sensors, accelerometers, audio sensors, infrared sensors, radar sensors, lidar sensors, ultrasonic sensors, distance measuring devices, or other proximity sensors, and the like.

[0017] In some embodiments, the system includes one or more sensors and a remote controller that receives data from the sensors and controls the sensors over an Ethernet network. The controller may include, for example, the system's central processor unit (CPU), a graphics processing unit (GPU), or any other suitable type of processor. In some embodiments, the controller may be integrated into a switch of the Ethernet network. A given sensor is typically connected to the network through a "sensor bridge." The sensor bridge communicates with the controller over the network using Ethernet and with the sensors using one or more local interfaces. The type of local interface may vary from one sensor type to another. As described below, in the disclosed embodiments, primary control of the sensors is offloaded from a remotely located controller to the sensor bridge, and primary control of the sensors is performed locally by the sensor bridge.

[0018] In an embodiment, the link between the sensor bridge and the Ethernet network is typically configured for asymmetric communication to reduce power consumption: the direction in which data originating from the sensor is sent to the controller to carry high speed data is referred to as the "downlink", and the direction in which commands originating from the controller are sent to the sensor to carry low speed commands is referred to as the "uplink".

[0019] In a typical conventional system, the sensor sends both image data and auxiliary data related to the image data to the controller, including any suitable data that may be useful for rapid generation of commands to control the sensor, such as statistical information about the sensor data, e.g., a histogram of image pixels.

[0020] In principle, the remotely located controller may generate control commands for controlling and configuring the sensors based on the auxiliary data, and send the control commands again to the sensors via the sensor bridge over the Ethernet network. However, this control scheme has several shortcomings as described herein. For example, the above control scheme undesirably interrupts the controller to analyze the auxiliary data and generate control commands. As another example, since both the auxiliary data in the downlink direction and the commands in the uplink direction are sent over the Ethernet network, the overall response time may be affected (i.e., lengthened) by the travel time for communication between the sensors and the remotely located controller. Also, the controller typically sends many commands to the sensors, which (i) may overload the network, causing congestion and further increasing latency, and (ii) increases power consumption in the bridge.

[0021] Consider an apparatus for controlling an imaging sensor in a vehicle, the apparatus including an Ethernet transceiver, a sensor interface, and a processor local to the apparatus. The Ethernet transceiver is configured to communicate with a remote processor, such as a central controller, over an in-vehicle Ethernet network. The sensor interface is configured to communicate with the imaging sensor. The local processor is configured to receive image data and auxiliary data related to the image data from the imaging sensor via the sensor interface, and to transmit at least the image data to the remote processor via the Ethernet transceiver. The local processor is further configured to locally generate control commands for controlling operational aspects of the imaging sensor based on the auxiliary data, independent of the remote processor, and to transmit the control commands to the imaging sensor via the sensor interface.

[0022] In some embodiments, the local processor is configured to generate at least some of the control commands locally on behalf of the remote processor, thereby reducing the traffic load on the Ethernet network generated by the remote processor sending control commands for controlling the sensors over the Ethernet network.

[0023] In some embodiments, the receiver of the Ethernet transceiver must be authorized for communication in the uplink direction and is preferably powered down when not needed. In such an embodiment, the local processor powers down the receiver in response to detecting that no command has been received from the remote processor for a predetermined period of time. In an embodiment, the local processor schedules the powering on and off of the receiver according to an Energy-Efficient Ethernet (EEE) protocol to reduce power consumption. For example, EEE is specified in IEEE Standard 802.3ch-2020, entitled "IEEE Standard for Ethernet - Amendment 8: Physical Layer Specifications and Management Parameters for 2.5 Gb / s, 5 Gb / s, and 10 Gb / s Automotive Electrical Ethernet," July 2020, which is incorporated herein by reference.

[0024] In some embodiments, the imaging sensor generates the image data by capturing individual image frames and includes a camera for providing auxiliary data for the individual image frames, and the local processor locally generates the control instructions by processing the auxiliary data for the individual image frames. For example, in embodiments, the individual image frames include a plurality of image pixels, the auxiliary data includes a histogram of brightness levels associated with the image pixels of a given image frame, and the local processor locally generates the control instructions by processing the histogram for the given image frame.

[0025] In an embodiment, the local processor generates control instructions to control the quality level of individual image frames by controlling one or more operational aspects of the camera selected from a list including exposure time, contrast, color balance, aperture and sensor sensitivity.

[0026] Providing commands to the sensors by a local processor instead of a remote processor significantly reduces response time, which is essential for safety in some use cases, such as for autonomous driving. In one embodiment, the local processor locally provides a given control command to the sensor with a first latency that is less than a second latency incurred when providing the given control command remotely by the remote processor.

[0027] The local processor may selectively notify the remote processor of control commands generated by the local processor, in an example embodiment, the local processor signals the remote processor to avoid sending commands already generated by the local processor.

[0028] In the uplink direction, the local processor forwards commands sent by the remote processor over the Ethernet network to the sensor. The local processor receives from the remote processor given control commands generated by the remote processor based on one or both of the image data and the auxiliary data, and transmits the received given control commands to the imaging sensor. For example, the sensor includes a camera that captures multiple image frames, and the given control commands include focus commands generated by the remote processor by analyzing two or more of the image frames to control a focus setting of the camera.

[0029] In the disclosed technique, the sensor bridge communicates locally with the sensors via a local link and remotely with a remote controller through an Ethernet network. The sensors send sensor data, such as image frames and ancillary data (e.g., statistical information about the image frames), to the bridge. The sensor bridge locally generates commands to control various operational aspects of the sensors. The controller may also generate commands to control the sensors, e.g., commands that the sensor bridge cannot generate due to insufficient computing power.

[0030] The control loop in which the sensor bridge provides commands to the sensors locally is much more efficient and has reduced latency compared to a conventional control loop involving an Ethernet network and a remote controller. Reducing latency is important for safety in automotive applications. Also, generating at least some commands locally reduces the number of commands that the remote controller needs to generate, thereby reducing the traffic load on the Ethernet network. Power consumption is also reduced, as the reduced rate of commands allows the Ethernet uplink to be down for longer periods of time.

[0031] 1 is a block diagram that generally illustrates an automotive Ethernet communication system 20 with local sensor control, according to embodiments described herein. In the embodiments described herein, the communication system 20 is installed in a vehicle 24 and includes a number of sensors 28 in communication with and controlled by a controller 32. In other embodiments (not shown), the system 20 may be installed in an industrial network, a monitoring network, or other suitable communication network.

[0032] In various embodiments, the sensors 28 may include any suitable type of sensor. Some non-limiting examples of sensors include video cameras, speed sensors, accelerometers, audio sensors, infrared sensors, radar sensors, lidar sensors, ultrasonic sensors, range finders, or other proximity sensors, and the like. The controller 32 may include any suitable type of processor, such as a CPU or GPU.

[0033] The sensors 28 and controller 32 communicate over an Ethernet network that includes multiple network links 40 and one or more Ethernet switches 44. The Ethernet links 40 may include, for example, twisted pair cables. The sensors 28 connect to the Ethernet network through one or more sensor bridges 36. A given sensor bridge may connect a single sensor or multiple sensors to the Ethernet network.

[0034] The sensor bridge 36, the switch 44, and the controller 32 may communicate over the network link 40 at any suitable bit rate. Example bit rates are 2.5 Gb / s, 5 Gb / s, or 10 Gb / s per the IEEE 802.3ch-2020 standard. In some embodiments, the sensor bridge 36 and the controller 32 communicate using EEE per the IEEE 802.3ch-2020 standard listed above.

[0035] The inset on the left hand side of Figure 1 shows an example implementation of the controller 32 and the sensor bridge 36 in an embodiment. In this example, the controller 32 and the sensor bridge 36 communicate over an in-vehicle Ethernet network 48 that supports EEE. Figure 1 provides a high level description of the sensor data flow and command flow in the system.

[0036] 1, controller 32 includes an Ethernet transceiver (TCVR) 52 for communication over network 48, and a processor 56 for performing various controller tasks. Processor 56 includes an analytical device 60 (also referred to as a "remote analytical device"), among other possible components. Sensor bridge 36 includes an Ethernet transceiver (TCVR) 72 for communication over network 48, a processor 76 for performing various sensor bridge tasks, and a sensor interface 80 for communicating with sensors 28 over a local link. Sensor interface 80 may be, for example, an Inter-Integrated Circuit (IIC) interface. 2 C) Bus, Improved Inter-Integrated Circuit (IIC) 3 The sensor interface 80 may exchange control data with the sensor 28 using a PCI Express (PCIe) bus, a General-Purpose Input-Output (GPIO) interface, a Peripheral Component Interconnect express (PCIe) bus, a Serial Peripheral Interface (SPI), or any other suitable local link. Additionally, the sensor interface 80 receives sensor data and ancillary data related to the sensor data from the sensor 28 using a suitable data interface. The processor 76 includes an analyzer 88 (also referred to herein as a “local analyzer”), among other possible components.

[0037] In a typical flow beginning with a sensor 28, the processor 76 receives sensor data and ancillary data related to the sensor data in the form of Ethernet packets from the sensor 28 via the sensor interface 80 using the Ethernet transceiver 72, and transmits the sensor data and the ancillary data to the Ethernet network 48. Based on analysis of the ancillary data by the local analysis device 88, the processor 76 generates control commands, also referred to herein as “local commands,” and transmits the local commands directly to the sensor 28 via the sensor interface 80.

[0038] The Ethernet transceiver 52 receives Ethernet packets carrying the sensor data and optionally the auxiliary data and forwards them to the processor 56. The remote analysis device 60 analyzes the sensor data and / or the auxiliary data and based on the analysis, the processor 56 generates control commands for controlling the sensors. The control commands generated by the processor 56 are also referred to herein as "remote commands" to distinguish them from "local commands" generated by the processor 76 of the sensor bridge.

[0039] The controller 60 transmits the remote commands to the network 48 in the form of Ethernet packets using the Ethernet transceiver 52. The Ethernet transceiver 72 receives the packets carrying the remote commands from the network, and the processor 76 transmits the remote commands to the sensor 28 using the sensor interface 80.

[0040] In an embodiment, the controller 32 may simultaneously perform the above processes for multiple Ethernet packets conveying control data to different sensors 28 via different sensor bridges 36. Similarly, the sensor bridge 36 may simultaneously perform the above processes for multiple Ethernet packets conveying control data to different sensors 28 connected thereto.

[0041] 2 is a block diagram that generally illustrates a communication system 100 including implementation details of a central processor 104 and a sensor bridge 108, according to an embodiment described herein. The system 100 may be used to implement the communication system 20 of FIG.

[0042] In the exemplary communication system 100 , the sensors 28 include digital cameras that communicate data and control information with a central processor 104 in the vehicle 24 over the Ethernet network 48 via a sensor bridge 108 .

[0043] In the example of FIG. 2, camera 28 transmits video data to the sensor bridge over a high speed link 106 and communicates control and management data with the sensor bridge over a low speed control link 110 .

[0044] The camera 28 generates the video data by capturing image frames at a required frame rate. Each image frame includes a number of image pixels arranged, for example, in a two-dimensional (2D) array. The image pixels carry intensity information in a suitable format, for example, gray intensity levels or color intensity levels according to a Red-Green-Blue (RGB) representation or a YUV model.

[0045] Camera 28 is controllable, for example, using control commands sent to the camera over control link 110. The commands control various operational aspects of the camera, such as, for example, frame rate, image resolution, field of view, zoom in and out, sensor sensitivity, aperture, exposure time, color balance (e.g., white balance), and focus. In response to receiving a command, the camera controls one or more of its operational aspects accordingly.

[0046] In some embodiments, in addition to video data, the camera 28 outputs auxiliary data related to the video data. The auxiliary data can be useful in generating control commands for the camera by the sensor bridge 36, the central processor 104, or both.

[0047] In some embodiments, the auxiliary data includes frame-by-frame statistics about the pixels in an image frame, such as a histogram of the brightness levels of the pixels in the image frame. Alternatively or additionally, the auxiliary data may include other frame-by-frame attributes, such as contrast levels, color shifts, saturation, and the like. In embodiments, some of the auxiliary data includes camera settings reported on demand by the camera, and acknowledgement notifications sent by the camera upon command.

[0048] In the sensor bridge 108, the data path includes a mapper / medium access control (MAC) module 112 and an Ethernet physical layer (PHY) transceiver 114, which includes a PHY transmitter 116 and a PHY receiver 118. In this example, the PHY transceiver 114 is coupled to the network 48 using a twisted pair link 166.

[0049] In the downlink direction, the mapper / MAC module 112 receives video data and auxiliary data from the camera 28 via the link 106 and applies mapping and MAC functions such as identifying video and control signals (in the headers in the video frames) and mapping them into a format that can be used by the MAC for encapsulation of the video data and auxiliary data through Ethernet packets. The mapper / MAC module transmits Ethernet packets carrying the video data and auxiliary data to the Ethernet network 48 using the PHY transmitter 116. In the uplink direction, the PHY receiver 118 receives Ethernet packets from the network 48 carrying commands that the remote controller generates to control the camera. The MAC module 112 depacketizes the received Ethernet packets to extract the commands and transmits the commands to the camera 28 via the control link 110.

[0050] The sensor bridge 108 further includes an identification device 130 and a local processor 134. The identification device 130 monitors the link 106 to identify auxiliary data and provides the identified auxiliary data to the local processor. The local processor processes the auxiliary data to locally generate commands for controlling the camera 28. The local processor transmits the locally generated commands directly to the camera via the control link 110.

[0051] In some embodiments, the auxiliary data includes a histogram (or multiple histograms, e.g., if YUV or RGB channels are provided separately) of an individual video frame. In such embodiments, the local processor may generate instructions for the sensor based on the histogram, e.g., by identifying populated histogram bins that may indicate a non-optimal sensor configuration. For example, in some embodiments, the local processor may generate instructions to adjust sensor exposure or white balance on the fly based on the histogram.

[0052] In an embodiment, the local processor generates the command based on the histogram using a lookup table (LUT). For example, the local processor sums the number of pixels in the histogram bins associated with overexposure (e.g., the histogram bins with the highest luminance levels) and accesses the LUT using the result of the summation that uses a normalized version of the result of the summation. In this example, the LUT entry stores the appropriate command to reduce the sensor exposure. In some embodiments, the local processor determines the final command based on the output from multiple LUTs, e.g., separate LUTs for the three RGB channels.

[0053] As another example, the sensor temperature may be embedded in the auxiliary data and, upon identification of a dangerously high temperature, the local processor may shut down or slow down the sensor (e.g., reduce frame rate and / or resolution) to reduce the temperature.

[0054] In some embodiments, the sensor bridge supports downloading and / or configuring programs that the local processor executes to generate instructions. This feature allows the user the flexibility to apply modifications to the functionality of the local processor as needed.

[0055] The central processor 104 comprises a PHY transceiver 150, which includes a PHY transmitter 156 and a PHY receiver 158, and a MAC module 162. The PHY transceiver 150 is coupled to the Ethernet network 48 using a twisted pair link 166. In the downlink direction, the MAC module 162 receives, via the PHY receiver 158, Ethernet packets carrying video data (e.g., a sequence of image frames) and ancillary data originating from the camera.

[0056] The MAC module 162 decapsulates the packets to extract the image frames and ancillary data, and transfers the image frames using the DMA module 170 for storage in memory 174 and the ancillary data to a controller 178. A data processor 182 reads the image frames from memory 174 and processes them, e.g., for display and for controlling the operation of the vehicle.

[0057] Memory 174 may include any suitable type of memory. For example, memory 174 may include volatile memory, such as random access memory (RAM), or non-volatile memory. Memory 174 may reside within the central processor, as shown, or external to the central processor.

[0058] The controller 178 processes the image frames, and optionally the auxiliary data, to generate control commands for controlling the camera. The controller 178 sends these commands to the MAC module 162 for packetization, and the PHY transmitter 156 transmits the packets to the network 48.

[0059] In some embodiments, the local processor 134 generates commands for the camera that would otherwise be generated by a remote controller, in this sense the local processor generates these commands on behalf of the remote controller.

[0060] In general, the local processor 134 controls the sensor so that the sensor output is valid and meaningful. For example, the local processor controls the sensor to output video frames with proper exposure, white balance, contrast, and the like. The local processor may also handle safety-related issues such as high temperatures. In contrast, the remote central processor (104) processes the actual sensor data, such as video frames or LiDAR data. The remote central processor may analyze the video frames, for example, to identify specific objects in the scene.

[0061] In some embodiments, controller 178 generates instructions for controlling camera 28 based on the image frames, the auxiliary data, or both. Controller 178 may generate instructions that are different from those generated locally by local processor 134, such as instructions that override instructions generated by the local processor, and / or instructions that the local processor cannot generate due to limited computational resources.

[0062] In an embodiment, the local processor 134 may inform the controller 178 of at least some of the locally generated instructions, e.g., to avoid sending redundant instructions by the controller 178. The local processor 134 may selectively inform the controller 178 of at least some of the locally generated instructions, e.g., instructions that the local processor generates frequently.

[0063] In some embodiments, the controller 178 has higher computing power than the local processor 134 of the sensor bridge. As a result, the local processor 134 may generate commands based only on auxiliary data and not on image frames. In contrast, the controller 178 has sufficient resources to generate commands based on complete image frame information (and possibly on auxiliary data). For example, the controller 178 may detect, based on video data (image frames), that the number of saturated pixels in one or more video frames exceeds a specified threshold number and generate one or more commands for the camera to reduce exposure time, aperture, or both. Commands that may be generated by the controller 178 rather than the local processor 134 may include commands to modify one or more camera settings, such as frame rate, image resolution, field of view angle, zoom in and out, and focus.

[0064] Some types of cameras 28 are affected by temperature and may even fail at high temperatures. In an embodiment, to prevent such camera failure, local processor 134 monitors the temperature, and in response to detecting that the temperature exceeds a specified temperature limit, local processor 134 may act quickly to reduce the temperature, for example, by shutting down the camera.

[0065] In some embodiments, the sensor bridge and the camera are physically close to each other, in which case the camera may absorb heat generated by the sensor bridge. By reducing the power dissipation in the sensor bridge due to the reduced rate of commands sent by the remote controller (as described above), the camera temperature is also reduced.

[0066] FIG. 3 is a flow diagram that generally illustrates a method for local generation of commands for controlling a camera in a vehicle, according to an embodiment described herein.

[0067] We now describe a method performed by the elements of the sensor bridge 108 of Figure 2 above. Assume that the sensor bridge is coupled to a sensor that includes a video camera.

[0068] The method begins with the sensor bridge 108 receiving video frames captured by the camera and ancillary data related to the video frames from the sensor 28 in a receiving operation 200. In an identifying operation 204, the identifying device 130 monitors the data received in operation 200 to identify the ancillary data and transmits the identified ancillary data to the local processor 134.

[0069] In a forwarding operation 208, the sensor bridge transmits the video frames (and optionally the auxiliary data) received in operation 200 over the network 48 to the controller 178. In a local processing operation 212, the local processor 134 processes the auxiliary data to locally generate one or more commands for controlling the camera. In a local control operation 216, the sensor bridge transmits the locally generated commands directly to the camera. In a notify operation 220, the sensor bridge optionally notifies the controller 178 of the locally generated commands, e.g., to avoid generating the same commands by both the local processor 134 and the controller 178.

[0070] Following operation 220, the method loops back to operation 200 to receive subsequent video frames and ancillary data from the camera.

[0071] FIG. 4 is a flow diagram that generally illustrates a method for relaying commands generated by a remote controller for controlling a camera in a vehicle, according to embodiments described herein.

[0072] We now describe a method performed by the elements of the sensor bridge 108 of Figure 2 above. Assume that the sensor bridge is coupled to a sensor that includes a video camera.

[0073] The method begins with a receive command operation 250, where the sensor bridge 108 receives a command over the network 48 to control a camera, where the command was generated by a remote controller 178 (and not generated locally by the sensor bridge). In a relay command operation 254, the sensor bridge transmits the remotely generated command to the camera.

[0074] Following operation 254, the method loops back to operation 250 where a subsequent command is received from the remote controller.

[0075] In an embodiment, the methods of Figures 3 and 4 above may be performed in parallel with each other.

[0076] The above described embodiments are given as examples, and other suitable embodiments may be used. For example, the above described embodiments employ the EEE protocol, but other suitable protocols for connecting between sensors and an Ethernet network may be used. Examples of such protocols include the Mobile Industry Processor Interface (MIPI) (A-PHY) protocol and the Automotive SerDes Alliance (ASA) protocol.

[0077] The configurations of the various systems, controllers, and sensor bridges shown in Figures 1 and 2 are merely example configurations shown for purposes of clarity. In alternative embodiments, any other suitable configurations may be used. Different elements of the various systems, controllers, and sensor bridges described herein may be implemented using dedicated hardware or firmware, such as using hardwired or programmable logic in an Application-Specific Integrated Circuit (ASIC) or Field-Programmable Gate Array (FPGA). Additionally or alternatively, some functions of the disclosed systems, controllers, and sensor bridges may be implemented in software and / or using a combination of hardware and software elements. Elements that are not essential to an understanding of the disclosed techniques are omitted from the figures for purposes of clarity.

[0078] In some embodiments, some functions of the disclosed system, controller, and sensor bridge, e.g., functions of processors 56, 76, 134 and / or 182, identification device 130 and / or controller 178, may be implemented in one or more programmable processors, e.g., one or more central processing units (CPUs), microcontrollers, and / or digital signal processors (DSPs), that are programmed in software to perform the functions described herein. For example, the software may be in electronic form and downloaded over a network to any of the processors, or alternatively or additionally, the software may be provided and / or stored on a non-transitory tangible medium, such as a magnetic, optical or electronic memory.

[0079] It should be noted that the above-described embodiments are cited as examples, and the present invention is not limited to what has been specifically shown and described above. Rather, the scope of the present invention includes both combinations and subcombinations of the various features described above, as well as variations and modifications thereof not disclosed in the prior art that would occur to a person skilled in the art upon reading the foregoing description. Documents incorporated by reference into this patent application shall be considered as an integral part of this application. However, to the extent that any term is defined in these incorporated documents in a manner that contradicts a definition expressly or impliedly made herein, only the definition in this specification shall be considered.

Claims

1. 1. An apparatus for controlling an imaging sensor in a vehicle, comprising: an Ethernet transceiver configured to communicate with a remote processor over an in-vehicle Ethernet network; a sensor interface configured to communicate with the imaging sensor; and a local processor local to the device and located remotely from the remote processor; the local processor comprising: receiving image data and ancillary data related to the image data from the imaging sensor via the sensor interface and transmitting at least the image data to the remote processor via the Ethernet transceiver; generating control instructions for controlling operational aspects of the imaging sensor locally and independently of the remote processor based on the auxiliary data; and Transmitting the control command to the imaging sensor via the sensor interface. It is configured as follows: Device.

2. 2. The apparatus of claim 1, wherein the local processor is configured to locally generate at least some of the control instructions on behalf of the remote processor, thereby reducing traffic load on the in-vehicle Ethernet network generated by the remote processor sending control instructions for controlling the imaging sensor over the in-vehicle Ethernet network.

3. 3. The apparatus of claim 1, wherein the Ethernet transceiver includes a transmitter and a receiver, and the local processor is configured to power down the receiver in response to detecting that no commands are received from the remote processor for a predetermined period of time.

4. 3. The apparatus of claim 1 or 2, wherein the imaging sensor includes a camera configured to generate the image data by capturing individual image frames and to provide the auxiliary data for the individual image frames, and the local processor is configured to locally generate the control instructions by processing the auxiliary data for the individual image frames.

5. 5. The apparatus of claim 4, wherein the individual image frames include a plurality of image pixels, the auxiliary data includes a histogram of brightness levels associated with the plurality of image pixels of a given image frame, and the local processor is configured to locally generate the control instructions by processing the histogram of the given image frame.

6. 5. The apparatus of claim 4, wherein the local processor is configured to generate the control instructions for controlling a quality level of the individual image frames by controlling one or more operational aspects of the camera selected from a list including exposure time, contrast, color balance, aperture and sensor sensitivity.

7. 3. The apparatus of claim 1 or 2, wherein the local processor is configured to locally provide the given control command to the imaging sensor with a first latency that is shorter than a second latency that occurs when the given control command is provided remotely by the remote processor.

8. 3. The apparatus of claim 1 or 2, wherein the local processor is configured to selectively communicate the control commands generated by the local processor to the remote processor.

9. 3. The apparatus of claim 1 or 2, wherein the local processor is configured to receive from the remote processor a given control command generated by the remote processor based on one or both of the image data and the auxiliary data, and to transmit the received given control command to the imaging sensor.

10. 10. The apparatus of claim 9, wherein the imaging sensor includes a camera that generates the image data by capturing a plurality of image frames, and the given control command includes a focus command generated by the remote processor by analyzing two or more of the plurality of image frames for controlling a focus setting of the camera.

11. 1. A method for controlling an imaging sensor in a vehicle, comprising:

1. An apparatus for communicating with an imaging sensor locally via a sensor interface and with a remote processor over an in-vehicle Ethernet network using an Ethernet transceiver, comprising: receiving image data and ancillary data related to the image data from the imaging sensor via the sensor interface and transmitting at least the image data over the vehicle Ethernet network using the Ethernet transceiver to the remote processor; generating control commands locally at the device based on the auxiliary data and independent of the remote processor for controlling operational aspects of the imaging sensor; and transmitting the control command to the imaging sensor via the sensor interface; A method for providing the above.

12. 12. The method of claim 11, wherein generating the control instructions includes locally generating at least some of the control instructions on behalf of the remote processor, thereby reducing traffic load on the in-vehicle Ethernet network generated by the remote processor transmitting control instructions for controlling the imaging sensor over the in-vehicle Ethernet network.

13. 13. The method of claim 11 or 12, wherein the Ethernet transceiver includes a transmitter and a receiver, and further comprising powering off the receiver in response to detecting that no commands are received from the remote processor for a predetermined period of time.

14. 13. The method of claim 11 or 12, wherein the imaging sensor includes a camera that generates the image data by capturing individual image frames and that provides the auxiliary data for the individual image frames, and wherein locally generating the control instructions includes processing the auxiliary data for the individual image frames.

15. 15. The method of claim 14, wherein the individual image frames include a plurality of image pixels, the auxiliary data includes a histogram of brightness levels associated with the plurality of image pixels of a given image frame, and generating the control instructions includes processing the histogram of the given image frame.

16. 15. The method of claim 14, wherein generating the control instructions for controlling a quality level of the individual image frames comprises controlling one or more operational aspects of the camera selected from the list including exposure time, contrast, color balance, aperture, and sensor sensitivity.

17. 13. The method of claim 11 or 12, comprising providing the given control command locally to the imaging sensor with a first latency that is less than a second latency incurred when providing the given control command remotely by the remote processor.

18. 13. A method according to claim 11 or 12, comprising the step of communicating the control commands generated by a local processor to the remote processor.

19. 13. A method according to claim 11 or 12, comprising receiving from the remote processor a given control command generated by the remote processor based on one or both of the image data and the auxiliary data, and transmitting the received given control command to the imaging sensor.

20. 20. The method of claim 19, wherein the imaging sensor includes a camera that generates the image data by capturing a plurality of image frames, and the given control command includes a focus command generated by the remote processor by analyzing two or more of the plurality of image frames for controlling a focus setting of the camera.

Images