Mode switching for partial integrity security operation

Abstract

The processing circuitry is coupled to the imaging device through a plurality of data communication links. Each data communication link couples at least one imaging device with the processing circuitry. The processing circuitry is configured to: determine when a change is required in a first data protection configuration that protects first image data transmitted over a first data communication link; and determining a second data protection configuration to be used when second image data is transmitted from the first imaging device to the image processing circuitry over the first data communication link. The first data protection configuration or the second data protection configuration provides a partial integrity mode of data protection for image data frames sent over the first data communication link. The partial integrity mode uses a message authentication code to protect some but not all of the image data frames sent over the first data communication link.

Description

[0001] Cross-reference to related applications

[0002] This patent application claims priority to pending U.S. nonprovisional application No. 18 / 188,652, filed March 23, 2023, which has been assigned to the assignee of this application and is expressly incorporated herein by reference, as fully set forth below and for all applicable purposes. Technical Field

[0003] This disclosure relates in general to data protection of image data via wired or wireless connections, and more specifically to the protection of data transmitted between sensors and controllers in a vehicle.

[0004] background

[0005] Vehicles, including cars, trucks, airplanes, and ships, are increasingly equipped with complex electronic systems for system management, control, and safety. For example, vehicles commonly include navigation systems, which in many instances support the automated operation of the vehicle. In this latter example, the control system may communicate not only with various sensors providing feedback for controlling the vehicle's operation, but also with the navigation system, steering system, and powertrain or other propulsion systems. The various systems within a vehicle can be implemented using a variety of components such as circuit boards, integrated circuit (IC) devices, application-specific integrated circuit (ASIC) devices, and / or system-on-a-chip (SoC) devices. Component types may include processing circuitry, user interface components, storage devices, and other peripheral components. Communication infrastructure may be provided to support data communication within and between various systems or subsystems. This communication infrastructure may include some combination of wireless and wired communication links.

[0006] Examples of standards that define or control wireless communications that can be deployed or supported within vehicles include certain standards defined by the Institute of Electrical and Electronics Engineers (IEEE) 802.11 Working Group and those defined by the Bluetooth Special Interest Group (SIG). Standards and radio access standards defined by the 3rd Generation Partnership Project (3GPP).

[0007] Examples of standards that define or control wired communication include those for inter-integrated circuits (I2C or I2C). 2 C) Related standards, standards defined by the Mobile Industry Processor Interface (MIPI) Alliance, and the CAN bus standard published by the International Organization for Standardization (ISO).

[0008] As device technology improves, increased functionality can be serviced using data communication links configured for point-to-point or multi-point operation. Mission critical functionality often requires secure and reliable data communication links. Accordingly, there is a continuing need to improve the resiliency and reliability of data communication links. SUMMARY

[0009] Certain aspects of the present disclosure relate to systems, apparatus, methods, and techniques that enable mobile communication devices and other portable devices to discard a number of data packets after an imaging device has been turned on. The number of data packets discarded can be calculated or configured to ensure that corrupted data packets are discarded before processing of data packets begins.

[0010] In various aspects of the present disclosure, a sensor management system includes a plurality of imaging devices deployed within a vehicle, processing circuitry including at least one controller, and a plurality of data communication links. Each data communication link couples at least one imaging device of the plurality of imaging devices with the processing circuitry. The processing circuitry is configured to determine a change is needed in a first data protection configuration that protects first image data sent over a first data communication link and determine a second data protection configuration to use when second image data is sent from a first imaging device of the plurality of imaging devices to the image processing circuitry over the first data communication link. The first data protection configuration or the second data protection configuration provides a partial integrity mode of data protection for frames of image data sent over the first data communication link.

[0011] In various aspects of the present disclosure, a method for managing a plurality of imaging devices deployed within a vehicle includes determining a change is needed in a first data protection configuration that protects first image data sent over a first data communication link and determining a second data protection configuration to use when second image data is sent from a first imaging device of the plurality of imaging devices to the image processing circuitry over the first data communication link. The first data protection configuration or the second data protection configuration provides a partial integrity mode of data protection for frames of image data sent over the first data communication link.

[0012] In various aspects of the present disclosure, an apparatus includes means for determining a change is needed in a first data protection configuration that protects first image data sent over a first data communication link and means for determining a second data protection configuration to use when second image data is sent from a first imaging device to the image processing circuitry over the first data communication link. The first data protection configuration or the second data protection configuration provides a partial integrity mode of data protection for frames of image data sent over the first data communication link.

[0013] In various aspects of the disclosure, a processor-readable storage medium includes code that, when executed by processing circuitry, causes the processing circuitry to determine that a change is needed in a first data protection configuration that protects first image data sent over a first data communication link and determine a second data protection configuration to use when second image data is sent from a first imaging device deployed within a vehicle to image processing circuitry over the first data communication link. The first data protection configuration or the second data protection configuration provides a partial integrity mode of data protection for image data frames sent over the first data communication link.

[0014] In certain aspects, the second data protection configuration provides a full integrity mode of data protection for the image data frames sent over the first data communication link. The second data protection configuration can cause the image data frames to be encrypted before being sent over the first data communication link.

[0015] In certain aspects, the partial integrity mode of data protection protects a first portion of the image data frames sent over the first data communication link using a message authentication code (MAC). A second portion of the image data frames sent over the first data communication link is not protected by the MAC used to protect the first portion of the image data frames or is not protected by a different MAC. The partial integrity mode can be configured with a skip pattern that specifies image data rows in the first portion of the image data frames to be protected by the MAC. The partial integrity mode can be configured with a skip pattern that specifies blocks of pixels in the first portion of the image data frames to be protected by the MAC.

[0016] In one example, the second data protection configuration can be determined based on a computational capacity of the sensor management system, a power consumption budget or a thermal budget defined for the sensor management system. In another example, the second data protection configuration can be determined or provided to accommodate a change in an operating mode of the vehicle. In one example, the change is determined to be needed in the first data protection configuration after detecting an increase in temperature or power consumption caused by the sensor management system. In another example, the change is determined to be needed in the first data protection configuration after detecting a high value scene or scene segment in the first image data.

[0017] In certain examples, an artificial intelligence (AI) agent is used to determine the change needed in the first data protection configuration. Initially, the AI agent can be trained using measurements of power or heat obtained for a plurality of operating conditions. The AI agent can be retrained using real-time system feedback. The change needed in the first data protection configuration can be determined based on inferences or predictions received from the AI agent. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 Examples of systems in a vehicle that can be adapted, configured, or operated in accordance with certain aspects of the present disclosure are illustrated.

[0019] Figure 2 Examples of devices in a vehicle subsystem that can be adapted in accordance with certain aspects of the present disclosure are illustrated.

[0020] Figure 3 A system including an imaging device in accordance with certain aspects of the present disclosure is illustrated.

[0021] Figure 4 Systems that can be deployed in certain automotive applications are illustrated.

[0022] Figure 5 Examples of data security paradigms that can be adapted in accordance with certain aspects of the present disclosure are illustrated.

[0023] Figure 6 is a graphical representation of a series of data security modes provided by a scalable data security scheme implemented in accordance with certain aspects of the present disclosure.

[0024] Figure 7 Certain examples of step-cross patterns that can be configured in accordance with certain aspects of the present disclosure are illustrated.

[0025] Figure 8 Examples of frame-related transactions that are partitioned in accordance with certain aspects of the present disclosure are illustrated.

[0026] Figure 9 One example of a scalable data security scheme applied to frame partitioning in accordance with certain aspects of the present disclosure is illustrated.

[0027] Figure 10 is a flowchart illustrating secure variant mode switching in accordance with certain aspects of the present disclosure.

[0028] Figure 11 One example of a device employing processing circuitry that can be adapted in accordance with certain aspects disclosed herein is illustrated.

[0029] Figure 12 is a flowchart illustrating a method for operating bus interface circuitry in accordance with certain aspects disclosed herein.

[0030] Figure 13 A first example of a hardware implementation of a bus interface device adapted in accordance with certain aspects disclosed herein is illustrated. DETAILED DESCRIPTION

[0031] The detailed description set forth below, in connection with the appended drawings, is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts described herein can be practiced. The detailed description includes specific details for the purpose of providing a thorough understanding of various concepts. However, it will be apparent to those skilled in the art that these concepts can be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form in order to avoid obscuring such concepts.

[0032] Several aspects of the application will now be presented with reference to various apparatus and methods. These apparatus and methods will be described in the following detailed description and illustrated in the accompanying drawings by various blocks, modules, components, circuits, steps, processes, algorithms, etc. (collectively referred to as “elements”). These elements can be implemented using electronic hardware, computer software, or any combination thereof. Whether such elements are implemented as hardware or software depends on the particular application and design constraints imposed on the overall system.

[0033] Certain examples described herein can be implemented using wired communication links to interconnect different parts of a vehicle, including a multi-point serial bus that operates according to a standardized or proprietary protocol. In one example, the serial bus can operate according to an Inter-Integrated Circuit (I2C or I 2 C) communication protocol. The I2C bus is configured as a multi-point bus and was developed for connecting low-speed peripheral devices to a processor. Two wires of the I2C bus include a serial data line (SDA) that carries a data signal and a serial clock line (SCL) that carries a clock signal. The serial bus can operate according to a multi-master protocol such that one or more devices can be designated as a bus master or host device for the serial bus. A device can function as a bus master or host in some transmissions and as a slave or slave device in other transmissions.

[0034] In some examples, a high-speed serial bus operates according to standards defined by the Mobile Industry Processor Interface (MIPI) Alliance, such as the Improved Inter-Integrated Circuit (I3C) standard, the Radio Frequency Front-End (RFFE) standard, the System Power Management Interface (SPMI) standard, the Camera Serial Interface (CSI) standard, and the Display Serial Interface (DSI) standard. In some examples, a Controller Area Network (CAN) vehicle bus standard can be used in vehicles to provide a message-based protocol that supports prioritized multi-point operation in which bus contention is resolved by allowing the highest priority contending device to transmit while devices with lower priority avoid transmitting according to a backoff procedure. Certain implementations of the CAN bus are described in the CAN standards published by the International Organization for Standardization (ISO).

[0035] Some or all of the examples described can be found in accordance with the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard, the IEEE 802.15 standard, and as defined by the Bluetooth Special Interest Group (SIG). The described implementation can be implemented in any device, system, or network that transmits and receives radio frequency (RF) signals using one or more of the following standards or those published by the 3rd Generation Partnership Project (3GPP): Long Term Evolution (LTE), 3G, 4G, or 5G (New Radio (NR)). The specific implementation described can be implemented in any device, system, or network capable of transmitting and receiving RF signals according to one or more of the following technologies or techniques: Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal FDMA (OFDMA), Single Carrier FDMA (SC-FDMA), Single User (SU) Multiple Input Multiple Output (MIMO), and Multi User (MU) MIMO. The specific implementation described can also be implemented using other wireless communication protocols or RF signals suitable for use in one or more of the following: Wireless Personal Area Network (WPAN), Wireless Local Area Network (WLAN), Wireless Wide Area Network (WWAN), or Internet of Things (IoT) networks.

[0036] Wired data communication links are used to facilitate the description of certain concepts herein. However, in many specific implementations, these concepts can also be applied to wireless communication links. For example, a physical communication interface may be implemented using wired and wireless transceivers or some combination of wired and wireless transceivers. Physical connections between devices may include wired or wireless propagation paths.

[0037] The data integrity architecture provided by certain aspects of this disclosure offers configurable integrity modes for data communication links that adapt to power budgets while ensuring data security for mission-critical functionalities within a vehicle. Data security typically relies on the availability of data communication links that can reliably transmit data within and between discrete systems deployed within a vehicle. For some systems or applications, data security may involve data encryption and data integrity services to prevent unauthorized access to information by malicious actors and / or the introduction of malicious data, commands, or other information into the control system from unauthorized sources.

[0038] Certain systems and / or processing circuitry can be used to automate features and functionality of a vehicle. For example, camera-based systems, proximity indicators, speed and velocity detectors, and other systems can provide images and other information that can be used to assist an operator of an automobile. In one example, the images and other information enable identification of potential hazards, proximate objects, presence of pedestrians, and other traffic and / or road conditions. In some instances, a combination of systems and / or processing circuitry can enable automated operation of an automobile, and can provide feedback and other information to safety and vehicle access management systems, as well as to driving assistance systems, including forward collision warning systems, lane departure warning systems, rear cross-traffic warning systems, blind spot warning systems, vehicle back-up imaging systems, parking assistance, and other driving assistance systems. These and other systems can use some combination of cameras, sensors, and positioning systems, including radio detection and ranging ("radar") based systems, light detection and ranging ("lidar") based systems, and / or ultrasonic detection systems.

[0039] The examples of sensor management systems provided throughout this disclosure use cameras as a primary example of a type of sensor. However, the concepts, architectures, systems, apparatuses, techniques, and methods disclosed in these examples are equally applicable to other types of sensors and positioning systems, including radar, lidar, and / or ultrasonic transducer or sensor based systems.

[0040] Certain systems and / or processing circuitry can support one or more displays provided within a vehicle. For example, display subsystems can be used as instrument clusters, entertainment or media centers, one or more view panels accessible to passengers, heads-up displays that control information and instruments, and so on. Certain systems and / or processing circuitry can manage, configure, and control environmental conditions of a vehicle's cabin, including heating, cooling, lighting, and seating configurations.

[0041] Figure 1An example of an automobile 100 including a system that can be adapted, configured, or operated in accordance with certain aspects of the present disclosure is illustrated. The automobile 100 can be equipped with a number of imaging or sensing devices, including the illustrated cameras or sensors 102, 104, 106, 108, 112, 114. The automobile 100 can include sensors such as tire pressure or brake sensors 116, 118. The automobile 100 can also include one or more antennas 110 for radio frequency reception, wireless communication, and / or radio navigation using the global positioning system (GPS). A central controller 120 can be coupled to each of the cameras 102, 104, 106, 108, 112, 114, sensors 116, 118, and antennas 110. The central controller 120 can configure and manage automated systems and / or drive assist systems. In some implementations, the central controller 120 can be configured to operate as an engine control unit that manages the operation and performance of an engine, one or more motors, or other power systems in the automobile 100. In some examples, the central controller 120 can be embodied in a SoC.

[0042] Robust data communication links are needed to support the large number of cameras deployed within the automobile 100. In some examples, 20 to 30 cameras can be deployed to support automated and drive assist systems. Each camera can be capable of generating data at a rate of between 1 gigabit per second to 10 gigabits per second (Gbps), resulting in an aggregate data rate of up to 300 Gbps. Communication of this amount of data can be expected to result in a high level of power consumption and the generation of associated heat from the interface and data protection and processing circuitry. In traditional systems, the data rate can be reduced to control power consumption and heat generation, resulting in a loss of image quality.

[0043] Figure 2 An example of an apparatus 200 that can be incorporated in a vehicle subsystem including sensors and at least one controller that can be adapted in accordance with certain aspects of the present disclosure is illustrated. In the illustrated example, the apparatus 200 includes a number of devices 202 and 2220-222 N . The devices 202 and 2220-222 N may be implemented in one or more semiconductor IC devices, such as an application processor, SoC, or ASIC. In various implementations, certain of the devices 202 and 2220-222 N may include, support, or operate as modems, signal processing devices, display drivers, cameras, user interfaces, sensors, sensor controllers, media players, transceivers, and / or other such components or devices. In some examples, one or more of the devices 2220-222 Nmay be used to control, manage, or monitor sensor devices. Communication between the device 202 and 2220-222 N

[0044] In one example, the host device 202 can include an interface controller 204 that can manage access to the serial bus, configure dynamic addresses for slave devices, and / or generate a clock signal 228 to be sent on the clock line 218 of the serial bus 220. The host device 202 can include configuration registers 206 or other storage 224 and other control logic 212 configured to handle protocols and / or higher level functions. The control logic 212 can include processing circuitry such as a state machine, sequencer, signal processor, or general purpose processor. The host device 202 includes a transceiver 210 and line drivers / receivers 214a and 214b. The transceiver 210 can include a receiver, a transmitter, and common circuitry that can include timing, logic, and storage circuitry and / or devices. In one example, the transmitter encodes and transmits data based on timing in the clock signal 228 provided by the clock generation circuit 208. Other timing clocks 226 can be used by the control logic 212 and other functions, circuitry, or modules.

[0045] At least one device 2220-222 N may be configured to operate as a slave device on the serial bus 220 and can include circuitry and modules to support a display, image sensor, and / or control and communicate with one or more sensors that measure environmental conditions. In one example, a device 2220 configured to operate as a slave device can provide control functions, physical layer circuitry 232 including circuitry and modules to support a display, image sensor, and / or control and communicate with one or more sensors that measure environmental conditions. In this example, the device 2220 can include configuration registers 234 or other storage 236, control logic 242, a transceiver 240, and line drivers / receivers 244a and 244b. The control logic 242 can include processing circuitry such as a state machine, sequencer, signal processor, or general purpose processor. The transceiver 240 can include a receiver, a transmitter, and common circuitry that can include timing, logic, and storage circuitry and / or devices. In one example, the transmitter encodes and transmits data based on timing in the clock signal 248 provided by the clock generation and / or recovery circuit 246. In some instances, the clock signal 248 can be derived from the signal received from the clock line 218. Other timing clocks 238 can be used by the control logic 242 and other functions, circuitry, or modules.​

[0046] Serial bus 220 can operate according to CAN, Ethernet, RFFE, I2C, I3C, SPMI, or other suitable protocol. In some instances, two or more devices 202, 2220 through 222 N may be configured to operate as a host device on serial bus 220. In some instances, apparatus 200 includes multiple serial buses 220, 252a and / or 252b that couple two or more of devices 202, 2220 through 222 N may be configured to operate as a host device on serial bus 220. In some instances, apparatus 200 includes multiple serial buses 220, 252a and / or 252b that couple two or more of devices 202, 2220 through 222 N one of devices 202, 2220 through 222

[0047] Figure 3 A system 300 including imaging devices is illustrated in accordance with certain aspects of the present disclosure. The illustrated system 300 includes a SoC 302 and a camera subsystem 304 communicatively coupled using a high-speed serial bus 306 and a low-speed serial bus 308. In the illustrated example, the SoC 302 can include multiple processors including a central processing unit or image processor 312, one or more microcontrollers, digital signal processors, finite state machines, or other sequential logic components. The SoC 302 also includes high-speed physical interface circuitry 314 and low-speed physical interface circuitry 316. Various types of communication protocols can be used to control or manage the high-speed serial data link and the low-speed serial data link. One or more serial data links can be aggregated if needed to meet performance requirements.

[0048] The camera subsystem 304 includes a sensor controller 322 that can be configured to control and configure the operation of an image sensor 330. The sensor controller 322 can include one or more processing circuits including, for example, one or more microcontrollers, digital signal processors, finite state machines, or other sequential logic components. The sensor controller 322 can include a high-speed physical interface 324 coupled to the high-speed serial bus 306 and a low-speed physical interface 326 coupled to the low-speed serial bus 308.

[0049] Figure 4 A system 400 that can be deployed in automotive applications is illustrated. In the illustrated example, a multi-drop data link is used to couple multiple cameras 4021 through 402 Mcoupled to multiple application processors or SoCs 4041-404 N In the illustrated example, each camera 4021-402 M streams data over a corresponding multipoint data link 4061-406 M transmit image data. In some instances, one of the cameras 4021-402 M may transmit a data stream to one of the SoCs 4041-404 N hosting an application configured to provide assistance to a driver when parking a vehicle, and that camera can be one of the SoCs 4041-404 N used by a guidance system during automated operation of the vehicle. In some instances, various cameras 4021-402 M may transmit data streams to one or more of the SoCs 4041-404 N used to manage vehicle security, including for access control and / or to support an anti-theft system. In some instances, images generated by one or more of the cameras 4021-402 M may be transmitted concurrently to multiple SoCs 4041-404 N or application processors that cooperate to provide redundant autonomous driving capabilities, where each of the SoCs 4041-404 N may reliably operate when one or more of the other SoCs 8041-404 N fail, produce errors or invalid results, or enter an idle state. The multipoint data links 4061-406 M may be used to support these and other implementations in which a single camera 4021-402 M may be monitored by multiple SOC 4041-404 N processors. An application processor, SoC, or other video processor that receives a data stream from one or more can be referred to herein as a video processing circuit, whether or not the application processor, SoC, or other video processor uses, modifies, or operates on the data stream or merely relays the data stream to another device.

[0050] Certain aspects of the present disclosure relate to cryptographic data protection of image data transmitted over wired or wireless connections. Certain aspects of the present disclosure provide systems, apparatuses, and processes that can ensure the integrity of transmitted data within constraints defined or specified for an application. In one example, the transmitted data can include image data generated by a camera-based system and / or a radar or lidar system, which can be transmitted to a controller in an automobile. Constraints associated with data communication systems deployed within an automobile include power consumption and / or thermal management limitations.

[0051] For purposes of the present disclosure, data protection and / or data security involves authentication, data integrity, and encryption. Users, applications, systems, and devices involved in a data communication transaction can be authenticated to establish trust between a sender and a recipient of data. In one example, a sender can authenticate a recipient prior to sending data. In another example, mutual authentication of a sender and a recipient is desirable.

[0052] For purposes of the present disclosure, data integrity can also be referred to as data or message authentication. Data integrity is implemented to ensure that data exchanges between the central controller 120 and other systems are unaltered during transmission. Data integrity can be accomplished by providing a message authentication code (MAC) to each data packet or transaction. Encryption services are used to protect information including images and sensor data from unauthorized access. In certain examples, messages sent over a data communication link can be encrypted and encapsulated within datagrams, packets, or other units of transmission.

[0053] Certain aspects of the present disclosure relate to systems that can utilize, adapt, or extend data security standards. In one example, the Institute of Electrical and Electronics Engineers (IEEE) 802.1AE MACsec standard defines a data security standard based on a link-based paradigm. In another example, the Camera Service Extension (CSE) standard published by the MIPI Alliance defines a data security standard based on an application-based paradigm.

[0054] Figure 5 Examples of a link-based data security paradigm and an application-based data security paradigm are illustrated. In a first illustrated example 500, data security is provided according to a link-based paradigm (link-based security). When link-based security is implemented, a data communication link 530 is protected at the lowest protocol layer. Typically, data protection is provided by link layer circuits and modules 522, 524, 526, 528, which can also configure and operate connections between components 502, 504, 506, 508. Data protection processes are typically performed prior to physical transmission of data 518. In the illustrated example 500, data 518 is generated by a sensor control module (data source 514) in a sensor device 504. All data transactions between the SoC 502 and the sensor device 504 are sent over a data communication link 530 that is protected using link-based security. In a conventional system, all transactions are protected, with no exceptions, and with no selectivity as to the source or nature of data traffic. In the illustrated example 500, link layer circuits and modules 522 are responsible for protecting data sent or received by the SoC 502, and link layer circuits and modules 524 are responsible for protecting data sent or received by the sensor device 504.

[0055] Link-based security can reduce communication overhead at the SoC 502 and the sensor device 504. For example, an application that receives data from the sensor device 504 or configures and manages the operation of the sensor device 504 need not generate a MAC or encrypt data packets. In some systems, accordingly, the link layer circuit and module 524 encrypts data sent by the sensor device 504, and the link layer circuit and module 522 decrypts data received by the SoC 502. An application that serves as a data sink 512 in the SoC need not know that data 516 received from the sensor device 504 has been encrypted during transmission, or even that a data communication link 530 has been used to receive the data 516 from.

[0056] A virtual data link 510 can be established between the SoC 502 and the sensor device 504 using the data communication link 530. The data communication link 530 can be established using multiple segments 532a, 532b, 532c. When link-based security is employed, the link layer circuit and module 522 in the SoC 502 is responsible only for the first segment 532a of the protected data communication link 530, and the sensor device 504 can be involved in protecting the last segment 532c. In the illustrated example 500, intervening devices 506, 508 are responsible for protecting the segment 532b through which they are connected. In some instances, certain endpoint devices can not understand link-based security, and can be connected by an unprotected physical segment. For example, the SoC 502 and / or the sensor device 504 in the illustrated example can be endpoint devices that do not understand link-based security, and the corresponding segments 532a and / or 532c can not be protected by link-based security.

[0057] Certain advantages of the link-based paradigm derive from the automatic protection of all data traffic entering the link layer. For example, no cryptographic processing is required at higher protocol layers including the application layer. In another example, no associated processing circuitry in the SoC 502 and the sensor device 504 is required to manage, configure, or control cryptographic operations for any data communication link other than the first segment 532a or 532c, respectively, of the data communication link 530 that is coupled to the SoC 502 or the sensor device 504.

[0058] In a second illustrated example 540, data security is provided according to an application-based paradigm (application-based security). When application-based security is implemented, data communication links 570 are generally protected within the application layer of the communication protocol. For example, the Secure Standard and Protocol defined by the MIPI Alliance defines application layer data security features for data communication links.

[0059] According to certain aspects of the present disclosure, a partial integrity mode of data protection can be applied when employing an application-based paradigm. The partial integrity mode provides data security with some degree of granularity, whereby, for example, only a portion of a video frame is protected. In one example, data originating at one data source 554 is protected, while data originating at another data source can not be protected. In another example, data 558 to be provided to one data sink 552 can be protected, while data directed to other data sinks can not be protected. In another example, certain portions of data 558 originating at a specified data source 554 and directed to a specified data sink 552 can be protected, while other data sent over the data communication link 570 can not be protected. Certain data types sent over the data communication link 570 can be protected, while other data types can not be protected. Data types can include data types that can be used to represent or identify image data itself in a particular format, such formats being based on a red-green-blue (RGB) color model or based on a YUV color model, in which pixel data is encoded according to luminance (Y) and chrominance (U and V). Certain data types can be used to represent or identify auxiliary or embedded data associated with image data, such as a data valid or data invalid status of the image data or a GPS location of the image data.

[0060] According to certain aspects of the present disclosure, a partial integrity mode of data protection enabled under an application-based paradigm can be optimized based on a tradeoff between power consumption and security requirements, providing a system that includes a scalable data integrity configuration. In one example, an application-based paradigm enables a reduction in power consumption by limiting cryptographic and other processing operations to particular portions of image frames or data types, sources, and sinks as selected by an application or system designer. Certain elements of data security can be provided independent of an underlying transport medium or protocol. For example, a data integrity mode implemented under an application-based paradigm enables an application to establish a partially protected data link 550 between SoC 542 and sensor device 544 that can be used to send data over wired and wireless communication links without modifying link layer circuitry and modules 562, 564, 566, 568. The partially protected data link 550 can be established without modifying or reconfiguring a communication protocol used to manage, configure, and control the data communication link 570. The reduction in power consumption resulting from selective and / or scaled protection of data 558, 556 can enable power and thermal budgets to be met. Power and thermal budgets often assume a high priority in embedded systems, battery-powered systems, and systems with limited heat dissipation capabilities. Independence from an underlying transport mechanism enables an application to establish a partially protected data link with another application or controller without considering physical layer aspects of the data link (e.g., wired or wireless) or a communication protocol used to manage, configure, and control the physical data link.

[0061] In some systems, the use of a link-based paradigm can improve the performance of the overall system or device. In one example, link initialization and authentication for a data communication link protected according to a link-based paradigm can be completed more quickly than link initialization and authentication for a data communication link protected according to an application-based paradigm. When a link-based paradigm is employed, multiple segments of a data communication link can be initialized and authenticated concurrently. When a multi-segment and / or multi-drop data link is protected according to an application-based paradigm, an application processor or SoC is typically required to authenticate each component in sequence. Sequential authentication can thereby be further slowed down when slow types of data communication are used to control or manage communication over one or more segments. In one example, I2C-based data communication can be considered a slow type of data communication.

[0062] Data security can include authentication, integrity, and confidentiality components. Authentication involves the establishment of trust between parties of a transaction. Parties of a transaction can include a transmitter of data and one or more receivers of data. For a data communication link, one-way authentication is desirable to ensure that a receiver, such as a controller, is confident that the receiver trusts the source of transmitted data, such as a sensor. Additionally, mutual or two-way authentication is often desirable to ensure that a transmitter, such as a sensor, trusts the source of data from a controller, such as control data. Integrity involves guaranteeing that data is not altered by an “attacker,” often referred to as a “man-in-the-middle attacker,” who intentionally modifies and / or replays data at the time of transition. In some examples, data integrity can be protected by sending a message authentication code (MAC), which can be derived from transmitted data that has not been altered, and can be used by a receiver to test for differences between transmitted and received data. Confidentiality involves protecting the content or meaning of data from unauthorized access. In some implementations, confidentiality protection is obtained through message encryption.

[0063] In conventional systems, data security provisions are uniformly implemented for all communication over a data communication link. For example, when enabled, data integrity provisions typically apply to all data transmitted over a data communication link, or when disabled, do not apply to any data transmitted over a data communication link. In some instances, some data can be encrypted for transmission over a data communication link, while other data is not encrypted. In one example, the MIPI Alliance defines a flexible security paradigm, which can be referred to as the Source Selective Partial Integrity and Encryption (SSPIE) standard and protocol.

[0064] Certain aspects of the present disclosure relate to systems, methods, and techniques that provide a highly "granular" or "selective" and / or "scalable" security mode. In one aspect, a system can employ a combination of link-based security and application-based security to protect data communications over a single data communication link. In one aspect, a scalable security paradigm is provided that enables partial integrity, whereby certain portions of transmitted data are protected to a higher degree of integrity than other portions of transmitted data. In one example, steps are taken to ensure the integrity of a first portion of transmitted data, while a second portion of transmitted data is not actively protected.

[0065] Certain aspects of the present disclosure relate to protecting the transmission of video or image data. In one aspect, different security protection is applied to different portions or partitions of a video frame. In one example, certain portions of data corresponding to a video frame can be transmitted with a MAC that enables detection of loss of data integrity during transmission, and other portions of data corresponding to the video frame can be transmitted without first generating a corresponding MAC. In another example, certain portions of data corresponding to a video frame can be encrypted prior to transmission, while other portions of data corresponding to the video frame can be transmitted unencrypted. The MAC mode configured in accordance with the present disclosure determines the frequency of MAC computation and transmission in the transmitter. The generation of MACs and encryption of data can involve certain compute-intensive processes. The ability to provide security mode selectivity can reduce power consumption and / or can limit heat generation in system components involved in MAC or cryptographic computation and / or transmission. In some applications, the power consumption and heat generation issues can be exacerbated (worsened) at higher camera resolutions and frame rates.

[0066] Implementation of selective security modes in accordance with certain aspects of the present disclosure can include source selective aspects and partial integrity aspects. Source selective security, in the context of video frame or image data transmission and for the purposes of the present disclosure, refers to the implementation of different levels of security for different portions or partitions of a video frame. Partial integrity transmission can refer to a payload, where the integrity of a selected portion of the payload is protected, while some portion of the payload is not actively protected, within a selected portion of a frame. Full integrity transmission can refer to a payload where the entire payload is protected for integrity. In some implementations, full integrity transmission can be encrypted. Encryption can be enabled or disabled depending on the security protection required by the security mode or video partition in operation.

[0067] Certain aspects of the present disclosure provide systems, apparatus, and techniques that enable selection, configuration, and adjustment of security modes, including support for application and operational requirements via artificial intelligence (AI) agents. A security mode can define a set of security options that can be selected by an application or during system integration or initialization. An example of a car environment is used to describe various aspects of the present disclosure. In a car environment that can be equipped with twenty or thirty cameras, a large amount of image data can be sent over wired or wireless connections. Cryptographic data protection, such as integrity protection, can be provided for transmissions involving at least some of the cameras, and other data protection modes can be provided for transmissions involving other cameras. In some instances, transmissions involving one or more cameras can not be actively protected. Cameras deployed within a car can produce video camera, radar, or lidar images that are sent to a controller, which can be referred to as a central controller or electronic control unit or ECU. Some combination of authentication, data integrity, and encryption can be used to protect data transmissions involving the ECU. Authentication refers to a process used to establish trust between devices, while data integrity involves message authentication of data exchanged between devices.

[0068] In certain examples, each camera deployed within a car can produce image data at a rate between 1 gigabit per second and 10 gigabits per second (1 Gbps to 10 Gbps). In certain vehicle operating modes, the cameras can produce transmissions up to 300 Gbps. Receiving and processing data at such data rates can result in high power consumption and associated high levels of heat generation.

[0069] Figure 6 is a graphical representation 600 of a range of data security modes provided in accordance with a scalable data security scheme implemented in accordance with certain aspects of the present disclosure. In the illustrated example, five data security variants 602 (x-axis) can select different relative data security levels 604 (y-axis). In other examples, the number of defined security variants can be greater than or less than five. Different relative data security ranges 610a-610e can be supported or associated with each data security variant 602. Data integrity modes can also be selected according to MAC modes 606 (z-axis). The data integrity mode can define whether a MAC is computed: per frame; or per selected or identified portion of a frame; per data type; or for a complete message that can include metadata, control, and / or status information in addition to image data.

[0070] According to one aspect of the disclosure, a relative data safety level for data transmission can be selected or switched according to the identified data type, data source, data sink, transmission mode, available power, temperature constraints, and / or operating mode of the vehicle or system in which the transmitted data is generated or processed. The data safety level 604 can be selected or configured by an ECU or other controller that monitors or defines vehicle operating modes, or that is responsible for power consumption, thermal management, and / or safety and physical safety of the vehicle and occupants. The data safety level 604 can be selected or configured by an AI agent that resides in an ECU, controller, or SoC.

[0071] In the illustrated example, an ECU or other controller can configure a data safety level 604 for transactions between the application and sensors or peripherals, and / or for one or more physical data communication links within the vehicle or system, by selecting a data safety variant 602. The safety levels available for the data safety variant 602 and MAC mode 606 can also be configured by the ECU or other controller. In some instances, data communication links can be grouped by function and / or operating mode of the vehicle. In some examples, two or more data communication links can provide the same level of data protection when the vehicle is configured for a first operating mode, and can operate at different levels of data protection from each other when the vehicle is configured for a second operating mode.

[0072] In some implementations, an ECU or other controller references one or more lookup tables to determine the data safety level 604 to be configured or selected for a transaction or data link. In some implementations, an ECU or other controller can determine a data safety level based on an analysis of sensor readings, vehicle operating states, environmental conditions, battery or fuel levels, and other parameters. In one example, the analysis can include the contribution of an AI agent or other machine learning algorithm. In the latter case, the AI agent can operate in phases known in the AI practice, including an AI training phase performed using a controlled data set, such as based on measurements of power and / or heat based on operating conditions, followed by an AI feedback phase performed instantaneously or periodically with the help of real-time system feedback, followed by a “inference” or “prediction” phase that identifies the next system configuration.

[0073] In the illustrated example, five data security variants 602 are available for selecting partial integrity modes that provide a range of protection levels from a maximum data integrity mode with data encryption to a minimum integrity mode that provides no additional data integrity beyond that which can be provided by a data communication link management protocol. Data exchanges under the first security variant (SV-1) can provide maximum data security for each MAC mode 606 with data encryption and full data integrity protection enabled. Data integrity can be applied on a frame, per data type, or per message basis depending on the selected MAC mode 606. Data encryption is disabled under the other four security variants (SV-2, SV-3, SV-4, and SV-5). Full data integrity protection is enabled and applied on a frame, per data type, or per message basis depending on the selected MAC mode 606 under the second security variant (SV-2). No additional data integrity protection is provided under the fifth security variant (SV-5), and two modes of partial data integrity protection are provided under the third and fourth security variants (SV-3 and SV-4). The partial data integrity protection modes can be characterized by skip patterns that define portions of data transmissions that are provided increased or decreased relative data integrity protection.

[0074] Figure 7 Examples of skip patterns that can be configured in accordance with certain aspects of the present disclosure are illustrated. A first skip pattern 700 can be applied on a row-by-row basis to image data. Image data can comprise pixel data and can be transmitted in a sequence of data packets along with other data and command codes. For purposes of the present disclosure, a row refers to an amount of image data representing a horizontal row in an image captured by a sensor. In other examples, a row can refer to an amount of image data representing a row in an image captured by a sensor. For each row, data integrity protection is applied to all image data or not applied to any image data. Skip pattern 700 defines a combination and arrangement of rows for which data integrity protection is applied. In some implementations, skip pattern 700 is symmetrical with a determinable cadence. Skip pattern 700 can be considered symmetrical when data integrity protection rows are consistently applied to row groups 702, 706 that include the same number of rows and are followed by unprotected row groups 704, 708 that include a minimum number of rows. Skip pattern 700 can be considered to have a determinable cadence when data integrity protection rows are applied at a fixed interval through frames and / or associated or embedded data. In some instances, an irregular skip pattern can be defined in which data integrity protection is applied to specified or selected rows or row groups that include image data within a frame and / or selected rows or row groups that include metadata or embedded data.

[0075] The second stride pattern 720 can be applied to the image data on a pixel-by-pixel basis within one or more rows or one or more vertical blocks of pixel data within the frame of image data. A block of pixel data can include contiguous data occupying a portion of one or more rows. In some examples, a block of pixel data can correspond to a vertical or horizontal row of pixels, including when rows and portions of rows can be included in the stride pattern 720. In some examples, the randomized stride pattern 730 is applied to protect the integrity of a region of interest (RoI) or arbitrarily selected block of pixel data. The stride patterns 720, 730 define the size and location of the blocks of pixels for which data integrity protection is applied. In some implementations, the stride patterns 720, 730 are symmetrical with a determinable stride frequency. The stride patterns 720, 730 can be considered symmetrical when data integrity protection is applied to data corresponding to a plurality of blocks of pixels having the same size and shape and positioned throughout the image data and / or metadata or embedded data. The stride patterns 720, 730 can be considered to have a determinable stride frequency when data integrity protection is applied at a fixed interval through the image data and / or metadata or embedded data. In some instances, an irregular stride pattern can be defined in which data integrity protection is applied to blocks of image data within an image frame having various sizes and / or without an identifiable repeating pattern.

[0076] In some implementations, a transaction involving the transmission of an image frame can include defined frame partitions including data packets carrying command and control sequences, portions of image data, and / or metadata or embedded data. A stride pattern can be defined independently for each frame partition within a frame. For purposes of this description, the structure of a transaction and associated data packets can be defined by a data communication protocol.

[0077] Figure 8 An example of a frame-related transaction 800 partitioned according to certain aspects of the present disclosure is illustrated. The frame-related transaction 800 can include the transmission of data packets configured according to a standard-based protocol or a proprietary protocol. One example of a standard-based protocol is the CSI-2 protocol defined by the MIPI Alliance. A first frame partition 802 includes at least one short packet 812 indicating the start of the frame-related transaction 800. The short packet 812 can include parameters defining certain aspects of the structure of the frame-related transaction 800. A second frame partition 804 includes one or more packets 814i-814 M The embedded data can include information defining certain characteristics of the captured image, including size, resolution, aperture and focus settings, and other image-related information. A third frame partition 806 includes one or more packets 816i-816 KImage data may include one or more bytes of data for each pixel in the image frame. The fourth frame partition 805 includes one or more packets 814 carrying second embedded data associated with the image frame. M+1 Up to 814 N The fifth frame partition 810 includes at least one short packet 818 indicating the end of frame transmission. Embedded data may include information identifying the timing of image capture or the sequence number of the image. In some implementations, the embedded data may include data integrity information, including information related to the stride pattern associated with the image data transmitted in frame-related transactions 800.

[0078] Figure 9 An example of a scalable data security scheme 900 is illustrated, which can be selectively applied to frame partitions 902, 904, 906, 908, and 910 in frame-related transactions according to certain aspects of this disclosure. A series of data security modes can be implemented for use with packets carrying command and control information, image data, embedded data, and other information. Frame partitions 902, 904, 906, 908, and 910 can correspond to... Figure 8 The illustrated frame partitions are 802, 804, 806, 808, and 810, and the security variants (SV-1, SV-2, SV-3, SV-4, and SV-5) correspond to... Figure 6 The illustrated data security variant 602. Certain aspects of the scalable data security scheme 900 may correspond to features of standards-based protocols, but... Figure 9 The concepts illustrated are not limited to any particular protocol or standard.

[0079] In the illustrated data security scheme 900, short packets sent in the first frame partition 902 and the fifth frame partition 910 are provided with full integrity data protection across all frame-related transactions, without encryption. This level of data protection corresponds to the data protection provided under the SV-2 security variant. In addition to providing [specific protection details], other security variants may also be available. Figure 7 Any security variant other than the SV-4 variant of the illustrated second straddle pattern 720 is used to protect embedded data transmitted in the second frame partition 904 or the fourth frame partition 908, the second straddle pattern being applied on a pixel-by-pixel basis to image data, pixel data blocks within image data, or regions of interest in an image. Embedded data may be transmitted with encrypted or unencrypted full integrity data protection (SV-1 and SV-2); unencrypted non-integrity data protection (SV-5); or unencrypted line-based partial integrity data protection (SV-3). The data protection mode configured for embedded data transmitted in the second frame partition 904 may differ from the data protection mode configured for embedded data transmitted in the fourth frame partition 908.

[0080] Image data sent in the third frame partition 906 can be protected according to any of the safety variants (SV-1, SV-2, SV-3, SV-4, or SV-5) and can have full integrity data protection with or without encryption, no integrity data protection without encryption, or partial integrity data protection without encryption.

[0081] Certain aspects of the present disclosure enable an AI, ECU, or other controller disposed in a vehicle to determine when to initiate a switch between data protection modes and determine which data protection mode to select when a switch between data protection modes is indicated. Systems configured or adapted according to certain aspects of the present disclosure can implement data security protection at the link layer and, in some instances, at the application layer in SoCs, ECUs, and / or sensor devices. Systems configured or adapted according to certain aspects of the present disclosure can configure and manage data integrity modes, including selecting and configuring partial integrity modes. An ECU or other controller managing operation of a vehicle can also configure and initiate data protection modes for multiple data communication links and can initiate changes in data protection modes in devices coupled to data communication links. The ECU or other controller can select data protection modes based on the type and combination of data to be communicated, the source and destination of data to be communicated, power budget, and thermal budget. The ECU or other controller can switch between data protection modes based on changes in power consumption, heat generation and heat dissipation, and other operating conditions. The ECU or other controller can switch between data protection modes in response to changes in operating modes of the vehicle.

[0082] According to certain aspects of the present disclosure, an ECU or other controller disposed in a vehicle can be configured to weigh competing operational priorities when selecting or switching between data protection modes. In some examples, the ECU or other controller can determine data protection modes to implement based on tradeoffs between operational parameters such as a desired or specified relative security level, cryptographic computations required to achieve the desired or specified relative security level, expected power consumption associated with performance of the cryptographic computations, power budget, expected thermal impact of performing the cryptographic computations, and thermal budget. In some instances, the ECU or other controller is configured to apply weights or priorities to the operational parameters and to comply with limits and thresholds defined by system specifications. In some implementations, tradeoffs are made between hardwired functionality, power consumption, and silicon space during design of an IC device including the ECU or other controller, and the ECU or other controller can allocate certain functionality throughout the vehicle according to power or thermal efficiency attributed to different IC devices.

[0083] A scalable data security scheme implemented in accordance with certain aspects of the present disclosure can initiate partial integrity modes for some data transactions in order to meet power budget limitations and thermal budget limitations. Partial integrity modes that provide reduced data protection can be implemented for data transactions that are deemed to have a low priority during a current operating mode of the vehicle. An ECU or other controller can estimate, calculate, or otherwise determine a probability of data loss associated with a partial integrity mode, and can modify the partial integrity mode selection based on potential performance loss or based on the impact of system-level errors from such data loss. In some instances, the probability of data loss can be measured based on previous operation of the vehicle monitored under various data protection modes. The ECU or other controller can also calculate or determine a limited probability of attack by a malicious actor when not all data is protected under the selected partial integrity mode.

[0084] In accordance with certain aspects of the present disclosure, a controller in a vehicle can modify a relative security level of a data transaction based on an operating mode of the vehicle or a subsystem of the vehicle. In the following examples, reference will be made to the illustrated automobile 100. A central controller 120 in the automobile can be implemented as an ECU or another vehicle control system. The central controller 120 can cooperate with other controllers or processors disposed in the automobile 100. Figure 1

[0085] In one example, the central controller 120 can switch between partial integrity modes based on the degree of motion or change in the scene. The central controller 120 or other controller can determine that image data corresponding to a static scene does not need to be sent in a full integrity mode. When a moving or changing scene captured by a camera becomes static or slow moving, the central controller 120 can switch the data protection mode from a full integrity mode to a partial integrity mode. In some instances, when a slow moving scene captured by a camera transitions to a static or substantially static scene, the central controller 120 can switch the data protection mode from a partial integrity mode that employs a row-wise stepping pattern to a partial integrity mode that employs a pixel-wise stepping pattern, where a substantially static scene includes areas in which some changes or movements are occurring. When a static or slow moving scene captured by a camera begins to become a moving or changing scene, the central controller 120 can switch the data protection mode from a partial integrity mode to a full integrity mode.

[0086] ​In another example, the central controller 120 can enforce a higher level of security on data transmissions associated with the forward-facing cameras 102, 104, 106 when the car 100 is traveling forward. The higher level of security can be enforced to protect data used for autonomous driving purposes. A higher level of security can be desirable to protect data used to provide automated driving assistance. In some instances, the higher level of security can be enforced on data transmissions associated with rear-facing or side-facing cameras when the car 100 is parked or traveling in reverse. The higher level of security can include providing data encryption on certain data communication links and providing full integrity transmission of image frames. The central controller 120 can switch the data protection mode for image data generated by the forward-facing cameras 102, 104, or 106 to an encrypted full integrity mode upon determining that forward motion of the car 100 has begun or been commanded. The central controller 120 can switch the data protection mode for image data generated by the forward-facing cameras 102, 104, or 106 to a non-encrypted partial integrity mode upon determining that forward motion of the car 100 has terminated. In some implementations, the central controller 120 can switch the data protection mode for image data generated by other cameras 108, 112, 114 in the car 100 to a non-encrypted partial integrity mode upon determining that forward motion of the car 100 has begun or been commanded. In these latter implementations, switching the data protection mode for image data that is not used for autonomous or assisted driving can reduce aggregate cryptographic computation overhead, power consumption, and heat generation.

[0087] In another example, the central controller 120 can modify the level of security for data transmissions when power consumption exceeds a power budget of the car 100 or a power budget of a subsystem of the car 100. In some implementations, the central controller 120 can be configured to switch the data protection mode for image data generated by one or more forward-facing cameras 102, 104, 106 from an encrypted full integrity mode to a non-encrypted partial integrity mode upon determining that power consumption has exceeded a threshold defined for the one or more forward-facing cameras 102, 104, 106. Initially, the central controller 120 can reduce the data protection mode for image data generated by other cameras 108, 112, 114 in the car 100 to compensate for the excessive power consumption of the one or more forward-facing cameras 102, 104, 106.

[0088] In another example, the central controller 120 can modify the security level of data transmission when the temperature of a camera or camera subsystem exceeds a threshold value specified by the car 100 design. In some implementations, the central controller 120 can be configured to switch the data protection mode for image data generated by one or more forward-facing cameras 102, 104, 106 from an encrypted full-integrity mode to an unencrypted partial-integrity mode upon determining that the temperature has exceeded a threshold value defined for the one or more forward-facing cameras 102, 104, 106. The central controller 120 can be configured to activate other forward-facing cameras 102, 104, 106 to enable the overheated camera to be cooled.

[0089] According to certain aspects of the present disclosure, the partial-integrity setting can be configured based on system functionality or operational mode. In the example of autonomous or assisted driving, the operational focus of the system in the vehicle is on having a view of the road ahead that involves Figure 1 The prioritized data exchange of the forward-facing cameras 102, 104, 106 is exemplified on forward-facing drive monitoring. In many implementations, the image data produced by the forward-facing cameras 102, 104, 106 is prioritized for data protection and delivery. As such, data protection circuits and modules can be required to process the large amount of data quickly in order to deliver the image data in time to the autonomous or assisted driving subsystems. The availability of the partial-integrity mode can enable the data protection circuits and modules to meet the timing specification within the power and thermal budget. The partial-integrity mode provides protection to a portion of the data generated by the forward-facing cameras 102, 104, 106, and the portion of data that is protected can be selected according to operational needs. For example, some image data can have a higher importance in certain driving environments, while having a reduced importance in other driving environments. In some implementations, the image data produced by some forward-facing cameras 102, 104, 106 can be assigned a higher priority in high-speed driving environments compared to the image data produced by some side-facing cameras 112, 114 or rear-facing cameras 108. In these implementations, the priority difference between the forward-facing cameras 102, 104, 106 and the side-facing cameras 112, 114 can be reduced in urban driving environments, where the need for awareness of pedestrians and bicyclists increases.

[0090] In another example, the partial integrity settings can be configured based on the current operating environment of the automobile 100. The central controller 120 can be configured to define data protection modes for image data based on data sources and driving environments that enable the entire vehicle system to operate within a compute budget, a power budget, and a thermal budget in accordance with the present disclosure. The central controller 120 can switch between partial integrity configurations to reduce power or accumulated heat, and / or dynamically reallocate power consumption and heat generation throughout the physical extent of the automobile 100. The ability to configure levels of partial integrity for multiple data communication links and image data sources or sinks enables tradeoffs to be made with respect to power consumption or heat generation. In some instances, AI or machine learning (ML) algorithms, systems, or modules can be used to dynamically evaluate and select these tradeoffs, which can be trained to find an optimal allocation of partial integrity settings among the data communication links and data sources or sinks. The AI or ML algorithms, systems, or modules can be trained to meet generalized goals, such as maintaining a minimum safety level while maintaining power consumption at or near a minimum value and limiting current consumption on the electric vehicle battery or power source to a target level. In some instances, the learning process can enable the minimum safety level to increase, the minimum power consumption level to decrease, and the target level of current consumption to be lowered. The learning process can involve monitoring and recording the effects of changes to the partial integrity configurations on meeting the minimum values and targets and the ability of the system to adequately protect mission-critical or other important data.

[0091] In another example, the partial integrity setting can be configured as a component to manage power consumption as well as reduce temperature or heat generation. In this example, the central controller 120 configured according to certain aspects of the present disclosure can review or analyze the expected impact of the change in data protection on power consumption and thermal management. For example, the central controller 120 can determine that a change in data protection configuration is needed as the automobile 100 transitions from a parked environment to a driving environment. The central controller 120 can identify further changes to the data protection configuration for one or more data communication links, camera subsystems of processing circuitry after reviewing the impact on power consumption and thermal management. The further changes can include implementing or changing partial integrity parameters (SV-3, SV-4) or stride patterns for image data sent by one or more cameras. In some instances, the further changes result in an overall decrease in data protection coverage for image data. In some instances, the further changes result in a redistribution of the strength of data protection coverage for image data by decreasing data protection for low priority image data and increasing data protection for high priority image data. In some instances, the further changes result in an overall increase in data protection coverage for image data when, for example, headroom (additional capacity) is found to be available in a thermal budget and power consumption budget or a minimum level of safety is not expected to be met by the planned switch in partial integrity settings. In the latter instance, headroom can be obtained by redistributing computational or other processing overhead throughout the physical range of the automobile 100.

[0092] In some implementations, the central controller 120 implements a reduction in data protection coverage by determining partial integrity parameters including SV3 / SV4 and stride pattern parameters in order to reduce integrity computations. In some instances, the central controller 120 can turn off encryption when a previously configured data protection configuration provides the highest level of data safety (i.e., SV-1). The central controller 120 can use a lookup table (LUT) or multiple LUTs when reviewing or analyzing the expected impact of the change in data protection on power consumption and thermal management. The one or more LUTs can cross-reference data protection settings with operational modes for one or more components of the vehicle system. After determining a final data protection configuration, the central controller 120 can initiate a switch in data protection for certain components of the vehicle system. For example, the central controller 120 can send commands to the components affected by the change over a shared data communication bus. The central controller 120 can monitor temperature, power consumption data rate, data communication error rate (including rate of dropped packets), retransmission rate, and transmission latency in order to determine the efficacy of the change in data protection configuration.

[0093] In some implementations, the central controller 120 can implement or cooperate with an AI inference engine that uses one or more LUTs to review or analyze the expected impact of changes to data protection on power consumption and thermal management. The AI inference engine can make decisions about configuration changes based on initial training and real-time training operations.

[0094] In another example, the partial integrity setting can be configured based on the content of the image data and changes to the content. In this example, the image processing subsystem or the central controller 120 can monitor spatial or temporal dynamic aspects of the scene captured by the camera. The central controller 120 can determine that a switch to a partial integrity mode or a reconfiguration of the partial integrity mode is possible or desirable when the scene or a segment of the scene by the camera crosses a spatial or temporal dynamic threshold. In some instances, the central controller 120 determines that a reduction in the relative security level protection is possible when the scene or a segment of the scene by the camera falls below a spatial or temporal dynamic threshold. In some instances, the central controller 120 determines that an increase in the relative security level protection is needed when the scene or a segment of the scene by the camera rises above a spatial or temporal dynamic threshold.

[0095] The central controller 120 can initiate a switch in data protection for certain components of the vehicle system based on a transition across a spatial or temporal dynamic threshold. The central controller 120 can send commands to the components affected by the change over the shared data communication bus. The central controller 120 can continue to monitor the state of the image data output by the affected camera to determine whether further changes to the data protection configuration are needed.

[0096] In another example, the partial integrity setting can be configured based on security processing capacity. In one example, the vehicle system can be designed or configured to provide a processing capacity capable of handling up to 100 Gbps of generated image data across all cameras in the system. In this example, the central controller 120 can determine that a change in the driving environment or other system operating mode has occurred and can determine the impact of a switch in data protection configuration on the image data processing capacity. In some implementations, the autonomous or assisted driving subsystem can indicate the occurrence of a high value or high priority scene or scene portion that triggers a switch to a higher data integrity mode. For example, the autonomous or assisted driving subsystem can be configured to identify the presence of a child or a stop sign as requiring full integrity data protection.

[0097] When the vehicle system is operating at or near its rated capacity, the central controller 120 can reduce processing and computational overhead by reducing the data integrity mode provided for another camera in order to provide sufficient capacity for data communication associated with high value or high priority scenes or scene portions. Another camera can be configured to a partial integrity mode by assigning an appropriate safety value (SV-3, SV-4, or SV-5) and, in some instances, defining a cross-step pattern for image data generated by all affected cameras to reduce the data integrity mode.

[0098] The central controller 120 can initiate a switch in data protection for certain components of the vehicle system based on detection of a high value or high priority scene or scene portion. The central controller 120 can send commands to the components affected by the change over the shared data communication bus. The central controller 120 can continue to monitor the status of image data output by the affected cameras to determine if further changes in data protection configuration are needed.

[0099] Figure 10 is a flowchart 1000 illustrating a safe variant mode switch according to certain aspects of the present disclosure. The safe variant mode switch can be performed by a central controller configured to implement and manage a scalable data safety scheme provided according to certain aspects of the present disclosure. The central controller can be an ECU and SoC or other processing circuitry deployed as part of a vehicle control or management system. The central controller can initialize the vehicle control or management system following a power-on or reset event. In some instances, the central controller can initialize the vehicle control or management system in response to a command received from an application. At block 1002, and during initialization, the central controller can enumerate and inventory cameras provided as part of the vehicle control or management system. The cameras can include different types of cameras each configured to provide data to an image processor participating in at least one vehicle control or management function. The central controller can identify capabilities and locations of each of the cameras. The capabilities of each camera can be characterized by a supported image size, an image resolution expressed as a number and density of pixels, a color depth, and a frame rate. The central controller can further determine a topology of a network through which the central controller is coupled to the cameras. The network can include some combination of point-to-point and multi-point data links. One or more cameras can be coupled to the central controller through multiple data links or data link segments and through one or more bridges or gateways.

[0100] At block 1004, and during initialization, the central controller can determine the configuration, capabilities, and secure processing capacity of various data protection controllers, processors, circuits, and modules. The data protection components can be embodied in one or more SoCs, and can include the central controller. Certain data protection components can be embodied in link layer circuits, and can include data encryption circuits. The processing capacity of the data protection components can be expressed as the number of bytes or pixels that can be processed per second. One or more bytes of data can be required to fully represent a single pixel.

[0101] At block 1006, and during initialization, the central controller can establish security specifications for the vehicle control or management system. The security specifications can define a minimum data protection level for each data communication link and / or segment. The security specifications can define a minimum data protection level for each type of data communicated over the data communication links. The security specifications can define a minimum data protection level for data generated by each camera, including according to an identified operating camera configuration, such as a set of forward-facing cameras, and according to an operating vehicle mode, such as driving above a threshold speed. The security specifications can define a minimum data protection level for each operating mode of the vehicle and / or for each operating mode of each camera. The security specifications can define a priority for each camera or for a set of cameras. The security specifications can define a priority for each data link in each operating mode of the vehicle. Each data protection level can be expressed as a relative security level (RSL). In some implementations, the security specifications can be recorded in an AI function that relates the minimum data protection levels and priorities to operating modes of the vehicle and / or operating modes of each camera. In some implementations, the security specifications can be recorded in one or more LUTs that relate the minimum data protection levels and priorities to operating modes of the vehicle and / or operating modes of each camera. The LUTs can account for operating parameters, including power consumption, processing capacity, and thermal conditions within the vehicle or within the vehicle control or management system. The processing capacity can be expressed as the number of pixels that can be processed during integrity calculations. The processing capacity can be expressed as the number of CSI-2 messages that can be processed during integrity calculations. The LUTs can be preconfigured based on design or simulation during system integration.

[0102] The RSL override for type selection of data communication links or image data transactions can impact operational parameters, including processing overhead, power consumption, and temperature over time. At block 1008, the central controller can monitor operational parameters and can trigger a change in data protection mode to offset the effects of observed or anticipated increased temperature or power consumption. For example, the central controller can trigger a change in data protection mode when power savings are needed, when thermal mitigation (temperature reduction) is needed, or when a high value camera scene or scene segment is identified. The central controller can also trigger a change in data protection mode when a change in vehicle operating mode is detected or commanded. For example, a change in city driving to highway driving mode can cause the central controller to trigger a change in data protection mode.

[0103] At block 1010, the central controller determines whether a change in data protection mode is to be triggered. When no change is to be triggered, then the central controller can loop back to block 1008. When a change in data protection mode is to be triggered, then at block 1012, the central controller can determine the minimum data protection requirements for the new data protection mode and can select initial data integrity configuration parameters for the affected or identified cameras. The new data protection mode can be expressed as RSL values for one or more data communication links and / or for one or more cameras. The RSL values can be derived from one or more LUTs. The central controller can determine the safe processing capacity of the affected data protection components to ensure that sufficient capacity is available for processing the new data protection mode. In one example described herein, the vehicle control or management system has a capacity to process a total of 100 Gbps of image related data. In some instances, the capacity of individual data protection components can be limited even when the overall capacity nominally sufficient to process the new data protection mode.

[0104] At block 1014, the central controller determines whether the new data protection mode can be processed by the vehicle control or management system and the affected components of the vehicle control or management system. When the vehicle control or management system and the affected components of the vehicle control or management system are nominally capable of processing the new data protection mode, then the central controller can implement the new data protection mode at block 1018 and then resume monitoring at block 1008. When the vehicle control or management system or the affected components of the vehicle control or management system are determined to be incapable of processing the new data protection mode, then the central controller proceeds to block 1016.

[0105] At block 1016, the central controller can modify the RSL values defining a new data protection mode. In one example, the central controller can turn off encryption for one or more data communication links. In another example, the central controller can modify a partial integrity mode defined for image data generated by one or more cameras or communicated over one or more data communication links. In another example, the central controller can modify a partial integrity setting defined for image data generated by one or more cameras or communicated over one or more data communication links. The partial integrity setting can include a cross-step pattern associated with image data generated by one or more cameras or communicated over one or more data communication links. In another example, the central controller can modify a partial integrity mode or a partial integrity setting associated with image data generated by one or more lower-priority cameras or communicated over one or more lower-priority data communication links. In some examples, the central controller can define a data protection mode that provides a lower level of protection compared to the security specification established for the vehicle control or management system at block 1006.

[0106] The central controller can return to block 1014 to determine whether the modified data protection mode can be handled by the vehicle control or management system and affected components of the vehicle control or management system.

[0107] Examples of processing circuitry and methods

[0108] Figure 11 FIG. 11 is a diagram illustrating an example of a hardware implementation for an apparatus 1100. In some examples, the apparatus 1100 can perform one or more of the functions disclosed herein. In accordance with various aspects of the disclosure, an element, any portion of an element, or any combination of elements can be implemented with the processing circuit 1102 as disclosed herein. The processing circuit 1102 can include one or more processors 1104 controlled by some combination of hardware and software modules. Examples of processors 1104 include microprocessors, microcontrollers, digital signal processors (DSPs), SoCs, ASICs, field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, sequencers, gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functionality described throughout this disclosure. The one or more processors 1104 can include special-purpose processors that perform specific functions and can be configured, enhanced, or controlled by one of the software modules 1116. The one or more processors 1104 can be configured by a combination of software modules 1116 loaded during initialization and can be further configured by loading or unloading one or more software modules 1116 during operation.

[0109] In the illustrated example, the processing circuit 1102 can be implemented with a bus architecture, as represented generally by the bus 1110. The bus 1110 can include any number of interconnecting buses and bridges depending on the specific application of the processing circuit 1102 and the overall design constraints. The bus 1110 links together various circuits including the one or more processors 1104 and the storage device 1106. The storage device 1106 can include memory devices and mass storage devices, and in this document can be referred to as computer- and / or processor-readable media. The bus 1110 can further link together various other circuits such as timing sources, timers, peripherals, voltage regulators, and power management circuits. A bus interface 1108 can provide an interface between the bus 1110 and one or more transceivers 1112a, 1112b. A transceiver 1112a, 1112b can be provided for each networking technology supported by the processing circuit. In some instances, multiple networking technologies can share some or all of the circuitry or processing modules found in the transceivers 1112a, 1112b. Each transceiver 1112a, 1112b provides a means for communicating with various other apparatus over a transmission medium. In one example, the transceiver 1112a can be used to couple the apparatus 1100 to a multi-drop bus. In another example, the transceiver 1112b can be used to connect the apparatus 1100 to a radio access network. Depending on the nature of the apparatus 1100, a user interface 1118 (e.g., keypad, display, speaker, microphone, joystick) can also be provided and can be communicatively coupled to the bus 1110, either directly or through the bus interface 1108.

[0110] The processor 1104 can be responsible for managing the bus 1110 and for general processing that can include the execution of software stored in computer-readable media that can include the storage device 1106. In this regard, the processing circuit 1102, including the processor 1104, can be used to implement any of the methods, functions, and techniques disclosed herein. The storage device 1106 can be used to store data that is manipulated by the processor 1104 when executing software, and the software can be configured to implement certain methods disclosed herein.

[0111] One or more processors 1104 in processing circuitry 1102 can execute software. Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, algorithms, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise. The software can reside on storage 1106 or an external computer readable medium. External computer-readable media and / or storage 1106 can include a non-transitory computer-readable medium. A non-transitory computer-readable medium includes, for example, a magnetic storage device (e.g., hard disk, floppy disk, magnetic strips), an optical disk (e.g., compact disk (CD), digital versatile disk (DVD)), a smart card, a flash memory device (e.g., card, stick, key drive), RAM, ROM, programmable read-only memory (PROM), erasable PROM (EPROM, including EEPROM), registers, removable disk, and any other suitable medium for storing software and / or instructions that can be accessed and read by a computer. The computer-readable medium and / or storage 1106 can also include, for example, a carrier wave, a transmission line, and any other suitable medium for transmitting software and / or instructions that can be accessed and read by a computer. The computer-readable medium and / or storage 1106 can reside in the processing circuitry 1102, the processor 1104, external to the processing circuitry 1102, or be distributed across multiple entities including the processing circuitry 1102. The computer-readable medium and / or storage 1106 can be embodied in a computer program product. By way of example, a computer program product can include a computer-readable medium in packaging material. Those skilled in the art will recognize how best to implement the described functionality presented throughout this disclosure depending on the particular application and the overall design constraints presented by the overall system.

[0112] Storage 1106 can maintain software that is maintained and / or organized in loadable code segments, modules, applications, programs, etc., which can be referred to herein as software modules 1116. Each of the software modules 1116 can include instructions that, when installed or loaded on processing circuitry 1102 and executed by one or more processors 1104, facilitate the runtime image 1114 that controls operation of the one or more processors 1104. Certain instructions, when executed, can cause processing circuitry 1102 to perform functions in accordance with certain methods, algorithms and processes described herein.

[0113] Some of the software modules 1116 can be loaded during initialization of the processing circuitry 1102, and these software modules 1116 can configure the processing circuitry 1102 to perform various functionalities disclosed herein. For example, some of the software modules 1116 can configure internal devices and / or logic circuits 1122 of the processor 1104, and can manage access to external devices, such as the transceiver 1112a, 1112b, the bus interface 1108, the user interface 1118, timers, math co-processors, etc. The software modules 1116 can include control programs and / or operating systems that interact with interrupt handlers and device drivers, and control access to various resources provided by the processing circuitry 1102. Resources can include memory, processing time, access to the transceiver 1112a, 1112b, the user interface 1118, etc.

[0114] The one or more processors 1104 of the processing circuitry 1102 can be multi-functional, whereby some of the software modules 1116 are loaded and configured to perform different functions or different instances of the same function. The one or more processors 1104 can additionally be adapted to manage background tasks initiated in response to inputs from, for example, the user interface 1118, the transceiver 1112a, 1112b, and device drivers. To support performance of multiple functions, the one or more processors 1104 can be configured to provide a multi-tasking environment, whereby each of the multiple functions is implemented as a set of tasks serviced by the one or more processors 1104 as needed or desired. In one example, the multi-tasking environment can be implemented using a time-sharing program 1120 that passes control of the processor 1104 between different tasks, whereby each task returns control of the one or more processors 1104 to the time-sharing program 1120 upon completion of any outstanding operations and / or in response to inputs such as interrupts. When a task has control of the one or more processors 1104, the processing circuitry is effectively dedicated to the purpose addressed by the function with which the controlling task is associated. The time-sharing program 1120 can include an operating system, a main loop that transfers control on a loop basis, a function that allocates control of the one or more processors 1104 according to prioritization of functions, and / or an interrupt-driven main loop that provides control of the one or more processors 1104 to processing functions to respond to external events.

[0115] Figure 12is a flowchart 1200 of a method for managing a plurality of imaging devices deployed within a vehicle in accordance with certain aspects of the present disclosure. At block 1202 in the illustrated method, a change is determined to be needed in a first data protection configuration that protects first image data sent over a first data communication link. At block 1204 in the illustrated method, a second data protection configuration is determined to be used when second image data is sent from a first imaging device of the plurality of imaging devices to image processing circuitry over the first data communication link. The first data protection configuration or the second data protection configuration provides a partial integrity mode of data protection for image data frames sent over the first data communication link.

[0116] In some examples, the second data protection configuration provides a full integrity mode of data protection for image data frames sent over the first data communication link. The second data protection configuration can cause image data frames to be encrypted before being sent over the first data communication link.

[0117] In some examples, the partial integrity mode of data protection protects a first portion of image data frames sent over the first data communication link using a MAC. A second portion of image data frames sent over the first data communication link is not protected by the MAC used to protect the first portion of image data frames or is not protected by a different MAC. The partial integrity mode can be configured with a skip pattern that specifies image data rows in the first portion of image data frames to be protected by the MAC. The partial integrity mode can be configured with a skip pattern that specifies blocks of pixels in the first portion of image data frames to be protected by the MAC.

[0118] In one example, the second data protection configuration can be determined based on a computing capacity of the sensor management system, a power consumption budget or a thermal budget defined for the sensor management system. In one example, the second data protection configuration can be determined or provided to accommodate a change in an operating mode of the vehicle.

[0119] In one example, the change is determined to be needed in the first data protection configuration after detecting an increase in temperature or power consumption caused by the sensor management system. In one example, the change is determined to be needed in the first data protection configuration after detecting a high value scene or scene segment in the first image data.

[0120] In certain implementations, an AI agent is used to determine the change needed in the first data protection configuration. Initially, the AI agent can be trained using measurements of power or heat obtained for a plurality of operating conditions. The AI agent can be retrained using real-time system feedback. The change needed in the first data protection configuration can be determined based on inferences or predictions received from the AI agent.

[0121] Figure 13 is a diagram illustrating a first example of a hardware implementation for an apparatus 1300 employing a processing circuit 1302. The processing circuit is often having one or more microprocessors, microcontrollers, digital signal processors, sequencers, and / or state machines, often represented by processor 1316. The processing circuit 1302 can be implemented with a bus architecture, often represented by bus 1320. The bus 1320 can include any number of interconnecting buses and bridges, depending on the specific application of the processing circuit 1302 and the overall design constraints. The bus 1320 links together various circuits including the multiple processor 1316, modules or circuits 1304, 1306, and 1308, and the processor-readable storage medium 1318. Bus interface circuitry and / or modules 1314 can be provided to support communications over multiple serial links 1312. The bus 1320 can also link various other circuits such as timing sources, peripherals, voltage regulators, and power management circuits, which are well known in the art, and therefore, will not be further described.

[0122] The processor 1316 can be responsible for general processing, including the execution of software, code, and / or instructions stored on the processor-readable storage medium 1318. The processor-readable storage medium 1318 can include a non-transitory storage medium. The software, when executed by the processor 1316, causes the processing circuit 1302 to perform the various functions described above for any particular apparatus. The processor-readable storage medium can be used for storing data that is manipulated by the processor 1316 when executing software. The processing circuit 1302 further includes at least one of the modules 1304, 1306, and 1308. The modules 1304, 1306, and 1308 can be software modules running in the processor 1316, resident / stored in the processor-readable storage medium 1318, one or more hardware modules coupled to the processor 1316, or some combination thereof. The modules 1304, 1306, and 1308 can include microcontroller instructions, state machine configuration parameters, or some combination thereof.

[0123] In one configuration, the apparatus 1300 includes a module and / or circuit 1304 configured or adapted to configure and manage data protection of image data. Data protection can be provided in different modes, including: a full integrity mode including encryption; a full integrity mode without encryption; and a plurality of partial integrity modes. In some implementations, encryption is performed at a link layer and can be performed by the bus interface circuit and / or module 1314. Partial integrity modes can be able to be distinguished by a stride pattern used to identify lines or blocks of pixels of image data to be protected. The apparatus 1300 can also include a module and / or circuit 1306 configured or adapted to process image data. In one example, these modules and / or circuits 1306 can perform certain functions as part of an autonomous or assisted driving subsystem. The apparatus 1300 can also include a module and / or circuit 1308 configured or adapted to monitor a state and operation of a vehicle, a state and operation of a vehicle management system, and / or image data, as well as the module and / or circuit 1306 configured or adapted to process image data. The state monitoring module and / or circuit 1308 can generate events indicating a need for a change to data protection provided for one or more data exchanges, for transactions conducted over one or more data communication links, or for transactions involving two or more devices.

[0124] The apparatus 1300 can include means for determining a need for a change in a first data protection configuration that protects first image data sent over a first data communication link and means for determining a second data protection configuration to be used when second image data is sent from a first imaging device of a plurality of imaging devices to image processing circuitry over the first data communication link. The first data protection configuration or the second data protection configuration can provide a partial integrity mode of data protection for frames of image data sent over the first data communication link.

[0125] In some examples, the second data protection configuration provides a full integrity mode of data protection for frames of image data sent over the first data communication link. The second data protection configuration can cause the frames of image data to be encrypted before being sent over the first data communication link.

[0126] In some examples, a partial integrity mode of data protection uses a MAC to protect a first portion of image data frames sent over the first data communication link. A second portion of image data frames sent over the first data communication link is not protected by the MAC used to protect the first portion of image data frames or is not protected by a different MAC. The partial integrity mode can be configured with a stride pattern that specifies image data rows in the first portion of image data frames to be protected by the MAC. The partial integrity mode can be configured with a stride pattern that specifies blocks of pixels in the first portion of image data frames to be protected by the MAC.

[0127] In one example, the second data protection configuration can be determined based on a computing capacity of the sensor management system, a power consumption budget or a thermal budget defined for the sensor management system. In one example, the second data protection configuration can be determined or provided to accommodate a change in an operating mode of the vehicle.

[0128] In one example, a change in the first data protection configuration is determined to be needed after detecting an increase in temperature or power consumption caused by the sensor management system. In one example, a change in the first data protection configuration is determined to be needed after detecting a high value scene or scene segment in the first image data.

[0129] In certain implementations, an AI agent is used to determine a change in the first data protection configuration is needed. Initially, the AI agent can be trained using measurements of power or heat obtained for a plurality of operating conditions. The AI agent can be retrained using real-time system feedback. A change in the first data protection configuration needed can be determined based on an inference or prediction received from the AI agent.

[0130] In one aspect, processing circuitry 1302 is coupled to a plurality of imaging devices deployed within a vehicle using a plurality of data communication links. Processing circuitry 1302 can be configured to determine when a change in a first data protection configuration is needed, the first data protection configuration protecting first image data sent over a first data communication link; and determine or configure a second data protection configuration to be used when second image data is sent over the first data communication link from a first imaging device of the plurality of imaging devices to the image processing circuitry. The first data protection configuration or the second data protection configuration can provide a partial integrity mode of data protection for image data frames sent over the first data communication link.

[0131] In one example, the second data protection configuration provides a full integrity mode of data protection for image data frames sent over the first data communication link. In this example, the second data protection configuration causes image data frames to be encrypted before being sent over the first data communication link.

[0132] In certain examples, the partial integrity mode of data protection uses a MAC to protect a first portion of image data frames sent over the first data communication link, and a second portion of image data frames sent over the first data communication link is not protected by the same or a different MAC. In some instances, the first portion of image data frames can be defined by a stride pattern that specifies lines of image data to be protected by the MAC. In some instances, the first portion of image data frames can be defined by a stride pattern that specifies blocks of pixels in the image data to be protected by the MAC.

[0133] In some implementations, the processing circuit 1302 is further configured to determine the second data protection configuration based on a computing capacity of the sensor management system, a power consumption budget or a thermal budget defined for the sensor management system.

[0134] In some implementations, the processing circuit 1302 is further configured to determine the second data protection configuration to accommodate a change in an operating mode of the vehicle.

[0135] In some implementations, the processing circuit 1302 is further configured to determine that a change is needed in the first data protection configuration after detecting an increase in temperature or power consumption caused by the sensor management system.

[0136] In some implementations, the processing circuit 1302 is further configured to determine that a change is needed in the first data protection configuration after detecting a high value scene or scene segment in the first image data.

[0137] In certain implementations, the processing circuit 1302 is further configured to determine that a change is needed in the first data protection configuration using an AI agent. The processing circuit 1302 can be further configured to train the AI agent using measurements of power or heat obtained for a plurality of operating conditions, retrain the AI agent using real-time system feedback, and determine the change needed in the first data protection configuration based on inferences or predictions received from the AI agent.

[0138] The processor-readable storage medium 1318 can include code or instructions that, when executed by the processing circuit 1302, cause the processing circuit 1302 to determine that a change is needed in a first data protection configuration that protects first image data sent over a first data communication link, and determine a second data protection configuration to use when second image data is sent over the first data communication link from a first imaging device deployed within a vehicle to image processing circuitry. The first data protection configuration or the second data protection configuration provides a partial integrity mode of data protection for image data frames sent over the first data communication link.

[0139] In some examples, the second data protection configuration provides a full-integrity mode of data protection for image data frames sent over the first data communication link. The second data protection configuration can cause image data frames to be encrypted before being sent over the first data communication link.

[0140] In some examples, the partial-integrity mode of data protection uses a MAC to protect a first portion of image data frames sent over the first data communication link. A second portion of image data frames sent over the first data communication link is not protected by the MAC used to protect the first portion of image data frames or is not protected by a different MAC. The instructions can cause the processing circuitry 1302 to configure the partial-integrity mode with a skip pattern that specifies image data rows in the first portion of image data frames to be protected by the MAC. The instructions can cause the processing circuitry 1302 to configure the partial-integrity mode with a skip pattern that specifies blocks of pixels in the first portion of image data frames to be protected by the MAC.

[0141] In one example, the instructions can cause the processing circuitry 1302 to determine the second data protection configuration based on a computing capacity of the sensor management system, a power consumption budget or a thermal budget defined for the sensor management system. In one example, the instructions can cause the processing circuitry 1302 to determine or provide the second data protection configuration to accommodate a change in an operating mode of the vehicle.

[0142] In one example, the instructions can cause the processing circuitry 1302 to determine that a change is needed in the first data protection configuration after detecting an increase in temperature or power consumption caused by the sensor management system. In one example, the instructions can cause the processing circuitry 1302 to determine that a change is needed in the first data protection configuration after detecting a high-value scene or scene segment in the first image data.

[0143] In certain examples, the instructions can cause the processing circuitry 1302 to determine that a change is needed in the first data protection configuration using an AI agent. The code can further cause the processing circuitry 1302 to train the AI agent using measurements of power or heat obtained for a plurality of operating conditions, to retrain the AI agent using real-time system feedback, and to determine the change needed in the first data protection configuration based on inferences or predictions received from the AI agent.

[0144] Some implementation examples are described in the following numbered clauses:

[0145] 1. A sensor management system comprising: a plurality of imaging devices disposed within a vehicle; processing circuitry comprising a controller; and a plurality of data communication links each coupling at least one imaging device of the plurality of imaging devices with the processing circuitry, wherein the processing circuitry is configured to: determine a change in a first data protection configuration is needed, the first data protection configuration protecting first image data sent over a first data communication link; and determine a second data protection configuration to use when second image data is sent from a first imaging device of the plurality of imaging devices to the image processing circuitry over the first data communication link, wherein the first data protection configuration or the second data protection configuration provides a partial integrity mode of data protection for image data frames sent over the first data communication link.

[0146] 2. The sensor management system of clause 1, wherein the second data protection configuration provides a full integrity mode of data protection for the image data frames sent over the first data communication link.

[0147] 3. The sensor management system of clause 2, wherein the second data protection configuration causes the image data frames to be encrypted before being sent over the first data communication link.

[0148] 4. The sensor management system of any one of clauses 1-3, wherein the partial integrity mode of data protection uses a message authentication code (MAC) to protect a first portion of the image data frames sent over the first data communication link, and wherein a second portion of the frames.

[0149] 5. The sensor management system of clause 4, wherein the first portion of the image data frames is defined by a stride pattern that specifies lines of image data to be protected by the MAC.

[0150] 6. The sensor management system of clause 4, wherein the first portion of the image data frames is defined by a stride pattern that specifies blocks of pixels in image data to be protected by the MAC.

[0151] 7. The sensor management system of any one of clauses 1-6, wherein the processing circuitry is further configured to: determine the second data protection configuration based on a computational capacity of the sensor management system, a power consumption budget or a thermal budget defined for the sensor management system, or determine the second data protection configuration to accommodate a change in an operating mode of the vehicle.

[0152] 8. The sensor management system of any of clauses 1-7, wherein the processing circuit is further configured to determine that the change is needed in the first data protection configuration after detecting an increase in temperature or power consumption caused by the sensor management system or after detecting a high-value photo scene or scene segment in the first image data.

[0153] 9. The sensor management system of any of clauses 1-8, wherein the processing circuit is further configured to determine that the change is needed in the first data protection configuration using an artificial intelligence agent.

[0154] 10. The sensor management system of clause 9, wherein the processing circuit is further configured to train the artificial intelligence agent using measurements of power or heat obtained for a plurality of operating conditions, retrain the artificial intelligence agent using real-time system feedback, and determine the change needed in the first data protection configuration based on inferences or predictions received from the artificial intelligence agent.

[0155] 10. The sensor management system of any of clauses 1-10, wherein the processing circuit is further configured to determine that the change is needed in the first data protection configuration after detecting a high-value photo scene or scene segment in the first image data.

[0156] 11. A method for managing a plurality of imaging devices deployed within a vehicle, the method comprising: determining that a change is needed in a first data protection configuration that protects first image data transmitted over a first data communication link; and determining a second data protection configuration to use when second image data is transmitted from a first imaging device of the plurality of imaging devices to image processing circuitry over the first data communication link, wherein the first data protection configuration or the second data protection configuration provides a partial integrity mode of data protection for image data frames transmitted over the first data communication link.

[0157] 12. The method of clause 11, wherein the second data protection configuration provides a full integrity mode of data protection for the image data frames transmitted over the first data communication link.

[0158] 13. The method of clause 12, wherein the second data protection configuration causes the image data frames to be encrypted before being transmitted over the first data communication link.

[0159] 14. The method of any of clauses 11-13, wherein the partial integrity mode of data protection uses a message authentication code (MAC) to protect a first portion of the image data frames sent over the first data communication link, and

[0160] wherein a second portion of the image data frames sent over the first data communication link is not protected by a MAC.

[0161] 15. The method of clause 14, further comprising defining a stride pattern that specifies image data lines in the first portion of the image data frames to be protected by the MAC.

[0162] 16. The method of clause 14, further comprising defining a stride pattern that specifies blocks of pixels in the first portion of the image data frames to be protected by the MAC.

[0163] 17. The method of any of clauses 11-16, further comprising determining the second data protection configuration based on a computing capacity of a sensor management system, a power consumption budget or a thermal budget defined for the sensor management system, or determining the second data protection configuration to accommodate a change in an operating mode of the vehicle.

[0164] 18. The method of any of clauses 11-17, further comprising determining that the change is needed in the first data protection configuration after detecting an increase in temperature or power consumption or after detecting a high-value scene or scene segment in the first image data.

[0165] 19. The method of any of clauses 11-18, further comprising using an artificial intelligence agent to determine that the change is needed in the first data protection configuration.

[0166] 20. The method of clause 19, further comprising training the artificial intelligence agent using measurements of power or heat obtained for a plurality of operating conditions, retraining the artificial intelligence agent using real-time system feedback, and determining the change needed in the first data protection configuration based on inferences or predictions received from the artificial intelligence agent.

[0167] 21. A processor-readable storage medium comprising code that, when executed by processing circuitry, causes the processing circuitry to determine that a change is needed in a first data protection configuration that protects first image data sent over a first data communication link and determine a second data protection configuration to use when second image data is sent from a first imaging device deployed within a vehicle to image processing circuitry over the first data communication link, wherein the first data protection configuration or the second data protection configuration provides a partial integrity mode of data protection for image data frames sent over the first data communication link.

[0168] 22. The processor-readable storage medium of clause 21, wherein the second data protection configuration provides a full integrity mode of data protection for the image data frames sent over the first data communication link.

[0169] 23. The processor-readable storage medium of clause 22, wherein the second data protection configuration causes the image data frames to be encrypted before being sent over the first data communication link.

[0170] 24. The processor-readable storage medium of any one of clauses 21 to 23, wherein the partial integrity mode of data protection uses a message authentication code (MAC) to protect a first portion of the image data frames sent over the first data communication link, and wherein a second portion of the image data frames sent over the first data communication link is not protected by a MAC.

[0171] 25. The processor-readable storage medium of clause 24, wherein the code causes the processing circuitry to define a stride pattern that specifies image data rows in the first portion of the image data frames to be protected by the MAC.

[0172] 26. The processor-readable storage medium of clause 24, wherein the code causes the processing circuitry to define a stride pattern that specifies blocks of pixels in the first portion of the image data frames to be protected by the MAC.

[0173] 27. The processor-readable storage medium of any one of clauses 21 to 26, wherein the code causes the processing circuitry to:

[0174] determine the second data protection configuration based on a computing capacity of a sensor management system, a power consumption budget or a thermal budget defined for the sensor management system, or determine the second data protection configuration to accommodate a change in an operating mode of the vehicle.

[0175] 28. The processor-readable storage medium of any of clauses 21-27, wherein the code causes the processing circuitry to:

[0176] determine that the change is needed in the first data protection configuration after detecting an increase in temperature or power consumption or after detecting a high value scene or scene segment in the first image data.

[0177] 29. The processor-readable storage medium of any of clauses 21-28, wherein the code causes the processing circuitry to:

[0178] determine that the change is needed in the first data protection configuration using an artificial intelligence agent.

[0179] 30. The processor-readable storage medium of clause 29, wherein the code causes the processing circuitry to train the artificial intelligence agent using measurements of power or heat obtained for a plurality of operating conditions, retrain the artificial intelligence agent using real-time system feedback, and determine the change needed in the first data protection configuration based on inferences or predictions received from the artificial intelligence agent.

[0180] It should be understood that the particular order or hierarchy of steps in processes disclosed is an example. It should be appreciated that depending on the implementation, the particular order or hierarchy of steps can be re-arranged or reordered. Furthermore, some steps can be optional. The accompanying method claims present elements of the various steps in a sample order, and are not meant to be limited to the specific order or hierarchy presented.

[0181] The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects. Thus, the claims are not intended to be limited to the aspects shown herein, but is to be accorded the full scope consistent with the language claims, wherein reference to an element in the singular is not intended to mean "one and only one" unless specifically so stated, but rather "one or more." Unless specifically stated otherwise, the term "some" refers to one or more. All structural and functional equivalents to aspects described herein that are known or later come to be known to those of ordinary skill in the art are expressly incorporated herein by reference and intended to be encompassed by the claims. Moreover, any combination of the aspects described herein can be implemented with no intent or purpose to cause a withdrawal from the scope of the claims. Further, any claims set forth in this application are not intended to cause a withdrawal of any other patent application filed by the Applicant or a 3rd party, named inventor hereof that claims common priority. No disclaimer hereof is intended. Nothing herein is to be construed as a disclaimer of the statutory patent term reset under 35 U.S.C. 21 7. The embodiments described herein can be implemented in software and / or hardware. The embodiments can be implemented in firmware and / or software running on a digital signal processor, microprocessor, microprocessing unit, programmable, or general purpose processor.

Claims

1. A sensor management system, the sensor management system comprising: a plurality of imaging devices, the plurality of imaging devices being disposed within a vehicle; processing circuitry, the processing circuitry comprising a controller; and a plurality of data communication links, each data communication link coupling at least one imaging device of the plurality of imaging devices with the processing circuitry, wherein the processing circuitry is configured to: determine that a change is needed in a first data protection configuration, the first data protection configuration protecting first image data sent over a first data communication link; and determine a second data protection configuration to use when second image data is sent from a first imaging device of the plurality of imaging devices to the image processing circuitry over the first data communication link, wherein the first data protection configuration or the second data protection configuration provides a partial integrity mode of data protection for image data frames sent over the first data communication link.

2. The sensor management system of claim 1, wherein the second data protection configuration provides a full integrity mode of data protection for the image data frames sent over the first data communication link.

3. The sensor management system of claim 2, wherein the second data protection configuration causes the image data frames to be encrypted before being sent over the first data communication link.

4. The sensor management system of claim 1, wherein the partial integrity mode of data protection protects a first portion of the image data frames sent over the first data communication link using a message authentication code (MAC), and wherein a second portion of the image data frames sent over the first data communication link is not protected by a MAC.

5. The sensor management system of claim 4, wherein the first portion of the image data frames is defined by a stride pattern that specifies lines of image data to be protected by the MAC.

6. The sensor management system of claim 4, wherein the first portion of the image data frames is defined by a stride pattern that specifies blocks of pixels in image data to be protected by the MAC.

7. The sensor management system of claim 1, wherein the processing circuitry is further configured to: determine the second data protection configuration based on a computational capacity of the sensor management system, a power consumption budget or a thermal budget defined for the sensor management system, or determine the second data protection configuration to accommodate a change in an operating mode of the vehicle.

8. The sensor management system of claim 1, wherein the processing circuitry is further configured to: determine that the change is needed in the first data protection configuration after detecting an increase in temperature or power consumption caused by the sensor management system or after detecting a high value scene or scene segment in the first image data.

9. The sensor management system of claim 1, wherein the processing circuitry is further configured to: determine that the change is needed in the first data protection configuration using an artificial intelligence agent. ​ ​ 10. The sensor management system of claim 9, wherein the processing circuit is further configured to: train the artificial intelligence agent using measurements of power or heat obtained for a plurality of operating conditions; retrain the artificial intelligence agent using real-time system feedback; and determine that the change is needed in the first data protection configuration based on inferences or predictions received from the artificial intelligence agent.

11. A method for managing a plurality of imaging devices deployed within a vehicle, the method comprising: determining that a change is needed in a first data protection configuration, the first data protection configuration protecting first image data transmitted over a first data communication link; and determining a second data protection configuration to use when second image data is transmitted from a first imaging device of the plurality of imaging devices to image processing circuitry over the first data communication link, wherein the first data protection configuration or the second data protection configuration provides a partial integrity mode of data protection for image data frames transmitted over the first data communication link.

12. The method of claim 11, wherein the second data protection configuration provides a full integrity mode of data protection for the image data frames transmitted over the first data communication link.

13. The method of claim 12, wherein the second data protection configuration causes the image data frames to be encrypted before being transmitted over the first data communication link.

14. The method of claim 11, wherein the partial integrity mode of data protection protects a first portion of the image data frames transmitted over the first data communication link using a message authentication code (MAC), and wherein a second portion of the image data frames transmitted over the first data communication link is not protected by a MAC.

15. The method of claim 14, further comprising: defining a striding pattern that specifies image data lines in the first portion of the image data frames to be protected by the MAC.

16. The method of claim 14, further comprising: defining a striding pattern that specifies blocks of pixels in the first portion of the image data frames to be protected by the MAC.

17. The method of claim 11, further comprising: determining the second data protection configuration based on a computing capacity of a sensor management system, a power consumption budget or a thermal budget defined for the sensor management system, or determining the second data protection configuration to accommodate a change in an operating mode of the vehicle.

18. The method of claim 11, further comprising: determining that the change is needed in the first data protection configuration after detecting an increase in temperature or power consumption or after detecting a high value scene or scene segment in the first image data.

19. The method of claim 11, further comprising: determining that the change is needed in the first data protection configuration using an artificial intelligence agent. ​ 20. The method of claim 19, further comprising: training the artificial intelligence agent using measurements of power or heat obtained for a plurality of operating conditions; retraining the artificial intelligence agent using real-time system feedback; and determining the change needed in the first data protection configuration based on inferences or predictions received from the artificial intelligence agent.

21. A processor-readable storage medium comprising code that, when executed by processing circuitry, causes the processing circuitry to: determine that a change is needed in a first data protection configuration that protects first image data sent over a first data communication link; and determine a second data protection configuration to use when second image data is sent from a first imaging device deployed within a vehicle to image processing circuitry over the first data communication link, wherein the first data protection configuration or the second data protection configuration provides a partial integrity mode of data protection for image data frames sent over the first data communication link.

22. The processor-readable storage medium of claim 21, wherein the second data protection configuration provides a full integrity mode of data protection for the image data frames sent over the first data communication link.

23. The processor-readable storage medium of claim 22, wherein the second data protection configuration causes the image data frames to be encrypted before being sent over the first data communication link.

24. The processor-readable storage medium of claim 21, wherein the partial integrity mode of data protection protects a first portion of the image data frames sent over the first data communication link using a message authentication code (MAC), and wherein a second portion of the image data frames sent over the first data communication link is not protected by a MAC.

25. The processor-readable storage medium of claim 24, wherein the code causes the processing circuitry to: define a striding pattern that specifies image data lines in the first portion of the image data frames to be protected by the MAC.

26. The processor-readable storage medium of claim 24, wherein the code causes the processing circuitry to: define a striding pattern that specifies blocks of pixels in the first portion of the image data frames to be protected by the MAC.

27. The processor-readable storage medium of claim 21, wherein the code causes the processing circuitry to: determine the second data protection configuration based on a computational capacity of a sensor management system, a power consumption budget or a thermal budget defined for the sensor management system, or determine the second data protection configuration to accommodate a change in an operating mode of the vehicle.

28. The processor-readable storage medium of claim 21, wherein the code causes the processing circuitry to: ​ determining that the change is needed in the first data protection configuration after detecting an increase in temperature or power consumption or after detecting a high value scene or scene segment in the first image data.

29. The processor-readable storage medium of claim 21, wherein the code causes the processing circuitry to: determine that the change is needed in the first data protection configuration using an artificial intelligence agent.

30. The processor-readable storage medium of claim 29, wherein the code causes the processing circuitry to: train the artificial intelligence agent using measurements of power or heat obtained for a plurality of operating conditions; retrain the artificial intelligence agent using real-time system feedback; and determine the change needed in the first data protection configuration based on inferences or predictions received from the artificial intelligence agent.

31. The processor-readable storage medium of claim 21, wherein the code causes the processing circuitry to: determine that the change is needed in the first data protection configuration using a machine learning model.

32. The processor-readable storage medium of claim 31, wherein the code causes the processing circuitry to: train the machine learning model using measurements of power or heat obtained for a plurality of operating conditions; retrain the machine learning model using real-time system feedback; and determine the change needed in the first data protection configuration based on inferences or predictions received from the machine learning model.

Images