Method and apparatus for image frame freeze detection

Abstract

Methods, apparatus, systems, and articles of manufacture are described for image frame freeze detection. An example hardware accelerator (204) includes a core logic circuit (208) that generates second image data based on first image data associated with a first image frame, the second image data corresponding to processed image data, transformed image data, or one or more image data statistics; a load / store engine (LSE) (206) coupled to the core logic circuit, the LSE determining a first CRC value based on the second image data obtained from the core logic circuit, and a first interface (230) coupled to a second interface (224), the second interface coupled to a memory (212), the first interface transmitting the first CRC value obtained from the memory to a host device (214A; 214B).

Description

Technical Field

[0001] This specification relates generally to image processing, and more specifically to methods and apparatus for detecting image frame freeze. Background Technology

[0002] Safety protocols are used to ensure the safety of electrical and / or electronic systems. For example, the International Organization for Standardization (ISO) 26262 is an international standard for the functional safety of electrical and / or electronic systems in automobiles. Such safety protocols analyze the risks associated with electronic failures (e.g., a combination of the frequency and severity of injuries). Failures corresponding to electronic equipment can be random or systematic. Random failures typically correspond to hardware-related permanent or temporary failures due to the loss of function of system components. Systematic failures typically correspond to design flaws, incorrect specifications, and / or non-functional errors in software. Such safety protocols can analyze electrical risks associated with signal processing subsystems (e.g., visual imaging subsystems) that can process signals to improve vehicle safety. Attached Figure Description

[0003] Figure 1 An example implementation of a data processing pipeline associated with a vehicle imaging system is described.

[0004] Figure 2 An example implementation of a system-on-a-chip (SOC) including a hardware accelerator is described, which includes a load / store engine and a kernel for detecting freezes of image frames associated with a vehicle imaging system.

[0005] Figure 3 Depicting and Figure 2 Example workflow associated with the SOC.

[0006] Figure 4 Depicting Figure 2 Example implementation of a hardware accelerator.

[0007] Figure 5 Depicting and Figure 4 An example workflow associated with an example implementation of a hardware accelerator.

[0008] Figure 6 Depicting and Figure 4 Another example workflow associated with the example implementation of the hardware accelerator.

[0009] Figure 7 Depicting Figure 2 An example implementation of the load / store engine.

[0010] Figure 8 Depicting and Figure 7Example workflow associated with an example implementation of the load / store engine.

[0011] Figure 9 Depicting and Figure 7 Another example workflow associated with the example implementation of the load / store engine.

[0012] Figure 10 Depicting Figure 2 Another example implementation of the load / store engine.

[0013] Figure 11 Depicting Figure 2 Example implementation of the core.

[0014] Figure 12 Depicting what can be made by Figure 2 A schematic diagram of example image data processed by SOC.

[0015] Figure 13 Depicting what can be made by Figure 2 Example transformation workflow performed by the SOC.

[0016] Figure 14 It means that it can be generated by Figure 2 SOC, Figure 10 Example implementations of the load / store engine and / or Figure 11 A flowchart of example machine-readable instructions executed by an example implementation of the kernel to detect frame freeze conditions.

[0017] Figure 15 It means that it can be generated by Figure 2 SOC and / or Figure 11 The flowchart illustrates an example machine-readable instruction executed by an example implementation of the kernel to detect frame freeze conditions based on different pixel processing operations.

[0018] Figure 16 It is constructed to execute Figures 14-15 Example machine-readable instructions for implementation Figure 10 A block diagram of an example processing platform for an example implementation of the load / store engine.

[0019] Figure 17 It is constructed to execute Figure 15 Example machine-readable instructions to implement Figure 11 A block diagram of an example processing platform for an example implementation of the core.

[0020] Descriptors such as "first," "second," and "third" are used to identify multiple elements or components that can be individually referenced. Unless otherwise stated or understood based on their context of use, these descriptors do not assign any priority or chronological order; they are merely labels used to refer to multiple elements or components separately for ease of understanding of the described examples. In some examples, the descriptor "first" may be used to refer to an element in a specific embodiment, while different descriptors, such as "second" or "third," may be used in the claims to refer to the same element. In such cases, these descriptors are only used for convenience in referencing multiple elements or components. Detailed Implementation

[0021] With technological advancements, the complexity of computing and / or electronic devices has increased significantly. Such devices comprise hardware, software, and / or firmware that perform one or more specific functions. If the hardware, software, and / or firmware malfunctions, such devices may be unable to perform one or more specific functions or may perform poorly when doing so. This inability to perform and / or poor performance can affect the outcomes of the related systems. For example, an autonomous driving computing device that facilitates the autonomous operation of vehicles (e.g., air-based vehicles, land-based vehicles, etc.) can acquire raw image data from sensors and transmit the raw image data to a visual imaging subsystem (VISS) (e.g., a visual image processing subsystem) to process the raw image data and / or render images based on the raw image data. Once the autonomous driving computing device receives the rendered image, it can make navigation decisions based on further analysis of the rendered image. To ensure appropriate navigation decisions are made, safety protocols (e.g., safety hooks) can be implemented to ensure the proper functioning of electrical and / or electronic systems (e.g., to mitigate risks) and the availability of appropriate data for the decision-making process. In some cases, safety protocols can be used in conjunction with a VISS.

[0022] Due to the efficiency of such subsystems, VISS can be combined with a processor to execute signal processing protocols. For example, VISS can process input signals (e.g., data from an image source) and transmit the processed input signals to the processor. In some cases, the processor can detect frame freeze (e.g., image frame freeze) conditions at the image data source (e.g., camera). For example, frame freeze may occur due to camera damage, destruction, etc., and the erroneous generation of an output image that is substantially similar to the previously generated output image.

[0023] In some cases, the processor can detect frame freeze conditions based on delivery or transmission failures, where the same image data is repeatedly transmitted to the VISS. In other cases, the processor can detect frame freeze conditions based on VISS processing failure outputs, where the current image frame from the VISS does not reflect the current condition captured by the image data source. In this case, the VISS may generate erroneous outputs due to errors in the hardware, software, and / or firmware associated with the VISS causing the generation of identical or substantially similar rendered images.

[0024] In some cases, hardware accelerators can be used to detect frame freezes. However, such hardware accelerators are often too far from the image data source in the data processing pipeline (e.g., too far in terms of latency, processing distance, etc.) to detect frame freezes within the desired timeframe. Therefore, a hardware accelerator may not detect frame freezes during the manageable timeframe of a vehicle or other equipment.

[0025] In some cases, a processor communicating with a hardware accelerator can detect frame freezes in software by comparing the current image frame with the previous image frame pixel by pixel. However, due to various factors (e.g., sensor noise, changes in the external environment, etc.), individual pixels can flip between image frames, so even if a frame freeze occurs, the processor may generate a false negative. Furthermore, this comparison leads to computational overhead and reduced read bandwidth at the interface between the VISS and the image data source. The computational resources required to perform the comparison in software may delay the reception of new image frames, thus increasing computational overhead and reducing read bandwidth. Therefore, this comparison increases latency in frame freeze detection and reduces the security of the relevant system.

[0026] The examples described herein describe the detection of image frame freeze. In some examples, at least one processor detects image frame freeze by comparing raw image data of two different image frames (e.g., two different image frames captured sequentially, in order, etc.). For example, the raw image data may correspond to image data generated and / or otherwise acquired from one or more cameras, one or more sensors, etc. In some examples, at least one processor detects image frame freeze by comparing processed image data of two different image frames (e.g., two different image frames captured sequentially, in order, etc.). For example, at least one processor may detect image frame freeze by comparing a first image frame processed by VISS and a second image frame processed by VISS.

[0027] In some examples, at least one processor detects image frame freeze by comparing signatures associated with two different sets of raw image data. For example, at least one processor may compare (1) a first signature (e.g., a first cyclic redundancy check (CRC) signature) generated based on a first set of raw image data from the camera with (2) a second signature (e.g., a second CRC signature) generated based on a second set of raw image data from the camera. In some examples, at least one processor detects image frame freeze by comparing signatures associated with two different sets of processed image data (e.g., processed image data based on raw image data processed by VISS). For example, at least one processor may compare (1) a first signature (e.g., a first CRC signature) generated based on a first set of processed image data processed by VISS with (2) a second signature (e.g., a second CRC signature) generated based on a second set of processed image data from VISS.

[0028] In some examples, image frame comparison logic in the beginning or front-end portion of the data processing pipeline detects image frame freezes. For instance, a hardware accelerator may acquire image data from an image data source via an image data interface. Example hardware accelerators may execute signature capture algorithms or schemes, such as CRC signature capture on one or more output channels of the hardware accelerator. Example hardware accelerators may execute signature capture algorithms to generate unique signatures that can be retained until a new image frame is processed.

[0029] In some examples, the hardware accelerator calculates and / or otherwise determines image data statistics based on data transmitted from the output channel of the hardware accelerator. In some examples, at least one processor can detect image frame freezes by comparing signatures associated with one or more image data statistics. For example, at least one processor can (1) generate a first signature (e.g., a first CRC signature) based on a first image data statistic associated with a first data transmitted from the output channel of the hardware accelerator and (2) generate a second signature (e.g., a second CRC signature) based on a second image data statistic associated with a second data transmitted from the output channel.

[0030] In some examples, the hardware accelerator transforms image data (e.g., raw image data, processed image data, etc.) by performing one or more transformation operations (e.g., neighborhood-aware pixel transformation, census transform, or census transition, etc.). In some examples, at least one processor can detect image frame freezes by comparing signatures associated with the transformed image data. For example, at least one processor can (1) generate a first signature (e.g., a first CRC signature) based on a first transformed image data transmitted through the output channel of the hardware accelerator and (2) generate a second signature (e.g., a second CRC signature) based on a second transformed image data transmitted through the output channel.

[0031] Advantageously, compared to alternative techniques, the example hardware accelerator can increase the processing bandwidth of one or more processors and reduce latency in detecting frame freezes by comparing CRC signatures (e.g., 32-bit CRC signatures) instead of the entire image frame (e.g., 2 megabytes (MB), 3 MB, etc.). Advantageously, compared to alternative techniques, the example hardware accelerator can reduce or, in some cases, eliminate the occurrence of false negatives. For example, the hardware accelerator can reduce and / or otherwise eliminate false negatives by comparing CRC signatures generated based on one or more image data statistics, transformed image data, etc., and / or combinations thereof.

[0032] Figure 1 An example implementation of a data processing pipeline 100 associated with an example vehicle imaging system 102 is depicted. The vehicle imaging system 102 may correspond to one or more camera systems for a vehicle (e.g., a car, aircraft, ship, etc.), including a front-view camera system, a rear-view camera system (e.g., a backup camera system), and / or a side-view camera system. A vehicle may use one or more camera systems to perform vehicle maneuvers, such as aerial maneuvers (e.g., increasing or decreasing altitude and / or speed, turning an aircraft, etc.), land maneuvers (e.g., moving forward on a road surface, turning on a road surface, etc.). In some examples, the vehicle imaging system 102 is included in and / or otherwise associated with an autonomous vehicle. For example, an autonomous vehicle may use the vehicle imaging system 102 to adjust the vehicle's orientation and / or speed relative to one or more objects or vehicles in front of or to the side of the vehicle. In some examples, an autonomous vehicle uses a backup camera system to perform object-related vehicle maneuvers (e.g., autonomous vehicle maneuvers), such as reversing toward another object or vehicle.

[0033] Figure 1The vehicle imaging system 102 includes an example image data source 104 coupled to a data processing pipeline 100 to capture one or more images and transmit the captured images (one or more) to the data processing pipeline 100 for processing. The image data source 104 may be a single camera (e.g., a monocular camera, a video camera, etc.) to capture images of the vehicle's surroundings (e.g., a single image, one or more image frames, one or more video frames, etc.) or a real-time feed. In some examples, the image data source 104 corresponds to two or more cameras to capture images of the vehicle's surroundings (e.g., stereo images) or a real-time feed. Alternatively, the image data source 104 may correspond to any other image capture device, such as a thermal imaging camera, an infrared camera, etc.

[0034] Figure 1 The vehicle imaging system 102 includes a data processing pipeline 100 to transform, convert, and / or otherwise process image frames (e.g., raw image data, image data, etc.) from an image data source 104 to generate first example output image data 106 and / or second example output image data 108. For example, the first output image data 106 may be transmitted to an example vehicle electronic control unit (ECU) 110 to influence the operation of the vehicle. In other examples, the second output image data 108 may be transmitted to an example display panel (e.g., a vehicle display panel) 112 to display rendered images on one or more displays or monitors included in and / or otherwise associated with the vehicle.

[0035] Figure 1 The data processing pipeline 100 includes an example image pipeline 114 coupled to the image data source 104. Figure 1 In this context, the image tube 114 may correspond to a hardware circuit (e.g., discrete and / or integrated analog and / or digital circuits, field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), field-programmable logic devices (FPLDs), application-specific integrated circuits (ASICs), comparators, operational amplifiers (op-amps), logic circuits, etc.) configured to perform corresponding operations without executing software or firmware, but other structures are equally suitable.

[0036] The data processing pipeline 100 includes an image pipe 114 to process example input image data 116 from an image data source 104 to generate an example processed image frame 117. For example, the input image data 116 may correspond to image data captured by the image data source 104, such as pixel data. In some examples, the image pipe 114 acquires the input image data 116 from the image data source 104 and extracts data or information of interest from the input image data 116. For example, the image pipe 114 may determine values ​​associated with pixels of an image frame captured by the image data source 104 based on the input image data 116. In such examples, the input image data 116 may correspond to an image frame, such as an RGB image or an RGB image frame, where pixel values ​​correspond to the red, green, and blue components that define each pixel. In other examples, the input image data 116 may include values ​​corresponding to image characteristics such as brightness, hue, and / or saturation. In yet another example, the input image data 116 may include a set of image data parameters with corresponding values, wherein the image data parameters may include the number of rows (e.g., image frame height), the number of columns (e.g., image frame width), the number of bands, bits per pixel (bpp), file type, etc. Additionally or alternatively, the input image data 116 from the image data source 104 may be any other image format (such as Portable Bitmap (PBM), Portable Pixelmap (PPM), etc.) and / or may be represented using any other data representation (such as binary values, hexadecimal values, ASCII characters, etc.) and / or combinations thereof.

[0037] In some examples, image tube 114 can determine one or more parameters (e.g., diagnostic parameters, image data parameters, image data statistics, image tube statistics, etc.) associated with the input image data 116. For example, image tube 114 can perform a signature capture algorithm or scheme (such as Cyclic Redundancy Check (CRC) signature capture) on one or more output channels of image tube 114 to generate a unique signature that can be retained until a new image frame is processed. In such examples, image tube 114 can transmit one or more CRC signatures to at least one of example analysis tube 118 or example display controller 120 for further processing. For example, analysis tube 118 can detect and / or otherwise determine frame freeze status based on one or more CRC signatures.

[0038] Figure 1 The data processing pipeline 100 includes an analysis pipeline 118 coupled to the image pipeline 114 and the vehicle ECU 110. Figure 1In this context, the analyzer tube 118 may correspond to one or more programmable processors, one or more programmable controllers, and / or one or more hardware circuits configured to perform corresponding operations without executing software or firmware, but other configurations are equally suitable.

[0039] The data processing pipeline 100 includes an analysis pipe 118 to acquire processed image frames 117 from the image pipe 114 and perform additional computational tasks on the processed image frames 117, such as computer vision tasks, deep learning tasks (e.g., convolutional neural network (CNN) tasks), etc. In some examples, the analysis pipe 118 generates first output image data 106 based on the processed image frames 117 from the image pipe 114. For example, the first output image data 106 may include and / or otherwise correspond to the processed image frames 117. In other examples, the first output image data 106 may include and / or otherwise correspond to alarms, notifications, etc., indicating a frame freeze status associated with the input image data 116. In yet another example, the first output image data 106 may include and / or otherwise correspond to one or more parameters (e.g., diagnostic parameters, image data statistics, image pipe statistics, etc.) associated with the input image data 116.

[0040] Figure 1 The vehicle ECU 110 is coupled to the analysis tube 118. In some examples, the vehicle ECU 110 generates vehicle commands (e.g., adjusting the steering wheel position, adjusting the vehicle speed, etc.) based on the first output image data 106. For example, the vehicle ECU 110 may instruct one or more systems of the vehicle to take appropriate action in response to determining that a frame freeze condition has been detected. Figure 1 In this context, the vehicle ECU 110 may correspond to an embedded controller or system that monitors and / or controls one or more components (e.g., electrical, mechanical, etc.), electrical subsystems, electrical systems, etc. of a vehicle.

[0041] Figure 1 The data processing pipeline 100 includes a display controller 120 coupled to the image tube 114 and the display panel 112. Figure 1 In this context, the display controller 120 may correspond to one or more programmable processors, one or more programmable controllers, and / or one or more hardware circuits configured to perform corresponding operations without executing software or firmware, but other configurations are equally suitable.

[0042] In some examples, display controller 120 generates second output image data 108 based on data from image tube 114. For example, second output image data 108 may correspond to an image rendered by display controller 120 for display on display panel 112. For example, display panel 112 may correspond to a display device (e.g., a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, a liquid crystal display (LCD), a side-panel switching (IPS) display, a touchscreen, etc.). Alternatively, vehicle imaging system 102 may include more than one display panel 112. In other examples, display controller 120 may determine that second output image data 108 includes alarms, notifications, etc., indicating a frame freeze condition associated with input image data 116. In such examples, display controller 120 may instruct display panel 112 not to display images or video to the user until the frame freeze condition has been resolved.

[0043] Advantageously, by detecting frame freeze conditions in image tube 114 compared to another point in data processing pipeline 100 (e.g., analysis tube 118, display controller 120, etc.), the delay can be reduced from the first example detection time 122 to the second example detection time 124. Figure 1 In this context, the first detection time 122 can correspond to a first duration from the start of capturing an image frame from the image data source 104 to the end of displaying the captured image frame on the display panel 112. Figure 1 In this context, the second detection time 124 can correspond to a second duration that is shorter than the first duration, wherein the second duration begins when an image frame is captured and ends when the image tube 114 finishes processing the captured image frame. For example, compared to alternative techniques, Figure 1 The data processing pipeline 100 can detect frame freeze conditions associated with captured image frames in less time and with less computational resources.

[0044] Figure 2An example frame freeze detection system 200 is depicted for detecting freezes in image frames associated with a vehicle imaging system. The frame freeze detection system 200 includes an example implementation of a system-on-chip (SOC) 202, which includes an example implementation of a hardware accelerator (HWA) 204, which includes a load / store engine (LSE) 206 and one or more cores 208. The LSE 206 performs and / or otherwise performs data unpacking and packing for the cores 208 and provides other interface data processing functions. For example, the LSE 206 may correspond to LSE logic circuitry, LSE hardware, or hardware logic, etc. The cores 208 execute one or more algorithms or core logic on the data processed by the LSE 206. For example, the cores 208 may include one or more cores and / or otherwise correspond to one or more core logic circuitry, core hardware, or hardware logic, etc., to execute one or more algorithms. Figure 2 A System-on-Chip (SoC) 202 can correspond to one or more integrated circuits that are incorporated in a compact form into components of a computer or other electronic system. For example, an SoC 202 can correspond to a combination of a programmable processor, hardware logic, and hardware peripherals and interfaces.

[0045] exist Figure 2 In the illustrated example, SOC 202 from Figure 1 Image data source 104 acquisition Figure 1 The input image data 116. In some examples, the SOC 202 instructs the example display panel 210 to display an image rendered by the display controller 228 based on the input image data 116. In some examples, the SOC 202 stores the output image data in the example double data rate (DDR) memory 212. For example, the DDR memory 212 may be DDR, DDR2, DDR3, DDR4, mobile DDR (mDDR), etc. In some examples, the SOC 202 generates an alarm to one or more example host devices 214A-B indicating that a frame freeze condition has been detected.

[0046] exist Figure 2 In the example shown, SOC 202 includes an example image data interface (I / F) 216 coupled to HWA 204 and an example data movement structure 218. In some examples, image data interface 216 acquires input image data 116 from image data source 104. Image data interface 216 may correspond to one or more logic circuits to implement a bidirectional interface for acquiring input image data 116. For example, image data interface 216 may correspond to one or more logic circuits that facilitate operation and / or other implementation of a two-wire circuit (I2C) bus, a universal asynchronous receiver-transmitter (UART) bus, a serial peripheral interface (SPI) bus, etc.

[0047] exist Figure 2 In the illustrated example, SOC 202 includes HWA 204 coupled to image data interface 216 and data movement fabric 218. HWA 204 may correspond to one or more hardware accelerators. For example, HWA 204 may include and / or otherwise correspond to one or more hardware circuits, hardware logic, hardware-implemented state machines, and / or combinations thereof.

[0048] In some examples, HWA 204 acquires image data (e.g., image data parameters, pixel values, etc.) from image data interface 216 and determines CRC signatures (e.g., unique CRC signatures) for one or more output channels (e.g., output data channels) of HWA 204. For example, LSE 206 can acquire image data and transmit the image data to core 208. Core 208 can calculate and / or otherwise determine image data statistics based on the image data. For example, core 208 can determine one or more image data statistics for a set of pixel values, such as the minimum pixel value, maximum pixel value, sum of pixel values, sum of squared values, etc. In such examples, core 208 can determine one or more image data statistics based on performing a neighborhood-aware pixel transformation (e.g., census transformation, census conversion, etc.) on a set of pixel values ​​or other image-related data. For example, core 208 can determine one or more image data statistics based on the following: Figure 12 The image data is transformed into pixel data (e.g., pixel data from a census transformation), as follows: Figure 13 As described, core 208 can transfer image data statistics to LSE 206 on one or more output channels. LSE 206 can calculate CRC signatures on one or more output channels. In some examples, LSE 206 stores one or more CRC signatures in memory included in LSE 206. In some examples, LSE 206 transfers one or more CRC signatures to DDR memory 212 for storage via data movement structure 218.

[0049] In some examples, core 208 determines one or more image data statistics based on image data corresponding to previously acquired and / or otherwise processed image frames. For example, core 208 may determine a flow vector or light vector for consecutive image frames. In such an example, core 208 may (1) determine a first image data statistic based on a pixel value at a first position in a first image frame, and (2) determine a second image data statistic based on a pixel value at a second position in a second image frame. In some examples, LSE 206 may determine one or more CRC signatures based on the first and second image data statistics. In such an example, LSE 206, one or both of host devices 214A-B, etc., may determine how a pixel moves relative to the first image frame in the second image frame.

[0050] In some examples, core 208 determines one or more image data statistics based on depth information. For example, Figure 1 The camera 104 may be a stereo or stereoscopic camera. In such an example, core 208 may (1) determine one or more first image data statistics based on first image data associated with a first image frame captured by the stereo camera at a first time; and (2) determine one or more second image data statistics based on second image data associated with a second image frame captured by the stereo camera at a first time. In some examples, LSE 206 may determine one or more CRC signatures based on one or more first image data statistics and / or one or more second image data statistics.

[0051] exist Figure 2 In the illustrated example, SOC 202 includes an example digital signal processor (DSP) 220 coupled to a data movement structure 218. In some examples, DSP 220 performs tasks and / or otherwise supports computer vision processing (such as object detection and / or classification).

[0052] exist Figure 2 In the illustrated example, SOC 202 includes a data movement structure 218 to coordinate the timing of data transfers between various hardware components of SOC 202 (e.g., HWA 204, image data interface 216, DSP 220, etc.). Figure 2In this configuration, data movement structure 218 is an interconnect. For example, data movement structure 218 may include and / or otherwise correspond to a direct memory access (DMA) controller. In other examples, the DMA controller may be external to and / or otherwise coupled to data movement structure 218. In some examples, data movement structure 218 allows hardware components of SOC 202 to access example SOC random access memory (RAM) 222, access DDR memory 212 via example DDR memory interface 224, etc.

[0053] exist Figure 2 In the illustrated example, SOC RAM 222 is coupled to data movement structure 218. SOC RAM 222 may correspond to volatile memory, such as Synchronous Dynamic Random Access Memory (SDRAM) or Dynamic Random Access Memory (DRAM). Dynamic Random Access Memory Wait. In Figure 2 In this configuration, the DDR memory interface 224 is coupled to the data movement structure 218 and the DDR memory 212. The DDR memory interface 224 may correspond to one or more hardware circuits, logic circuits, etc., to facilitate a bidirectional interface between the data movement structure 218 and the DDR memory 212.

[0054] exist Figure 2 In the illustrated example, SOC 202 includes an example vision processor 226 coupled to data movement structure 218. Figure 2 In this context, the vision processor 226 can correspond to a vector processor adapted for computer vision processing, such as gradient calculation, orientation binning, histogram normalization, etc.

[0055] exist Figure 2 In the illustrated example, the example shows controller 228 coupled to data movement structure 218 and display panel 210. Display controller 228 may correspond to... Figure 1 The display controller 120. The display panel 210 can correspond to Figure 1 The display panel 112. For example, the display controller 228 can transmit rendered image data to the display panel 210 to display the image on the display panel 210.

[0056] exist Figure 2 In the illustrated example, SOC 202 includes a first example host device 214A and an example host device interface 230. Figure 2 In this configuration, host device interface 230 is coupled to data movement structure 218 and first host device 214A. Figure 2 In the example, the second host device 214B is external to the SOC 202. Figure 2In this configuration, the second host device 214B is coupled to the SOC 202 via a host device interface 230. The host device interface 230 may correspond to one or more logic circuits to implement a bidirectional interface to facilitate data transfer between the data movement structure 218 and one or more of the host devices 214A-B. Figure 2 In this context, one or both of the host devices 214A-B may correspond to one or more programmable processors, one or more programmable controllers, etc.

[0057] Although Figure 2 The diagram illustrates an example of implementing SOC 202, but... Figure 2 One or more of the elements, processes, and / or devices illustrated in the diagram may be combined, divided, rearranged, omitted, eliminated, and / or implemented in any other way. Furthermore, examples include HWA 204, LSE 206, core 208, image data interface 216, data movement structure 218, DSP 220, SOCRAM 222, DDR memory interface 224, vision processor 226, display controller 228, host device interface 230, and / or more generally... Figure 2Example SOC 202 can be implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. Thus, for example, example HWA 204, example LSE 206, example core 208, example image data interface 216, example data movement structure 218, example DSP 220, example SOC RAM 222, example DDR memory interface 224, example vision processor 226, example display controller 228, example host device interface 230, and / or more generally, example SOC 202 can be implemented by one or more analog or digital circuits, logic circuits, one or more programmable processors, one or more programmable controllers, one or more graphics processing units (GPUs), one or more DSPs, one or more ASICs, one or more PLDs, and / or one or more FPLDs. When reading any of the device or system claims of this patent to cover purely software and / or firmware implementations, at least one of the following examples is expressly defined herein as including non-transitory computer-readable storage devices or disks (such as non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, etc.) and volatile memory (e.g., SDRAM, DRAM, etc.). (etc., and / or any other type of RAM device), which includes software and / or firmware. Furthermore, Figure 2 Example SOC 202 includes, except for or instead of Figure 2 One or more elements, processes, and / or devices other than those shown in the illustrations, and / or may include any or all of the illustrated elements, processes, and devices. As used herein, the phrase “in communication” (including variations thereof) covers direct communication and / or indirect communication via one or more intermediate components, and does not require direct physical (e.g., wired) communication and / or continuous communication, but also includes selective communication at periodic intervals, predetermined intervals, non-periodic intervals, and / or one-off events.

[0058] Figure 3 Depicting and Figure 2 The example workflows 300a and 300b associated with SOC 202. Figure 3 In the examples, workflows 300a and 300b include a first example workflow 300a and a second example workflow 300b. Figure 3In this context, the first workflow 300a corresponds to operations 1A, 2A, 3A, and 4A. In Figure 3 In this context, the second workflow 300b corresponds to operations 1B and 2B.

[0059] In the first workflow 300a, during the first operation 1A, input image data 116 is stored in DDR memory 212. For example, image data interface 216 obtains input image data 116 from image data source 104. In such an example, image data interface 216 may (1) process input image data 116 by extracting raw image data from the transport layer of one or more data packets transmitting input image data 116, and (2) transfer the raw image data to data movement structure 218. In response, data movement structure 218 transfers the raw image data to DDR memory 212 via DDR memory interface 224.

[0060] In the first workflow 300a, during the second operation 2A, HWA 204 retrieves raw image data from DDR memory 212 via data movement structure 218 and DDR memory interface 224. During the third operation 3A of the first workflow 300a, core 208 can determine one or more image data statistics and / or transform the raw image data from two-dimensional (2-D) image data to another domain by performing example transformation operations (e.g., image data transformation operations), such as performing a census transformation on the raw image data to generate transformed pixel data (e.g., census transformed data).

[0061] During the third operation 3A, LSE 206 may determine a CRC signature based on at least one of (1) the original image data, (2) one or more image data statistics, or (3) the transformed pixel data. For example, LSE 206 may determine the CRC signature value of the original image data, the corresponding image data statistics in one or more image data statistics, and / or the transformed pixel data. During the third operation 3A, HWA 204 transmits the original image data, one or more image data statistics, the transformed pixel data, and / or (one or more) CRC signature values ​​to data movement structure 218. During the third operation 3A, data movement structure 218 transmits the original image data, one or more image data statistics, the transformed pixel data, and / or (one or more) CRC signature values ​​to DDR memory 212 via DDR memory interface 224. Alternatively, data movement structure 218 may transmit the original image data, one or more image data statistics, the transformed pixel data, and / or (one or more) CRC signature values ​​to one or both of host devices 214A-B via host device interface 230. For example, one or both of host devices 214A-B can detect frame freeze based on one or more CRC signature values ​​and / or one or more image data statistics. In such an example, one or both of host devices 214A-B can detect frame freeze based on one or more CRC signature values ​​and / or one or more image data statistics for two consecutive or sequential image frames. Alternatively, one or both of host devices 214A-B can obtain one or more CRC signature values ​​and / or one or more image data statistics from DDR memory 212 via host device interface 230, data movement structure 218, and DDR memory interface 224.

[0062] In the first workflow 300a, during the fourth operation 4A, raw image data from the DDR memory 212 is transferred to the display panel 210 224 via the DDR memory interface, data movement structure 218, and display controller 228. For example, the display panel 210 may display one or more image frames captured by the image data source 104 and rendered by the display controller 228.

[0063] In the second workflow 300b, during the first operation 1B, the input image data 116 is processed by the HWA 204 before being stored in the DDR memory 212. For example, the image data interface 216 may acquire the input image data 116, extract raw image data from the input image data 116, and process the raw image data to generate processed image data. During the first operation 1B, the HWA 204 may instruct the core 208 to determine one or more image data statistics and / or transform the processed image data from two-dimensional image data to another domain by performing example transformation operations (such as neighborhood-aware pixel transformation) to generate transformed pixel data.

[0064] During the first operation 1B, HWA 204 may instruct LSE 206 to determine one or more CRC signatures on at least one of (1) the processed image data, (2) one or more image data statistics, or (3) the transformed pixel data. HWA 204 may transfer the processed image data, one or more image data statistics, transformed pixel data, and / or one or more CRC signature values ​​to DDR memory 212 via data movement structure 218 and DDR memory interface 224. In some examples, LSE 206 may store one or more CRC signatures in LSE 206 and may detect frame freeze conditions by comparing one or more CRC signatures with one or more previously generated and / or stored CRC signatures (e.g., one or more CRC signatures previously generated by LSE 206, one or more CRC signatures previously stored in LSE 206, etc.). Advantageously, with Figure 3 Compared to the first workflow 300a, the second workflow 300b reduces the detection time of frame freeze conditions by determining image data statistics and / or (one or more) CRC signature values ​​at the front end of the SOC 202 before storing the input image data 116 in the DDR memory 212.

[0065] Figure 4 Depicting Figures 2-3 An example implementation of HWA 204 is provided. HWA 204 includes an example HWA VISS 402, which includes LSE 206 and core 208. HWA VISS 402 is a subsystem of HWA 204 (e.g., one or more hardware and / or logic circuits) configured to perform computational and / or computational tasks related to visual imaging processing.

[0066] Figure 4 HWA VISS 402 includes Figures 2-3 LSE 206 from Figures 2-3 Image data interface 216 acquires input image data. Figure 4 The HWAVISS 402 includes core 208 to acquire input image data from LSE 206. Core 208 calculates and / or otherwise determines parameters such as one or more image data statistics. Core 208 may transfer one or more image data statistics to LSE 206 to invoke LSE 206 to calculate and / or otherwise determine the CRC signature value of each of the one or more image data statistics.

[0067] Figure 4 The LSE 206 includes one or more example CRC logic circuits 404 for determining CRC signature values ​​for one or more output channels of the LSE 206. For example, the CRC logic circuits 404 may compute a first CRC signature value on a first output channel transmitting processed image data, a second CRC signature value on a second output channel transmitting image data statistics, a third CRC signature value on a third output channel transmitting transformed pixel data, and / or combinations thereof. Figure 4 In this context, one or more CRC logic circuits 404 determine a CRC signature value with a 32-bit data size. Alternatively, one or more CRC logic circuits 404 can determine that the CRC signature value has any other data size. In some examples, one or more CRC logic circuits 404 can determine a 32-bit CRC signature value based on the following 32-bit CRC polynomial:

[0068] x 32 +x 26 +x 23 +x 22 +x 16 +x 12 +x 11 +x 10 +x 8 +x 7 +x 5 +x 4 +x 2 Alternatively, +x+1 can be used with any other CRC polynomial.

[0069] In some examples, one or more CRC logic circuits 404 can update and store the CRC signature value until the entire image frame has been processed. In response to the entire image frame being processed, one or more CRC logic circuits 404 can store the CRC signature value in SRAM 410 via SRAM interface 408. In some examples, Figures 2-3 The host device 214A-B is connected via SRAM interface 408, HWA DMA 416, Figures 2-3 Data movement structure 218 and Figures 2-3 The host device interface 230 obtains the CRC signature value from SRAM 410.

[0070] Figure 4 One or more CRC logic circuits 404 can determine the CRC signature value of the first image frame, and in some examples, the CRC signature value can be compared with a previously calculated CRC signature value of a second image frame or a previous image frame. When the CRC signature value is the same as the previously calculated CRC signature value, a frame freeze detection condition can be detected, which indicates a frame freeze from... Figures 1-3 Image data source 104 captures and / or processes errors in the software, hardware, and / or firmware associated with the image.

[0071] exist Figure 4 In the illustrated example, HWA 204 includes an example HWA configurator 406 coupled to HWA VISS 402. Alternatively, HWA 204 may not include HWA configurator 406. For example, HWA 204 may be coupled to HWA configurator 406.

[0072] In some examples, the HWA configurator 406 configures the LSE 206 to compute and / or otherwise calculate CRC values ​​on one or more output channels of the LSE 206. For example, the HWA configurator 406 can configure the LSE 206 by selecting one or more output channels for CRC calculation. In some examples, one or more output channels correspond to processed image data channels. For example, one or more output channels may include a first output channel corresponding to luminance data, a second output channel corresponding to chrominance or chromaticity data, a third output channel corresponding to RGB data, etc., where the data size of one or more output channels is 8 bytes, 12 bytes, etc.

[0073] In some examples, the HWA configurator 406 configures core 208 to compute and / or otherwise calculate one or more types of image data statistics. For example, the HWA configurator 406 may configure a first core in core 208 to compute a first image data statistic corresponding to the maximum value of a set of pixels (e.g., the maximum value is the largest among the values). In other examples, the HWA configurator 406 may configure a second core in core 208 to determine a second image data statistic corresponding to the minimum value of a set of pixel values ​​(e.g., the minimum value is the smallest among the values). In some examples, the hardware configurator 406 may base its calculations on data from... Figures 2-3 The host devices 214A-B use commands, configurations, instructions, etc. to configure at least one of LSE 206 or core 208.

[0074] exist Figure 4In the illustrated example, HWA 204 includes an example static RAM (SRAM) interface 408 coupled to HWA VISS 402 and example SRAM 410. Figure 4 In this context, SRAM 410 may correspond to one or more SRAM devices. Alternatively, SRAM 410 may be any other type of memory. Figure 4 In this context, SRAM interface 408 may correspond to one or more logic circuits to implement a bidirectional interface to facilitate data reading and / or writing to SRAM 410. For example, SRAM interface 408 may correspond to one or more logic circuits to facilitate the operation of the memory controller and / or otherwise implement the memory controller.

[0075] exist Figure 4 In the illustrated example, HWA 204 includes, according to example HWA subsystems 412A-C, HWA subsystem-A, HWA subsystem-B, and HWA subsystem-C. In some examples, HWA subsystems 412A-C are copies or instances of HWA VISS 402. In some examples, one or more of HWA subsystems 412A-C are different from the different subsystems in HWA subsystems 412A-C. For example, one or more of HWA subsystems 412A-C may include LSE 206 and core 208 of HWA VISS 402. Alternatively, one or more of HWA subsystems 412A-C may be different from HWA VISS 402. For example, one or more of HWA subsystems 412A-C may perform and / or otherwise perform image processing-related functions, such as scaling, distortion correction, etc.

[0076] Each of the HWA subsystems 412A-C is coupled to an SRAM interface 408 and an example HWA thread scheduler (HTS) 414. The HTS 414 controls data flow scheduling between the HWA subsystems 412A-C and data flow scheduling from / to external memory of the HWA subsystems 412A-C. For example, the HTS 414 may manage a set of state machines for each HWA subsystem 412A-C and coordinate data processing / transfer scheduling within the HWA subsystems 412A-C via dedicated start / complete signals. For example, the HTS 414 may perform and / or otherwise facilitate start / complete task synchronization signal processing, programmable completion mask generation schemes, end-of-process (EOP) generation, and / or debug-ready state signal generation for debug / pause states.

[0077] Further described as being included Figure 4 In HWA 204, it is coupled to SRAM interface 408, HTS 414 and Figures 2-3 The data movement structure 218 is an HWA DMA 416. In Figure 4 In this context, the HWA DMA 416 can correspond to one or more hardware circuits, logic circuits, etc., to facilitate... Figures 2-3 A bidirectional interface between the data movement structure 218, HTS 414 and / or SRAM interface 408.

[0078] Figure 5 Depicting and Figure 4 Example workflow 500 associated with the example implementation of HWA 204. Figure 5 The workflow 500 can correspond to Figure 3 The first workflow 300a. In Figure 5 In the process, workflow 500 corresponds to operations 1-4, which include the first operation (1), the second operation (2), the third operation (3), and the fourth operation (4).

[0079] Figure 5 The workflow 500 starts with the first operation, which will come from Figures 2-3 Image data interface 216 data via Figures 2-3 Data movement structure 218 is transmitted to Figures 2-3 The DDR memory 212. During the second operation, data is retrieved from the DDR memory 212 via the data movement structure 218. For example, Figure 4 The HWA DMA 416 can acquire image data from the data movement structure 218 and store the acquired image data in the SRAM 410.

[0080] exist Figure 5 In the illustrated example, during the third operation, the image data stored in SRAM 410 is... Figures 2-4The image data is acquired by LSE 206. LSE 206 transmits image data to core 208 for processing. For example, core 208 can generate processed image data, transformed pixel data, and / or one or more image data statistics based on the image data. In such an example, core 208 can transmit the processed image data, transformed pixel data, and / or one or more image data statistics to one or more output channels of LSE 206. LSE 206 can invoke one or more CRC logic circuits 404 to calculate one or more CRC signature values ​​based on the processed image data, transformed pixel data, and / or image data statistics. When an updated value is received from core 208, one or more CRC logic circuits 404 can update (e.g., iteratively update) the CRC signature value. In response to processing the entire image frame, core 208 can notify LSE 206 (e.g., by generating an end signal and transmitting it to LSE 206, transmitting an end bit to LSE 206, etc.) that the entire image frame has been processed.

[0081] exist Figure 5 Upon receiving notification, one or more CRC logic circuits 404 may store and / or otherwise retain the final or last value of the CRC signature in LSE 206. For example, LSE 206 may store the final or last value of the CRC signature in a memory-mapped register, flip-flop, etc. In some examples, upon receiving notification, one or more CRC logic circuits 404 may transfer the final or last value of the CRC signature to SRAM interface 408 to store the final or last value of the CRC signature in SRAM 410. For example, the final or last value of the CRC signature may be transferred from SRAM 410 to SRAM 410 via a data path that includes at least SRAM interface 408 and HWA DMA 416. Figures 2-3 One or both of the host devices 214A-B.

[0082] Figure 6 Depicting and Figure 4 Example workflow 600 associated with the example implementation of HWA 204. Figure 6 The workflow 600 can correspond to Figure 3 The second workflow 300b. In Figure 6 In the middle, when LSE 206 from Figures 2-3 Workflow 600 begins when the image data interface 216 acquires input image data. Advantageously, this is achieved by acquiring input image data from the image data interface 216 instead of as described above. Figure 5 The described method of acquiring input image data from DDR memory 212 can reduce the detection time for image frame freezes. For example, Figure 5The first and second operations of workflow 500 can be eliminated, thereby reducing latency and thus reducing the time for detecting image frame freeze conditions.

[0083] exist Figure 6 In the illustrated example, as shown above... Figure 5 As described, LSE 206 transmits received image data to core 208 for processing and receives processed image data, transformed pixel data, and / or (one or more) image data statistics from core 208. Therefore, (one or more) CRC logic circuits 404 can store the final or last value of (one or more) CRC signatures based on the processed image data, transformed pixel data, and / or (one or more) image data statistics in LSE 206. Alternatively, the final or last value of (one or more) CRC signatures can be stored in SRAM 410 and transmitted from SRAM 410 via a data path including at least SRAM interface 408 and HWA DMA 416 to Figures 2-3 One or both of the host devices 214A-B.

[0084] Figure 7 Depicting Figures 2-6 An example implementation of LSE 206. In Figure 7 In this example, LSE 206 includes a sample thread 702, a sample core interface 704, a sample output channel multiplexer 706, a sample output channel 708, a sample data multiplexer (MUX) 710, a sample memory interface 712, and a sample HWA configurator interface 714. Figure 7 Further description Figures 4-6 One or more CRC logic circuits 404.

[0085] exist Figure 7 In the illustrated example, thread 702 starts from... Figures 2-3 Image data can be acquired from the image data interface 216 and / or the memory interface 712. For example, thread 702 can obtain image data from the image data interface 216 and / or the memory interface 712. Figure 2 The DDR memory 212 acquires image data via the memory interface 712. Figure 7 In this context, thread 702 refers to a processing thread, HWA thread, logic circuit thread, etc., corresponding to a processing chain with its own HTS start / complete. For example, thread 702 can be a logical construct representing a hardware and / or software processing thread and may not be a physical entity. Alternatively, LSE 206 may include more than one thread 702. In some examples, thread 702 executes a series of hardware logic operations that begin with a start signal and end with a complete signal.

[0086] exist Figure 7In the illustrated example, thread 702 includes an example image data buffer 716, an example input channel 718, an example input channel multiplexer 720, and an example data merger 722. The image data buffer 716 stores data from... Figures 2-3 The image data buffer of the image data interface 216. For example, LSE 206 can access the image data from the image data buffer 716. Figures 1-3 Image data source 104 acquires image data in essentially real-time.

[0087] exist Figure 7 In the illustrated example, input channel 718 corresponds to two or more data channels, which facilitate reading and / or unpacking from... Figures 4-6 Image data (e.g., pixel data, pixel values, etc.) acquired by the SRAM 410 can later be transferred to one of the core interfaces 704. Figures 2-6 The core 208 performs the processing. Alternatively, thread 702 may have only one of the input channels 718 or only one of the input channels 718 configured to read image data.

[0088] exist Figure 7 In the illustrated example, the image data buffer 716 and the first input channel (input channel [0]) of the input channel 718 are coupled to the input channel multiplexer 720. Figure 7 In this circuit, the image data buffer 716 is coupled to the first input of the input channel multiplexer 720. Figure 7 In this configuration, the first input channel of input channel 718 is coupled to the second input of input channel multiplexer 720. In some examples, input channel multiplexer 720 determines whether to transfer (1) stored image data from SRAM 420 or (2) substantially real-time image data acquired by image data buffer 716 and / or stored by image data buffer 716 to core 208. In some examples, input channel multiplexer 720 may determine to transfer stored image data from SRAM 420 to core 208, which may correspond to... Figure 3 The first workflow 300a. In some examples, the input channel multiplexer 720 can determine the substantially real-time image data acquired and / or stored by the image data buffer 716 to be transferred to the core 208, which can correspond to Figure 3 The second workflow 300b.

[0089] exist Figure 7 In the illustrated example, the second input channel (input channel [N]) of input channel multiplexer 720 and input channel 718 is coupled to data combiner 722. Figure 7 In this circuit, the output of the input channel multiplexer 720 is coupled to the first input of the data combiner 722. Figure 7 In this configuration, the output of the second input channel of input channel 718 is coupled to the second input of data merger 722. Data merger 722 may correspond to one or more logic circuits that arrange and / or otherwise format the image data into a format readable and / or otherwise accessible by core 208. For example, data merger 722 may correspond to a first-in, first-out (FIFO) data buffer that can synchronously exchange and / or transfer data with core 208. In other examples, data merger 722 may append or remove data bits, data headers, etc., to the image data to facilitate core 208's acceptance of the image data.

[0090] exist Figure 7 In the illustrated example, data merger 722 is coupled to the first core interface of core interface 704 and the first input of output channel multiplexer 706. Figure 7 In the middle, the second core interface of the core interface 704 is coupled to the first output channel (output channel [0]) of the output channel 708 and the first CRC logic circuit of the CRC logic circuit 404. Figure 7 In this configuration, the third core interface of core interface 704 is coupled to the second input of output channel multiplexer 706. The output of output channel multiplexer 706 is coupled to the second output channel (output channel [N]) of output channel 708 and the second CRC logic circuit of CRC logic circuit 404. Alternatively, the second and third core interfaces of core interface 704 may be unrelated to the first and second output channels of output channel 708. For example, the second core interface of core interface 704 may be coupled to the first output channel of output channel 708 and the second input of output channel multiplexer 706.

[0091] exist Figure 7 In the illustrated example, core interface 704 may correspond to one or more logic circuits to implement a bidirectional interface for transmitting and / or receiving data from core 208. For example, core interface 704 may correspond to one or more logic circuits that facilitate operation and / or other implementation of a two-wire circuit (I2C) bus, a universal asynchronous receiver-transmitter (UART) bus, a serial peripheral interface (SPI) bus, etc.

[0092] exist Figure 7In the illustrated example, data multiplexer 710 is coupled to input channel 718, output channel 708, and memory interface 712. Data multiplexer 710 synchronizes and / or otherwise facilitates image data processing operations of LSE 206. For example, data multiplexer 710 may invoke memory interface 712 to retrieve image data from SRAM 420 for processing by thread 702. In other examples, data multiplexer 710 may invoke memory interface 712 to store processed data (e.g., CRC signature values, image data statistics, etc.) in SRAM 420.

[0093] exist Figure 7 In the illustrated example, the HWA configurator interface 714 facilitates LSE 206 and Figures 4-6 Communication between the HWA configurator 406 and the HWA configurator interface 714. For example, the HWA configurator interface 714 can obtain configuration information from the HWA configurator 406 to configure and / or otherwise adjust the operation of the thread 702. In such an example, the thread 702 can enable or disable one or more input channels 718 based on the configuration information. In other examples, the thread 702 can invoke the input channel multiplexer 720 to select data from either the image data buffer 716 or one of the input channels 718.

[0094] Figure 8 Depicting and Figure 7 Example workflow 800 associated with the example implementation of LSE 206. Figure 8 The workflow 800 can correspond to Figure 3 The first workflow 300a. In Figure 8 In the middle, LSE 206 via Figures 4-6 SRAM interface 408 from Figures 4-6 The SRAM 410 acquires image data. For example, Figure 7 The memory interface 712 can acquire image data and transmit it to... Figure 7 Thread 702. Thread 702 can unpack image data and format the unpacked image data into a format that can be used by... Figures 2-6The core 208 processes the image data in the specified format. Core 208 can acquire image data from LSE 206 via the first core interface in core interface 704. Core 208 can generate processed image data, transformed pixel data, and / or (one or more) image data statistics based on the image data. The core can transmit the processed image data, transformed pixel data, and / or (one or more) image data statistics to LSE 206 via the second core interface, third core interface, etc., in core interface 704. Core interface 704 can transmit the processed image data, transformed pixel data, and / or (one or more) image data statistics to one or more output channels 708.

[0095] exist Figure 8 In the illustrated example, one or more CRC logic circuits 404 calculate CRC signature values ​​on the corresponding output channels. For example, CRC logic circuits 404 may calculate CRC signature values ​​before and / or substantially simultaneously with the output channel 708 receiving data from core 208. In such an example, CRC logic circuits 404 may calculate a first CRC signature value on the processed image data transmitted to output channel [0], a second CRC signature value on the transformed pixel data transmitted to output channel [1], a third CRC signature value on the first image data statistic transmitted to output channel [2], a fourth CRC signature value on the Nth image data statistic transmitted to output channel [N], and / or combinations thereof. In response to acquiring the processed image data, transformed pixel data, and / or (one or more) image data statistics, one or more output channels 708 may transmit the processed image data, transformed pixel data, and / or (one or more) image data statistics to SRAM 410 via a data path including data multiplexer 710 and memory interface 712. In some examples, the input channel multiplexer 720, the output channel multiplexer 706, and / or combinations thereof can be configured, enabled, etc., to bypass the input channel multiplexer. Figure 2 The image data interface 216 acquires image data. In such an example, the CRC logic circuit 404 can process the input image data (e.g., the raw input image data) from the image data interface 216.

[0096] Figure 9 Depicting and Figure 7 Example workflow 900 associated with the example implementation of LSE 206. Figure 9 The workflow 900 can correspond to Figure 3 The second workflow 300b. In Figure 9 In the process, image data is stored Figures 4-6 Prior to SRAM 410, LSE 206 was from Figures 2-3Image data is acquired via image data interface 216. For example, image data buffer 716 can acquire image data instead of the combination described above. Figure 8 The input channel described is [0]. The LSE 206 can be based on the input channel described. Figures 2-6 The CRC signature value is calculated using the processed image data, transformed pixel data, and / or (one or more) image data statistics determined by kernel 208, as described above. Figure 8 As described herein. Advantageously, compared to other workflows described herein, the LSE 206 can process image data earlier in the image data processing pipeline, thus enabling processing from the image data interface 216 instead of as described above. Figure 8 The described method of acquiring image data from SRAM 410 can reduce the time required to detect image frame freeze conditions.

[0097] Figure 10 An example implementation of the LSE controller 1000 is described. Figure 10 The LSE controller 1000 can correspond to Figure 2 Another example implementation of LSE 206. In Figure 10 In the illustrated example, the LSE controller 1000 includes an example thread controller 1010, an example core interface 1020, an example memory interface 1030, an example HWA configurator interface 1040, an example data multiplexer 1050, an example CRC logic controller 1060, and an example database 1070.

[0098] exist Figure 10 In the illustrated example, the LSE controller 1000 includes a thread controller 1010 to synchronize data exchange and / or transmission with different devices, controllers, and / or control logic. In some examples, the thread controller 1010 acquires data from an image data source (e.g., Figure 1 Image data source 104) captures and / or otherwise obtains from image data sources (e.g., Figure 1 Image data obtained from image data source 104 (e.g., Figure 1 Input image data 116, from Figure 2 Image data or processed image data, etc., from the image data interface 216.

[0099] In some examples, thread controller 1010 arranges and / or otherwise formats the image data in a manner that can be combined with the following. Figure 11The described example core controller 1100 reads and / or otherwise acquires the image data in a specific format. For example, thread controller 1010 may append or remove data bits, data headers, etc., to image data to facilitate acceptance of the image data by different controllers and / or control logic. In some examples, thread controller 1010 corresponds to and / or is implemented in other ways. Figure 7 Image data buffer 716, one or more of input channels 718, input channel multiplexer 720 and / or data combiner 722.

[0100] exist Figure 10 In the illustrated example, the LSE controller 1000 includes a core interface 1020 for communication with different controllers and / or control logic (such as...). Figure 2 Core 208, Figure 11 The core interface 1020 transmits data to and / or receives data from different controllers and / or control logic (such as the core controller 1100). In some examples, the core interface 1020 preprocesses, formats, transforms, and / or otherwise converts data from a first data format used by the LSE controller 1000 to a second data format different from the first data format before transmission. In some examples, the core interface 1020 converts data of the second data format received from different controllers and / or control logic into the first data format. In some examples, the core interface 1020 corresponds to and / or otherwise implements... Figure 7 One or more of the 704 core interfaces.

[0101] exist Figure 10 In the illustrated example, the LSE controller 1000 includes a memory interface 1030 for reading data from memory or other storage devices and / or writing data to memory or other storage devices. In some examples, the memory interface 1030 is a memory controller that, when invoked, can facilitate data transfer between memory and the LSE controller 1000. In some examples, the memory interface 1030 corresponds to... Figure 7 The memory interface 712 and / or implemented in other ways Figure 7 The memory interface 712.

[0102] exist Figure 10 In the illustrated example, the LSE controller 1000 includes an HWA configurator interface 1040 to obtain configuration information for configuring the LSE controller 1000. In some examples, the HWA configurator interface 1040 can configure the thread controller 1010 to obtain image data from a specified source (e.g., from memory, an image data source, etc.) based on the configuration information. In some examples, the HWA configurator interface 1040 obtains image data from... Figure 4 HWA Configurator 406 Figure 2The host device 214A-B, or one or both of them, obtains configuration information. In some examples, the HWA configurator interface 1040 corresponds to and / or is implemented in other ways. Figure 7 HWA Configurator Interface 714.

[0103] exist Figure 10 In the illustrated example, the LSE controller 1000 includes a data multiplexer 1050 to control read and / or write data operations. In some examples, the data multiplexer 1050 selects a source to acquire image data based on configuration information obtained from the HWA configurator interface 1040. In some examples, the data multiplexer 1050 corresponds to and / or is implemented in other ways. Figure 7 710 data multiplexer.

[0104] exist Figure 10 In the illustrated example, the LSE controller 1000 includes a CRC logic controller 1060 to calculate and / or otherwise determine a CRC signature (e.g., a 32-bit CRC signature based on the 32-bit CRC polynomial described above). For example, the CRC logic controller 1060 may calculate the CRC signature based on processed image data generated by core 208, transformed pixel data generated by core 208, image data statistical parameters determined by core 208, etc. In some examples, the CRC logic controller 1060 compares CRC signature values ​​to detect image frame freeze conditions. For example, the CRC logic controller 1060 may detect image frame freeze conditions in response to determining differences between CRC signature values. In some examples, the CRC logic controller 1060 corresponds to... Figures 4-9 One or more CRC logic circuits 404 and / or otherwise implemented Figures 4-9 One or more CRC logic circuits 404.

[0105] exist Figure 10 In the illustrated example, the LSE controller 1000 includes a database 1070 to record data (e.g., configuration information, image data statistics, CRC signature values, etc.). The database 1070 may be stored in volatile memory (e.g., SDRAM, DRAM, etc.). Database 1070 may be implemented using (e.g., DDR) and / or non-volatile memory (e.g., ROM, EEPROM, flash memory, etc.). Database 1070 may be additionally or alternatively implemented using one or more DDR memories, such as DDR, DDR2, DDR3, DDR4, mDDR, etc. Database 1070 may be additionally or alternatively implemented using one or more mass storage devices such as (one or more) hard disk drives, (one or more) compact disk drives, (one or more) digital multifunction disk drives, (one or more) solid-state drives, etc. Although in the illustrated example, database 1070 is depicted as a single database, database 1070 may be implemented using any number and / or (one or more) types of databases. Furthermore, the data stored in database 1070 may be in any data format, such as binary data, comma-separated data, tab-separated data, Structured Query Language (SQL) structures, etc.

[0106] Although Figure 10 The diagram illustrates an example of implementing the LSE controller 1000, but... Figure 10 One or more of the elements, processes, and / or devices illustrated in the diagram may be combined, divided, rearranged, omitted, eliminated, and / or implemented in any other way. Furthermore, example thread controller 1010, example core interface 1020, example memory interface 1030, example HWA configurator interface 1040, example data multiplexer 1050, example CRC logic controller 1060, and example database 1070, and / or more generally, Figure 10The example LSE controller 1000 can be implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. Thus, for example, any one of the example thread controller 1010, example core interface 1020, example memory interface 1030, example HWA configurator interface 1040, example data multiplexer 1050, example CRC logic controller 1060, and example database 1070, and / or more generally, the example LSE controller 1000 can be implemented by one or more analog or digital circuits, logic circuits, (one or more) programmable processors, (one or more) programmable controllers, (one or more) graphics processing units (GPUs), (one or more) DSPs, (one or more) ASICs, (one or more) PLDs, and / or (one or more) FPLDs. When reading any of the device or system claims of this patent to cover purely software and / or firmware implementations, at least one of the example thread controller 1010, example core interface 1020, example memory interface 1030, example HWA configurator interface 1040, example data multiplexer 1050, example CRC logic controller 1060, and / or example database 1070 is hereby explicitly defined as including a non-transitory computer-readable storage device or disk (such as non-volatile memory (e.g., ROM, EEPROM, flash memory, etc.), volatile memory (e.g., SDRAM, DRAM, etc., and / or any other type of RAM device)), which includes software and / or firmware. Furthermore, Figure 10 Example LSE controller 1000 includes, except for or instead of Figure 10 One or more elements, processes and / or devices other than those shown in the illustrations, and / or may include more than one of any or all of the illustrated elements, processes and devices.

[0107] Figure 11 An example implementation of the nuclear controller 1100 is depicted. The nuclear controller 1100 may correspond to... Figure 2 Another example implementation of the core 208. In Figure 11 In the illustrated example, the core controller 1100 includes an example LSE interface 1110, an example core configurator 1120, an example image data processor 1130, and an example parameter calculator 1140.

[0108] exist Figure 11 In the illustrated example, the nuclear controller 1100 includes an LSE interface 1110 to communicate with different controllers and / or control logic (such as... Figure 2 LSE 206 Figure 10The LSE interface 1110 transmits data to and / or receives data from different controllers and / or control logic (such as the LSE controller 1000). In some examples, the LSE interface 1110 preprocesses, formats, transforms, and / or otherwise converts data from a first data format used by the core controller 1100 to a second data format different from the first data format before transmission. In some examples, the LSE interface 1110 converts data in the first data format to the second data format before transmitting the second data format data to different controllers and / or control logic. Alternatively, the LSE interface 1110 can convert data from the first data format to a third data format, corresponding to a format suitable for I2C bus, UART bus, SPI bus, etc.

[0109] exist Figure 11 In the illustrated example, the core controller 1100 includes a core configurator 1120 to obtain configuration information to configure the core controller 1100. In some examples, the core configurator 1120 may configure a parameter calculator 1140 to determine image data statistics or other parameters of interest based on the configuration information. For example, the core configurator 1120 may configure the parameter calculator 1140 to calculate a first image data statistic corresponding to the maximum value of a set of pixels. In other examples, the core configurator 1120 may determine a second image data statistic corresponding to the minimum value of a set of pixel values. In some examples, the core configurator 1120 obtains configuration information from the parameter calculator 1140 to determine the maximum value of a set of pixel values. Figure 4 HWA Configurator 406 Figure 2 The host device 214A-B or one or both of them can obtain configuration information.

[0110] exist Figure 11 In the illustrated example, the core controller 1100 includes an image data processor 1130 to extract data or information of interest from image data acquired by the LSE interface 1110. For example, the image data processor 1130 may be based on input image data (e.g., Figure 1 The input image data 116) is determined by the image data source (e.g., Figure 1 The image data processor 1130 captures the pixel-related values ​​of an image frame from the image data source 104. In such an example, the image data processor 1130 may extract and / or otherwise identify RGB pixel values ​​and / or values ​​associated with at least one of brightness, hue, or saturation. In other examples, the image data processor 1130 may determine the number of rows (e.g., image frame height), the number of columns (e.g., image frame width), the number of bands, bits per pixel (bpp), file type, etc., associated with the acquired image data. In such an example, the image data processor 1130 may determine when a complete or entire image frame has been processed.

[0111] In some examples, the image data processor 1130 generates processed image data by performing one or more image processing tasks on the acquired image data. For example, the image data processor 1130 may generate processed image data by identifying wide dynamic range features, performing defect pixel correction techniques, decompressing the acquired image data (e.g., converting the image data back to the linear domain), and / or combinations thereof. In some examples, the image data processor 1130 generates processed image data by performing and / or otherwise implementing lens shading correction, lens distortion correction, noise filtering, tone mapping, RGB conversion, color plane conversion, edge enhancement, histogram extraction, and / or combinations thereof on the acquired image data. In some examples, the image data processor 1130 may perform and / or otherwise implement any of the above processes, transform filters, etc., to... Figure 10 The CRC logic controller 1060 generates processed image data before performing a CRC signature task on the processed image data.

[0112] In some examples, image data processor 1130 transforms pixel data by performing transformation operations such as neighborhood-aware pixel transformation. For example, image data processor 1130 can generate a signature (e.g., a bit signature) by performing a census transformation operation on image data (e.g., data or information of interest extracted from image data acquired by LSE interface 1110). For example, image data processor 1130 can apply and / or otherwise perform a census transformation algorithm on pixel data to generate census transformed data. In such an example, image data processor 1130 can generate a bit signature by arranging the census transformed data into a bit string (e.g., a bit string of size or length of 8 bits, 16 bits, 32 bits, etc.). Image data processor 1130 can generate transformed pixel data by determining a decimal value corresponding to a binary value based on the bit string. In such an example, image data processor 1130 can determine one or more image data statistics based on the transformed pixel data (e.g., census transformed pixel data), which are advantageously robust to sensor noise, transmission noise, etc., which can be correlated with image data sources (such as...). Figure 1 The image data source 104 is associated with it.

[0113] exist Figure 11In the illustrated example, the core controller 1100 includes a parameter calculator 1140 to determine one or more parameters (e.g., diagnostic parameters, image data parameters, image data statistics, image tube statistics, etc.) associated with image data acquired by the LSE interface 1110. For example, the parameter calculator 1140 may determine the maximum, minimum, sum, etc., values ​​associated with a set of pixel values ​​associated with an image frame of interest to be processed. In some examples, the parameter calculator 1140 may determine one or more parameters associated with processed image data generated by the image data extractor 1130. For example, the parameter calculator 1140 may determine one or more parameters based on any output generated by the image data processor 1130, where the output may correspond to processed image data.

[0114] Although Figure 11 The diagram illustrates an example of implementing the nuclear controller 1100, but... Figure 11 One or more of the elements, processes, and / or devices illustrated in the diagram may be combined, divided, rearranged, omitted, eliminated, and / or implemented in any other way. Furthermore, the example LSE interface 1110, the example core configurator 1120, the example image data processor 1130, the example parameter calculator 1140, and / or more generally... Figure 11 The example core controller 1100 can be implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. Thus, for example, the example LSE interface 1110, the example core configurator 1120, the example image data processor 1130, the example parameter calculator 1140, and / or more generally, any of the example core controller 1100 can be implemented by one or more analog or digital circuits, logic circuits, one or more programmable processors, one or more programmable controllers, one or more graphics processing units (GPUs), one or more DSPs, one or more ASICs, one or more PLDs, and / or one or more FPLDs. When reading any of the device or system claims of this patent to cover purely software and / or firmware implementations, at least one of the example LSE interface 1110, example core configurator 1120, example image data processor 1130, and example parameter calculator 1140 is hereby explicitly defined as including a non-transitory computer-readable storage device or disk (such as non-volatile memory (e.g., ROM, EEPROM, flash memory, etc.), volatile memory (e.g., SDRAM, DRAM, etc., and / or any other type of RAM device)), which includes software and / or firmware. Furthermore, Figure 11 Example nuclear controller 1000 may include, in addition to or instead of Figure 11One or more elements, processes and / or devices other than those illustrated in the figures, and / or may include more than one of any or all of the illustrated elements, processes and devices.

[0115] Figure 12 A schematic diagram of example image data 1200 is depicted. Figure 12 The image data 1200 may correspond to HWA 204 and / or more generally, Figure 2 The frame freeze detection system 200 processes the image frames of interest to detect frame freeze conditions. For example, image data 1200 can be pixel data. Figure 12 In the image data 1200, there are example pixel windows 1202, each pixel window corresponding to a set of pixel values. For example, the pixel values ​​may correspond to RGB pixel values ​​or other types of pixel values. In other examples, the pixel values ​​may represent image characteristics such as brightness, hue, and / or saturation.

[0116] exist Figure 12 In the illustrated example, each pixel window 1202 has example pixel values ​​1204 arranged in a window having a size of pixel window height (PWH) multiplied by pixel window width (PWW). PWH is the height of the window corresponding to a first number of pixel values. PWW is the width of the window corresponding to a second number of pixel values. In some examples, the first and second number of pixel values ​​are the same, while in other examples, the first and second number of pixel values ​​are different.

[0117] exist Figure 12 In the illustrated example, image data 1200 may correspond to the image frame of interest to be processed (e.g., the entire image frame, the complete image frame, etc.). Figure 12 Image data 1200 is arranged in a window having a size of image frame height (IFH) multiplied by image frame width (IFW). IFH is the height of the image frame corresponding to a first number of pixel windows. IFW is the width of the image frame corresponding to a second number of pixel windows. In some examples, the first and second number of pixel windows are the same, while in other examples, the first and second number of pixel windows are different.

[0118] exist Figure 12 In the illustrated examples, HWA 204 and / or more generally, Figure 2 The frame freeze detection system 200 can process image data 1200 by starting with a window of pixels of interest. For example, HWA 204 and / or more generally, Figure 2 The frame freeze detection system 200 can begin processing image data 1200 by starting with an initial pixel window specified by the image frame start horizontal (IFSH) and image frame start vertical (IFSV). Figure 12 In this context, IFSH and IFSV correspond to the first pixel window in pixel window 1202. In such an example, HWA 204 and / or more generally, Figure 2 The frame freeze detection system 200 can process the first pixel window in pixel window 1202 and proceed from left to right and from top to bottom until the entire image data 1200 has been processed. Alternatively, IFSH and IFSV can be different ones in pixel window 1202 and / or different orders of pixel windows can be processed. In response to HWA 204 and / or more generally, the frame freeze detection system 200 processes... Figure 12 The entire image data 1200, HWA 204 and / or more generally, the frame freeze detection system 200 may calculate and / or otherwise determine one or more image parameters based on the processed image data 1200.

[0119] Figure 13 Example transformation workflow 1300 is described. Figure 13 The transformation workflow 1300 can be performed by Figure 2 Core 208 Figure 1 The nuclear controller 1100 and others perform and / or otherwise operate. In some examples, Figure 13 The transformation workflow 1300 can be handled by LSE 206, Figures 2-3 First host device 214A, Figures 2-3 Second host device 214B, Figures 2-3 The vision processor 226 and others execute and / or otherwise operate. Figure 13 In this example, the transformation workflow 1300 is an example implementation of a neighborhood-aware pixel transformation. For example, the transformation workflow 1300 may correspond to a census transformation or any other type of pixel transformation. Alternatively, any other image data transformation operation may be performed.

[0120] exist Figure 13 In the illustrated example, core 208 and / or core controller 1100 can acquire example image data 1302 and perform example neighborhood-aware pixel transformation operations (e.g., census transformation algorithm, census transformation, census transformation operation, etc.) 1304 on the image data 1302 to generate example transformed image data (e.g., census transformation data) 1306. Figure 13 In this context, image data 1302 corresponds to original image data, luminance data, chrominance data, image data associated with a grayscale image of 8, 10, or 12 bits per pixel (bpp), and / or data from... Figure 1 The image data source 104 is any other processed or extracted image data. For example, Figure 13 Image data 1302 can correspond to Figure 12 Image data 1200 Figure 12 A portion of the image data 1200 Figure 12 One or more of the pixel windows 1202, etc. In Figure 13 In the image data 1302, 25 pixel values ​​are arranged in a 5×5 grid or square. Alternatively, the image data 1302 and / or the transformed image data 1306 may have a different number of pixel values ​​and / or a different arrangement of pixel values.

[0121] exist Figure 13 In the illustrated example, core 208 and / or core controller 1100 can perform a neighborhood-aware pixel transformation operation 1304 to determine whether the pixel value of interest (e.g., one of the 25 pixel values ​​in image data 1302) is greater than the center pixel value. Figure 13 In the image data 1302, the center pixel value is 78. For example, core 208 and / or core controller 1100 can perform a neighborhood-aware pixel transformation operation 1304 to output a value "1" when the pixel value of interest is greater than or equal to 78 and an output value "0" when the pixel value of interest is not greater than 78. In such an example, core 208 and / or core controller 1100 can generate the image data 1302 by performing the neighborhood-aware pixel transformation operation 1304 on the pixel values ​​included in the image data 1302. Figure 13 Image data 1306 depicting the transformation.

[0122] exist Figure 13 In the illustrated example, core 208 and / or core controller 1100 can generate an example bit signature 1308 based on transformed image data 1306. Figure 13 In the image data 1306, the transformed image data is a 5×5 grid or square, with the center data value having an "X" or an irrelevant value. Core 208 and / or core controller 1100 can generate a bit signature 1308 by arranging the transformed image data 1306 into a bit string of size or length 24 bits, excluding the "X" value. Alternatively, the bit signature 1308 can be of a different size. Core 208 and / or core controller 1100 can generate bit data (e.g., pixel data of a census transform) 1310 for an example transform by replacing the center pixel value of image data 1302 (e.g., a pixel value of 78) with the value corresponding to the bit signature 1308. For example, core 208 and / or core controller 1100 can convert the binary value 110001100011000001101111 of the bit signature 1308 into the decimal value 12988527. In such an example, the decimal value 12988527 could correspond to a transformed pixel data value (e.g., a census transform value). Alternatively, the transformed pixel data 1310 could have a different number of pixel values ​​and / or a different arrangement of pixel values.

[0123] In some examples, core 208 and / or core controller 1100 generate one or more image data statistics by executing and / or otherwise running transformation workflow 1300. For example, core 208 and / or core controller 1100 can be Figure 12 The pixel window 1202 generates transformed pixel data values. In such an example, core 208 and / or core controller 1100 can determine a first transformed pixel data value corresponding to the first pixel in pixel window 1202, a second transformed pixel data value corresponding to the second pixel in pixel window 1202, and so on. In some examples, core 208 and / or core controller 1100 determines one or more image data statistics by determining the maximum, minimum, and sum values ​​of the transformed pixel data values. In some examples, core 208 and / or core controller 1100 can transfer one or more image data statistics based on the transformed pixel data values ​​to... Figure 2 LSE 206 Figure 10 LSE controller 1000, etc.

[0124] Advantageously, Figure 13 The transformation workflow 1300 can be pixel-invariant and / or pixel value neighborhood-aware operation, because the neighborhood-aware pixel transformation operation 1304 is for applications with sensor noise (e.g., relative to sensor noise). Figure 1 The image data source 104 is robust to minute changes in incoming pixel data (e.g., flipping in one or more pixel data bits) caused by associated electrical noise and / or signal noise, transmission noise, etc.

[0125] exist Figures 14-15 The diagram shows the representation used for implementation. Figure 10 LSE controller 1000, and / or Figure 11 The example hardware logic, machine-readable instructions, hardware-implemented state machines, and / or any combination thereof of the core controller 1100 are described. Machine-readable instructions can be used by a computer processor (such as those combined below). Figure 16 The processor 1612 shown in the example processor platform 1600 described below and / or in conjunction with the processor 1612 shown below Figure 17 The executable program or part of an executable program is executed by the processor 1712 shown in the example processor platform 1700 described. This program may be embodied on a non-transitory computer-readable storage medium (such as a CD-ROM, floppy disk, hard disk drive, digital versatile disc (DVD), Blu-ray disc, or similar media). Figure 16 Processor 1612 and / or Figure 17 The software is located in the memory associated with the processor 1712, but the entire program and / or parts thereof may alternatively be provided by other means. Figure 16 Processor 1612 and / or Figure 17The processor 1712 is executed on devices other than the processor itself and / or embodied in firmware or dedicated hardware. Furthermore, while the example program is for reference only... Figure 14-15 The illustrated flowcharts are used to describe the implementation, but many other methods of implementing the example LSE controller 1000 and / or example core controller 1100 can be used alternatively. For example, the execution order of the blocks can be changed, and / or some of the described blocks can be changed, eliminated, or combined. Additionally or alternatively, any or all blocks can be implemented by one or more hardware circuits (e.g., discrete and / or integrated analog and / or digital circuits, FPGAs, ASICs, comparators, operational amplifiers (op-amps), logic circuits, etc.) configured to perform the corresponding operations without executing software or firmware.

[0126] As mentioned above, Figures 14-15 Example processes can be implemented using executable instructions (e.g., computer and / or machine-readable instructions) stored on non-transitory computer and / or machine-readable media (such as hard disk drives, flash memory, read-only memory, optical disks, digital multifunction disks, caches, random access memory, and / or any other storage device or storage disk), where information can be stored for any duration (e.g., long periods, permanent, transient situations, for temporary buffering, and / or information caching). As used herein, the term non-transitory computer-readable media is explicitly defined to include any type of computer-readable storage device and / or storage disk and excludes propagation signals and transmission media.

[0127] "Comprising" (and all its forms and tenses) is an open-ended term. Therefore, as long as a claim uses any form of "comprising" or "including" (e.g., including, comprising, having, etc.) in its preamble or in any type of claim statement, additional elements, terms, etc., may be present without exceeding the scope of the corresponding claim or statement. As used herein, when the phrase "at least" is used as a transitional term, for example, in the preamble of a claim, it is open-ended in the same way as the term "comprising". The term "and / or", when used in the form of, for example, A, B, and / or C, refers to any combination or subset of A, B, C, such as (1) A alone, (2) B alone, (3) C alone, (4) A and B, (5) A and C, (6) B and C, and (7) A and B and C. As used herein in the context of describing structures, components, items, objects, and / or things, the phrase "at least one of A and B" means an implementation including any of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, as used herein in the context of describing structures, components, items, objects, and / or things, the phrase "at least one of A or B" refers to an implementation that includes any of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. As used herein in the context of describing the implementation or execution of processes, instructions, actions, activities, and / or steps, the phrase "at least one of A or B" refers to an implementation that includes any of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, as used herein in the context of describing the implementation or execution of processes, instructions, actions, activities, and / or steps, the phrase "at least one of A or B" refers to an implementation that includes any of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B.

[0128] Figure 14 This is a flowchart representing example machine-readable instructions 1400 that can be executed to detect frame freeze conditions. In some examples, Figure 14 The machine-readable instructions 1400 can be executed to implement LSE206, and / or more generally, Figure 2 HWA 204. In some examples, Figure 14 The machine-readable instructions 1400 can be executed to implement Figure 10 The LSE controller 1000.

[0129] Figure 14Machine-readable instructions 1400 begin at box 1402, where the hardware accelerator (HWA) is configured to compute cyclic redundancy check (CRC) values ​​on one or more output channels of the image frame of interest. For example, HWA configurator interface 714 ( Figure 7 The HWA configurator interface 1040 can obtain configuration information and configure output channel 708 based on the configuration information to calculate (one or more) CRC values ​​of the first image frame of interest (e.g., image frame [n]). In other examples, the HWA configurator interface 1040... Figure 10 The thread controller 1010 can be configured. Figure 10 ) or CRC logic controller 1060 ( Figure 10 At least one of them is used to determine the CRC value of image frame [n].

[0130] At box 1404, obtain pixel output data. For example, one or more core interfaces 704 ( Figure 7 ) can be obtained from core 208 ( Figure 2 ) to obtain pixel output data (e.g., pixel values, image data statistics, etc.). In other examples, the core interface 1020 ( Figure 10 ) can be controlled from different controllers and / or control logic (such as the core controller 1100). Figure 11 Get pixel output data.

[0131] At box 1406, calculate (one or more) CRC values. For example, (one or more) CRC logic circuits 404 ( Figure 4 The controller can generate corresponding CRC values ​​for one or more output channels 708. Output channels 708 may correspond to processed image data, census transformation data, and / or one or more image data statistics. In other examples, the CRC logic controller 1060 can calculate CRC values ​​for one or more output channels of interest, each of which may correspond to data obtained from different controllers and / or control logic.

[0132] At box 1408, it is determined whether the end of the image frame of interest has been reached. For example, one or more CRC logic circuits 404 and / or CRC logic controller 1060 can determine whether the frame corresponding to the CRC logic has been processed. Figure 12 The entire image frame of image data 1200. In such an example, one or more CRC logic circuits 404 and / or CRC logic controller 1060 can acquire the frame end signal, frame end data bits, etc., associated with the corresponding output channel.

[0133] If the end of the image frame of interest has not been reached at box 1410, control returns to box 1404 to obtain additional pixel output data. If the end of the image frame of interest has been reached at box 1410, then at box 1410, one or more CRC values ​​are stored as one or more image frame [n] CRC values. For example, one or more CRC logic circuits 404 can store the corresponding values ​​in SRAM 410. Figure 4 The CRC values ​​of one or more image frames processed in the CRC logic controller 1060 are stored and associated (e.g., their associated CRC values ​​are stored) with the image frame[n]. In other examples, the CRC logic controller 1060 may store one or more CRC values ​​in a database 1070. Figure 10 In this context, the stored CRC values ​​(one or more) are associated (e.g., their associations are stored) with the corresponding image frame [n].

[0134] At box 1412, one or more CRC values ​​are transmitted to the host device to compare one or more CRC values ​​of image frame [n] with one or more CRC values ​​of image frame [n-1]. For example, LSE 206 may store one or more CRC values ​​of image frame [n] (e.g., the first image frame) in SRAM 410 for subsequent transmission to the first host device 214A via the data stream path. Figure 2 The data flow path includes the SRAM interface 408. Figure 4 HWA DMA 416 Figure 4 ), Data movement structure 218 ( Figure 2 ) or host device interface 230 ( Figure 2 At least one of the following. In other examples, the CRC logic controller 1060 may obtain from the database 1070 one or more first CRC values ​​associated with image frame [n] and one or more second CRC values ​​associated with image frame [n-1] (e.g., a second image frame acquired and / or processed prior to the first image frame). Alternatively, the CRC logic controller 1060 may transmit one or more first CRC values ​​and one or more second CRC values ​​to different devices for analysis.

[0135] At box 1414, a comparison by the host device determines whether a matching indication has been obtained. For example, the first host device 214A may determine that one or more CRC values ​​corresponding to image frame [n] match one or more CRC values ​​corresponding to image frame [nl]. In such an example, the first host device 214A may generate an alarm, indication, etc., indicating that a potential frame freeze condition has been detected based on CRC values ​​associated with the processed image data, census transformation data, and / or (one or more) image data statistics. In other examples, the CRC logic controller 1060 may determine that one or more first CRC values ​​associated with image frame [n] and one or more second CRC values ​​associated with image frame [nl] match. In such an example, the CRC logic controller 1060 may generate an alarm, indication, etc., indicating that a potential frame freeze condition has been detected.

[0136] If no matching indication based on the host device comparison is obtained at box 1414, control returns to box 1404 to obtain additional pixel output data. If a matching indication based on the host device comparison has been obtained at box 1414, then at box 1416, image frame [n] is identified as an image frame freeze candidate. For example, the first host device 214A may obtain additional information (e.g., diagnostic parameters, image data statistics, etc.) from SRAM 410 for further processing. In other examples, the CRC logic controller 1060 may obtain additional information from database 1070 for further processing.

[0137] At box 1418, an alarm indicating image frame freeze detection is generated. For example, the first host device 214A can determine that an image frame freeze condition has been detected based on at least one of a CRC value match or additional information from SRAM 410. In such an example, the first host device 214A can instruct the display controller 228 ( Figure 2 Display an alarm on display panel 210 or take other corrective or mitigating measures (e.g., instruct display panel 210 not to use data from...). Figure 1 The image data source 104 continues until the image frame freeze condition is no longer detected and / or has been resolved. In response, an alarm indicating image frame freeze detection is generated at box 1418. Figure 14 The machine-readable instruction 1400 ends.

[0138] Figure 15 This is a flowchart representing example machine-readable instruction 1500, which can be executed to detect frame freeze conditions based on different pixel processing operations. In some examples, Figure 15 The machine-readable instructions 1500 can be executed to implement Figure 2 HWA 204. In some examples, Figure 15 The machine-readable instructions 1500 can be executed to implement Figure 10 LSE controller 1000 and / or Figure 11 The nuclear controller 1100.

[0139] Figure 15 The machine-readable instruction 1500 begins at box 1502, where pixel data is acquired. For example, LSE206 can acquire pixel data from image data interface 216. Figure 2 SRAM 410 Figure 4 ) etc., to acquire image data corresponding to and / or otherwise including pixel data. In other examples, thread controller 1010 ( Figure 10 ) can be obtained from image data source 104 ( Figure 1 ), Database 1070 ( Figure 10 (e.g., to obtain pixel values)

[0140] At box 1504, one or more first CRC values ​​are determined based on pixel data. For example, one or more CRC logic circuits 404 may determine one or more first CRC values ​​based on pixel data from image data interface 216, SRAM 410, etc. In other examples, CRC logic controller 1060 ( Figure 10 The first CRC value (one or more) can be determined based on pixel data from image data source 104, database 1070, etc.

[0141] At box 1506, the pixel data acquired at box 1502 is transformed. For example, core 208 ( Figure 2 It can be based on image data 1200 ( Figure 12 ) Generate transformed pixel data 1310 ( Figure 13 In other examples, image data processor 1130 may generate transformed pixel data 1310 based on image data 1200.

[0142] At box 1508, one or more second CRC values ​​are determined based on the transformed pixel data. For example, one or more CRC logic circuits 404 can be based on... Figure 13 The transformed pixel data 1310 determines one or more second CRC values. In other examples, the CRC logic controller 1060 may determine one or more second CRC values ​​based on the transformed pixel data 1310.

[0143] In response to at least one of acquiring pixel data at box 1502 or transforming pixel data at box 1506, one or more image data statistics are generated at box 1510 based on the pixel data. For example, kernel 208 and / or parameter calculator 1140 may generate one or more image data statistics based on the pixel data acquired at box 1502. In other examples, kernel 208 and / or parameter calculator 1140 may generate one or more image data statistics based on the pixel data transformed at box 1506. In yet another example, kernel 208 and / or parameter calculator 1140 may generate one or more image data statistics based on at least one of the pixel data acquired at box 1502 or transformed pixel data at box 1506.

[0144] In response to generating one or more image data statistics based on pixel data at block 1510, determining one or more third CRC values ​​based on the image data statistics at block 1512. For example, one or more CRC logic circuits 404 may determine one or more third CRC values ​​based on one or more image data statistics from core 208. In other examples, CRC logic controller 1060 may determine one or more third CRC values ​​based on one or more image data statistics determined by parameter calculator 1140 of core controller 1100.

[0145] At box 1514, a frame freeze condition is detected based on at least one of one or more first CRC values, one or more second CRC values, or one or more third CRC values. For example, the first host device 214A ( Figure 2 It can detect, identify, and / or otherwise recognize images from data source 104. Figure 1The associated frame freeze condition. In such an example, the first host device 214A may compare one or more first CRC values ​​of a first image frame of interest with one or more CRC values ​​of a second image frame of interest that was acquired and / or otherwise processed prior to the first image frame of interest. The first host device 214A may detect a frame freeze condition when the one or more CRC values ​​do not match and / or are otherwise different. In other examples, the first host device 214A and / or the CRC logic controller 1060 may (1) perform a first comparison by comparing one or more first CRC values ​​of the first image frame of interest with one or more first CRC values ​​of a second image frame of interest that was acquired and / or otherwise processed prior to the first image frame of interest, (2) perform a second comparison by comparing one or more second CRC values ​​of the first image frame of interest with one or more second CRC values ​​of the second image frame of interest, and / or (3) perform a third comparison by comparing one or more third CRC values ​​of the first image frame of interest with one or more third CRC values ​​of the second image frame of interest. In such an example, the first host device 214A and / or the CRC logic controller 1060 can detect a frame freeze based on at least one of a first comparison, a second comparison, or a third comparison of the differences in the generated CRC values. In response to detecting a frame freeze at block 1514 based on at least one of one or more first CRC values, one or more second CRC values, or one or more third CRC values, Figure 15 The machine-readable instruction 1500 ends.

[0146] Figure 16 It is constructed to execute Figure 14 and / or Figure 15 Instructions to be implemented Figure 10 Block diagram of an example processor platform 1600 for the LSE controller 1000. The processor platform 1600 can be, for example, a server, personal computer, workstation, self-learning machine (e.g., neural network), mobile device (e.g., cellular phone, smartphone, such as iPad). TM Tablets, personal digital assistants (PDAs), headphones or other wearable devices or any other type of computing device.

[0147] The illustrated example processor platform 1600 includes a processor 1612. The illustrated example processor 1612 is hardware. For example, the processor 1612 can be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor can be a semiconductor-based (e.g., silicon-based) device. In this example, processor 1612 is implemented... Figure 10 Example thread controller 1010, example data multiplexer 1050 and example CRC logic controller 1060.

[0148] The illustrated example processor 1612 includes local memory 1613 (e.g., cache). The illustrated example processor 1612 communicates via bus 1618 with main memory, which includes volatile memory 1614 and non-volatile memory 1616. Volatile memory 1614 may be SDRAM, DRAM, etc. It may be implemented using flash memory and / or any other type of random access memory device. The non-volatile memory 1616 may be implemented using flash memory and / or any other desired type of memory device. Access to the main memory 1614, 1616 is controlled by the memory controller.

[0149] The illustrated example processor platform 1600 also includes interface circuitry 1620. Interface circuitry 1620 can be implemented using any type of interface standard, such as an Ethernet interface, Universal Serial Bus (USB), etc. Interfaces, Near Field Communication (NFC) interfaces and / or PCI Fast Interfaces.

[0150] In the illustrated example, one or more input devices 1622 are connected to interface circuitry 1620. The input devices 1622 allow the user to type data and / or commands into processor 1612. The input devices 1622 can be implemented, for example, audio sensors, microphones, cameras (still or video), keyboards, buttons, mice, touchscreens, touchpads, trackballs, isotope devices, and / or voice recognition systems.

[0151] One or more output devices 1624 are also connected to the interface circuitry 1620 of the illustrated example. The output devices 1624 may be implemented, for example, as display devices (e.g., LED, OLED, LCD, CRT, IPS, touchscreen, etc.), haptic output devices, printers, and / or speakers. Therefore, the interface circuitry 1620 of the illustrated example may include a graphics driver, a graphics driver chip, and / or a graphics driver processor.

[0152] The illustrated example interface circuit 1620 also includes communication devices (such as transmitters, receivers, transceivers, modems, residential gateways, wireless access points, and / or network interfaces) to facilitate data exchange with external machines (e.g., any kind of computing device) via network 1626. Communication can be via, for example, Ethernet connections, Digital Subscriber Line (DSL) connections, telephone line connections, coaxial cable systems, satellite systems, field wireless systems, cellular telephone systems, etc. In this example, interface circuit 1620 is implemented... Figure 10Example core interface 1020, example memory interface 1030, and example HWA configurator interface 1040.

[0153] The illustrated example processor platform 1600 also includes one or more mass storage devices 1628 for storing software and / or data. Examples of such mass storage devices 1628 include floppy disk drives, hard disk drives, compact disk drives, Blu-ray disc drives, redundant array of independent disks (RAID) systems, and DVD drives. In this example, one or more mass storage devices 1628 are implemented... Figure 10 Example database 1070.

[0154] Figures 14-15 The machine-executable instructions 1632 can be stored in a mass storage device 1628, volatile memory 1614, non-volatile memory 1616 and / or a removable non-transitory computer-readable storage medium such as a CD or DVD.

[0155] Figure 17 It is constructed to execute Figure 15 Instructions to be implemented Figure 11 A block diagram of an example processor platform 1700 with a core controller 1100. The processor platform 1700 can be, for example, a server, personal computer, workstation, self-learning machine (e.g., neural network), mobile device (e.g., cellular phone, smartphone), or tablet (e.g., iPad). TM Personal digital assistants (PDAs), headsets or other wearable devices, or any other type of computing device.

[0156] The illustrated example processor platform 1700 includes a processor 1712. The illustrated example processor 1712 is hardware. For example, the processor 1712 can be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor can be a semiconductor-based (e.g., silicon-based) device. In this example, processor 1712 is implemented... Figure 11 Example core configurator 1120, example image data processor 1130, and parameter calculator 1140.

[0157] The illustrated example processor 1712 includes local memory 1713 (e.g., cache). The illustrated example processor 1712 communicates via bus 1718 with main memory, which includes volatile memory 1714 and non-volatile memory 1716. The volatile memory 1714 may be SDRAM, DRAM, etc. It may be implemented using flash memory and / or any other type of random access memory device. The non-volatile memory 1716 may be implemented using flash memory and / or any other desired type of storage device. Access to the main memory 1714, 1716 is controlled by the memory controller.

[0158] The illustrated example processor platform 1700 also includes interface circuitry 1720. Interface circuitry 1720 can communicate via any type of interface standard (such as Ethernet, USB, etc.). It can be implemented using interfaces such as NFC interfaces and / or PCI fast interfaces.

[0159] In the illustrated example, one or more input devices 1722 are connected to interface circuitry 1720. The input devices 1722 allow the user to type data and / or commands into processor 1712. The input devices 1722 can be implemented, for example, audio sensors, microphones, cameras (still or video), keyboards, buttons, mice, touchscreens, touchpads, trackballs, isotope devices, and / or voice recognition systems.

[0160] One or more output devices 1724 are also connected to the interface circuitry 1720 of the illustrated example. The output devices 1724 may be implemented, for example, as display devices (e.g., LED, OLED, LCD, CRT monitors, IPS monitors, touchscreens, etc.), haptic output devices, printers, and / or speakers. Therefore, the interface circuitry 1720 of the illustrated example may include a graphics driver card, a graphics driver chip, and / or a graphics driver processor.

[0161] The illustrated example interface circuit 1720 also includes communication devices such as transmitters, receivers, transceivers, modems, residential gateways, wireless access points, and / or network interfaces to facilitate data exchange with external machines (e.g., any type of computing device) via network 1726. Communication can be conducted via, for example, Ethernet connections, DSL connections, telephone line connections, coaxial cable systems, satellite systems, field wireless systems, cellular telephone systems, etc. In this example, interface circuit 1720 is implemented... Figure 11 Example LSE interface 1110.

[0162] The illustrated example processor platform 1700 also includes one or more mass storage devices 1728 for storing software and / or data. Examples of such mass storage devices 1728 include floppy disk drives, hard disk drives, compact disk drives, Blu-ray disc drives, RAID systems, and DVD drives.

[0163] Figure 15The machine-executable instructions 1732 can be stored in mass storage devices 1728, volatile memory 1714, non-volatile memory 1716 and / or removable non-transitory computer-readable storage media (such as CDs or DVDs).

[0164] Based on the foregoing, it should be understood that example systems, methods, apparatuses, and articles of art have been described to improve the detection of image frame freeze conditions. Advantageously, the examples described herein determine CRC values ​​and can detect image frame freeze conditions based on the CRC values ​​of consecutive or sequential image frames. The described systems, methods, apparatuses, and articles of art improve the efficiency of using computing devices by comparing bit strings (e.g., 16-bit strings, 32-bit strings, etc.) associated with different image frames rather than comparing all image frames with each other. The described methods, apparatuses, and articles of art thus represent one or more improvements to computer capabilities because data comparisons are significantly reduced compared to alternative techniques (e.g., fewer computational resources required, fewer storage or memory resources required, etc.), and the examples described herein can facilitate substantially real-time detection or memory-to-memory image tube processing.

[0165] This article describes example methods, apparatuses, systems, and artifacts for image frame freeze detection. Other examples and combinations thereof include the following:

[0166] Example 1 includes a hardware accelerator comprising core logic circuitry for generating second image data based on first image data associated with a first image frame, the second image data corresponding to at least one of processed image data, transformed image data, or one or more image data statistics; a load / store engine (LSE) coupled to the core logic circuitry, the LSE determining a first CRC value based on the second image data; and a first interface coupled to a second interface coupled to a memory, the first interface transmitting the first CRC value obtained from the memory to a host device.

[0167] Example 2 includes the hardware accelerator of Example 1, wherein a first interface is used to cause a host device to generate an alarm in response to a first CRC value matching a second CRC value associated with a second image frame captured prior to the first image frame.

[0168] Example 3 includes the hardware accelerator of Example 1, wherein the first image data includes one or more pixel values, the core logic circuit generates a first image data statistic of one or more image data statistics by determining the sum of one or more pixel values, and the LSE determines a first CRC value based on the sum of one or more pixel values.

[0169] Example 4 includes the hardware accelerator of Example 1, wherein the first image data includes one or more pixel values, and wherein core logic circuitry is configured to generate a first image data statistic of one or more image data statistics by determining the minimum value of one or more pixels, and to generate a second image data statistic of one or more image data statistics by determining the maximum value of one or more pixels, and LSE determines a first CRC value based on the first image data statistic or the second image data statistic.

[0170] Example 5 includes the hardware accelerator of Example 1, wherein the first image data includes pixel values, the core logic circuit performs a neighborhood-aware pixel transformation on the pixel values ​​to generate transformed image data, determines a binary value based on a bit string corresponding to the transformed image data, generates the transformed pixel data by replacing the center pixel value of the pixel values ​​with a decimal value corresponding to the binary value, and the LSE determines a first CRC value based on the transformed pixel data.

[0171] Example 6 includes the hardware accelerator of Example 5, where the neighborhood-aware pixel transformation is a census transformation.

[0172] Example 7 includes the hardware accelerator of Example 1, wherein the LSE includes a logic circuit thread, the logic circuit thread including an image data buffer for acquiring first image data, multiple input channels and a multiplexer, the image data buffer being coupled to a first input of the multiplexer, the first input channel of the multiple input channels being coupled to a second input of the multiplexer, the multiplexer being responsive to selecting the image data buffer, the multiplexer facilitating the transfer of the first image data to the core logic circuit.

[0173] Example 8 includes the hardware accelerator of Example 1, wherein the LSE includes a third interface coupled to a core logic circuit, a plurality of output channels including a first output channel, and a plurality of CRC logic circuits including a first CRC logic circuit, a corresponding one of the plurality of CRC logic circuits being coupled to a corresponding one of the third interface and a corresponding one of the plurality of output channels, and the first CRC logic circuit determining a first CRC value based on second image data obtained from the first output channel.

[0174] Example 9 includes a system comprising a camera for capturing image frames, and an integrated circuit coupled to the camera, the integrated circuit including a hardware accelerator (HWA) for generating first image data based on second image data associated with the image frames, the first image data corresponding to at least one of processed image data, transformed image data, or one or more image data statistics, and determining a CRC value based on the first image data; and a host device interface for transmitting the CRC value to a host device to detect image frame freeze conditions based on the CRC value.

[0175] Example 10 includes the system of Example 9, wherein the image frame is a first image frame and the CRC value is a first CRC value, and also includes a host device that generates an alarm in response to a first CRC value matching a second CRC value, the second CRC value being associated with a second image frame captured by a camera prior to the first image frame.

[0176] Example 11 includes the system of Example 9, further including an image data interface for acquiring image frames, and wherein the integrated circuit includes a data movement structure coupled to the image data interface and the HWA, a memory interface coupled to the data movement structure, a memory coupled to the memory interface, the memory for storing second image data, and the HWA for acquiring the second image data from the memory before generating the first image data.

[0177] Example 12 includes the system of Example 9, further including an image data interface for acquiring image frames, and wherein the integrated circuit includes a data movement structure coupled to the image data interface and HWA, a memory interface coupled to the data movement structure, a memory coupled to the memory interface, and HWA is used to store at least one of the first image data or CRC values ​​in the memory after determining at least one of the first image data or CRC values.

[0178] Example 13 includes the system of Example 9, wherein the second image data includes one or more pixel values, and HWA generates a first image data statistic of one or more image data statistics by determining the minimum or maximum value among the one or more pixel values, and determines a CRC value based on the minimum or maximum value of the one or more pixel values.

[0179] Example 14 includes the system of Example 9, wherein the second image data includes pixel values, and HWA is used to perform a transformation operation on the pixel values ​​to generate transformed image data, determine binary values ​​based on the bit string corresponding to the transformed image data, generate transformed pixel data by replacing the center pixel value of the pixel values ​​with a decimal value corresponding to the binary value, and determine a CRC value based on the transformed pixel data.

[0180] Example 15 includes the system of Example 9, further including a display panel coupled to an integrated circuit, and the integrated circuit includes an image data interface for acquiring image frames from a camera; a data moving structure coupled to the image data interface, the data moving structure acquiring image frames from the image data interface; a memory interface coupled to the data moving structure, the memory interface acquiring image frames from the data moving structure; a memory coupled to the memory interface, the memory storing the image frames acquired from the memory interface; and a display controller coupled to the data moving structure, the display controller transmitting image frames from the data moving structure to the display panel.

[0181] Example 16 includes a method comprising generating first image data based on second image data associated with a first image frame captured by a camera, the first image data corresponding to at least one of processed image data, transformed image data, or one or more image data statistics; determining a first CRC value based on the first image data; comparing the first CRC value with a second CRC value associated with a second image frame captured by the camera prior to the first image frame; and generating an alarm indicating that an image frame freeze condition has been detected in response to matching the first CRC value with the second CRC value based on the comparison.

[0182] Example 17 includes the method of Example 16, which further includes displaying an alarm on a display panel and displaying a third image frame captured by the camera after the first image frame in response to the absence of detected image frame freezing.

[0183] Example 18 includes the method of Example 16, wherein the second image data includes one or more pixel values, and further includes generating one or more image data statistics by determining the sum of one or more pixel values, the minimum of one or more pixel values, or the maximum of one or more pixel values, and determining a first CRC value based on the sum of one or more pixel values, the minimum of one or more pixel values, or the maximum of one or more pixel values.

[0184] Example 19 includes the method of Example 16, wherein the second image data includes pixel values, and further includes performing a transformation operation on the pixel values ​​to generate transformed image data, determining a binary value based on a bit string corresponding to the transformed image data, generating the transformed pixel data by replacing the center pixel value of the pixel values ​​with a decimal value corresponding to the binary value, and determining a first CRC value based on the transformed pixel data.

[0185] Example 20 includes the method of Example 16, wherein a comparison of a first CRC value with a second CRC value is a first comparison, and further includes determining a third CRC value based on second image data, determining a fourth CRC value based on pixel data of a first transform associated with the second image data, performing a second comparison of the third CRC value with a fifth CRC value associated with the second image frame, the fifth CRC value being based on the third image data associated with the second image frame, performing a third comparison of the fourth CRC value with a sixth CRC value associated with the second image frame, the sixth CRC value being based on pixel data of a second transform associated with the third image data, and generating an alarm in response to at least one of the first comparison, the second comparison, or the third comparison indicating a difference between the corresponding CRC values.

[0186] Example 21 includes the method of Example 16, wherein the vehicle electronic control unit compares a first CRC value with a second CRC value, and further includes not displaying a first image frame on a display panel in response to generating an alarm, and displaying the first image frame on a display panel in response to a mismatch between the first CRC value and the second CRC value.

[0187] Although certain example systems, methods, apparatuses, and articles of manufacture have been described herein, the scope of this patent is not limited thereto. Rather, this patent covers all systems, methods, apparatuses, and articles of manufacture that fall fully within the scope of the claims of this patent.

Claims

1. A hardware accelerator, comprising: A core logic circuit is configured to generate census transformation data by performing at least a census transformation on the pixel values ​​of the first image data based on first image data associated with a first image frame, arrange the census transformation data into a bit string having binary values, and generate transformed pixel data by replacing selected pixel values ​​of the pixel values ​​with decimal values ​​corresponding to the binary values, thereby generating second image data. The load / store engine, or LSE, is coupled to the core logic circuitry and is configured to determine a first cyclic redundancy check value, or a first CRC value, based on the transformed pixel data. as well as An interface configured to transmit the first CRC value to a host device.

2. The hardware accelerator of claim 1, wherein the interface is configured to cause the host device to generate an alarm in response to a match between the first CRC value and a second CRC value, and the second CRC value is associated with a second image frame captured prior to the first image frame.

3. The hardware accelerator of claim 1, wherein the core logic circuit is configured to generate a first image data statistic by determining the sum of the pixel values, and the LSE is configured to determine a second CRC value based on the sum of the pixel values.

4. The hardware accelerator of claim 1, wherein the core logic circuit is configured to: A first image data statistic is generated by determining the minimum value of the pixel value; and A second image data statistic is generated by determining the maximum value of the pixel; and The LSE is configured to determine the first CRC value based on the first image data statistics or the second image data statistics.

5. The hardware accelerator of claim 1, wherein the selected pixel value is the center pixel value of the pixel value.

6. The hardware accelerator of claim 1, wherein the core logic circuit is configured to generate image data statistics by determining the sum of squared values ​​of the pixel values, and wherein the LSE is configured to determine a second CRC value based on the sum of squared values.

7. The hardware accelerator of claim 1, wherein the LSE includes a logic circuit thread, the logic circuit thread comprising: An image data buffer configured to acquire the first image data; Multiple input channels; as well as Multiplexer The image data buffer is coupled to a first input of the multiplexer, the first input channel of the plurality of input channels is coupled to a second input of the multiplexer, and the multiplexer is configured to facilitate the transmission of the first image data to the core logic circuit in response to selecting the image data buffer.

8. The hardware accelerator of claim 1, wherein the interface is a first interface, and the LSE comprises: The second interface is coupled to the core logic circuit; Multiple output channels, including a first output channel; as well as Multiple CRC logic circuits, including a first CRC logic circuit. One of the plurality of CRC logic circuits is coupled to one of the second interface and one of the plurality of output channels, and the first CRC logic circuit is configured to determine the first CRC value based on the second image data obtained from the first output channel.

9. A system for image processing, comprising: A camera, which is configured to capture image frames; An image data interface, coupled to the camera, and configured to acquire the image frames from the camera; as well as An integrated circuit coupled to the image data interface, the integrated circuit comprising: A data movement structure coupled to the image data interface; The hardware accelerator, or HWA, is coupled to the data movement structure and configured to: Based on first image data associated with the image frame, census transformation data is generated by performing at least a census transformation on the first image data, the census transformation data is arranged in a bit string, and second image data is generated by determining binary values ​​based on the bit string; and The cyclic redundancy check value, i.e., the CRC value, is determined based on the binary value generated from the census transformation. A memory interface coupled to the data movement structure; The memory coupled to the memory interface; and A host device interface is configured to transmit the CRC value to a host device to detect image frame freeze status based on the CRC value.

10. The system of claim 9, wherein the image frame is a first image frame and the CRC value is a first CRC value, wherein the system further includes the host device configured to generate an alarm in response to the first CRC value matching a second CRC value, and wherein the second CRC value is associated with a second image frame captured by the camera prior to the first image frame.

11. The system of claim 9, wherein the memory is configured to store the first image data; and The HWA is configured to retrieve the first image data from the memory before generating the second image data.

12. The system of claim 9, wherein the HWA is configured to store at least one of the second image data or the CRC value in the memory after determining at least one of the second image data or the CRC value.

13. The system of claim 9, wherein the CRC value is a first CRC value, the first image data includes pixel values, and the HWA is configured to: A first image data statistic is generated by determining the minimum or maximum value of the pixel value; and The second CRC value is determined based on the minimum or maximum value of the pixel value.

14. The system of claim 9, wherein the first image data comprises pixel values, and the HWA is configured to: Transformed pixel data is generated by replacing the center pixel value of the pixel value with a decimal value corresponding to the binary value; and The CRC value is determined based on the transformed pixel data.

15. The system of claim 9, further comprising a display panel coupled to the integrated circuit, wherein the data movement structure is configured to acquire the image frame from the image data interface; The memory interface is configured to acquire the image frame from the data movement structure; and The memory is configured to store the image frames acquired from the memory interface; and The integrated circuit further includes a display controller coupled to the data movement structure, the display controller being configured to transmit the image frame from the data movement structure to the display panel.

16. A method for image processing, comprising: Based on first image data associated with a first image frame captured in the first time, census transformation data is generated by performing at least a census transformation on the pixel values ​​of the first image data, the census transformation data is arranged into a bit string with binary values, and transformed pixel data is generated by replacing selected pixel values ​​of the pixel values ​​with decimal values ​​corresponding to the binary values, thereby generating second image data. The first cyclic redundancy check value, i.e., the first CRC value, is determined based on the binary value generated from the census transformation. The first CRC value is compared with a second CRC value, which is associated with a second image frame captured at a second time prior to the capture of the first image frame; and In response to matching the first CRC value with the second CRC value based on comparison, an alarm indicating that an image frame freeze condition has been detected is generated.

17. The method of claim 16, further comprising: The alarm is displayed on the display panel; as well as In response to the absence of detected image frame freezing, a third image frame captured by the camera after the first image frame is displayed.

18. The method of claim 16, wherein the first image data comprises pixel values, and wherein the method further comprises: Determine the sum of the pixel values, the minimum value of the pixel values, or the maximum value of the pixel values; The third CRC value is determined based on the sum of the pixel values, the minimum value of the pixel values, or the maximum value of the pixel values; The third CRC value is compared with a fourth CRC value, which is associated with the second image frame captured before the first image frame was captured; and In response to a comparison of the third CRC value with the fourth CRC value, an alarm indicating that an image frame freeze condition has been detected is generated.

19. The method of claim 16, wherein the selected pixel value is the center pixel value of the pixel value.

20. The method of claim 16, wherein the comparison between the first CRC value and the second CRC value is a first comparison, and wherein the method further comprises: Determine the third CRC value based on the first image data; A fourth CRC value is determined based on pixel data of a first transform associated with the first image data; A second comparison is performed between the third CRC value and a fifth CRC value associated with the second image frame, the fifth CRC value being based on the third image data associated with the second image frame; A third comparison is performed between the fourth CRC value and a sixth CRC value associated with the second image frame, the sixth CRC value being based on pixel data of a second transformation associated with the third image data; and The alarm is generated in response to at least one of the first comparison, the second comparison, or the third comparison indicating a difference between the corresponding CRC values.

21. The method of claim 16, wherein the vehicle electronic control unit is configured to compare the first CRC value with the second CRC value, and wherein the method further comprises: In response to the generation of the alarm, the first image frame is not displayed on the display panel; and In response to a mismatch between the first CRC value and the second CRC value, the first image frame is displayed on the display panel.

Images