Electronic device and method for object region determination based on analysis of pixels within acceptance range
By analyzing pixels within a programmable acceptance range to determine object regions, the electronic device reduces computational resources, enhancing real-time object detection and classification efficiency.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-10-01
- Publication Date
- 2026-04-09
AI Technical Summary
Conventional neural network-based object detection models consume high computational resources, impacting system performance during real-time object region identification.
An electronic device analyzes pixels within a programmable acceptance range to determine object regions using column and row activation profiles, reducing the need for extensive computational resources by selecting and processing only relevant pixels.
This approach minimizes computational requirements and power consumption while enabling efficient real-time object detection and classification, leveraging minimal processing time and memory usage.
Smart Images

Figure IB2024059610_09042026_PF_FP_ABST
Abstract
Description
Attorney Docket No. GCI1001ELECTRONIC DEVICE AND METHOD FOR OBJECT REGION DETERMINATION BASED ON ANALYSIS OF PIXELS WITHIN ACCEPTANCE RANGECROSS-REFERENCE TO RELATED APPLICATIONS / INCORPORATION BY REFERENCE
[0001] None.FIELD OF TECHNOLOGY
[0002] Various embodiments of the disclosure relate to object region determination from image pixels. More specifically, various embodiments of the disclosure relate to an electronic device and a method for object region determination based on analysis of image pixel content within an acceptance range.BACKGROUND
[0003] Recent advancements in the field of image processing have led to development of various techniques related to object detection from images. Typically, various machine learning, neural network, or deep-learning based techniques or models have evolved to detect and classify objects within one or more image frames. Generally, such models (such as Convolutional Neural Networks (CNN)) run a trained model on sensed images in which each grid in the images is scanned to determine object locations and classes. In certain cases, the images are split into a fixed number of overlapping grids and a classification algorithm (like neural network based) runs on each grid, where each grid may be assigned with one or more classes. In certain cases, a region proposal network (such as Region-based Convolutional Neural Network (R-CNN)) operates on the image to select one or more candidate regions which are further processed by a secondary CNN to perform the classification / detection. However, such neural network based models (like those used to select or identify candidate regions) may consume a high amount of computational resources (like high processing time and memory space). This may further impact the performance of a system (including such models) during their operational phase (for example during identification of object regions in real-time).
[0004] Further limitations and disadvantages of conventional and traditional approaches will become apparent to one of skill in the art, through comparison of suchAttorney Docket No. OCI1001 systems with some aspects of the present disclosure as set forth in the remainder of the present application with reference to the drawings.SUMMARY
[0005] A system and a method for determination of object regions based on analysis of pixels within an acceptance range, are provided substantially as shown in and / or described in connection with at least one of the figures, as set forth more completely in the claims.
[0006] These and other features and advantages of the present disclosure may be appreciated from a review of the following detailed description of the present disclosure, along with the accompanying figures in which like reference numerals refer to like parts throughout.BRIEF DESCRIPTION OF THE DRAWINGS
[0007] FIG. 1 is a diagram of an exemplary network environment including an electronic device for determination of an object region based on analysis of pixels within a programmable acceptance range, in accordance with an embodiment of the disclosure.
[0008] FIG. 2 is a block diagram that illustrates an electronic device of FIG. 1 for determination of an object region based on analysis of pixels within a programmable acceptance range, in accordance with an embodiment of the disclosure.
[0009] FIG. 3A and FIG. 3B are diagrams which illustrates a highlighted set of pixels within the programmable acceptance range, a column activation profile, and a row activation profile, in accordance with an embodiment of the disclosure.
[0010] FIG. 4 is a diagram that illustrates exemplary operations performed by the electronic device of FIG. 1 for determination of object region based on analysis of pixels within acceptance range, in accordance with an embodiment of the disclosure.
[0011] FIG. 5 is a diagram that illustrates object regions determined based on analysis of the set of pixels, in accordance with an embodiment of the disclosure.
[0012] FIG. 6 is a flowchart that illustrates exemplary operations for determination of object regions based on analysis of pixels, in accordance with an embodiment of the disclosure.Attorney Docket No. OCI1001DETAILED DESCRIPTION
[0013] The following described implementations may be found in an electronic device and a method for automatic object region determination (and object estimation) based on analysis of pixels within acceptance range with the utilization of minimal computational resources. Exemplary aspects of the disclosure provide an electronic device that may include a sensing array (for example image sensing array) which may further include a plurality of pixels. The sensing array may be configured to select a set of pixels from the plurality of pixels, where each pixel of the set of pixels may be within an acceptance range of pixel values (i.e. programmable or user defined pixel value or intensity). The sensing array may further output the set of pixels within the acceptance range. The sensing array may include inbuilt pixel circuitry and a pixel memory. The electronic device may further include circuitry configured to read a pixel value (for example intensity of imaging pixel) from each of the set of pixels or from each of the plurality of pixels. The circuitry may readout the set of pixels for further processing and may not read other pixels of the plurality of pixels which may not be within the acceptance range, thus reducing the computational resources to read and process all the pixels. The circuitry may be configured to determine activation profiles (like a column activation profile and / or a row activation profile) based on the set of pixels within the acceptance range. Each of the column activation profile and the row activation profile may correspond to a one dimensional vector which may indicate either a number of the set of pixels in each column / row or a sum of pixel values of each pixel of the set of pixels in each column / row. In some embodiments, the circuitry may be configured to select the set of pixels from the plurality of pixels read from the sensing array, where each pixel of the set of pixels is within the acceptance range of pixel values (i.e. programmable).
[0014] The circuitry may be configured to determine column peak information and row peak information based on the determined column activation profile and the row activation profile. The column peak information and the row peak information may indicate a column and a row with local maxima of number of pixels (within the acceptance range) and information about a starting column / row and about height and width of a region formed by the set of pixels. The circuitry may be further configured to determine one or more object regions (i.e. candidate regions) in a captured image (including the plurality of pixels) based on the determined column peak information and the row peak information. The one or more object regions determined based on the analysis of pixels (i.e. determination of theAttorney Docket No. OCI1001 set of pixels, activation profiles (i.e. one dimensional vectors) and column / row peak information) by the disclosed electronic device may utilize lesser computational resources (like processing time or memory usage) as compared to a traditional system that may utilize a trained neural network or machine learning model to determine object region from a captured image. The circuitry may be further configured to transmit information about the determined one or more object regions to an external processing device which may further estimate, detect, or classify objects based on the received information about one or more objection regions determined by the disclosed electronic device. The information received from the electronic device may further facilitate the external processing device to consume lesser computational resources for the detection or classification of the objects from a captured image on real time basis.
[0015] FIG. 1 is a diagram of an exemplary network environment including an electronic device for determination of object region based on analysis of pixels within a programmable acceptance range. With reference to FIG. 1 , there is shown a network environment 100. The network environment 100 may include an electronic device 102 which may further include a sensing array 104. The sensing array 104 may further include a plurality of pixels 106 arranged in a form of a matrix including a plurality of rows and a plurality of columns, as shown in FIG. 1. The plurality of pixels 106 may include different pixels (such a pixel 106A). Each pixel (like the pixel 106A) of the plurality of pixels 106 may further include different components, but limited to, a sensor 107A (for example a photodiode), an electronic front-end logic 107B, pixel circuitry 107C, and a pixel memory 107D as shown in FIG. 1. The electronic device 102 may further include circuitry 108. The network environment 100 may further include an external processing device 110 communicably coupled with the electronic device 102 via a communication network 112. The electronic device 102 and the external processing device 110 are shown as two separate devices; however, in some embodiments, the entire functionality of the external processing device 110 may be included in the electronic device 102, without a deviation from scope of the disclosure.
[0016] The electronic device 102 may comprise suitable logic, circuitry, interfaces, and / or code that may be configured to determine object regions from one or more captured image frames. The electronic device 102 may be configured to determine a set of pixels from the plurality of pixels 106, where each of the set of pixels may be within a programmable acceptance range. The determination of the set of pixels is described, forAttorney Docket No. OCI1001 example, in FIG. 3A, FIG. 3B, and FIG. 4 (at step 402). The electronic device 102 may be further configured to determine activation profiles (like the column activation profile and the row activation profile) as described further, for example, in FIG. 3A, FIG. 3B, and FIG. 4 (at step 404). The electronic device 102 may be further configured to determine the column peak information and the row peak information based on the determined activation profiles as described, for example, in FIG. 4 (at step 406). The electronic device 102 may be further configured to determine one or more object regions in the captured frames based on the analysis of the plurality of pixels 106 (i.e. determination of the set of pixels, the activation profiles, the column / row pixel information) as described, for example, in FIG. 4 (at step 408). The electronic device 102 may be further configured to estimate one or more objects in the captured image frame (or in the plurality of pixels 106) based on the determined one or more object regions as described, for example, in FIG. 4 (at step 410). In some embodiments, the electronic device 102 may control the external processing device 110 to estimate (i.e. detect or classify) the one or more objects based on information (about the determined one or more object regions) transmitted by the electronic device 102 to the external processing device 110 as described, for example, in FIG. 4 (at step 410). Examples of the electronic device 102 may include, but are not limited to, an imaging device, an image sensing device, an image capture device, a camera, an image sensor, a Light Detection and Ranging (LIDAR) system, an object region identification device, an object detection device, a low processing computing device, a wireless device, a controller system, a server, a mainframe machine, a computer work-station, a smartphone, a cellular phone, a mobile phone, and / or a consumer electronic (CE) device. Examples of the electronic device 102 may further include, but are not limited to, a wide-angle camera, an action camera, a closed-circuit television (CCTV) camera, a camcorder, a digital camera, camera phones, a time-of-flight camera (ToF camera), a night-vision camera, and / or other image capture devices.
[0017] The sensing array 104 may include suitable logic, circuitry, interfaces and / or code that may be configured to receive light from one or more objects being captured or present in a field of view (FOV) of the electronic device 102. The light may be received or captured by the sensor 107A (for example a photo diode, a photo voltaic cell, or light sensing element) in each of the plurality of pixels 106. The sensor 107A may generate an electrical signal (or analog voltage) equivalent to the light received. The sensing array 104 may further convert the received light into pixel values in terms of light intensity (forAttorney Docket No. OCI1001 example a brightness or contrast value) to further form one or more image frames. Each pixel (like the pixel 106A) may include the electronic front-end logic / circuit 107B that may be configured to receive the electrical signal from the sensor 107A and perform various electronic processing on the received electric signal, such as, but is not limited to, an analog-digital conversion (ADC), gain control, and the like. The sensing array 104 may be in a matrix form as shown in FIG. 1 and may include the plurality of pixels 106 arranged in a plurality of columns and a plurality of rows. The image frame formed by the sensing array 104 may include pixel values related to the plurality of pixels 106. Each pixel value related to the pixel (for example the pixel 106A) may correspond to an output generated by the electronic front-end logic 107B. Each of the plurality of pixels 106 may represent the smallest unit of information in an image frame captured by the electronic device 102 or by an image sensor (not shown) associated with the electronic device 102. Each of the plurality of pixels 106 may be light sensitive units (including the sensor 107A) of a particular shape that may be arranged in a form of grid or matrix which may include the plurality of rows and the plurality of columns (as shown in FIG. 1. In an embodiment, each pixel of the sensing array 104 may further have a logic or circuitry (like the pixel circuitry 107C) that may process the pixel value received from the electronic front-end logic 107B for each pixel. The pixel circuitry 107C of each pixel may read the corresponding pixel value of each of the plurality of pixels and determine the set of pixels within the acceptance range. The pixel circuitry 107C of each pixel may determine whether the pixel value of the corresponding pixel may be included in the set of pixels or may be within the acceptance range. The pixel circuitry 107C may further provide the pixel values of the determined set of pixels to the circuitry 108 for further processing.
[0018] It may be noted that the pixel circuitry 107C shown in each pixel is merely presented as an example. In some embodiments, the sensing array 104 may include one pixel circuitry (like the pixel circuitry 107C) for all the plurality of pixels 106 or for a group of pixels (such as region-of-interest (ROI), without deviation from the scope of the disclosure. In some embodiments, the pixel circuitry 107C may output the pixel values of the plurality of pixels 106 to the circuitry 202 for further processing.
[0019] The pixel value of each of the set of pixels (i.e. determined within the acceptance range) or the plurality of pixels 106 may be stored in the pixel memory 107D for any further processing inside the sensing array 104. In an embodiment, the pixel memory 107D may store information about the programmable acceptance range orAttorney Docket No. OCI1001 different thresholds defined for each of the plurality of pixels 106 or store information about the set of pixels determined from the plurality of pixels 106. The pixel memory 107D may further store pixel values of the set of pixels or the plurality of pixels 106 of one or more previous frames (like consecutive image frames). Such stored pixel values inside the sensing array 104 may facilitate quick calculation of pixel value differences (i.e. between current or previous frames) inside the sensing array 104. In some embodiment, the pixel memory 107D may store a template value related to a reference pixel value, based on which pixel value differences may be determined. It may be noted that the pixel memory 107D shown in each pixel is merely presented as an example. In some embodiments, the sensing array 104 may include one pixel memory (like the pixel memory 107D) for all the plurality of pixels 106 or for a group of pixels (such as region-of-interest (ROI), without deviation from the scope of the disclosure.
[0020] The circuitry 108 may be configured to read or receive pixel values from the pixel circuitry 107C or from the pixel memory 107D. The circuitry 108 may read or receive pixel values based on horizontal and vertical scanning and selection of at least one of the plurality of pixels 106. In some embodiments, the circuitry 108 may select few pixels (like selected region of interest) to read out the respective pixel values. Each pixel of the plurality of pixels 106 may have a specific color (like white, black, or RGB) and brightness value. The number of pixels in an image determines the resolution and detail of the image captured. Higher number of pixels in the plurality of pixels 106 may provide higher- resolution images with fine details. The plurality of pixels 106 may be monochromatic or may have color filters (like RGB) or may have different wavelength filters. Examples of the sensing array 104 may include, but are not limited to, an image sensor, a charge-coupled device (CCD), a CMOS based image sensor, a photo diode, and other image capture sensors. The sensing array 104 may be classified in different manner, for example, based on their structure, chroma, shutter type, resolution, frame rate, pixel size, and / or sensor format.
[0021] It may be noted here that the position, orientation, arrangement, and shape of the plurality of pixels 106 shown in FIG. 1 is presented merely as an example. The present disclosure may also be applicable to other positions, orientations, arrangements, and shapes of the plurality of pixels 106, without deviation from the scope of the disclosure. Further, six number of columns and rows of the plurality of pixels 106 shown in FIG. 1 is presented merely as an example. The plurality of pixels 106 may include only four pixelsAttorney Docket No. OCI1001 in 2 * 2 matrix or more pixels in different pixel resolutions, for example, but not limited to, 320 * 240 pixels matrix, 640 x 480 pixels matrix, 1024 * 768 pixels matrix, 1024 * 1024 pixels matrix, 1280 * 720 pixels matrix, 1280 * 960 pixels matrix, 1280 * 1024 pixels matrix, 1600 * 1280 pixels matrix, 1920 * 1080 pixels matrix, 3840 * 2160 pixels matrix, 4096 * 2160 pixels matrix, or any other pixel configuration for image capture applications, without deviation from the scope of the disclosure. For the sake of brevity, only six of rows and columns have been shown in FIG. 1. However, in some embodiments, there may be more than six number of rows and columns may be present in the matrix formed by the plurality of pixels 106, without limiting the scope of the disclosure.
[0022] The circuitry 108 may include suitable logic, circuitry, interfaces and / or code that may be configured to execute program instructions associated with different operations to be executed by the electronic device 102. For example, some of the operations may include, but not limited to, receipt of pixels values from the sensing array (including the pixel circuitry 107C), determination of the set of pixels, determination of the column activation profile and the row activation profile, determination of the column peak information and the row peak information, determination of the one or more object regions, and control for the estimation of the objects within the determined one or more object regions. In some embodiments, the circuitry 108 may include one or more specialized processing units, which may be implemented as a separate processor. In an embodiment, the one or more specialized processing units may be implemented as an integrated processor or a cluster of processors that perform the functions of the one or more specialized processing units, collectively. The circuitry 108 may be implemented based on a number of processor technologies known in the art. Examples of implementations of the circuitry 108 may be an X86-based processor, a Graphics Processing Unit (GPU), a Reduced Instruction Set Computing (RISC) processor, an Application-Specific Integrated Circuit (ASIC) processor, a Complex Instruction Set Computing (CISC) processor, a microcontroller, a central processing unit (CPU), and / or other control circuits.
[0023] The external processing device 110 may include suitable logic, circuitry, interfaces and / or code that may be configured to receive information about the detected one or more object regions from the electronic device 102. The external processing device 110 may further detect or classify one or more objects in the image frames captured by the image sensor (not shown) associated with the electronic device 102. The external processing device 110 may detect or classify the objects based on the received informationAttorney Docket No. OCI1001 (i.e. about the detected one or more object regions) from the electronic device 102. In an embodiment, the external processing device 110 may include a trained machine learning or neural network model to detect or classify the object based on the information received from the electronic device 102. The external processing device 110 may not determine the object or candidate regions (or may not include or utilize the trained machine learning or neural network model to determine the object regions), due to which the external processing device 110 may utilize lesser computational resources to detect or classify the objects in the captured image frames received from the electronic device 102. Examples of the external processing device 110 may include, but are not limited to, an object detection device, an object classification device, a computing device, a smartphone, a cellular phone, a mobile phone, a gaming device, a mainframe machine, a server, a computer work-station, an autonomous vehicle system and / or a consumer electronic (CE) device.
[0024] Further, in an embodiment, the external processing device 110 may comprise, but is not limited to, a server, for example, a database server, a file server, a web server, a media server, an application server, a mainframe server, or a cloud computing server. In such case, the external processing device 110 may be configured to store information about, but is not limited to, the detected objects, captured image frames, information about the set of pixels, determined activation profiles, the column peak information, the row peak information or different neural network or machine learning models. In some embodiments, the external processing device 110 may be configured to store information about the acceptance ranges to determine the set of pixels and different thresholds (programmable) based on which the set of pixels may be determined. In some embodiments, the external processing device 110 may be configured to store video content including multiple image frames from which the one or more object regions to be determined. The external processing device 110 may communicate with the electronic device 102 to provide any requested information for the automated determination of the object regions and object classification.
[0025] The communication network 112 may include a communication medium through which the electronic device 102 and the external processing device 110 may communicate with each other. The communication network 112 may be one of a wired connection or a wireless connection Examples of the communication network 112 may include, but are not limited to, the Internet, a cloud network, a Wireless Fidelity (Wi-Fi)Attorney Docket No. OCI1001 network, a Personal Area Network (PAN), a Local Area Network (LAN), or a Metropolitan Area Network (MAN). Various devices in the network environment 100 may be configured to connect to the communication network 112 in accordance with various wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, at least one of a Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), Zig Bee, EDGE, IEEE 802.11 , light fidelity (Li-Fi), 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless access point (AP), device to device communication, mobile / cellular communication protocols, and Bluetooth (BT) communication protocols.
[0026] In some embodiments, the communication network 112 may correspond to a wireless network that may include a medium through which two or more wireless nodes may communicate with each other. Also, the wireless network may include a medium through which the external processing device 110 and / or other network-related devices in the wireless network may provide communication with each other. The wireless network may be established in accordance with Institute of Electricals and Electronics Engineers (IEEE) standards for infrastructure mode (Basic Service Set (BSS) configurations), or in some specific cases, in ad hoc mode (Independent Basic Service Set (I BSS) configurations). The wireless network may be a Wireless Sensor Network (WSN), a Mobile Wireless Sensor Network (MWSN), a wireless ad hoc network, a Mobile Ad-hoc Network (MANET), a Wireless Mesh Network (WMN), a Wide Area Network (WAN), a Wireless Local Area Network (WLAN), a cellular network, a Long Term Evolution (LTE) network, an Evolved High Speed Packet Access (HSPA+), a 3G network, a 4G network, a 5G network, and the like. The wireless network may operate in accordance with IEEE standards, such as 802 wireless standards or a modified protocol, which may include, but are not limited to, 802.3, 802.15.1 , 802.16 (Wireless local loop), 802.20 (Mobile Broadband Wireless Access (MBWA)), 802.11-1997 (legacy version), 802.15.4, 802.11a, 802.11 b, 802.11g, 802.11e, 802.11i, 802.11f, 802.11c, 802.11 h (specific to European regulations) 802.11 n, 802.11j (specific to Japanese regulations), 802.11p, 802.11ac, 802.11ad, 802.11ah, 802.11aj, 802.11ax, 802.11ay, 802.11az, 802.11 hr (high data rate), 802.11af (white space spectrum), 802.11-2007, 802.11-2008, 802.11-2012, 802.11-2016.
[0027] In operation, the disclosed electronic device 102 may receive a request (for example, either from the user or from the external processing device 110) to determineAttorney Docket No. OCI1001 object regions and / or to estimate the objects present in the one or more image frames (including the plurality of pixels 106). In some embodiments, the request may be internally generated by the electronic device 102 on periodic intervals or whenever one or more objects enters a current image frame. The circuitry 108 of the electronic device 102 may be configured to receive one or more image frames (which may include the plurality of pixels 106) from the sensing array 104. The sensing array 104 or the pixel circuitry 107C (i.e. related to one or more pixels) may determine the set of pixels from the plurality of pixels 106, where the pixel values of the determined set of pixels are within the programmable acceptance range. The pixel circuitry 107C may retrieve the information about the predefined acceptance range (i.e. stored in the pixel memory 107D) and further determine the set of pixels from the plurality of pixels 106. The circuitry 108 of the electronic device 102 may further receive the pixel values of such set of pixels within the acceptance range for further processing. Other pixels of the plurality of pixels 106 (i.e. not within the acceptance range) may not be read or received by the circuitry 108 from the pixel circuitry 107C or from the sensing array 104. In some embodiments, the circuitry 108 may be further configured to determine the set of pixels from the plurality of pixels 106 read from the sensing array. The details of the determination of the set of pixels and the acceptance range are further provided, for example, in FIG. 3A, FIG. 3B, and FIG. 4 (at step 402). The circuitry 108 may be further configured to determine the column activation profile and the row activation profile based on the determined set of pixels, where each of the column activation profile and the row activation profile may correspond to a one-dimensional array. The details of the determination of the column activation profile and the row activation profile are further provided, for example, in FIG. 3A, FIG. 3B, and FIG. 4 (at step 404). The circuitry 108 may be further configured to determine the column peak information and the row peak information based on the determined column activation profile and the row activation profile, where the column peak information and the row peak information may indicate a pair (“X”, “W”) and a pair (“Y”, “H”), as further described, for example, in FIG. 4 (at step 406).
[0028] The circuitry 108 may be further configured to determine one or more object regions based on the determined column peak information and the row peak information. The determined one or more object regions are further described, for example, in FIG. 4 (at step 408) and FIG. 5. The circuitry 108 may be further configured to estimate one or more objects present in the image frame(s) based on the determined one or more objectAttorney Docket No. OCI1001 regions as further described, for example, in FIG. 4 (at step 410). In some embodiments, the circuitry 108 may be configured to transmit information about the determined one or more object regions (i.e. candidate regions or all pixel values in each region) and / or about the pixel values of the set of pixels to the external processing device 110 which may further estimate or classify the objects (like as a human, animal, vehicle, etc.) based on the information transmitted by the circuitry 108 of the disclosed electronic device 102. The estimation of the one or more objects by the external processing device 110 is further described, for example, in FIG. 4 (at step 412).
[0029] FIG. 2 is a block diagram that illustrates an electronic device of FIG. 1 for determination of object region based on analysis of pixels within a programmable acceptance range, in accordance with an embodiment of the disclosure. FIG. 2 is explained in conjunction with elements from FIG. 1. With reference to FIG. 2, there is shown a block diagram 200 of the electronic device 102 that may be coupled to the external processing device 110, via the communication network 112. The electronic device 102 may further include circuitry 202, a memory 204, an Input / output (I / O) device 206, and a network interface 208. The electronic device 102 may further include the sensing array 104 as described for example, in FIG. 1. In some embodiments, the electronic device 102 may be communicably coupled to the sensing array 104 and the sensing array 104 may not be integrated in the electronic device 102. The electronic device 102 may connect to the communication network 112, via the network interface 208.
[0030] The circuitry 202 may include suitable logic, circuitry, interfaces and / or code that may be configured to execute program instructions associated with different operations to be executed by the electronic device 102. The functions of the circuitry 202 may be same as the functions of the circuitry 108 described, for example, in FIG. 1. Therefore, the description of the circuitry 202 is omitted from the disclosure for the sake of brevity.
[0031] The memory 204 may comprise suitable logic, circuitry, interfaces and / or code that may be configured to store pixel values captured from the plurality of pixels 106, the acceptance range to determine the set of pixels, the determined column activation profile, the row activation profile, the determined column peak information and the row peak information, aspect ratio information, information about the one or more object regions determined in the capture image frame(s), and information about shape of the one or more object regions. In some embodiments, the memory 204 may be configured to differentAttorney Docket No. OCI1001 threshold values. Examples of implementation of the memory 204 may include, but are not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Hard Disk Drive (HDD), a Solid- State Drive (SSD), a CPU cache, and / or a Secure Digital (SD) card.
[0032] The I / O device 206 may comprise suitable logic, circuitry, interfaces, and / or code that may be configured to act as an I / O channel / interface between a user (not shown) and the electronic device 102. The I / O device 206 may comprise various input and output devices, which may be configured to communicate with different operational components of the electronic device 102. For example, the I / O device 206 may receive information about the acceptance range (or thresholds) to determine the set of pixels or may receive video information (or other media content) to determine one or more object regions of the objects included in the video information. In some embodiments, the electronic device 102 may receive information to select or discard a particular object region from a user, via the I / O device 206. Further, the I / O device 206 may output information about the determined one or more object regions, detected objects or determined activation profiles or peak information (for either column or row). Examples of the I / O device 206 may include, but are not limited to, a touch screen, a keyboard, a mouse, a joystick, a microphone, and a display screen. The display screen may be a touch screen which may enable a user to provide a user-input via the display screen. The touch screen may be at least one of a resistive touch screen, a capacitive touch screen, or a thermal touch screen. The display screen may be realized through several known technologies such as, but not limited to, at least one of a Liquid Crystal Display (LCD) display, a Light Emitting Diode (LED) display, a plasma display, or an Organic LED (OLED) display technology, or other display devices. In accordance with an embodiment, the display screen may refer to a display screen of a head mounted device (HMD), a smart-glass device, a see-through display, a projectionbased display, an electro-chromic display, or a transparent display.
[0033] The network interface 208 may comprise suitable logic, circuitry, interfaces, and / or code that may be configured to facilitate communication with the external processing device 110, with other on-chip circuits, or with other network devices, via the communication network 112. The network interface 208 may be implemented by use of various known technologies to support wired or wireless communication with the communication network 112. The network interface 208 may include, but is not limited to, an antenna, a radio frequency (RF) transceiver, one or more amplifiers, a tuner, one orAttorney Docket No. OCI1001 more oscillators, a digital signal processor, a coder-decoder (CODEC) chipset, a subscriber identity module (SIM) card, or a local buffer circuitry. The network interface 208 may be configured to communicate via wireless communication with networks, such as the Internet, an Intranet, a wireless network, a cellular telephone network, a wireless local area network (LAN), or a metropolitan area network (MAN). The wireless communication may be configured to use one or more of a plurality of communication standards, protocols and technologies, such as Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), wideband code division multiple access (W-CDMA), Long Term Evolution (LTE), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g or IEEE 802.11 n), voice over Internet Protocol (VoIP), light fidelity (Li-Fi), Worldwide Interoperability for Microwave Access (Wi-MAX), a protocol for email, instant messaging, and a Short Message Service (SMS).
[0034] The functions or operations executed by the electronic device 102, as described in FIG. 1 , may be performed by the circuitry 202. The operations executed by the circuitry 108 or the circuitry 202 are described in detail, for example, in FIGs. 3-6.
[0035] FIG. 3A and FIG. 3B are diagrams which illustrates a highlighted set of pixels within the programmable acceptance range, a column activation profile, and a row activation profile, in accordance with an embodiment of the disclosure. FIG. 3A and FIG. 3B are explained in conjunction with elements from FIG. 1 and FIG. 2. With reference to FIG. 3A, there is shown a diagram 300. As shown, the diagram 300 may indicate the plurality of pixels 106 which may be included in the sensing array 104 as described, for example, in FIG. 1. The plurality of pixels 106 may be arranged in the form of matrix and may include the plurality of columns (i.e. columns “C1” to “C6”) and the plurality of rows (i.e. rows "R1" to "R6") as shown, for example, in FIG. 3A.
[0036] At a time of capture of an image frame, each of the plurality of pixels 106 may capture or indicate a pixel value (i.e. intensity of light brightness or a color value). The pixel value may be generated by the corresponding pixel based on light received or reflected from one or more objects present in the field of view (FOV) of the electronic device 102. For example, as shown in FIG. 3A, pixel P11 (i.e. at column “C1” and row “R1”) may indicate a pixel value of “1”, pixel P12 (i.e. at column “C2” and row “R1”) may indicate a pixel value of “4”, pixel P13 (i.e. at by column “C3” and row “R1”) may indicate a pixel value of “16”, and so on. Similarly, pixel P21 (i.e. at column “C2” and row “R1”) may indicate aAttorney Docket No. OCI1001 pixel value of “3”, pixel P22 (i.e. at column “C2” and row “R2”) may indicate a pixel value of “5”, pixel P23 (i.e. at by column “C3” and row “R2”) may indicate a pixel value of “16”, and so on. It may be noted that the pixel value of each pixel of the plurality of pixels 106 shown in FIG. 1 is presented merely as an example. The pixel value of different pixels may vary based on, but is not limited to, received light, a type of image sensor, angle at which image is captured, resolution of the pixels, color information of objects being captured, distance between the image sensor and objects, or other imaging parameters. The present disclosure may also be applicable to other pixel values. A description based on other pixel values has been omitted from the disclosure for the sake of brevity.
[0037] In an embodiment, the circuitry 202 of the disclosed electronic device 102 may be configured to receive or read pixel values of different pixels from the sensing array 104 or from the pixel circuitry 107C of different pixels. For example, the pixel values of the set of pixels within the programmable acceptance range may be read or received by the circuitry 2020 from the sensing array 104. The sensing array 104 (including the pixels) may include a logic or another circuitry (like the pixel circuitry 107C) that may be configured to determine the set of pixels from the plurality of pixels 106 and further provide the determined set of pixels (i.e. only pixels within the acceptance range) to the circuitry 202 for further processing. In some embodiments, the circuitry 202 may retrieve the pixel values of the set of pixels stored in the corresponding pixel memory (like the pixel memory 107D inside the sensing array 104). In another embodiment, the circuitry 202 may receive information about the set of pixels (like pixel coordinates or column / row numbers) either from the pixel circuitry 107C or from the pixel memory 107D of different pixels, to further read-out the pixels values of the set of pixels based on the received information.
[0038] In another embodiment, the circuitry 202 may be configured to select or determine the set of pixels from the plurality of pixels 106, where the pixel value of each of the set of pixels is within the acceptance range. The acceptance range may be predefined or may be received from a user (not shown) of the electronic device 102. The user may define the acceptance range of the pixel based on application area for the object region detection or object classification or based on other factors (for example lighting conditions, environmental conditions, type of image sensor integrated or coupled to the electronic device 102, or based on region of interest (ROI) selected / defined for the determination of the set of pixels or object regions). The acceptance range may define a range of pixel values based on which the set of pixels may be detected or read-out fromAttorney Docket No. OCI1001 all the plurality of pixels 106. For example, the acceptance range may define a minimum pixel value and a maximum pixel value, and in case a real-time pixel value of a particular pixel is with the defined acceptance range, only that particular pixel may be read-out by the circuitry 202 and processed further by the disclosed electronic device 102 for the detection of the object region and object classification. The acceptance range may be stored within the sensing array 104, for example the pixel memory 107D may store the acceptance range for each pixel or for a particular region of pixels. For example, the predefined acceptance range may indicate a pixel value range from “5” to “14”, then each of the pixel values of the plurality of pixels 106 within the acceptance range may be selected as the set of pixels and further read-out by the sensing array 104 (or the pixel circuitry 107C) or by the circuitry 202 for further processing. As shown in FIG. 3A, for example, pixel values of different pixels, like P22 (with pixel value “5”), P32 (with pixel value “6”), P33 (with pixel value “11”), P34 (with pixel value “14”), P43 (with pixel value “5”), P44 (with pixel value “9”), and P53 (with pixel value “7”) are within the acceptance range (“5” to “14”). Therefore, such pixels (i.e. P22, P32, P33, P34, P43, P44, and P53) may be determined as the set of pixels from the plurality of pixels 106 (i.e. “P1” to “P66”). As shown in FIG. 3A, for example, the set of pixels (i.e. P22, P32, P33, P34, P43, P44, and P53) are highlighted. The circuitry 202 (or the sensing array 104) may be configured to determine the set of pixels from the plurality of pixels 106 and the circuitry 202 may further read-out only the set of pixels out of the plurality of pixels 106. In an embodiment, the circuitry 202 may flag the set of pixels (or their locations in the plurality of pixels 106) to read-out for further processing (like determination of the activation profiles, column / row peak information and object region). The circuitry 202 may not read-out other pixels (i.e. other than the set of pixels) from the plurality of pixels 106. In other words, the circuitry 202 may not further process other pixels (i.e. other than the set of pixels) from the plurality of pixels 106. Therefore, the selection and read-out of only the set of pixels (which are within the acceptance range) reduce the computation resources (like processing time, memory space, etc.) to process the selected pixels, rather than processing of all the plurality of pixels 106. This may further reduce power consumption and latency for the electronic device 102, especially during operational phase or real-time applications. In an embodiment, the acceptance range may be programmable and may be modified based on different application areas or situations. The set of pixels may be referred to as programmable acceptance interval (PAI) pixels. In some embodiment, the acceptance range may be an absolute threshold minimum value (for example stored within the sensingAttorney Docket No. OCI1001 array 104, like within the pixel memory 107D). The sensing array 104 (or the pixel circuitry 107C) may determine all the pixels as the set of pixels for which the corresponding pixel value is equal or higher than the predefined absolute threshold minimum value. In another embodiment, the acceptance range may be an absolute threshold maximum value (for example stored within the sensing array 104, like within the pixel memory 107D). The sensing array 104 (or the pixel circuitry 107C) may determine all the pixels as the set of pixels for which the corresponding pixel value is equal or lower than the predefined absolute threshold maximum value. The absolute threshold minimum value or the absolute threshold maximum value may be same or different for all the plurality of pixels 106.
[0039] In an embodiment, the circuitry 202 may be configured to update the acceptance range (as programmable) for one or more of the plurality of pixels 106 based on different factors (like, but not limited to, user inputs, lighting conditions of surroundings, weather conditions, ambient temperature, number of objects presented in the field of view, types of objects to be detected, color information of the objects, resolution of pixels, type of image sensor, focus or zoom related to the image sensor of the electronic device 102, or application areas (like medical, surveillance, vehicular navigation, sports, entertainment, and the like). In some embodiments, the circuitry 202 may update (or adjust) the acceptance range based on pixel values of the plurality of pixels 106. For example, in case there is a selection of region of interest (ROI) or selection of a particular set of column or rows, and a number of the set of pixels (with the acceptance range) may be lesser than a specific threshold, then the circuitry 202 may update the acceptance range to increase the number of the set of pixels or to determine more number of set of pixels within the selected ROI or within a particular set of columns / rows. Therefore, the circuitry 202 or the sensing array 104 may be configured to update the acceptance range based on the pixel values of the set of pixels (or all pixel values of the plurality of pixels 106) being read-out. The updated acceptance range may be further stored inside the sensing array 104 (for example within the pixel memory 107D of each pixel, like the pixel 106A) for further determination of the set of pixels (i.e. PAI pixels). In an embodiment, the acceptance range may be defined or different for each pixel or for different groups of pixels of the plurality of pixels 106. In such case, the sensing array 104 or the circuitry 202 may compare the captured pixel value of a particular pixel with the acceptance range pre-defined for that pixel and accordingly select that pixel in the set of pixels within the acceptance range. In another embodiment, the circuitry 202 may be configured to update the acceptance range for oneAttorney Docket No. OCI1001 or more of the plurality of pixels 106 based on the determined column activation profile and / or the row activation profile as described later in the present disclosure. In some embodiments, the circuitry 202 may update different threshold values based on the pixel values of the set of pixels (or pixel values of one or more pixels of the plurality of pixels 106) or based on user inputs. Such threshold values may be utilized to select the captured pixels to read-out and further processing. The updated threshold values may be further stored in the pixel memory 107D in the sensing array 104 to determine the set of pixels or activated pixels.
[0040] In an embodiment, the circuitry 202 may be configured to select the region of interest (ROI) from the plurality of pixels 106, The ROI may be selected based on the user inputs or based on application area or real-time situations (like estimation of a presence of one or more objects in the image frame, number of anticipated objects, size of objects, specific region to focus). For example, in case the electronic device 102 needs to detect the entry of an object from left direction, then the circuitry 202 may select the region of interest (ROI) as per initial set of columns (like “C1”, “C2”, and so on). The circuitry 202 may be configured to select particular number of rows or columns to select a particular region of interest (ROI). In certain situations, different regions of interest (ROIs) may be selected or updated for different image frames based on aforementioned factors related to presence of objects or object regions. Information about the selected region of interest (ROI) may be stored in the pixel memory 107D, such that the sensing array 104 or the pixel circuitry 107C related to the selected region of interest (ROI) may determine whether the corresponding pixel value is within the acceptance range or not. In an embodiment, multiple regions of interest (ROIs) may be selected or defined for one frame based on the expected number of objects to be present in the image frame. The sensing array 104 (or the circuitry 202) may be further configured to determine the set of pixels (within the acceptance range) from the pixels included in the selected region of interest (ROI) for a particular frame. The circuitry 202 may further perform further processing (like determination of activation profiles, column / row peak information, and the object regions, and so on). In an embodiment, the circuitry 202 may be configured to update one or more regions of interest (ROIs) for a particular image frame(s) based on the captured pixel values of each pixel of the set of pixels within the acceptance range or based on analysis of the captured pixel values. For example, in case the majority of pixel values of the selected region of interest (ROI) are not within the acceptance range, then the circuitryAttorney Docket No. OCI1001 202 may change the region of interest (ROI) (or may update the acceptance range) to select additional number of pixels (to be part of the set of pixels) from the plurality of pixels 106 or from the updated region-of-interest. In another example, if the number of the set of pixels (within the acceptance range) is lesser than a predefined threshold for a particular selected region of interest (ROI), the circuitry 202 may modify the region of interest (ROI) to determine a new set of pixels from the plurality of pixels 106 presented in the modified region of interest (ROI).
[0041] In the diagram 300, there is further shown an object region 302 that may indicate a region (also referred as candidate region or a bounding box) which may further include all the determined set of pixels, for example, pixels (P22, P32, P33, P34, P43, P44, and P53) as shown in FIG. 3A. For example, the object region 302 may be the region of interest (ROI). There is further shown a first activation graph 304 and a second activation graph 306. The first activation graph 304 may indicate a number of pixels from the set of pixels (within the acceptance range) in different columns of the plurality of columns. For example, as shown in the first activation graph 304, a first column C1 includes zero pixels of the set of pixels, a second column C2 includes two pixels of the set of pixels, a third column C3 includes three pixels of the set of pixels, a fourth column C4 includes two pixels of the set of pixels, a fifth column C5 includes zero pixels of the set of pixels, and a sixth column C6 includes two pixels of the set of pixels. Similarly, the second activation graph 306 may indicate a number of pixels from the set of pixels (within the acceptance range) in different rows of the plurality of rows. For example, as shown in the second activation graph 306, a first row R1 includes zero pixels of the set of pixels, a second row R2 includes one pixel of the set of pixels, a third row R3 includes three pixels of the set of pixels, a fourth row R4 includes two pixels of the set of pixels, a fifth row R5 includes one pixel of the set of pixels, and a sixth row R6 includes two pixels of the set of pixels.
[0042] The first activation graph 304 may represent a column activation graph to generate a column activation profile of an image frame formed by the plurality of pixels 106. Similarly, the second activation graph may represent a row activation graph to generate a row activation profile of the image frame formed by the plurality of pixels 106. As per the first activation graph 304 and the second activation graph 306 formed based the image frame (including the plurality of pixels 106), the circuitry 202 may be configured to determine the column activation profile and the row activation profile. Each of the column activation profile and the row activation profile may correspond to a one-dimensional array.Attorney Docket No. OCI1001 For example, as per the first activation graph 304, the one-dimensional array of the column activation profile may indicate array values of [0, 2, 3, 2, 0, 0] as shown in FIG. 3A. In other words, the one-dimensional array of the column activation profile may indicate a number of pixels from the set of pixels, in each column of the plurality of columns (“C1” to “C6”) or may indicate the number of activated pixels (within the acceptance range) in each column of a set of columns (“C2, “03”, and “04”) from the plurality of columns (“01” to “06”). In other words, the column activation profile may indicate the number of pixels which passed acceptance range / level in different columns.
[0043] In an embodiment, for example, the disclosed electronic device 102 may utilize following pseudo code or pseudo instructions to determine the column activation profile that may indicate the number of pixels in the set of pixels for each column: col_profile = [] / / initialize one-dimensional array for column fori in range(0, number_of_cols): col_profile[i] = 0 for j in range (0, number_of_rows): if pixel[j,i] is in acceptance_range: col_profile[i] = col_profile[i] + 1
[0044] In some embodiments, rather than processing of all the columns and rows, the circuitry may process limited number of columns and / or rows (like a region of interest (ROI)) to determine the column activation profile. The disclosed electronic device 102 may utilize following pseudo code or pseudo instructions to determine the column activation profile (indicating the number of pixels for the set of pixels) based on limited number of columns and / or rows:0 <= N1 , N2 <= number_of_cols0 <= M1 , M2 <= number_of_rows col_profile = [ ] for i in range(N1 , N2): col_profile[i] = 0Attorney Docket No. OCI1001 for j in range(M1 , M2): if pixel[j, i] is in acceptance_range: col_profile[i] = col_profile[i] + 1
[0045] Similarly, for example, as per the second activation graph 306, the onedimensional array of the row activation profile may indicate array values of [0, 1 , 3, 2, 1 , 0] as shown in FIG. 3A. In otherwords, the one-dimensional array of the row activation profile may indicate a number of pixels from the set of pixels, in each row of the plurality of rows (“R1” to “R6”) or may indicate the number of activated pixels (within the acceptance range) in each row of a set of rows (“R2, “R3”, “R4”, and “R5”) from the plurality of rows (“R1” to “R6”). In other words, the row activation profile may indicate a number of pixels which passed acceptance range / level in different rows. The number of pixels (within the acceptance range) for different columns for different object regions is further described, for example, in FIG. 5.
[0046] In an embodiment, for example, the disclosed electronic device 102 may utilize following pseudo code or pseudo instructions to determine the row activation profile that may indicate the number of pixels in the set of pixels for each row: row_profile = [] / / initialize one-dimensional array for row forj in range(0, number_of_rows): row_profile[j] = 0 fori in range(0, number_of_cols): if pixel[j,i] is in acceptance_range: row_profile[j] = row_profile[j] + 1
[0047] In some embodiments, rather than processing of all the columns and rows, the circuitry may process limited number of columns and / or rows (like a region of interest (ROI)) to determine the row activation profile. The disclosed electronic device 102 may utilize following pseudo code or pseudo instructions to determine the row activation profile (indicating the number of pixels for the set of pixels) based on limited number of columns and / or rows:0 <= N1 , N2 <= number_of_colsAttorney Docket No. OCI10010 <= M1 , M2 <= number_of_rows row_profile = [ ] for j in range(M1 , M2): row_profile[j] = 0 for i in range(N1 , N2): if pixel[j, i] is in acceptance_range: row_profile[j] = row_profile[j] + 1
[0048] In some embodiments, the column activation profile (or related one-dimensional array) may indicate a sum of the pixel value of each of the set of pixels in each column of the set of columns (“C1” to “C6”). For example, in such case, the one-dimensional array of the column activation profile may indicate array values as [0, 11 , 23, 23, 0, 0] which may indicate a sum of pixels values (“5” and “6”) of pixels P22 and P32 for the second column C2, a sum of pixels values (“11”, “5”, and “7”) of pixels P33, P43, and P53 for the third column C3, and a sum of pixels values (“14” and “9”) of pixels P34 and P44 for the fourth column C4. In an embodiment, for example, the disclosed electronic device 102 may utilize following pseudo code or pseudo instructions to determine the column activation profile that may indicate the sum of the pixel value of each of the set of pixels for each column: col_profile = [ ] for i in range(0, number_of_cols): col_profile[i] = 0 forj in range(0, number_of_rows): if pixel[j, i] is in acceptance_range: col_profile[i] = col_profile[i] + pixel[j,i]
[0049] Similarly, the row activation profile (or related one-dimensional array) may indicate a sum of the pixel value of each of the set of pixels in each row of the set of rows (“R1” to “R6”). For example, in such case, the one-dimensional array of the row activation profile may indicate array values as [0, 5, 31 , 14, 7, 0] which may indicate a sum of pixels values (“5”) of pixel P22 for the second row R2, a sum of pixels values (“6”, “11”, and “14”)Attorney Docket No. OCI1001 of pixels P32, P33, and P34 for the third row R3, a sum of pixels values (“5” and “9”) of pixels P43 and P44 for the fourth row R4, and a sum of pixels values (“7”) of pixel P53 for the fifth row R5. In an embodiment, for example, the disclosed electronic device 102 may utilize following pseudo code or pseudo instructions to determine the row activation profile that may indicate the sum of the pixel value of each pixel of the set of pixels for each row: row_profile = [ ] forj in range(0, number_of_rows): row_profile[j] = 0 for i in range(0, number_of_cols): if pixel[j, i] is in acceptance_range: row_profile[j] = row_profile[j] + pixel[j,i]
[0050] The circuitry 202 may be configured to determine the sum of pixels values of the set of pixels for the determination of the column activation profile for limited number of columns and / or rows, rather than processing all the columns / rows as per following pseudo code or pseudo instructions:0 <= N1 , N2 <= number_of_cols0 <= M1 , M2 <= number_of_rows col_profile = [ ] for i in range(N1 , N2): col_profile[i] = 0 forj in range(M1 , M2): if pixel[j, i] is in acceptance_range: col_profile[i] = col_profile[i] + pixel[j,i]
[0051] The circuitry 202 may be configured to determine the sum of pixels values of the set of pixels for the determination of the row activation profile for limited number of columns and / or rows, rather than processing all the columns / rows as per following pseudo code or pseudo instructions:Attorney Docket No. OCI10010 <= N1 , N2 <= number_of_cols0 <= M1 , M2 <= number_of_rows row_profile = [ ] for j in range(M1 , M2): row_profile[j] = 0 for i in range(N1 , N2): if pixel[j, i] is in acceptance_range: row_profile[j] = row_profile[j] + pixel[j,i]
[0052] As previously described, the circuitry 202 may be configured to update the acceptance range based on the pixel values of the set of pixels (or all pixel values of the plurality of pixels 106) being read-out. As the column activation profile and the row activation profile are determined based on the pixel values of the set of pixels, the circuitry 202 may update the acceptance range based on the determined column activation profile and / or the determined row activation profile. The updated acceptance range may be further stored in a memory (like the pixel memory 107D) inside the sensing array 104. Similarly, the circuitry 202 may be configured to select or modify the region of interests (ROIs) based on the determined column activation profile and / or the determined row activation profile which indicates the pixel values of the set of pixels in different columns and rows of the plurality of columns and the plurality of rows, respectively.
[0053] In an embodiment, the circuitry 202 may be configured to process multiple image frames for further processing. For example, the circuitry 202 may select the plurality of pixels 106 from at least two or more image frames and further determine the set of pixels (within the acceptance range), the column activation profile, and the row activation profile based on the pixel values related to the plurality of pixels 106 of the two or more image frames. In certain cases, the circuitry 202 may calculate an average pixel value of a particular pixel from different image frames (like consecutive image frames) and compare the average pixel value with the acceptance range to determine whether the particular pixel has to be included in the set of pixels or not. In certain cases, the circuitry 202 may calculate a pixel value difference of a particular pixel from different image frames (for example, a previous frame with a pixel value) and compare the pixel value difference withAttorney Docket No. OCI1001 the acceptance range (i.e. indicating acceptable pixel differences) and accordingly determine whether the particular pixel has to be included in the set of pixels or not. For example, based on the pixel differences between consecutive image frames, moving objects or regions (like moving humans) may be determined.
[0054] With respect to FIG. 3B, there is shown a current image frame 308 and a previous image frame 310 which may correspond to consecutive image frames. Each of the current image frame 308 and the previous image frame 310 may be captured by the sensing array 104 and their pixel values may be stored with the sensing array 104. As shown, for example, in FIG. 3B, each of the current image frame 308 and the previous image frame 310 may indicate the plurality of pixels formed in the matrix (i.e. with the plurality of columns and the plurality of rows) and each pixel may indicate a pixel value. For example, the pixel memory 107D may store the pixel values of corresponding pixel (such as the pixel 106A) for both the current image frame 308 and the previous image frame 310. The pixel circuitry 107C within the sensing array 104 or the pixel may be configured to determine the pixel value difference or the pixel value absolute difference (such as absolute pixel difference information 312 as shown, for example, in FIG. 3). For example, as shown in FIG. 3B, the pixel value absolute difference for the pixel P23 may be “9” where the corresponding pixel value in the current image frame 308 is “16” and the corresponding pixel value in the previous image frame 310 is “25” as shown, for example, in FIG. 3B. Similarly, as shown in FIG. 3B, the absolute pixel difference information 312 may indicate the pixel value absolute difference for each pixel of the plurality of pixels 106. The pixel value absolute difference between the current image frame 308 and the previous image frame 310 may indicate a change in light intensity (or color information) between consecutive frames. For example, in a situation of moving sensor (like location inside a moving vehicle) or moving objects in front of the sensing array 104, the objects within the field of view may change between two consecutive frames due to entry and exit of objects.
[0055] In an embodiment, the pixel circuitry 107C or the sensing array 104 may be configured to determine the set of pixels from the plurality of pixels based on the determined pixel difference information 312 and the predefined acceptance range (i.e. acceptable pixel differences). The sensing array 104 or the pixel memory 107D may be configured to store the acceptance range (or different thresholds) for the pixel value difference. The pixel circuitry 107C related to the sensing array 104 may determine whether a particular pixel may be within the set of pixels or not if the determined pixel valueAttorney Docket No. OCI1001 difference is within the predefined acceptance range. For example, in case the acceptance range indicates the pixel values between “7” and “9”, then the pixel value difference for pixels P23, P24, P25, P34, P44, P45, and P54 may be within the acceptance range as shown, for example, in FIG. 3B. The pixel circuitry 107C or the sensing array 104 may determine such pixels as the set of pixels (within the acceptance range) based on the pixel value differences or absolute differences which may be determined based on the pixel values of corresponding pixels of the current image frame 308 and the previous image frame 310. As shown in FIG. 3B, for example, an object region 314 (i.e. candidate region) may include such set of pixels (P23, P24, P25, P34, P44, P45, and P54). The pixel memory 107D may further store information about such set of pixels determined based on the pixel value difference between the current image frame 308 and the previous image frame 310. The circuitry 202 of the disclosed electronic device 102 may further only read the determined set of pixels from the pixel memory 107D related to each pixel or related to a group of pixels of the sensing array 104. The circuitry 202 may not read other pixels of the plurality of pixels 106 which may not be determined as the set of pixels (within the acceptance range). Therefore, the set of pixels (i.e. PAI pixels) within the acceptance range may be determined based on the absolute pixel value or based on the pixel value difference of the corresponding pixels.
[0056] In an embodiment, the sensing array 104 (including the pixel memory 107D) may be configured to store a template value. The template value may be an absolute static pixel value (i.e. predefined user value). The pixel circuitry 107C within the sensing array 104 may be configured to determine the pixel value difference between the pixel value of each pixel of the plurality of pixels 106 with the predefined stored template value, rather than with the pixel values of the previous image frame 310. The pixel circuitry 107C for each pixel or for group of pixels in the sensing array 104 may further determine the set of pixels (within the acceptance range) from the plurality of pixels 106 based on the pixel value difference determined based on the pixel values of the current image frame 308 and the stored template value. The circuitry 202 of the disclosed electronic device 102 may further only read the set of pixels determined within the sensing array 104 or stored in the pixel memory 107D within the sensing array 104. In an embodiment, the acceptance range may be updated based on different applications / situations and further stored in the sensing array 104 (for example in the pixel memory 107D). The pixel circuitry 107C and the circuitry 202 may further determine and read the set of pixels, respectively, based on the updatedAttorney Docket No. OCI1001 acceptance range. In an embodiment, the template value may be different for each pixels or may be different for particular columns / rows.
[0057] FIG. 4 is a diagram that illustrates exemplary operations performed by the electronic device of FIG. 1 for determination of object region based on analysis of pixels within acceptance range, in accordance with an embodiment of the disclosure. FIG. 4 is explained in conjunction with elements from FIG. 1 , FIG. 2, FIG. 3A, and FIG. 3B. With reference to FIG. 4, there is shown a block diagram 400. The exemplary operations of the block diagram 400 may be from 402 to 412 and may be performed by any computing system, for example, by the electronic device 102 of FIG. 1 , by the circuitry 108 of FIG. 1 , or by the circuitry 202 of FIG. 2. In an embodiment, step 402 may be performed inside the plurality of pixels 106 of the sensing array 104.
[0058] At 402, a set of pixels may be determined. The sensing array 104 may determine (using the pixel circuitry 107C related to each pixel or a group of pixels) the set of pixels from the plurality of pixels 106, where the pixel values of the determined set of pixels are within the programmable acceptance range. The plurality of pixels 106 may correspond to either one image frame or multiple image frames. The circuitry 108 may further receive or read-out the pixel values of such set of pixels within the acceptance range for further processing from the sensing array 104 (or from the pixel circuitry 107C or the pixel memory 107D). Other pixels of the plurality of pixels 106 (i.e. not within the acceptance range) may not be read or received by the circuitry 108 from the sensing array 104 including the plurality of pixels 106. In some embodiments, the circuitry 202 may be configured to determine the set of pixels from the plurality of pixels 106 which may correspond to either one image frame or multiple image frames. Each pixel of the set of pixels may be within the acceptance range which may be predefined or user defined. In certain situations, the acceptance range is different either for individual pixel of the plurality of pixels or for different set of rows / columns of the plurality of rows / columns. In an embodiment, the acceptance range may change based on pixel values of the plurality of pixels 106 or based on different application areas or situations. The set of pixels may be included in the object region 302 as shown in FIGs. 3A and 4. The determination of the set of pixels from the plurality of pixels 106, the acceptance range, and its update are described, for example, in FIG. 3A. To optimize the processing time and memory consumption, the circuitry 202 may only read-out the set of pixels out of the plurality ofAttorney Docket No. OCI1001 pixels 106 for further processing, like to determine the object regions and / or to estimate the objects present in the determined object regions.
[0059] At 404, activation profiles may be determined. In an embodiment, the circuitry 202 may be configured to determine the activation profiles (like the column activation profile and the row activation profile) based on pixel values of the determined set of pixels as described, for example, in FIG. 3A. Each of the column activation profile and the row activation profile may correspond to the one-dimensional array information that may indicate either the number of pixels of the set of pixels in each column / row, or sum of pixel values of the set of pixels in each column and row, respectively as described, for example, in FIG. 3A. In an embodiment, the column activation profile and / or the row activation profile may not be determined for each column or row but may be for a predefined set of columns or rows which may be defined by the user of the electronic device 102 or may vary based on different application areas / situations. Therefore, particular columns and / or rows may be selected for which the one-dimensional arrays of the column activation profile and the row activation profile may be determined, respectively. The determination of the onedimensional arrays of the activation profiles and further processing based on the determined one-dimensional arrays, rather than processing of two-dimensional array information (like formed by the plurality of pixels 106), may further optimize the processing computations of the disclosed electronic device 102 to determine the objects regions (i.e. candidate regions) and / or to estimate the objects present in the determined object regions.
[0060] As described, for example, in FIG. 3A, the circuitry 202 may update the programmable acceptance range based on the pixel values of the set of pixels or based on the pixel values of the plurality of pixels 106. In some embodiments, the circuitry 202 may update the programmable acceptance range for complete image frame or for certain columns / rows or for different group of pixels based on the determined column activation profile and / or the row activation profile. For example, in case the circuitry 202 determines that number (or sum) of pixels in different columns / row indicated by either of the column activation profile or the row activation profile indicates a value lesser than a predefined threshold, then the circuitry 202 may update the acceptance range to increase the number (or sum) of pixels in the set of pixels.
[0061] At 406, peak information may be determined. In an embodiment, the circuitry 202 may be configured to determine the peak information (i.e. column peak information and row peak information) based on the determined activation profiles. For theAttorney Docket No. OCI1001 determination of the peak information, the circuitry 202 may determine a peak column and / or a peak row based on the determined one-dimensional arrays of the column activation profile and the row activation profile. The column peak information may indicate the peak column from the plurality of columns and the peak column may indicate a local maxima of the number of pixels of the set of pixels, among the plurality of columns. For example, based on the one-dimensional array (i.e. [0, 2, 3, 2, 0, 0] as shown in FIG. 3A) of the column activation profile, the circuitry 202 may determine the third column C3 as a peak column. In another embodiment, the peak column may indicate a local maxima of a sum of pixel values of the set of pixels, among the plurality of columns.
[0062] Similarly, the row peak information may indicate the peak row from the plurality of rows, where the peak row may indicate a local maxima of the number of pixels of the set of pixels, among the plurality of rows. For example, based on the one-dimensional array (i.e. [0, 1 , 3, 2, 1 , 0] as shown in FIG. 3A) of the row activation profile, the circuitry 202 may determine the third row R3 as a peak row for the set of pixels for a particular image frame being processed. In another embodiment, the peak row may indicate a local maxima of a sum of pixel values of the set of pixels, among the plurality of rows. The peak column and the peak row may further indicate a location in the set of pixels with maximum number of pixels in corresponding column or row which are within the acceptance range. In an embodiment, the circuitry 202 may perform a peak finding algorithm on the determined column activation profile and the row activation profile to determine the peak column and the peak row. For example, the circuitry 202 may execute following pseudo code or pseudo instructions, as the peak finding algorithm to determine the peak column and the peak row:# Find local maxima in 1 dimensional vector (find the peaks).# ©param x Vector of data values.# return Vector of peak locations. find_local_maxima_ 1d(x) : local_maxima = [] i = 1 i next = iAttorney Docket No. OCI1001 i_max = x.size()-1 while(i < i_max):# Check if current sample is greater than previous sample if (x[i] > x[i-1]): i_next = / + 1 # Find next sample that is not equal to current sample# This is to handle plateaus while (i_next < i_max && x[i_next] == x[i]): i_next = i_next + 1# Check if current sample is greater than next sample if (x[i]>x[i_next]): left = i right = i_next-1 mid = (right+left)»1 # divide by 2 local_maxima.append(mid) i = i_next i = i + 1 return local_maxima
[0063] In an embodiment, the column peak information may further indicate starting column information and width information of the set of pixels. The circuitry 202 may determine the starting column information based on the determined column activation profile (i.e. which indicates values of the one-dimensional array). For example, based on the one-dimensional array (i.e. [0, 2, 3, 2, 0, 0] as shown in FIG. 3A) of the column activation profile, the circuitry 202 may determine that a starting column of the set of pixels (or of the object region 302 including the set of pixels) may be the second column C2, as a non-zero value in the one-dimensional array starts from the second column C2 for currently processed image frame. Further, the circuitry 202 may determine the width information for the object region 302 or the set of pixels based on the determined columnAttorney Docket No. OCI1001 activation profile (i.e. which indicates values of the one-dimensional array). For example, based on the one-dimensional array (i.e. [0, 2, 3, 2, 0, 0] as shown in FIG. 3A) of the column activation profile, the width information may indicate “3” as there may be three columns (“02”, “03”, and “04”) with non-zero values in the one-dimensional array of the column activation profile determined for the current image frame being processed.
[0064] Similarly, the row peak information may further indicate starting row information and height information of the set of pixels. The circuitry 202 may determine the starting row information based on the determined row activation profile (i.e. which indicates values of the one-dimensional array). For example, based on the one-dimensional array (i.e. [0, 1 , 3, 2, 1 , 0] as shown in FIG. 3A) of the row activation profile, the circuitry 202 may determine that a starting row (i.e. starting row information) of the set of pixels (or of the object region 302 including the set of pixels) may be the second row R2, as a non-zero value in the one-dimensional array starts from the second row R2 for currently processed image frame. Further, the circuitry 202 may determine the height information for the object region 302 or the set of pixels based on the determined row activation profile (i.e. which indicates values of the one-dimensional array). For example, based on the onedimensional array (i.e. [0, 1 , 3, 2, 1 , 0] as shown in FIG. 3A) of the row activation profile, the height information may indicate “4” as there may be four row (“R2”, “R3”, “R4”, and “R5”) with non-zero values in the one-dimensional array of the row activation profile determined for the current image frame being processed.
[0065] In an embodiment, based on the determination of the column activation profile, the circuitry 202 may determine a pair of the starting column information and the width information as pair of (“X”, “W”), where “X” may represent the starting column information and “W” may represent the width information. In an embodiment, the circuitry 202 may perform the peak finding algorithm on the determined column activation profile (indicating different column peaks as the number of pixels within the acceptance range in each column) to determine the starting column information and the width information or the pair (“X”, “W”). The circuitry 202 may execute following pseudo code or pseudo instructions, for example, for the determination of the column peak, the starting column information and the width information as the pair of (“X”, “W’):# Find peaks in Column Activation Profile# Some properties to tune peak finder: height, relative_height,Attorney Docket No. OCI1001# width, distance, prominence col_peaks, col_pk_props = find_peaks(col_profile)# Extract column coordinates to consider x = col_pk_props['left_ips][c_idx] w = col_pk_props['width'][c_idx]
[0066] Similarly, based on the determination of the row activation profile, the circuitry 202 may determine a pair of the starting row information and the height information as pair of (“Y”, “H”), where “Y” may represent the starting row information and “H” may represent the height information. In an embodiment, the pair (“X”, “W”) of the starting column information and the width information and the pair (“Y”, “H”) of the starting row information and the height information may be also indicated by the column peak information and the row peak information, respectively.
[0067] In an embodiment, the circuitry 202 may determine the row activation profile based on the column peak information and the column activation profile. For each column in the column activation profile (for example [0, 2, 3, 2, 0, 0] defined by the pair (“X”, “W”) as (“2”, “3”), the circuitry 202 may process each row, determine the number of pixels that may be within (or pass) the acceptance range in that row to further determine the row activation profile (such as [0, 1 , 3, 2, 1 , 0] shown in FIG. 3A). In some embodiments, the circuitry 202 may process each row for the limited set of columns indicated by the pair (“X” “W”), rather than all the columns. Therefore, for example based on the (“2” “3”) as the pair (“X” “W”), the circuitry may process pixels of each row for only the second column C2, the third column C3, and the fourth column 04 to determine the row activation profile with the one-dimensional array values as [0, 1 , 3, 2, 1 , 0], Therefore, the row activation profile may be determined for the set of pixels (within the acceptance range) based on the determined column activation profile. In an embodiment, the circuitry 202 may determine the row activation profile and the row peak information (i.e. pair (“Y”, “H”)) based on the column activation profile and the column peak information (i.e. pair (“X”, “W”)) determined for each column using the following pseudo code or pseudo instructions:# Iterate over each peak and find associated Row Act Profile forc dx in range(0, len(col_peaks)):Attorney Docket No. OCI1001# Extract column coordinates to consider x = col_pk_props['left_ips][c_idx] w = col_pk_props['width'][c_idx]# Find Row Activation Profile for column peak row_profile = [] forj in range(O, rows): row_profile[j] = 0 fori in range(x, x+w): if pixel [j,i] is in acceptance_range: row_profile[j] += 1# Find peaks in Row Activation Profile row_peaks, row_pk_props = find_peaks(row_profile) forr dx in range(O, len(row_peaks)): y = row_pk_props['left_ips][r_idx] h = row_pk_props['width'][r_idx] regions. append( (x,y,w,h) )
[0068] In an embodiment, the circuitry 202 may be further configured to determine the pair (“Y”, “H”) of the starting row information and the height information based on the row activation profile (i.e. indicating the row peaks as the number of pixels within the acceptance range in each row). For a particular image frame, the determined row peak information (i.e. pair (“Y”, “H”)) may be implicitly associated with the column peak information (i.e. pair (“X”, “W”)). In an embodiment, the circuitry 202 may determine multiple instances of the column activation profile, the row activation profile, the column peak information, and the row peak information based on presence of multiple objects in an image frame. In such case, multiple set of pixels may be determined within the acceptance range, where each object region (like the object region 302) may include at least one set of pixels.Attorney Docket No. OCI1001
[0069] In an embodiment, the circuitry 202 may be configured to store the determined column peak information (i.e. information about the pair (“X”, “W”)) and the determined row peak information (i.e. information about the pair (“Y”, “H”)) in the memory 204 for further processing, for example, but not limited to), transmission to the external processing device 110 (or to a server, not shown, comparison with similar information determined for another similar object in later image frames, and the like.
[0070] In an embodiment, the circuitry 202 may be configured to process multiple image frames and determine the column peak information and the row peak information based on the activation profiles. For example, the sensing array 104 (or the circuitry 202) may select the plurality of pixels 106 from at least two or more image frames and further determine the set of pixels (within the acceptance range). The circuitry 202 may further determine the column activation profile, and the row activation profile based on the pixel values related to the set of pixels or the plurality of pixels 106 of the two or more image frames. The circuitry 202 may be further configured to determine the column peak information and the row peak information based on the determined the column activation profile, and the row activation profile for different objects presented in the two or more image frames (for example consecutive image frames).
[0071] At 408, object regions may be determined. In an embodiment, the circuitry 202 may be further configured to determine one or more object regions in an image frame (like current image frame being processed) based on the determined column peak information (like information about the pair (“X”, “W”)) and the determined row peak information (like information about the pair (“Y”, “H”)). The circuitry 202 may determine an object region (like boundary of an object present in the image frame) based on the determined column peak information and the row peak information. For example, the object region 302 (shown in FIGs. 3 and 4) which includes the set of pixels may be determined based on the starting column information (“C2” column), the width information (three columns), the starting row information (“R2” column), and the height information (four rows) indicated by the column peak information and the row peak information. The pair (“X”, “W”) and the pair (“Y”, “H”) may indicate a starting coordinate (“X”, “Y”) of an upper left point of the object region and the width (“W”) and the height (“H”) of the object region. The pair (“X”, “W”) and the pair (“Y”, “H”) may indicate exact locations of the object in the current image frame. Therefore, the circuitry 202 may be able to determine (using simplified computation) the object region based on the determined column peak information (i.e. information about the pair (“X”,Attorney Docket No. OCI1001 “W”)) and the determined row peak information (i.e. information about the pair (“Y”, “H”)) about the set of pixels (within the acceptance range). In other words, the circuitry 202 may be able to determine the object region based on simplified analysis of the set of pixels within the acceptance range. For multiple objects present in a particular image frame, the circuitry 202 may determine multiple pairs of (“X”, “W”) and (“Y”, “H”) information to determine multiple object regions.
[0072] The determined one or more object regions may include an object of different class types (like human, animal, vehicle, etc.) which may be captured in one or more image frames. As shown in FIG. 4, for an image frame 414, multiple object regions 416A-416N may be determined which may include multiple objects 418A-418N. Each of the multiple object regions 416A-416N may include the set of pixels within the acceptance range and may be determined based on the determined column activation profile, the row activation profile, the column peak information, and the row peak information determined for each object. Each of the object regions 416A-416N may indicate a bounding box around respective object (like objects 418A-418N) as shown, for example, in FIG. 4. Therefore, the object region (like object regions 416A-416N) may be a candidate region where a potential object (like objects 418A-418N) may be present or captured by an image sensor (not shown) of the disclosed electronic device 102.
[0073] The circuitry 202 may determine the one or more object regions (like the object regions 416A-416N i.e. candidate region) using the simplified computation achieved by the determination of the set of pixels, activation profiles (i.e. one-dimensional array), and column / row peak information rather than utilization of high computational neural network or machine learning models to determine object or candidate regions. Therefore, the disclosed electronic device 102 may replace the need for an integrated neural network model to propose candidate regions. Further, the processing of information related to the one-dimensional array (i.e. activation profiles) simplifies computation tasks for the determination of object regions, rather than processing of pixel data in form of an M*N matrix (i.e. where “M” and “N” represents the plurality of rows and the plurality of columns of the plurality of pixels 106, respectively) at different computational stages. The computations (steps 402-408) performed by the disclosed electronic device 102 may simplify both the processing as well as memory requirements at the end of the electronic device 102. In certain cases, as the electronic device 102 includes the sensing array with the plurality of pixels 106, such simplified computation may be performed as pre-Attorney Docket No. OCI1001 processing task(s), for example, within the pixel. The information about the determined object regions (i.e. results computed based on simplified processing) may be further provided to another model or device (like including the neural models) which may only focus on the estimation of the object classes using the information received from the disclosed electronic device 102.
[0074] At 410, one or more objects may be estimated. In an embodiment, the circuitry 202 of the disclosed electronic device 102 may be further configured to estimate one or more objects (for example the objects 418A-418N shown in FIG. 4 and FIG. 5) within the plurality of pixels 106 (or within the image frame (like image frame 414)) based on the determined one or more object regions (like the object regions 416A-416N). For the estimation of the objects, the circuitry 202 may determine aspect ratio information of the determined one or more object regions based on the determined column peak information and the row peak information. The aspect ratio information may be calculated based on the width information (“W”) and the height information (“H”) indicated by the row peak information and the column peak information, respectively. The aspect ratio information may be a ratio of the width information (“W”) and the height information (“H”). For example, in case the determined aspect ratio information indicates a value lesser than one (i.e. width of the object region is lesser than the height of the object region), the circuitry 202 may estimate the object as a human (like an object 418A) present in the object region 416A. In another example, in case the determined aspect ratio information indicates a value greater than one (i.e. width of the object region is greater than the height of the object region), the circuitry 202 may estimate the object as an animal (like an object 418B, i.e. cat, dog, tiger, etc.) present in the object region 416B.
[0075] The circuitry 202 may be further configured to estimate one or more objects within the image frame based on a pixels value of each pixel of the set of pixels (i.e. within the acceptance range) included in the determined object region. In some embodiments, the circuitry 202 may estimate one or more objects based on the pixel values and the determined column / row peak information (i.e. indicating the aspect ratio information in form width and height of the object region). For example, in case the determined aspect ratio information indicates a value substantially equal to one (i.e. width of the object region is substantially equal to the height of the object region), the circuitry 202 may estimate the object (for example a square shaped or a round shaped object) based on the pixel values of certain pixels within the acceptance range or based on the number of pixels (within theAttorney Docket No. OCI1001 acceptance range) at different locations of the determined object region (like at center or at edges of the object region). In certain case, the determined height of the object region may provide an estimate of a distance (“D”) between an actual object being captured and the image sensor, as the height of the object region (including the estimated object) may correspond to the distance (“D”) between the actual object and the image sensor related to the disclosed electronic device 102. In an embodiment, the circuitry 202 may estimate the object within the object region based on a shape of the determined object region (like square-shaped, round-shaped, rectangular-shaped, etc.). In some embodiments, the shape may correspond to the set of pixels which are within the acceptance range or to the remaining pixels in the plurality of pixels 106 (i.e. other than the set of pixels).
[0076] In certain situations, the determination of the size (i.e. width, height) and the shape of the object region may allow the object region to fully encompass potential objects of interest (for example human in motion, such as an object 418N within an object region 416N shown in FIG. 4 and 5). Therefore, the determination of different factors, but are not limited to, aspect ratio, size, pixel values, number of activated pixels at different locations, shape, and the like may facilitate the disclosed electronic device 102 to estimate the objects within the determined object regions to certain extent with simplified computation steps (as described in FIG. 3A, FIG. 3B, and FIG. 4 (at steps 402-410).
[0077] In some embodiments, the circuitry 202 may consider or discard the determined object regions for the estimation of the objects. Based on different application areas / situations and / or the target object classes of interest, the circuitry 202 may apply various filters / checks on the determined object regions, their sizes (width and / or height), shapes and / or aspect ratio, to determine if the object region should be preemptively discarded or not, for further estimation of the objects or precise determination of their object classes. In some embodiments, the circuitry 202 may discard the determined object region(s) based on the pixel values or the number of pixels of the set of pixels within the acceptance range. The circuitry 202 may also receive user inputs (via the I / O device 206) to select or discard any object region determined by the disclosed electronic device. In some embodiments, for the estimation of the object, the circuitry 202 may output information indicating whether a particular object class (like human, pet, vehicle, etc.) is present in a current or particular image frame or not. In some embodiments, the circuitry 202 may provide a particular confidence level of presence (or absence) of a particular object class in the determined object region. Such information may be determined basedAttorney Docket No. OCI1001 on the aspect ratio information and / or pixel values of the set of pixels. Such information may be further transmitted to the external processing device 110 for further processing (like object detection or classification).
[0078] At 412, region information may be transmitted. In an embodiment, the circuitry 202 may be configured to transmit the information about determined one or more object regions (like object regions 416A-416N) in a particular image frame (like image frame 414) to the external processing device 110. The transmitted information about the determined one or more object regions may indicate information about the determined pairs (“X”, “W”) and (“Y”, “H”) and / or bounding boxes (or their locations), as described, for example, at steps 402-408. The external processing device 110 may be a high computation device (for example including well trained neural network or machine learning model / algorithm) to precisely determine the object classes (like, but is not limited to, human, pet, wild animal, vehicle, etc.) based on the information about the determined object regions (like object regions 416A-416N) transmitted from the disclosed electronic device. In an embodiment, the circuitry 202 may be configured to transmit information about the pixel values of the set of pixels within the determined object regions to the external processing device 110. In an embodiment, the circuitry 202 may be configured to transmit information about all of the pixel values within the determined object regions to the external processing device 110.
[0079] The circuitry 202 may further control the external processing device 110 to either estimate one or more objects within the determined object regions or to determine the objects classes of the objects based on the information transmitted (about the determined object regions and the pixel values within the determined object regions) by the disclosed electronic device 102. Such information about the object regions and the pixel values (i.e. determined based on simplified computations (at steps 402-410) may further facilitate the external processing device 110 to determine the object classes with lesser or simplified computational processing. For example, by reducing the processing required at the end of the external processing device 110 to determine the object regions (i.e. candidate regions) which is accurately performed as pre-processing by the disclosed electronic device 102. Therefore, the external processing device 110 (i.e. with high computation resources or trained neural network or machine learning model) may avoid any initial processing (i.e. to process the captured images (including the plurality of pixels 106) and processing with respect to the determination of the candidate regions) and mainlyAttorney Docket No. OCI1001 rely on the information received from the disclosed electronic device 102. Therefore, the disclosed electronic device 102 not only optimizes (using the computations described at steps 402-410) the computation resources at its own end to determine the candidate regions and estimate the objects, but also support computational optimization at the end of the external processing device 110 to precisely estimate the object classes present in the capture image frames.
[0080] FIG. 5 is a diagram that illustrates object regions determined based on analysis of the set of pixels, in accordance with an embodiment of the disclosure. FIG. 5 is explained in conjunction with elements from FIG. 1 , FIG. 2, FIG. 3A, FIG. 3B, and FIG. 4. With reference to FIG. 5, there is shown a diagram 500 which may indicate an image frame 414 (similar to the image frame 414 of FIG. 4). The image frame 414 may include one or more object regions (such as the object regions 416A-416N as also shown in FIG. 4). The object regions 416A-416N may be determined based on the set of pixels, determined activation profiles, and the determined column / row peak information as described, for example, in FIG. 4 (steps 402-408).
[0081] In FIG. 5, there is further shown a graph 502 which may indicate the determined column activation profile for the image frame 414 shown in FIG. 5. For example, a total number of columns in the plurality of pixels 106 of the image frame 414 is 2000. The graph 502 may indicate a number of pixels (i.e. set of pixels within the acceptance range) in each column of the image frame 414. The graph 502 may include multiple graphs (or sub graphs), where each graph may correspond to different object regions determined in the image frame 414. For example, a sub-graph 504A that may correspond to the object region 416A (including the object 418A), a sub-graph 504B that may correspond to the object region 416B (including the object 418B), a sub-graph 504C that may correspond to the object region 416C (including the object 418C), and a sub-graph 504N that may correspond to the object region 416N (including the object 418N). Each of the sub-graph 504A, the sub-graph 504B, the sub-graph 504C, and the sub-graph 504N may indicate the number of the set of pixels (within the acceptance range) at different columns included in the corresponding object region. The set of pixels and the number of pixels of the set of pixels indicated by the column activation profile are described, for example, in FIG. 3A. The circuitry 202 may be configured to determine the graph for each object region based on corresponding set of pixels determined from the plurality of pixels 106 and pixel values of the set of pixels for different columns. The pixel value and their position (like column / row)Attorney Docket No. OCI1001 may further correspond to a shape of the activation profile (like the column activation profile) as shown, for example, by the multiple graphs in the graph 502. The shape of the activation profile may also be related to the set of pixels within the acceptance range and other remaining pixels of the plurality of pixels 106. Based on determined graphs (indicating the column activation profiles), the circuitry 202 may further determine the row activation profile, and column / row peak information to determine one of more object regions as described, for example, in FIG. 3A and 4. Similarly, for the row activation profile for different rows, the circuitry 202 may be configured to determine a graph (not shown) for each object region based on corresponding set of pixels (i.e. determined from the plurality of pixels 106) and pixel values of the set of pixels at corresponding rows.
[0082] It may be noted that four object regions illustrated in FIG. 4 and FIG. 5 is presented merely as an example. The image frame 414 may include less than four object regions (or objects) or more than four object regions (or objects) of same and different object classes, without deviation from the scope of the disclosure. For the sake of brevity, only four object regions have been shown in FIG. 4 and 5 for example.
[0083] FIG. 6 is a flowchart that illustrates exemplary operations for object region determination based on analysis of pixels, in accordance with an embodiment of the disclosure. FIG. 6 is explained in conjunction with elements from FIG. 1 , FIG. 2, FIG. 33A, FIG. 3B, FIG. 4, and FIG. 5. With reference to FIG. 6, there is shown a flowchart 600. The operations from 602 to 612 may be implemented, for example, by the electronic device 102 of FIG. 1 , the circuitry 108 of FIG. 1 , or the circuitry 202 of FIG. 2. The operations of the flowchart 600 may start at 602 and proceed to 604.
[0084] At 604, a set of pixels may be determined from a plurality of pixels, where a pixel value of each pixel of the set of pixels is within an acceptance range. In an embodiment, the circuitry 202 may be configured to determine the set of pixels from the plurality of pixels. The pixel value of each of the set of pixels is within the acceptance range as described, for example, in FIG. 3A, and FIG. 4 (at step 402). In another embodiment, the set of pixels may be determined by the sensing array 104 (including the pixel circuitry 107C related to each pixel or group of pixels) and further the circuitry 202 may read-out or receive the pixel values of the set of pixels from the sensing array 104.
[0085] At 606, a column activation profile and a row activation profile may be determined for the determined set of pixels. In an embodiment, the circuitry 202 may beAttorney Docket No. OCI1001 configured to determine the column activation profile and the row activation profile for the set of pixels with pixel values within the acceptance range, as described, for example, in FIG. 3A and FIG. 4 (at step 404).
[0086] At 608, column peak information and row peak information may be determined based on the determined column activation profile and the determined row activation profile. In an embodiment, the circuitry 202 may be configured to determine the column peak information and the row peak information based on the determined column activation profile and the determined row activation profile as described, for example, in FIG. 3A and FIG. 4 (at step 406).
[0087] At 610, one or more object regions may be determined in the plurality of pixels based on the determined column peak information and the determined row peak information. In an embodiment, the circuitry 202 may be configured to determine the one or more object regions in the plurality of pixels based on the determined column peak information and the determined row peak information as described, for example, in FIG. 4 (at step 408).
[0088] At 612, one or more objects may be estimated in the plurality of pixels based on the determined one or more object regions. In an embodiment, the circuitry 202 may be configured to estimate one or more objects in the plurality of pixels based on the determined one or more object regions as described, for example, at FIG. 4 (step 410). In another embodiment, the circuitry 202 may be configured to transmit information about the determined one or more object regions and the pixel values of the set of pixels to an external processing device and further control the external processing device to estimate the one or more objects in the plurality of pixels based on the transmitted information and the pixel values of the set of pixels as described, for example, at FIG. 4 (step 410).
[0089] Although the flowchart 600 is illustrated as discrete operations, such as 602, 604, 606, 608, 610, and 612, the disclosure is not so limited. Accordingly, in certain embodiments, such discrete operations may be further divided into additional operations, combined into fewer operations, or eliminated, depending on the particular implementation without detracting from the essence of the disclosed embodiments.
[0090] Various embodiments of the disclosure may provide a non-transitory computer- readable medium and / or storage medium having stored thereon, computer instructions (i.e. computer-executable instructions) that may be executable by a machine and / or aAttorney Docket No. OCI1001 computer to operate an electronic device (for example the electronic device 102). The instructions may cause the machine and / or computer to perform operations that may include determination of a set of pixels from a plurality of pixels, wherein a pixel value of each pixel of the set of pixels is within an acceptance range, wherein the plurality of pixels is in a sensing array as a matrix, and wherein the matrix includes a plurality of rows and a plurality of columns. The operations may further include determination of a column activation profile and a row activation profile based on the determined set of pixels. The operations may further include determination of column peak information and row peak information based on the determined column activation profile and the determined row activation profile. The operations may further include determination one or more object regions in the plurality of pixels based on the determined column peak information and the determined row peak information. The operations may further include estimation of one or more objects in the plurality of pixels based on the determined one or more object regions.
[0091] Exemplary aspects of the disclosure may include an electronic device (such as the electronic device 102) that may include a sensing array (for example the sensing array 104) and circuitry (for the circuitry 108 or the circuitry 202) coupled to the sensing array. The sensing array may include a plurality of pixels (for example the plurality of pixels 106) as a matrix, wherein the matrix includes a plurality of rows and a plurality of columns. The sensing array (including the pixel circuitry 107C for each pixel) may be configured to determine a set of pixels from the plurality of pixels, wherein a pixel value of each pixel of the set of pixels is within an acceptance range. The circuitry may be further configured to determine a column activation profile and a row activation profile based on the determined set of pixels. The circuitry may be further configured to determine column peak information and row peak information based on the determined column activation profile and the determined row activation profile. The circuitry may be further configured to determine one or more object regions (for example one or more object regions 416A-416N) in the plurality of pixels based on the determined column peak information and the determined row peak information. The circuitry may be further configured to estimate one or more objects (for example one or more objects 418A-418N) in the plurality of pixels based on the determined one or more object regions. In another embodiment, the circuitry may be configured to determine the set of pixels from the plurality of pixels wherein the pixel value of each pixel of the set of pixels is within the acceptance range.Attorney Docket No. OCI1001
[0092] In an embodiment, the circuitry may be further configured to determine the row activation profile for the set of pixels based on the determined column activation profile. The column activation profile may correspond to a one-dimensional array. The column activation profile may indicate at least one of: a number of pixels, from the set of pixels, in each column of a set of columns from the plurality of columns, or a sum of the pixel value of each pixel of the set of pixels in each column of the set of columns. The row activation profile may correspond to a one-dimensional array. The row activation profile may indicate at least one of: a number of pixels, from the set of pixels, in each row of a set of rows from the plurality of rows, or a sum of the pixel value of each pixel of the set of pixels in each row of the set of rows.
[0093] The column peak information may indicate starting column information and width information of the set of pixels. The row peak information may indicate starting row information and height information of the set of pixels. The column peak information may indicate a peak column from the plurality of columns, the peak column may indicate a local maxima for either a number of pixels of the set of pixels or a sum of pixel values of the set of pixels, among the plurality of columns. The row peak information may indicate a peak row from the plurality of rows, the peak row may indicate a local maxima for either a number of pixels of the set of pixels or a sum of pixel values of the set of pixels, among the plurality of rows.
[0094] In an embodiment, the circuitry may be further configured to determine aspect ratio information of the determined one or more object regions based on the column peak information and the row peak information, and estimate the one or more objects based on the determined aspect ratio information.
[0095] In an embodiment, the circuitry may be further configured to estimate the one or more objects based on: the pixel value of each pixel in the determined one or more object regions, and the column peak information and the row peak information related to the determined one or more object regions. In an embodiment, the circuitry may be further configured to estimate the one or more objects based on a shape of the determined one or more object regions. In an embodiment, the circuitry may be further configured to update the acceptance range for one or more of the plurality of pixels based on the determined column activation profile and the determined row activation profile.Attorney Docket No. OCI1001
[0096] In an embodiment, the plurality of pixels is related to two or more frames. The circuitry may be further configured to determine the set of pixels, the column activation profile, the row activation profile, the column peak information, and the row peak information for the two or more frames. In an embodiment, the circuitry may be further configured to select a region of interest (ROI) from the plurality of pixels, and determine the set of pixels from the selected ROI. In an embodiment, the circuitry may be further configured to modify the selected ROI based on at least one of the pixel value of each pixel of the set of pixels within the acceptance range, the determined column activation profile, or the determined row activation profile.
[0097] In an embodiment, the circuitry may be further configured to transmit information of the determined one or more object regions and the pixel value of each pixel within the determined one or more object regions to an external processing device, and further control the external processing device to estimate the one or more objects in the plurality of pixels based on the transmitted information and the pixel value of each pixel of the set of pixels.
[0098] The present disclosure may be realized in hardware, or a combination of hardware and software. The present disclosure may be realized in a centralized fashion, in at least one computer system, or in a distributed fashion, where different elements may be spread across several interconnected computer systems. A computer system or other apparatus adapted to carry out the methods described herein may be suited. A combination of hardware and software may be a general-purpose computer system with a computer program that, when loaded and executed, may control the computer system such that it carries out the methods described herein. The present disclosure may be realized in hardware that comprises a portion of an integrated circuit that also performs other functions.
[0099] The present disclosure may also be embedded in a computer program product, which comprises all the features that enable the implementation of the methods described herein, and which when loaded in a computer system is able to carry out these methods. Computer program, in the present context, means any expression, in any language, code or notation, of a set of instructions intended to cause a system with information processing capability to perform a particular function either directly, or after either or both of the following: a) conversion to another language, code or notation; b) reproduction in a different material form.Attorney Docket No. OCI1001
[0100] While the present disclosure is described with reference to certain embodiments, it will be understood by those skilled in the art that various changes may be made, and equivalents may be substituted without departure from the scope of the present disclosure. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the present disclosure without departure from its scope. Therefore, it is intended that the present disclosure is not limited to the particular embodiment disclosed, but that the present disclosure will include all embodiments that fall within the scope of the appended claims.
Claims
Attorney Docket No. OCI1001CLAIMSWhat is claimed is:
1. An electronic device, comprising: a sensing array which includes a plurality of pixels as a matrix, wherein the matrix includes a plurality of rows and a plurality of columns, and wherein the sensing array is configured to determine a set of pixels from the plurality of pixels, wherein a pixel value of each pixel of the set of pixels is within an acceptance range; and circuitry coupled to the sensing array, wherein the circuitry is configured to: determine a column activation profile and a row activation profile based on the determined set of pixels; determine column peak information and row peak information based on the determined column activation profile and the determined row activation profile; determine one or more object regions in the plurality of pixels based on the determined column peak information and the determined row peak information; and estimate one or more objects in the plurality of pixels based on the determined one or more object regions.
2. The electronic device according to claim 1 , wherein the circuitry is further configured to determine the row activation profile for the set of pixels based on the determined column activation profile.
3. The electronic device according to claim 1 ,Attorney Docket No. OCI1001 wherein the column activation profile corresponds to a one-dimensional array; and wherein the column activation profile indicates at least one of: a number of pixels, from the set of pixels, in each column of a set of columns from the plurality of columns, or a sum of the pixel value of each pixel of the set of pixels in each column of the set of columns.
4. The electronic device according to claim 1 , wherein the row activation profile corresponds to a one-dimensional array; and wherein the row activation profile indicates at least one of: a number of pixels, from the set of pixels, in each row of a set of rows from the plurality of rows, or a sum of the pixel value of each pixel of the set of pixels in each row of the set of rows.
5. The electronic device according to claim 1 , wherein the column peak information indicates starting column information and width information of the set of pixels.
6. The electronic device according to claim 1 , wherein the row peak information indicates starting row information and height information of the set of pixels.
7. The electronic device according to claim 1 ,Attorney Docket No. OCI1001 wherein the column peak information indicates a peak column from the plurality of columns, the peak column indicates a local maxima for either a number of pixels of the set of pixels or a sum of pixel values of the set of pixels, among the plurality of columns; and wherein the row peak information indicates a peak row from the plurality of rows, the peak row indicates a local maxima for either a number of pixels of the set of pixels or a sum of pixel values of the set of pixels, among the plurality of rows.
8. The electronic device according to claim 1 , wherein the circuitry is further configured to: determine aspect ratio information of the determined one or more object regions based on the column peak information and the row peak information; and estimate the one or more objects based on the determined aspect ratio information.
9. The electronic device according to claim 1 , wherein the circuitry is further configured to estimate the one or more objects based on: the pixel value of each pixel in the determined one or more object regions; and the column peak information and the row peak information related to the determined one or more object regions.
10. The electronic device according to claim 1 , wherein the circuitry is further configured to estimate the one or more objects based on a shape of the determined one or more object regions.Attorney Docket No. OCI100111. The electronic device according to claim 1 , wherein the circuitry is further configured to update the acceptance range for one or more of the plurality of pixels based on the determined column activation profile and the determined row activation profile.
12. The electronic device according to claim 1 , wherein the plurality of pixels is related to two or more frames, and wherein the circuitry is further configured to determine the column activation profile, the row activation profile, the column peak information, and the row peak information for the two or more frames.
13. The electronic device according to claim 1 , wherein the circuitry is further configured to: select a region of interest (ROI) from the plurality of pixels; and determine the set of pixels from the selected ROI.
14. The electronic device according to claim 13, wherein the circuitry is further configured to: modify the selected ROI based on at least one of the pixel value of each pixel of the set of pixels within the acceptance range, the determined column activation profile, or the determined row activation profile.
15. The electronic device according to claim 1 , wherein the circuitry is further configured to:Attorney Docket No. OCI1001 transmit information of the determined one or more object regions and the pixel value of each pixel within the determined one or more object regions to an external processing device; and control the external processing device to estimate the one or more objects in the plurality of pixels based on the transmitted information and the pixel value of each pixel within the determined one or more object regions.
16. The electronic device according to claim 1 , wherein the circuitry is further configured to determine the set of pixels from the plurality of pixels, and wherein the pixel value of each pixel of the set of pixels is within an acceptance range.
17. A method, comprising: in an electronic device: determining a set of pixels from a plurality of pixels, wherein a pixel value of each pixel of the set of pixels is within an acceptance range, wherein the plurality of pixels is in a sensing array as a matrix, and wherein the matrix includes a plurality of rows and a plurality of columns; determining a column activation profile and a row activation profile based on determined set of pixels; determining column peak information and row peak information based on the determined column activation profile and the determined row activation profile; determining one or more object regions in the plurality of pixels based on the determined column peak information and the determined row peak information; andAttorney Docket No. OCI1001 estimating one or more objects in the plurality of pixels based on the determined one or more object regions.
18. The method according to claim 17, further comprising: transmitting information about the determined one or more object regions and the pixel value of each pixel within the determined one or more object regions to an external processing device; and controlling the external processing device to estimate the one or more objects in the plurality of pixels based on the transmitted information and the pixel value of each pixel within the determined one or more object regions.
19. The method according to claim 17, further comprising: determining aspect ratio information of the determined one or more object regions based on the column peak information and the row peak information; and estimating the one or more objects based on the determined aspect ratio information.
20. A non-transitory computer-readable medium having stored thereon, computerexecutable instructions that when executed by an electronic device, causes the electronic device to execute operations, the operations comprising: determining a set of pixels from a plurality of pixels, wherein a pixel value of each pixel of the set of pixels is within an acceptance range, wherein the pluralityAttorney Docket No. OCI1001 of pixels is in a sensing array as a matrix, and wherein the matrix includes a plurality of rows and a plurality of columns; determining a column activation profile and a row activation profile based on determined set of pixels; determining column peak information and row peak information based on the determined column activation profile and the determined row activation profile; determining one or more object regions in the plurality of pixels based on the determined column peak information and the determined row peak information; and estimating one or more objects in the plurality of pixels based on the determined one or more object regions.
Citation Information
Patent Citations
Advanced region of interest function for image sensors
US20150358571A1
Imaging device, electronic device, and light detecting method
US20240236519A1
Using silhouette for fast object recognition
WO2019110824A1