Image feature extraction using entropy-based analysis

Abstract

The embodiment of the invention relates to image feature extraction using entropy-based analysis. Systems and methods for feature extraction from visual input are provided. Examples include calculating entropy information for a plurality of regions of the visual input and calculating a hyper-parameter for a clustering algorithm based on the entropy information. The clustering algorithm then segments the visual input into a plurality of clusters using hyper-parameters. A feature set of the visual input is extracted based on the plurality of clusters.

Description

Background Art

[0001] Computer vision is a field of artificial intelligence (AI) that enables machines such as computers and systems to extract information from images, videos, and other visual inputs. Machines can use this information to take actions or make recommendations. Computer vision trains machines how to distinguish objects in visual input, determine the distance between objects, determine if objects are moving, and determine if there are problems in the visual input.

[0002] Computer vision typically uses machine learning (ML) techniques such as deep learning, convolutional neural networks (CNNs) for images, and recurrent neural networks (RNNs) for videos to train machines to achieve the above functions. These ML techniques are embodied as algorithmic models. Visual data is applied to the algorithmic model to teach the machine the context of the visual data. Once enough data is provided through the algorithmic model, the machine can use the model to distinguish visual inputs. In the case of image inputs, CNNs and RNNs with CNNs teach machines by decomposing the visual input into pixels (i.e., given tags or labels) in the case of video inputs. It uses the labels to perform convolution (a mathematical operation on two functions to produce a third function) and predict the content in the visual input. CNN or CNN-RNN runs the convolution and checks the accuracy of its predictions in a series of iterations until the predictions match the ground truth. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] According to one or more different embodiments, the present disclosure is described in detail with reference to the following figures. The figures provided are for illustration purposes only and merely depict typical or example embodiments.

[0004] Figure 1 is a block diagram of an example feature extraction system according to an implementation of the present disclosure.

[0005] Figure 2 is a schematic diagram of another example feature extraction system according to an implementation of the present disclosure.

[0006] Figure 3A-3D is a visual depiction of an example expanded region algorithm performed on an example visual input in accordance with an implementation of the present disclosure.

[0007] Figure 4A and Figure 4B Depicted is a flow chart of an example process for feature extraction according to implementations of the present disclosure.

[0008] Figure 5A and Figure 5B Depicted are example images and a graphical representation of the entropy information calculated for each image.

[0009] Figure 6are example computing components that may be used to implement various features of feature extraction according to implementations disclosed herein.

[0010] Figure 7 is an example computer system that can be used to implement the various features of feature extraction of the present disclosure.

[0011] The drawings are not intended to be exhaustive or to limit the disclosure to the precise forms disclosed. DETAILED DESCRIPTION

[0012] As mentioned above, machines can use computer vision to take actions or make recommendations. Real-world applications of computer vision have penetrated industries such as healthcare, smart manufacturing, finance, and life sciences. Deep learning ML models (such as deep CNNs or visual transformers) trained on large visual datasets can be effectively used for image classification or object recognition, which can be used in applications such as healthcare applications for disease detection and autonomous systems for navigation.

[0013] However, datasets of visual data may often be unstructured. An unstructured dataset may refer to a dataset that is not stored in a structured database format. Although unstructured data may have an internal structure, this internal structure is not predefined through a data model. A unified representation of visual data may not exist in an unstructured dataset because, for example, there may not be uniqueness or consistency between the visual data of the dataset.

[0014] Thus, an ML model trained on an unstructured dataset may incorrectly and / or with low confidence classify visual data input (“visual input”) where the visual data input deviates from or otherwise differs from the training dataset. Thus, an ML model trained on one dataset consisting of one type of visual data may not be suitable for classifying visual input of a different type of visual data. For example, an ML model trained on a dataset of high-resolution images may not be able to classify low-quality images with consistency or accuracy. As another example, an ML model trained on a dataset of images of various objects (e.g., animals, things, machinery, etc.) may not be able to classify images of scans of human body parts.

[0015] Explanations about how / why a classification is wrong or how / why an ML model has low confidence in a classification can be provided through feature importance techniques. Feature importance techniques can provide information about which areas and / or features of the visual input caused the trained ML model to incorrectly classify the visual input or correctly classify the visual input with low confidence. Therefore, the machine can use feature importance to understand why the ML model performs classification in a suboptimal manner. Feature importance involves calculating a score of the importance of each feature to the classification or the importance of each feature in calculating the probability of classification for the features identified in the visual input. The score can be used to determine the relative importance of each feature's contribution to the classification, and can be used to rank features based on their contribution to the classification. This relative importance can tell the machine why the classification is suboptimal.

[0016] Feature importance relies on extracting features from visual input with both accuracy and granularity. As used herein, a feature may refer to a piece of information about the content of a visual input, such as information about whether a certain area of ​​a visual input has certain attributes. A feature may be a specific structure in a visual input, such as, but not limited to, a point, an edge, or an object. Other examples of features are the relative motion of content in a visual input between consecutive visual inputs, or shapes defined according to curvature or boundaries between different regions or structures. For a dataset for training computer vision, a feature may be a pixel or a group of pixels that constitutes a visual input and affects a prediction. As used herein, feature extraction refers to one or more processes that can be used to segment a visual input into various groups or features, which can make it easier and more accurate for ML models to analyze the content of a visual input. Features can be used to define areas in a visual input that are unique to a given category of visual input.

[0017] The aforementioned complexities of training ML algorithms on unstructured datasets of visual inputs can make feature extraction a challenging task. This is because, as mentioned above, ML algorithms trained on datasets of one type of visual input may not be well suited to classify different types of visual inputs, lest the ML model may incorrectly and / or with low confidence classify different types of visual inputs.

[0018] There are a variety of conventional techniques that can be used for feature extraction. However, conventional techniques present their own complexities and challenges in performing efficient and accurate feature extraction. For example, edge detector techniques have been used to identify boundaries in images by detecting discontinuities, such as sudden changes in pixel intensity. Discontinuities can be used for image segmentation and data extraction. However, edge detector techniques can be sensitive to noise, may not always be accurate, and may depend on control parameters that may require careful manual tuning. As an illustrative example, the Canny edge detector uses a Gaussian filter that can smooth edges, which can lead to inaccuracies in edge detection.

[0019] As another example, the Marching Squares technique can be used for feature extraction. This technique is an algorithm for generating contours from an image, where the contour levels may need to be manually specified as input parameters of the algorithm. Specifying the contour levels may require a manual iterative process, which may lead to inaccurate results if the input parameters are inaccurately assumed.

[0020] Clustering-based techniques can also be used for feature extraction (e.g., K-means model, Gaussian mixture model, etc.). However, each of these techniques utilizes manual specification of expected cluster counts to define the kernel hyperparameters of the model. Although techniques exist for determining cluster counts, such as Elbow Analysis, Silhouette Analysis, etc., these techniques may not be consistent, may be affected by outliers, and may not support accurate clustering of overlapping regions of an image.

[0021] Morphological operations (such as dilation and erosion), and energy-based techniques (such as active contours) are additional techniques that can aid in image segmentation. However, these techniques may require hyperparameters from an operator with visualization-based manual control that may present different results with different values.

[0022] Deep neural networks, such as SegNet or Fully Convolutional Network (FCN) semantic segmentation, can also be used for feature extraction. However, these models rely on the use of relevant image data and labels for training. This training process can be expensive in terms of computing resources and time. In addition, each implementation may require architectural changes between different image types, especially in the case of unstructured datasets.

[0023] In summary, the above conventional feature extraction techniques have technical deficiencies. For example, conventional feature extraction techniques are not universal and may require moderate to significant iterative tuning of manual user-defined input parameters and visualization-based prior understanding of input images for training. Tunable input parameters can often be determined based on a trial-and-error approach, where a user inputs different hyperparameters until the user locates a hyperparameter value that achieves the desired result. Therefore, conventional techniques may result in erroneous or inaccurate feature extraction.

[0024] The present disclosure provides a feature extraction method and system that overcomes the above-mentioned technical shortcomings. The implementation of the disclosed technology can determine the diffusion of entropy information in the visual input, analyze the diffusion of entropy information, and extract features based on the diffusion of entropy information. For example, a measure of the diffusion of entropy information can be obtained by calculating the entropy level in the visual input. The calculation can be performed on the visual input in the following manner: split the visual input into multiple regions, determine the entropy information for each region, and obtain the spatial distribution of the entropy information across the visual input. In various examples, the entropy information can be iteratively calculated on the region with increased or expanded size. That is, for example, a first region of the visual input can be defined, and an entropy value for the first region can be calculated. The size of the first region can be iteratively increased so that each iteration can correspond to the nth region (where n is an integer) in the multiple regions and the entropy value calculated for each nth region. The entropy information may include entropy values ​​calculated for multiple regions. In various implementations, this increased or expanded regional analysis can obtain local entropy across the visual input, which provides a spatial evaluation of the information in the visual input.

[0025] The entropy information can be used to determine hyperparameters as input to a clustering algorithm. In some examples, the entropy information can be used to automatically determine the hyperparameters. A clustering algorithm can be performed to extract features from the visual input based on the hyperparameters. In an example implementation, the clustering algorithm can be, for example, a mean shift clustering algorithm, where the hyperparameters for the mean shift clustering algorithm can be calculated based on the entropy information. Therefore, the entropy information calculated directly from the visual input can be used for the automatic derivation of the hyperparameters used in the clustering algorithm.

[0026] The clustering algorithm can then be applied to the visual input using the calculated hyperparameters to segment the visual input into clusters. These clusters can represent features in the visual input. Although mean shift clustering is described herein, the present disclosure is not intended to be limited to the mean shift clustering algorithm, and other clustering algorithms are considered to be within the scope of the present disclosure and can be implemented by the systems and methods disclosed herein.

[0027] The extracted feature set can be provided with an outline of each feature in the visual input derived from the cluster. The extracted feature set can be mapped to the outline, and the outline can be uniquely marked with attributes, such as color or other visual features, to be presented as an output image. For example, clusters can be identified and assigned different colors. Image content analysis (referred to as scanning) can be performed by visual input from edge to edge to locate the boundaries between clusters. When the cluster color changes during scanning, the change can be marked as an outline and assigned a unique attribute, such as a unique color or visual feature. The outline can then be stored as an extracted feature set. In some cases, the stored extracted feature set can be provided as an output image with outlines indicated according to unique attributes (e.g., highlighted according to a unique color in an example implementation).

[0028] Therefore, by determining the hyperparameters of feature extraction to be performed on the visual input based on entropy information derived from the visual input itself, the technology of the present disclosure overcomes the shortcomings of conventional feature extraction techniques. Therefore, the implementation of the disclosed technology does not require manual tuning, and feature extraction does not require user-defined input parameters. In addition, the present disclosure can be applied to any type of visual input because the diffusion of entropy information is calculated from the visual input itself and used to derive hyperparameters. Therefore, the feature extraction technology according to the present disclosure can be agnostic to the input type and can also be agnostic to the model algorithm used for feature importance or computer vision.

[0029] It should be noted that the terms "optimize", "optimal", etc., as used herein, may be used to mean making performance as efficient or perfect as possible. However, as one of ordinary skill in the art who reads this document will recognize, perfection cannot always be achieved. Therefore, these terms may also encompass achieving or realizing the best possible or efficient or practical performance under given circumstances, or achieving or realizing better performance than can be achieved using other settings or parameters.

[0030] Figure 1 is a block diagram of an example feature extraction system 100 according to an implementation of the present disclosure. The feature extraction system 100 may be implemented as a computer system, such as the following in combination with Figure 7 In another example, the feature extraction system can be implemented by any hardware processing circuit, such as but not limited to a combination of a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, etc. The feature extraction system 100 can be communicatively coupled to one or more input sources 130 and a storage medium 140 via a wired or wireless communication interface. Figure 7 To describe an example of a communication interface.

[0031] The feature extraction system 100 may refer to any computing infrastructure including control logic for performing feature extraction on visual input received from one or more input sources 130 and outputting a feature set of extracted features to a storage medium 140. In an example implementation, the feature extraction system 100 may be configured to calculate entropy information for multiple regions of the visual input and calculate hyperparameters based on the entropy information. The feature extraction system 100 may also segment the visual input into multiple clusters using a clustering algorithm and the calculated hyperparameters. Each of the multiple clusters may represent a feature in the visual input. Based on the multiple clusters, the feature extraction system 100 may extract a feature set of the visual input.

[0032] Traditionally, hyperparameters cannot be inferred by ML models because hyperparameters can be inputs used to control ML models. For example, some hyperparameters cannot be inferred by ML models because hyperparameters are used as inputs to control the speed and quality of the ML process. In the case of clustering-based ML models, examples of hyperparameters can be, but are not limited to, kernel functions, kernel bandwidth, and kernel density estimates. For example, a kernel function can be a hyperparameter input to a mean shift clustering algorithm that defines a window across which the mean shift clustering algorithm calculates the mean. The kernel bandwidth can be a hyperparameter that defines the distance or size of the window. The number of data points within the kernel bandwidth represents the kernel density.

[0033] The input source 130 may be one or more image capture devices, such as cameras, for capturing visual input. Visual input as used herein refers to a visual representation of an object or set of objects that constitutes the content of the visual input. Visual input may refer to an image (e.g., a digital image comprising a set of pixels that together depict the content of the digital image), a sequence of images that make up a video (e.g., an image frame), or any other visual input. In an example, an image (or an image frame of a video) may be a digital image comprising a set of pixels that together depict the content of the image. Each pixel includes a value that numerically represents the color, intensity, and / or grayscale of the pixel, and a spatial coordinate relative to other pixels within the image (e.g., represented by x, y on the x-axis and y-axis). In the case of an image capture device, the input source 130 may generate an input signal of the captured visual input, including pixels and their spatial coordinates, which may be transmitted to the feature extraction system 100 to perform feature extraction of the visual input.

[0034] In another example, input source 130 may be a storage medium for storing visual input. In this case, input source 130 may receive visual input from an external device such as an upstream computer system, an image capture device, and store the visual input. The visual input held in input source 130 may then be retrieved by feature extraction system 100 for performing feature extraction on the retrieved image. Examples of storage media that may be implemented as input source 130 include, but are not limited to, random access memory (RAM), cache, and / or other dynamic storage devices. In addition, the storage medium may be, for example, a read-only memory (ROM) or other static storage device.

[0035] The storage medium 140 can be communicatively coupled to the feature extraction system 100 and configured to receive output from the feature extraction system 100. In some examples, the output from the feature extraction system 100 can include a feature set of features extracted from the visual input. The feature set can be stored as a plurality of clusters derived from the visual input, as described in more detail below, where each cluster corresponds to an extracted feature of the feature set.

[0036] In some examples, the output from the feature extraction system 100 may include an output image representing the features extracted from the visual input. In this case, the output image may include contours represented according to uniquely assigned attributes. The assigned attributes can visually represent and distinguish the extracted features. For example, the output image can be uniquely colored and assigned to contours that highlight the extracted features. Each feature can correspond to a contour that is uniquely colored relative to other extracted features. The output image can provide a visual representation of the extracted features for a given visual input. As used herein, highlighting can refer to different colors, different gradients, different patterns (e.g., shadows, grids, lines, etc.), etc., for visually representing different areas within an image.

[0037] exist Figure 1 In an illustrative example of , the feature extraction system 100 includes a spatial information engine 110 configured to calculate entropy information for multiple regions of a visual input. For example, the spatial information engine 110 can determine the spread of entropy information in the visual input. According to various examples, the spatial information engine 110 can be configured to obtain the spread of entropy information by calculating the entropy level in the visual input. The spatial information engine 110 can split the visual input into multiple regions, determine an entropy value for each region, and obtain a spatial distribution of entropy values ​​on the visual input as entropy information.

[0038] In various examples, the spatial information engine 110 may be configured to iteratively calculate entropy information over regions that increase or expand in size. That is, for example, a first region of the visual input may be defined, and the spatial information engine 110 may calculate a first entropy value for the first region. The size of the first region may be iteratively expanded so that each iteration corresponds to an nth region (where n is an integer) of the plurality of regions, and the spatial information engine 110 may calculate an entropy value for each iterative region. This operation may be referred to herein as an entropy scan, which may be used by the spatial information engine 110 to obtain a spatial distribution of entropy information across the visual input.

[0039] The feature extraction system 100 may also include a feature extraction engine 120 configured to calculate hyperparameters based on entropy information. For example, the entropy information calculated by the spatial information engine 110 may be analyzed, and the feature extraction engine 120 may use the analysis results to calculate the hyperparameters. The feature extraction engine 120 may then use the hyperparameters as input to a clustering algorithm that may be executed to extract features from the visual input. The clustering algorithm may be, for example, a mean shift clustering algorithm, and the hyperparameters may be a kernel function and / or kernel bandwidth derived from the entropy information. In this example, the visual input may be applied to a mean shift clustering algorithm constrained according to the calculated hyperparameters, which segments the visual input to obtain clusters as output. These clusters may be segments representing features in the visual input. Although mean shift clustering is described herein, the present disclosure is not intended to be limited to mean shift clustering algorithms, and other clustering algorithms are considered to be within the scope of the present disclosure and may be implemented by the systems and methods disclosed herein.

[0040] Based on these clusters, the feature extraction engine 120 can be configured to extract a feature set of the visual input and map the clusters to contours. For example, the feature extraction engine 120 can be configured to derive contours from the clusters to provide a feature set of features extracted from the visual input. In an illustrative example, clusters can be identified and assigned different attributes (e.g., color or other visual features). Scanning can be performed by visual input from edge to edge. When the feature extraction engine 120 detects a change in cluster attributes during scanning, the feature extraction engine 120 can track the change as a contour and assign a unique attribute to the contour. The contour can then be stored as a feature set in, for example, a storage medium 140. In some cases, the feature extraction engine 120 can generate an output image as a visual representation of the feature set, and the output image can be stored in the storage medium 140. The output image can include a contour highlighted according to a unique attribute (e.g., color or other visual features) to visually depict the extracted features.

[0041] Figure 2is a schematic diagram of another example feature extraction system 200 according to an implementation of the present disclosure. The feature extraction system 200 can be similar to the above combined Figure 1 The feature extraction system 100 described above can therefore perform feature extraction on visual input as described above. Figure 2 In the illustrative example shown, feature extraction system 200 extracts information from a Figure 1 The input source 130 or other image source receives visual input 230. Figure 2 In the example of , the visual input 230 is provided as an x-ray image of a torso acquired from the National Institute of Health Clinical Center at https: / / nihcc.app.box.com / v / ChestXray-NIHCC. However, any visual input may be provided to a feature extraction system 200 according to the present disclosure. Furthermore, while the x-ray image is shown in grayscale, the implementations disclosed herein may be extended to color images having multiple planes spanning red, green, and blue (RGB). The feature extraction system 200 extracts features from the visual input 230 and generates an output image 240 in this example. The output image 240 is illustratively shown as a digital image that includes outlines highlighting the extracted features of the feature set.

[0042] The feature extraction system 200 includes a spatial information engine 210 and a feature extraction engine 220. The spatial information engine 210 and the extraction engine 220 may be similar to the spatial information engine 110 and the feature extraction engine 120, respectively, except that: Figure 2 Additional details of the spatial information engine 220 and the feature extraction engine 220 are depicted. For example, the spatial information engine 210 includes a spatial information analysis module 212 that can execute an extended area multi-scan module 214 and a per-area information calculation module 216. The extraction engine 220 includes a feature extraction module 222 that can execute a hyperparameter calculation module 224 and a cluster extraction and contour drawing module 226.

[0043] The spatial information engine 210 can be configured to calculate entropy information for multiple regions of the visual input 230. For example, the spatial information engine 210 can determine the spread of entropy information in the visual input 230 by calculating the entropy level in the visual input 230. For example, the extended region multi-scan module 214 can be executed to split the visual input 230 into multiple regions. The entropy calculation module 216 can be executed to determine the entropy information for each region and obtain the spatial distribution of the entropy information across the multiple regions.

[0044] Entropy may refer to a measure of the average uncertainty of information in a visual input, such as visual input 230. For example, a completely white or black image may have a normalized minimum entropy value of zero. However, an image with random noise including varying pixel intensities may have an entropy value close to a maximum value of 1. This quantitative entropy value may be used to obtain a measure of entropy information across visual input 230 that represents the distribution of uncertainty in the content contained in visual input 230.

[0045] An example of an entropy value is the Shannon entropy value. The Shannon entropy value can be used to obtain a quantifiable measure of the distribution of entropy information in the visual input and can be calculated as follows:

[0046]

[0047] Where E I is the entropy value of an image (I) (e.g., a two-dimensional image) with dimensions d1 and d2; n is the number of gray levels, which has a maximum number of gray levels of 2 in the case of an 8-bit image. 8 or 256; p k is the probability associated with the occurrence of a gray level, which ranges from 0 to n.

[0048] When equation 1 is applied to the entire image, the average calculation resulting from the summation in equation 1 may reduce the entropy value of regions in the image (I) having a large amount of uncertainty or a large amount of content, while the rest of the image has no or minimal uncertainty or content. That is, for example, a small region of the image (I) having a significant amount of uncertainty or content may have a reducing effect on the overall entropy value while the rest of the image has minimal (or no) uncertainty or information.

[0049] Thus, implementations disclosed herein are configured to determine a spread or distribution of entropy values ​​across visual input 230. For example, extended area multi-scan module 214 can execute an extended area algorithm that defines a plurality of regions across visual input 230, and entropy calculation module 216 can perform entropy calculations for each region of visual input 230. Extended area multi-scan module 214 and entropy calculation module 216 operate together to scan across visual input 230 and iteratively calculate entropy values ​​for each of the plurality of regions of visual input 230. The calculated entropy values ​​are collectively provided to provide a spread (e.g., spatial distribution) of entropy information for visual input 230.

[0050] According to some examples, the extended area multi-scanning module 214 may provide multiple areas as an increase or expansion area of ​​the visual input 230. For example, the extended area multi-scanning module 214 may define an initial first area of ​​the visual input 230 as a window W l ={w1 l , w2 l}, where w11 and w21 are the 2D sizes of the window. The sizes w11 and w21 can represent the number of pixels of the visual input 230 contained in the window defining the first area, and the spatial coordinates of the first area. For example, w11 and w21 can have a size of 2 pixels, and a spatial coordinate that positions the window at the lower left corner of the visual input 230. In one example, the lower left corner of the visual input 230 can define an origin with spatial coordinates 0,0. However, for a given implementation, other conventions can be used as needed. The size of the initial window can be defined according to the desired granularity of the analysis. That is, for example, a smaller initial size can provide increased granularity, but with increased processing time. However, a larger initial size can provide reduced processing time, but with reduced granularity.

[0051] From the first region, multiple regions can be provided by increasing one or two dimensions, so that each nth region can be defined as W n ={w1 n , w2 n}, where n is an integer. w1 n and w2 n One or both of can be increased for each subsequent window relative to the previous window according to the desired step size. The step size refers to the size difference between each iteration region, which can be defined as the amount used to increase w1 for each subsequent iteration. n and / or w2 n The number of pixels, applied to w1 n With w2 n The step size between each region can be defined according to the desired granularity of the analysis. That is, for example, a smaller step size may provide increased granularity, but with increased processing time. However, a larger step size may provide reduced processing time, but with reduced granularity.

[0052] Each window may be a crop of visual input 230 forming a corresponding region. n+1 ) can include the previous window W n That is, in one example, each region, except the initial region, covers all of the previous region, except for the initial region, because the initial region cannot include the previous window due to the absence of the previous window. In another example, each region may overlap at least partially with the previous region.

[0053] Figure 2 A schematic diagram 218 depicts two example methods of extending the region algorithm. Figure 3A-3D Provides a detailed explanation of example diagram 218 and other examples.

[0054] According to the multiple regions defined by the extended region multi-scan module 214, the entropy calculation module 216 may calculate entropy information as follows:

[0055] EnS{I}={r,E r Equation 2

[0056] Where EnS is an entropy scan of a visual input (e.g., visual input 230), r represents a set of regions of the visual input, and E r W represents the entropy information calculated for each region r. The set of regions can be provided as a plurality of regions, where each region is defined as W n {w1 n , w2 n}, where w1 n It is a*w1 n And w2 n It's b*w2 n In this example, a and b represent the scaling factors (e.g., step sizes) used to scale the window W of the region. In some examples, a and b can be the same scaling factor or different scaling factors, depending on the desired implementation. In addition, the size of the scaling (or step size) can be based on the granularity required for the implementation, as described above.

[0057] In various examples, E r It can be calculated based on applying Equation 1 to each region as follows:

[0058]

[0059] Where E In Represents a window {w1 n , w2 n As a result, the entropy scan (EnS{I}) can be provided as a set of regions r of the image I, which includes the entropy information of the region n with the window {w1 n , w2 n} and a set of entropy (E In ).

[0060] The entropy scan (EnS{I}) can be provided to the extraction engine 220 as entropy information for calculating hyperparameters for the clustering algorithm. In an example implementation, the entropy information of the entropy scan (EnS{I}) can be provided to the hyperparameter calculation module 224, which uses the entropy information to calculate quantile values ​​for estimating hyperparameters. A quantile refers to a point in a distribution that is related to the ranking order of the values ​​in the distribution. For a given sample, any quantile can be found by sorting the sample, where the middle value of the sorted sample (e.g., the middle quantile or the 50th percentile) represents the median, and the limits of the sample are the minimum and maximum values.

[0061] According to an illustrative example, the hyperparameter calculation module 224 uses a conversion function to calculate the quantile value Q of the entropy information. According to various examples, the conversion function provides an exponential valuation based on an entropy value selected from the entropy information to estimate the quantile value Q.

[0062] In an illustrative example, the conversion function may be a sigmoid function. In one example, the selected entropy value may be the maximum entropy value (Max E ). The maximum entropy value may indicate the minimum possible uniformity in the content of the visual input 230 (e.g., the maximum amount of information in the visual input) and the maximum granularity of the clustering of the clustering algorithm. In this example, the hyperparameter calculation module 224 locates the maximum entropy value (Max E ), and the quantile values ​​are calculated as follows:

[0063]

[0064] Among them, Es Q is the estimated quantile value; c is a factor set to normalize the entropy value to a range between 0 and 1, in this example, it can be set to 0.5; d is a tunable constant with a default optimal value for the sigmoid function, in this example, it is set to 5. The value of d can be tuned to adjust the slope of the sigmoid function, providing flexibility to the user. Tuning of c and d can be used to adjust the granularity of clustering (e.g., the number of clusters and their density). For example, increasing the slope by increasing d may result in an increase in the number of clusters and a decrease in the density of each cluster. Reducing d may result in the opposite tuning of the slope. Normalization can be increased by increasing c, which may result in a decrease in the number of clusters and an increase in the density of each cluster. In either case, the estimated quantile (Es Q ) can be used to derive hyperparameters for the clustering algorithm, as described below.

[0065] In the illustrative example, the clustering algorithm is a mean shift clustering algorithm. The mean shift clustering algorithm is a nonparametric density-based algorithm that uses a kernel density estimate for the underlying data distribution. The algorithm iteratively assigns data points to clusters by moving the data points toward the highest density data points. A high and thin kernel bandwidth results in large cluster counts (e.g., the number of data points per cluster) with a small density, while a short and fat kernel bandwidth results in fewer cluster counts with a large density.

[0066] The hyperparameter calculation module 224 may be implemented as an estimated quantile (Es) calculated according to Equation 4. Q) to automatically calculate the kernel bandwidth. For example, to automatically determine the kernel bandwidth, a k-nearest neighbor (k-NN) method can be used, where a parameter (number_of_neighbors) for the number of neighbors for each cluster can be set equal to the total number (number_of_data_points) of data points (e.g., pixels in a digital image) in the visual input 230 multiplied by a calculated quantile value calculated according to an entropy scan (EnS{I}), as described above. According to known k-NN techniques, the parameter (number_of_neighbors) can be used to estimate the k nearest neighbors. The estimated k-NN (estimate_k-NN) can be used to estimate the kernel bandwidth as follows:

[0067] Kernel bandwidth = estimate_k-NN(EsQ,I) Equation 5

[0068] Using the kernel bandwidth calculated in Equation 5, the contour drawing module 226 can be executed to segment the visual input 230 into clusters using a mean shift clustering algorithm as follows:

[0069] CS={c1,c2,...c k =meanshift_clustering(Kernel_bandwidth, I) Equation 6

[0071] Where CS is the cluster set, c k represents the kth cluster of the cluster set, and meanshift_clustering(Kernel_bandwidth, I) represents a mean shift clustering algorithm known in the art, which is applied to the visual input 230 using the kernel bandwidth calculated by the hyperparameter calculation module 224 according to Equation 4.

[0072] Once the cluster set (CS) is identified, the contour drawing module 226 can be executed to map each cluster of the cluster set (CS) to a contour. Based on the mapping, the contour drawing module 226 can generate an output image 240 in which the mapped contours are displayed using different attributes to distinguish each contour. The different attributes can include, but are not limited to, different colors, different gradients, different patterns (e.g., shadows, grids, lines, etc.) for visually distinguishing different areas within the output image 240, etc.

[0073] In an illustrative example, mapping each cluster to a contour may include performing contour detection around each cluster in the set of clusters. For example, contour drawing module 226 may be implemented to assign a different color as an example attribute to each point of a cluster (e.g., each cluster corresponds to an assigned color), and select a set of unique colors as example attributes of the contour (e.g., there is a unique color for each contour). Contour drawing module 226 may then scan the contents of visual input 230 and identify changes in cluster colors (e.g., transitions from one cluster to another). For example, contour drawing module 226 may examine each pixel of visual input 230 and detect to which cluster the examined pixel belongs by detecting the assigned color. Contour drawing module 226 assigns a unique color of the contour to the pixel in response to detecting a change in cluster color, and records the spatial coordinates of the identified change with the unique color. Contour drawing module 226 may then store the set of clusters as a feature belonging to visual input 230, represented as a colored contour. For example, as Figure 2 As shown, the output image 240 depicts various gray levels, where each gray level corresponds to an identified cluster and contour. For illustrative purposes, the output image 240 shows a subset of contours as contours 242a-242e.

[0074] Figure 3A-3D is a visual depiction of an example extended region algorithm executed on an example visual input according to an implementation of the present disclosure. An implementation of the extended region algorithm may be executed, for example, by the spatial information engine 110 and / or 210 (e.g., the extended region multi-scan module 214) to scan multiple regions extending in any direction. For example, a vertical scan such as Figure 3A As shown (referred to as horizontal window vertical scanning in this article), the vertical window horizontal scanning is as follows Figure 3B As shown in (referred to as vertical window horizontal scanning in this article), the outward scanning from the center point to the corner is as shown in Figure 3C As shown in (referred to as growth center scan in this article), the scan from one corner to another is as follows Figure 3D Each implementation of the extended region algorithm is shown as overlaid with an example image 330 (eg, an X-ray image in these examples), which may be Figure 2 An example of visual input 230 .

[0075] Figure 3AAn initial or first region 302a is depicted overlaid on an image 330. The first region 302a is shown as a window having dimensions w11 and w21, where w21 is the horizontal dimension of the image 330. The dimensions of the first region 302a extend vertically along a direction 306a to a second region 304a. The second region 304a is shown as a window having dimensions w12 and w21, where w12 is greater than and includes w11. The second region 304a may extend to a window having dimensions w12 and w21. n One or more subsequent regions of size 306a, which expands in direction 306a until w1 n is equal to the vertical dimension of image 330. Each subsequent region may include the previous region. Thus, Figure 3A The illustrated extended region algorithm may include a first region 302a, a second region 304a, and one or more subsequent regions.

[0076] Although Figure 3A Direction 306a is shown extending from the top edge of image 330 to the bottom edge of image 330, but other implementations are possible. For example, direction 306a may extend from the bottom edge to the top edge.

[0077] Figure 3B Another example is depicted in which the first region 302b extends horizontally in direction 306b (e.g., from one edge of the image 330 to the other edge). In this case, the first region 302b has dimensions w11 and w21, where w11 is the vertical dimension of the image 330. The first region 302b extends to a second region 304b having dimensions w11 and w22, where w22 is greater than and includes w21. The second region 304b may extend to a second region 304b having dimensions w22. n One or more subsequent regions of size extending in direction 306b until w2 n Equal to the width of image 330 .

[0078] Although Figure 3B A direction 306b extending from left to right is shown, but the expansion area may extend from right to left.

[0079] Figure 3C Another example is depicted in which a first region 302c is located within an image and extends in a direction 306c. In this case, the first region 302c is located at the center of 330. Here, the first region 302c has dimensions w11 and w21. The first region 302c extends to a second region 304c having dimensions w12 and w22, wherein w12 and w22 are greater than and include w11 and w21. The second region 304c may extend to a second region 304c having dimensions w12 and w22. n and w2 nFor one or more subsequent regions of size, the size expands in direction 306 c until the region includes the entire image 330 .

[0080] Although Figure 3C The first region 302c is depicted as being located at the center of the image 330, but any initial location of the first region 302c may be utilized. In the case of a non-central location, the scaling factor may be adjusted to ensure that the final region does not exceed the edge of the image 330. For example, Figure 3D An example is shown in which a first region 302d is located at the left corner of the image 330 extending in a direction 306d. As described above, the first region 302d has dimensions w11 and w21 and extends to a second region 304d having dimensions w12 and w22, wherein w12 and w22 are greater than and include w11 and w21. The second region 304d may extend to a second region 304d having dimensions w12 and w22. n and w2 n One or more subsequent regions of the image 330 are formed, which expand in direction 306d until the region includes the entire image 330. Other locations are also possible, such as the lower right corner, the upper left corner, the upper right corner, non-corner and non-center locations, etc.

[0081] Figure 4A and Figure 4B A flowchart of an example process 400 for feature extraction according to an implementation of the present disclosure is depicted. The process 400 may be implemented, for example, as instructions stored in a memory that, when executed by one or more processors, perform the operations of the process 400. The memory and / or one or more processors may be implemented as follows in combination Figure 7 4. In some examples, process 400 can be performed by feature extraction systems 100 and / or 200.

[0082] Figure 4A Depicted is a subprocess 410 that may be implemented to perform spatial information analysis of visual input. Subprocess 410 may be performed by spatial information engine 110 and / or spatial information engine 210, for example. Figure 4B Depicted is a subprocess 420 that may be implemented to perform feature extraction and segmentation of visual input based on analysis of spatial information from subprocess 410. Subprocess 420 may be performed by feature extraction engine 120 and / or extraction engine 220, for example.

[0083] Subprocess 410 includes an operation 412 of defining a plurality of regions by performing one or more extended region scans. Each of the one or more extended region scans provides a corresponding plurality of regions of visual input, and entropy information is calculated from the plurality of regions for each scan at operation 414, for example, as described above in conjunction with Figure 2At operation 418, the entropy information from operation 416 may be aggregated and sorted according to the entropy values. Based on the sorted entropy values, the maximum entropy value may be obtained for downstream use.

[0084] In more detail, operation 412 in the example of process 400 includes three operations 412a-412c, each of which can be performed in parallel. In another example, operations 412a-412c can be performed serially or partially serially according to the needs of a given application.

[0085] In operation 412a, the above combination may be performed. Figure 3B The vertical window horizontal scanning and the above combination Figure 3A The horizontal window scans vertically. The vertical window scans horizontally to provide a first plurality of regions of visual input, and the horizontal window scans vertically to provide a second plurality of regions.

[0086] In operation 414a, entropy information for each scan is calculated, for example, according to Equation 3. That is, for example, in a first iteration of operation 414, entropy information for a first region of the first plurality of regions defined in operation 412a is calculated and stored as an entry of an entropy scan (EnS).

[0087] At operation 416a, the region is incremented to the next region in the first plurality of regions. For example, in the case of providing a vertical window horizontal scan of the first plurality of regions, operation 416a increases the horizontal size (e.g., Figure 3B In the case of w2 N ) is extended to the next window to extend the region to the second region. Process 400 returns to operation 414a, and entropy information for the next region is calculated and stored as an entry of the entropy scan (EnS).

[0088] Similarly, in the case of a horizontal window scanning vertically, operation 416a performs the operation 416a by increasing the vertical size (e.g., Figure 3A In the case of w1 N ) is expanded to the next horizontal window to increment the region to the second region of the second plurality of regions. Process 400 returns to operation 414a, and entropy information for the second region is calculated and stored as an entry of the entropy scan (EnS) horizontal window vertical scan. Operations 414a and 416a are repeated until entropy is calculated for each region.

[0089] Additionally, in the example of process 400, operation 412 includes operation 412b, in which one or more spread angle scans are performed. Operation 412b may include performing a spread angle scan, such as in conjunction with Figure 3DOperation 412b may also include performing one or more additional extended angle scans, such as an angle scan for each of the other angles. Thus, operation 412b may include four extended angle scans: an upper left extended angle scan, an upper right extended angle scan, a lower left extended angle scan, and a lower right extended angle scan. In this case, a third plurality of regions, a fourth plurality of regions, a fifth plurality of regions, and a sixth plurality of regions may be generated corresponding to each of the extended angle scans.

[0090] In operation 414b, entropy information is calculated for each scan performed in operation 412b using each of the third to sixth plurality of regions, for example, according to Equation 3. As described above, entropy information is iteratively calculated for each region while the region is incremented to the next region of the corresponding scan in operation 416b. For each iteration, operation 414b records the entries in the entropy scan (EnS) for each region, thereby generating an entropy scan for each of the upper left spread angle scan, the upper right spread angle scan, the lower left spread angle scan, and the lower right spread angle scan. Operations 414b and 416b are repeated until entropy is calculated for each region.

[0091] In the example of process 400, operation 412 also includes operation 412c, in which an extended center scan is performed, such as in conjunction with Figure 3C In this case, a seventh plurality of regions corresponding to the extended central scan may be generated.

[0092] At operation 414c, for example, according to Equation 3, the seventh plurality of regions may be used to calculate entropy information for the extended center scan performed at operation 412c. As described above, entropy information is iteratively calculated for each region, and the region is incremented to the next region in the seventh plurality of regions at operation 416c. For each iteration, operation 414c records an entry for the extended center scan in the entropy scan (EnS). Operations 414c and 416c are repeated until entropy is calculated for each region.

[0093] In the above example, a total of seven growing region scans are performed. However, this is an illustrative example, and other implementations may include more or fewer scans as desired for a given implementation. For example, a subset of the above examples may be used, while another example may utilize one or more additional growing region scans having locations distributed around the visual input.

[0094] According to some implementations, the amount of time for performing operations 412-416 can be accelerated by parallel computing. For example, each of the extended area scan and entropy calculation can be independent of each other. By utilizing this independence, each scan can be performed individually and independently of each other, which allows the use of parallel processing and calculation to perform operations 412-416. One or more of operations 412-416 can be distributed across multiple CPUs or CPU cores in a multi-core environment, and executed in parallel or almost simultaneously. For example, operations 412a-416a can be performed on a CPU (or CPU core), and operations 412b-416b can be performed on another CPU. In addition, operations 412c-416c can be performed on another CPU (or CPU core). In addition, each scan (for example, horizontal scan and vertical scan) of operation 412a can be distributed across one or more CPUs or CPU cores. Similarly, each scan of operation 412b can be distributed across one or more CPUs or CPU cores. The number of CPUs or CPU cores and which scans are performed in parallel on which CPU or CPU core can be pre-specified.

[0095] In operation 418, the entropy information from operations 416a-416c is aggregated and sorted according to entropy values. That is, for example, each entropy scan contains an entropy value, and these entropy values ​​can be aggregated into a single entropy value set. The entropy values ​​can be sorted from the highest entropy value to the lowest entropy value. From the sorted entropy values, an entropy value can be selected for use by subprocess 420. For example, in operation 418, a maximum entropy (Max Entropy) value can be obtained from the sorted entropy values. E ). In some examples, the entropy value to be used by subprocess 420 may be tunable for adjusting to a desired granularity. In such cases, an entropy value other than the maximum entropy may be selected, such as the maximum entropy of one of the scans performed in operation 412. The selected entropy value may not be a global maximum (e.g., Max E ), but can be a local maximum for a particular scan.

[0096] Subprocess 420 includes operation 421, in which the entropy value selected in operation 418 (eg, the maximum entropy (Max E )) to calculate the quantile hyperparameter for the clustering algorithm performed in operation 424. Operation 421 includes operation 422, in which the quantile value is estimated using the selected entropy value, and operation 423, in which the hyperparameter is calculated as the kernel bandwidth based on the quantile value from operation 422. More specifically, as described above, operation 422 uses the mirror sigmoid function and the maximum entropy value (Max E ) to calculate the quantile value Q.

[0097] At operation 423, hyperparameters are calculated based on the quantile values ​​Q calculated at operation 422. In an example implementation of process 400, operation 423 calculates kernel bandwidth using the quantile values ​​from operation 422 and a k-nearest neighbor method for use in a mean shift clustering algorithm according to Equation 5 above.

[0098] The kernel bandwidth is supplied to operation 424, which performs clustering and contour mapping based on the hyperparameters (e.g., kernel bandwidth) calculated in operation 423. For example, operation 424 includes operation 425, in which a mean shift algorithm is run on the visual input using the kernel bandwidth from operation 423. Operation 424 segments the visual input into clusters c1, c2, ..., c k to provide a set of clusters CS, as described above in Equation 6. The clusters represent feature segments, so that the set of clusters can represent a feature set or an extracted feature set.

[0099] Operation 426 obtains cluster set CS and derives the outline around the cluster to map the cluster to the outline, and the derived outline is stored in a memory. For example, operation 426 assigns different attributes to each point in the points of the cluster (for example, each cluster corresponds to the assigned attribute), and selects a set of unique attributes for the outline (for example, selects a unique attribute for each outline). Operation 426 then scans the content of the visual input 230 and identifies the change of the cluster attribute (for example, from one cluster to another cluster and transition). For example, operation 426 can check each pixel of the visual input, and detect which cluster the pixel belongs to by detecting the assigned attribute. In response to detecting the change of the cluster attribute, the unique attribute of the outline can be assigned to the pixel, and the spatial coordinates of the identified change can be recorded with the unique attribute. The cluster set can be stored as a feature, represented as a contour represented according to the assigned attribute, belonging to the visual input. Operation 426 can then use the assigned attribute to generate an output image with a contour represented according to the unique attribute. The output image can be displayed to the user, for example, via the display of the computer system 700.

[0100] Figure 5A and Figure 5B Depicted are example images and a graphical representation of the entropy information calculated for each image.

[0101] Figure 5AA graphical representation 504a of entropy information calculated for an image 502a according to different strategies is depicted. For example, an average entropy is calculated for the entire image 502a, and entropy information for different extended area scans is also calculated and plotted as a graphical representation 504a. The graphical representation 504a plots entropy values ​​in bytes as a function of the number of regions (e.g., number of windows) of the image. An average entropy is calculated for the entire image, and various scans provide entropy as a function of the number of regions. In this case, both the horizontal and vertical sizes of the window remain unchanged. Figure 5B Similar to Figure 5A , except that entropy information is calculated for image 502b to provide graphical representation 504b.

[0102] Figure 5A and Figure 5B The importance of spatially evaluating entropy using the extended region algorithm disclosed herein is highlighted. As described above, the extended region algorithm according to the present disclosure can ensure that entropy information in an image is not lost due to the content of the image. The extended region algorithm disclosed herein can be implemented to give an improved representation of entropy in an image when the image is extended across a region that is greater than the average entropy over the entire image.

[0103] Figure 6 Example computing components that can be used to implement feature extraction according to various embodiments are shown. Figure 6 , computing component 600 may be, for example, a server computer, a controller, or any other similar computing component capable of processing data. Figure 6 In the example implementation of , computing component 600 includes a hardware processor 602 and a machine-readable storage medium 604 .

[0104] The hardware processor 602 may be one or more central processing units (CPUs), semiconductor-based microprocessors, and / or other hardware devices suitable for retrieving and executing instructions stored in the machine-readable storage medium 604. The hardware processor 602 may retrieve, decode, and execute instructions, such as instructions 606-612, to control a process or operation for feature extraction. As an alternative or in addition to retrieving and executing instructions, the hardware processor 602 may include one or more electronic circuits including electronic components for performing the functions of one or more instructions, such as a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or other electronic circuits.

[0105] A machine-readable storage medium, such as machine-readable storage medium 604, may be any electronic, magnetic, optical, or other physical storage device that contains or stores executable instructions. Thus, machine-readable storage medium 604 may be, for example, a random access memory (RAM), a non-volatile RAM (NVRAM), an electrically erasable programmable read-only memory (EEPROM), a storage device, an optical disk, etc. In some embodiments, machine-readable storage medium 604 may be a non-transitory storage medium, where the term "non-transitory" does not encompass transient propagating signals. As described in detail below, machine-readable storage medium 604 may be encoded with executable instructions, such as instructions 606-612.

[0106] Hardware processor 602 may execute instructions 606 to calculate entropy information for a plurality of regions of the visual input. In some examples, the plurality of regions may be generated by defining a first region of the visual input and iteratively expanding the first region for a subset of the plurality of regions, where the plurality of regions may include the first region and the subset of the plurality of regions. Additional examples and details may be found above, for example, as described above in conjunction with Figure 1-4B described.

[0107] The entropy information may be calculated by calculating an entropy value for each of the plurality of regions. In some examples, instruction 606 may include selecting an entropy value from the calculated entropy values, such as a maximum entropy value as described above. Other entropy values ​​may be selected based on the desired granularity of feature extraction. In an example implementation, the entropy value may be calculated based on the Shannon entropy value, as shown in equations 2 and 3 above.

[0108] The hardware processor 602 may execute instructions 608 to calculate a hyperparameter for a clustering algorithm based on the entropy information. For example, the hyperparameter may be calculated based on the entropy value selected in instruction 606. A quantile value may be determined based on the selected entropy value, which may be used to calculate the hyperparameter. In an example implementation, the clustering algorithm is a mean shift clustering algorithm, and the hyperparameter is the kernel bandwidth. In this case, the quantile value may be calculated as described in Equation 4 above, and the kernel bandwidth may be calculated as described in Equation 5 above.

[0109] Hardware processor 602 may execute instructions 610 to segment the visual input into a plurality of clusters using a clustering algorithm and hyperparameters. In some examples, each of the plurality of clusters represents a feature in the visual input. For example, a mean shift clustering algorithm may be applied to the visual input using the kernel bandwidth calculated by instruction 608 to return a set of clusters as described in conjunction with Equation 6.

[0110] The hardware processor 602 can execute instructions 612 to extract a feature set of the visual input based on the plurality of clusters. For example, from a set of clusters determined using a clustering algorithm, the clusters can be mapped to contours, which can be stored as a set of extracted features. These contours can be used to generate an output image that visually represents the extracted features as contours, which are represented as uniquely assigned attributes (e.g., color or other visual representations), for example, as described above in conjunction with Figure 1-Figure 2 and Figure 4B described.

[0111] Figure 7 A block diagram of an example computer system 700 is depicted in which various embodiments described herein may be implemented. The computer system 700 includes a bus 702 or other communication mechanism for communicating information, one or more hardware processors 704 coupled to the bus 702 for processing information. The hardware processor(s) 704 may be, for example, one or more general purpose microprocessors. In some examples, the hardware processor(s) 704 may be a multi-core processor including multiple processing cores.

[0112] Computer system 700 also includes a main memory 706, such as a random access memory (RAM), cache, and / or other dynamic storage device, coupled to bus 702 for storing information and instructions to be executed by processor 704. In some examples, process 400 may be provided as instructions to be executed by processor 704. Main memory 706 may also be used to store temporary variables or other intermediate information during execution of instructions to be executed by processor 704. When stored in a storage medium accessible to processor 704, such instructions cause computer system 700 to become a special-purpose machine customized to perform the operations specified in the instructions.

[0113] The computer system 700 also includes a read only memory (ROM) 708 or other static storage device coupled to the bus 702 for storing static information and instructions for the processor 704. A storage device 710, such as a magnetic disk, optical disk, or USB thumb drive (flash drive), etc., is provided and coupled to the bus 702 for storing information and instructions.

[0114] The computer system 700 may be coupled to a display 712, such as a liquid crystal display (LCD) (or touch screen), via the bus 702 and / or the communication interface 718, for displaying information to a computer user. An input device 714, including alphanumeric and other keys, is coupled to the bus 702 for communicating information and command selections to the processor 704. Another type of user input device is a cursor control 716, such as a mouse, trackball, or cursor direction keys, for communicating direction information and command selections to the processor 704 and for controlling cursor movement on the display 712. In some embodiments, the same direction information and command selections as a cursor control may be achieved by receiving touches on a touch screen without a cursor.

[0115] Another type of user input device is an image capture device, such as a camera, that is used to capture images; sequences of images that constitute image frames that make up a video; and other visual inputs. The image capture device may generate an image signal of the captured image, which may be a communication memory for storage, such as the main memory 706. The captured image may be a digital image composed of pixels that collectively depict the content of the captured image.

[0116] The computing system 700 may include a user interface module for implementing a GUI, which may be stored in a mass storage device as executable software code executed by the computing device(s). For example, this module and other modules may include components, such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, program code segments, drivers, firmware, microcode, circuit systems, data, databases, data structures, tables, arrays, and variables.

[0117] In general, the words "component", "engine", "system", "database", "data store", etc. used herein may refer to logic embodied in hardware or firmware, or to a collection of software instructions, which may have entry and exit points, written in a programming language (such as Java, C or C++). Software components may be compiled and linked into an executable program, installed in a dynamic link library, or may be written in an interpreted programming language (e.g., BASIC, Perl or Python). It should be understood that software components may be called from other components or from themselves, and / or may be called in response to detected events or interrupts. Software components configured to execute on a computing device may be provided on a computer-readable medium, such as a compact disc, digital video disc, flash drive, disk, or any other tangible medium, or provided as a digital download (and may be initially stored in a compressed or installable format that requires installation, decompression or decryption prior to execution). Such software code may be stored in part or in whole on a memory device of an executing computing device for execution by the computing device. Software instructions may be embedded in firmware, such as an EPROM. It should also be understood that hardware components may be composed of connected logic units, such as gates and flip-flops, and / or may be composed of programmable units, such as programmable gate arrays or processors.

[0118] The computer system 700 may implement the techniques described herein using custom hardwired logic, one or more ASICs or FPGAs, firmware, and / or program logic that, in combination with the computer system, causes the computer system 700 to become or program the computer system 700 to be a special purpose machine. According to one embodiment, the techniques herein are performed by the computer system 700 in response to the processor(s) 704 executing one or more sequences of one or more instructions contained in the main memory 706. Such instructions may be read into the main memory 706 from another storage medium such as the storage device 710. Execution of the sequences of instructions contained in the main memory 706 causes the processor(s) 704 to perform the processing steps described herein. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions.

[0119] The term "non-transient medium" and similar terms used herein refer to any medium that stores data and / or instructions, which cause the machine to operate in a specific manner. Such non-transient media can include non-volatile media and / or volatile media. Non-volatile media include, for example, optical disks or magnetic disks, such as storage device 710. Volatile media include dynamic memories such as main memory 706. Common forms of non-transient media include, for example, floppy disks, flexible disks, hard disks, solid-state drives, tapes or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with hole patterns, RAM, PROM and EPROM, FLASH-EPROM, NVRAM, any other memory chip or cassette memory, and network versions of the above.

[0120] Non-transient media are distinct from transmission media, but may be used in conjunction with transmission media. Transmission media participate in the transmission of information between non-transient media. For example, transmission media include coaxial cables, copper wire, and optical fiber, including wires, including bus 702. Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

[0121] Computer system 700 also includes a communication interface 718 (also referred to as a network interface) coupled to bus 702. Communication interface 718 provides a two-way data communication coupling to one or more network links connected to one or more local networks. For example, communication interface 718 can be an integrated services digital network (ISDN) card, a cable modem, a satellite modem, or a modem that provides a data communication connection to a telephone line of a corresponding type. As another example, communication interface 718 can be a local area network (LAN) card that is used to provide a data communication connection to a compatible LAN (or a WAN component that communicates with a WAN). Wireless links can also be implemented. In any such implementation, communication interface 718 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.

[0122] A network link typically provides data communications to other data devices through one or more networks. For example, a network link may provide a connection to a host computer or to data devices operated by an Internet Service Provider (ISP) through a local network. The ISP in turn provides data communications services through a global packet data communications network now commonly referred to as the "Internet." Both local networks and the Internet use electrical, electromagnetic, or optical signals that carry digital data streams. The signals through the various networks and on the network link and through the communication interface 718 are example forms of transmission media that transmit digital data to and from the computer system 700.

[0123] Computer system 700 can send messages and receive data, including program code, through the network(s), network links and communication interface 718. In the Internet example, a server can transmit the requested code for an application program through the Internet, an ISP, a local network and communication interface 718.

[0124] The received code may be executed by processor 704 as it is received, and / or stored in storage device 710 or other non-volatile storage for later execution.

[0125] Each of the processes, methods, and algorithms described in the preceding sections may be embodied in a code component executed by one or more computer systems or computer processors including computer hardware, and fully or partially automated by it. One or more computer systems or computer processors may also operate to support the execution of related operations in a "cloud computing" environment or as "software as a service" (SaaS). The processes and algorithms may be implemented in part or in whole in a dedicated circuit system. The various features and processes described above may be used independently of each other, or may be combined in various ways. Different combinations and sub-combinations are intended to fall within the scope of the present disclosure, and certain methods or process blocks may be omitted in some implementations. The methods and processes described herein are also not limited to any particular sequence, and the blocks or states associated therewith may be executed in other appropriate sequences, or may be executed in parallel, or in some other manner. Blocks or states may be added to or removed from the disclosed example embodiments. The execution of certain operations or processes may be distributed between computer systems or computer processors, not only residing in a single machine, but also deployed across multiple machines.

[0126] As used herein, circuits can be implemented using any form of hardware, software or a combination thereof. For example, one or more processors, controllers, ASICs, PLAs, PALs, CPLDs, FPGAs, logic components, software routines or other mechanisms can be implemented to form circuits. In implementation, the various circuits described herein can be implemented as discrete circuits, or the functions and features described can be partially or completely shared between one or more circuits. Even if various features or functional elements can be described or claimed as separate circuits separately, these features and functions can be shared between one or more common circuits, and this description should not require or imply that separate circuits are needed to implement such features or functions. In the case where the circuit is implemented in whole or in part using software, this software can be implemented to operate together with a computing or processing system (such as computer system 700) that can perform the functions described therefor.

[0127] As used herein, the term "or" may be interpreted as inclusive or exclusive. In addition, singular resource, operation, or structure descriptions should not exclude the plural. Unless otherwise specifically stated, or otherwise understood in the context of use, conditional language such as "can," "could," "might," or "may" is generally intended to convey that certain embodiments include and other embodiments do not include certain features, elements, and / or steps.

[0128] Unless expressly stated otherwise, the terms and phrases used in this document and their variations should be interpreted as open-ended, not restrictive. Adjectives such as "conventional," "traditional," "normal," "standard," "known," and terms of similar meaning should not be interpreted as limiting the items described to items available in a given time period or as of a given time, but should be understood to cover conventional, traditional, normal, or standard technologies that may be available or known at any time now or in the future. In some cases, the presence of broadening words and phrases such as "one or more," "at least," "but not limited to," or other similar phrases should not be understood as an intent or requirement to use a narrower case where such broadening phrases may be absent.

Claims

1. A method for extracting features from an image, comprising: Calculating entropy information for a plurality of regions of the image; Calculating hyperparameters for a clustering algorithm based on the entropy information; segmenting the image into a plurality of clusters using the clustering algorithm and the hyperparameters; as well as A feature set of the image is extracted based on the plurality of clusters. 2 . The method of claim 1 , wherein each cluster of the plurality of clusters represents a feature in the image.

3. The method according to claim 1, further comprising: generating the plurality of regions of the image, the plurality of regions defining a first region of the image and iteratively extending the first region for a subset of the plurality of regions, wherein the plurality of regions includes the first region and the subset of the plurality of regions, Calculating the entropy information includes calculating an entropy value of each of the multiple regions.

4. The method according to claim 3, further comprising: selecting an entropy value among the calculated entropy values, The hyperparameter is calculated based on the selected entropy value. The method of claim 4 , wherein selecting the entropy value comprises selecting a maximum entropy value among the calculated entropy values.

6. The method of claim 4, wherein calculating the hyperparameter comprises: Determining a quantile value according to the selected entropy value; as well as The hyperparameter is calculated according to the quantile value. The method of claim 1 , wherein the clustering algorithm is based on a mean shift clustering algorithm.

8. The method of claim 7, wherein the hyperparameter is kernel bandwidth.

9. The method of claim 1, wherein the entropy information is based on a Shannon entropy value for each of the plurality of regions.

10. A system for feature extraction, comprising: Memory, which stores instructions; as well as at least one processor coupled to the memory and configured to execute the instructions to: calculating entropy information by determining an entropy value for each of a plurality of regions of the image; determining hyperparameters for a clustering algorithm based on the entropy information; segmenting the image into a plurality of clusters using the clustering algorithm and the hyperparameters; as well as A feature set of the image is extracted based on the plurality of clusters. The system of claim 10 , wherein each cluster of the plurality of clusters represents a feature in the image.

12. The system of claim 10, wherein the at least one processor is further configured to: generating the plurality of regions of the image, the plurality of regions defining a first region of the image and iteratively extending the first region for a subset of the plurality of regions, wherein the plurality of regions includes the first region and the subset of the plurality of regions, Calculating the entropy information includes calculating an entropy value for each of the multiple regions.

13. The system of claim 10, further comprising: Selecting the maximum entropy value among the calculated entropy values, The hyperparameter is calculated based on the maximum entropy value.

14. The method of claim 13, wherein calculating the hyperparameter comprises: Determine the quantile value based on the selected entropy value; as well as The hyperparameter is calculated according to the quantile value.

15. The system of claim 10, wherein the clustering algorithm is based on a mean shift clustering algorithm. The method of claim 15 , wherein the hyperparameter is kernel bandwidth.

17. The system of claim 10, wherein the entropy information is based on a Shannon entropy value for each of the plurality of regions.

18. A non-transitory machine-readable medium that, when executed by at least one processor of a computing system, causes the computing system to perform a method comprising: performing a plurality of entropy scans of an image, the plurality of entropy scans providing entropy information by determining an entropy value for each of a plurality of regions of the image; as well as A hyperparameter for a clustering algorithm is derived based on the entropy information, and a feature set of the image is extracted by applying the clustering algorithm to the image using the derived hyperparameter.

19. The non-transitory medium of claim 18, wherein the plurality of entropy scans include a first entropy scan and a second entropy scan, wherein the first entropy scan and the second entropy scan are performed in parallel.

20. The non-transitory medium of claim 18, wherein the method further comprises generating the plurality of regions of the image by defining a first region of the image and iteratively extending the first region to a subset of the plurality of regions, wherein the plurality of regions includes the first region and the subset of the plurality of regions.

Images