Memory containing instances of hamming distance of a computational neural network and a data center application

Abstract

Examples of the systems and methods described herein enable processing of image codes (e.g., binary embeddings) at a memory die. Such image codes can be generated by various endpoint computing devices, such as Internet of Things (IoT) computing devices. Such devices can generate Hamming processing commands with image codes of the images to compare the representations of the images with other images (e.g., in a dataset of images) to identify matches or sets of neural network results. Advantageously, examples described herein can be used in neural networks to facilitate processing of datasets in order to increase the rate and amount of processing of such datasets. For example, comparisons of image codes can be performed on the memory die itself, as with a NAND memory device.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims priority to U.S. Application No. 17 / 016,074, filed September 09, 2020, which is incorporated by reference herein in its entirety for all purposes. BACKGROUND

[0003] A neural network can provide output data that represents a“signature” of input data. For example, in one type of neural network, input data is provided to a set of layers in which some portion of the input data is multiplied by a set of weights and accumulated in a transfer function, which can be non-linear, to provide neural network output data. The output data can be referred to as a signature, for example, in a deep learning neural network context. The signature can represent one or more aspects or features of the input data.

[0004] Deep learning neural network contexts can be interconnected among various neural networks, e.g., providing output of some neural networks as input to other neural networks. Typically, deep learning neural networks can utilize large data sets, e.g., a facial image deep learning neural network can be trained on 8 million facial images (as in FaceNet). Large data sets that are typically based on memory intensive content such as images or video require increased amounts of memory and increased data transfer requirements, such as bandwidth or number of memory connections to a processor, e.g., to retrieve data of the data set for training or processing on a deep neural network.

[0005] Image content can be represented using a hash algorithm or hash. For example, a hash can be applied to a particular image to represent the image as a binary embedding. That is, a representation of the image is“embedded” into a binary representation or binary code. A particular binary embedding can be used as a way to match the image to another image, e.g., the respective binary embeddings of two images are exactly the same.

[0006] According to information theory, a Hamming distance can be calculated among two numbers of equal length (e.g., binary numbers) by identifying the number of positions in which the numbers differ, e.g., different bits.

[0007] In addition, there is growing interest in developing wireless communications to“fifth generation” (5G) systems. 5G is expected to improve speed and ubiquity, but methods for handling 5G wireless communications have not yet been determined. In some implementations of 5G wireless communications, “Internet of Things” (IoT) devices can operate on a narrowband wireless communication standard that can be referred to as Narrowband-IoT (NB-IoT). For example, Release 13 of the 3GPP specification describes a narrowband wireless communication standard. SUMMARY

[0008] Example methods and apparatus are described herein. An example method may include: obtaining an image code associated with a Hamming processing command at an I / O control unit of a memory device; responding to the Hamming processing command, providing at least one memory access request to a plurality of memory addresses to obtain a plurality of stored image codes; and comparing each of the plurality of stored image codes with the image code associated with the Hamming processing command to calculate a corresponding Hamming distance among a plurality of Hamming distances.

[0009] Alternatively or additionally, the control logic of the memory device may further include obtaining control signals that indicate Hamming processing commands.

[0010] Alternatively, obtaining the control signal indicating the Hamming processing command may include obtaining the control signal indicating the Hamming processing command at the Command Latch Enable (CLE) pin of the control logic.

[0011] Alternatively or additionally, the control logic of the memory device may include an additional control signal indicating that image code will be written to the data register of the memory device.

[0012] Alternatively, additional control signals may be obtained at the control logic of the memory device to indicate that the image code will be written to the cache register of the memory device, including additional control signals obtained at the write enable (WE#) pin of the control logic.

[0013] Alternatively, it may further include a control signal that indicates a Hamming processing command, confirming the readiness / busy (R / B) pin of the control logic.

[0014] Alternatively or concurrently, in response to a Hamming processing command, providing at least one memory access request for a plurality of memory addresses to obtain a plurality of stored image codes includes: generating a plurality of memory addresses to be read based on the Hamming processing command; providing the generated plurality of memory addresses to the control logic via a command register; and generating at least one memory access request for the plurality of memory addresses at the control logic based on the generated plurality of memory addresses stored in the command register.

[0015] Alternatively or concurrently, in response to a Hamming processing command, providing at least one memory access request to a plurality of memory addresses to obtain a plurality of images includes providing at least one memory access request to row and column decoders of the memory device for reading the memory array of the memory device.

[0016] Alternatively or additionally, it may further include reading multiple memory addresses into the data register of the memory device at the memory array of the memory device to obtain multiple stored image codes on the data register of the memory device.

[0017] Alternatively, comparing each of the multiple stored image codes with the image code associated with the Hamming processing command to calculate the corresponding Hamming distance among the multiple Hamming distances includes: calculating the corresponding Hamming distance among the multiple Hamming distances at the data register based on each of the multiple stored image codes and the image code associated with the Hamming processing code.

[0018] Alternatively, calculating the corresponding Hamming distance among the multiple Hamming distances includes: for each corresponding stored image code among the multiple stored image codes: adding the corresponding bit of the image code associated with the Hamming processing command to the bit of each stored image code to produce the corresponding first bit result; performing a modulo-2 operation on each corresponding bit result to produce the corresponding second bit result; and summing the corresponding second bit results to produce the corresponding Hamming distance among the multiple Hamming distances.

[0019] Alternatively, multiple stored image codes are stored in the NAND memory of the memory device.

[0020] Alternatively or additionally, it may further include generating Hamming processing commands at the host computing device based on a user application performing Hamming distance calculations.

[0021] In another aspect of this disclosure, an example memory is disclosed. A memory may include: a processor; a memory array configured to store a plurality of stored image codes representing corresponding images among a plurality of images; and a non-transitory computer-readable medium storing executable instructions that, when executed by the processor, cause the memory to perform operations. The operations may include: obtaining an image code associated with a Hamming processing command at an I / O control unit of the memory; providing at least one memory access request, including a plurality of memory addresses, to the memory array in response to the Hamming processing command; and comparing each of the plurality of stored image codes with the image code associated with the Hamming processing command at a data register of the memory array to generate a corresponding Hamming distance among a plurality of Hamming distances.

[0022] Additionally or alternatively, the operation further includes: generating an image processing result based on a plurality of Hamming distances, comprising a result set indicating certain images among a plurality of images; and providing the image processing result to a host computing device for use by a neural network hosted on the host computing device.

[0023] Alternatively or additionally, the operation further includes: obtaining an image different from a plurality of images from an Internet of Things (IoT) computing device; and generating an image code associated with a Hamming processing command based on the image using a hash algorithm.

[0024] Alternatively or concurrently, an IoT computing device corresponds to at least one of a camera, a smartphone device, or an image capture device.

[0025] Examples of devices are described herein. Example devices include: a host computing device configured to acquire an image from an IoT computing device and provide a Hamming processing request based on the image; and a memory. The memory may include: a memory array configured to store a plurality of stored image codes representing corresponding images among a plurality of images; a processor; and at least one non-transitory computer-readable medium encoded with executable instructions that, when executed by the processor, cause the device to perform operations. The operations may include: obtaining an image code in a Hamming processing command based on a Hamming processing request at an I / O control unit of the memory; providing at least one memory access request including a plurality of memory addresses to the memory array in response to the Hamming processing command; and comparing each of the plurality of stored image codes with an image code associated with the Hamming processing command at a data register of the memory array to generate a corresponding Hamming distance among a plurality of Hamming distances.

[0026] Alternatively, the host computing device includes a memory controller configured to provide Hamming processing requests to the processor.

[0027] Additionally or alternatively, the operation further includes: obtaining a Hamming processing request from the memory controller via the NVMe memory bus; and generating a Hamming processing command based on the Hamming processing request, including image code to be provided to the I / O control unit of the memory. Attached Figure Description

[0028] Figure 1 A schematic diagram of a system arranged according to the examples described in this article.

[0029] Figure 2 This is a schematic illustration of a method based on an example described in this article.

[0030] Figure 3 This is a schematic illustration of a method based on an example described in this article.

[0031] Figure 4 A schematic diagram of a system arranged according to the examples described in this article.

[0032] Figure 5 This is a schematic illustration of a method based on an example described in this article.

[0033] Figure 6 A schematic illustration of a memory system arranged according to the examples described herein.

[0034] Figure 7This is a schematic illustration of a method based on an example described in this article.

[0035] Figure 8 Examples of systems according to aspects of this disclosure are shown. Detailed Implementation

[0036] Examples of the systems and methods described herein provide processing of image codes at various memory devices. Hamming image codes can be generated by various endpoint computing devices, such as Internet of Things (IoT) computing devices, which receive or acquire images of their surrounding environment. As an example, a smartphone or camera device can acquire images of vehicles such as cars or trucks. Such devices can generate Hamming processing requests with image codes for comparing the representation of the image with other images. In an example, an image of a vehicle can be compared with other vehicle images, for example, to identify the vehicle. Hamming processing requests can be obtained at the computing device itself (e.g., in this example, a smartphone) or at a computing device network (e.g., a data center), which can route the Hamming processing requests to one or more memory devices for processing the image codes. Thus, as part of processing the image codes, the memory devices can compare the image codes with other image codes of corresponding images stored on the memory devices. Advantageously, when processing Hamming processing requests at the memory device itself, the systems and methods described herein provide efficient processing of Hamming image codes, in contrast to, for example, a processor requesting and obtaining images at associated memory (e.g., cache memory) for comparison. This utilizes fewer memory requests in such systems because the comparison of image codes occurs at the memory device itself. This can be termed “edge” processing of image codes, since the processing involving the comparison of image codes occurs at a processing unit with a memory device, at a memory controller coupled to the memory device, or at the memory device itself.

[0037] Continuing with the example of the acquired vehicle image, an image code is generated based on various features of the acquired vehicle image. In this example, features may include the vehicle's color, words associated with the vehicle's model / manufacture (e.g., brand name), aspects of the image's background, the vehicle's size relative to the background image, or other aspects of the vehicle's size relative to other aspects of the vehicle or background. Therefore, the acquired vehicle image can be represented by various features. The features are used to compute the image code of the vehicle image. For example, a hash such as a Supervised Semantic Preserving Deep Hash (SSDH) algorithm can be used to represent the features as a binary embedding. This binary embedding can be referred to as the image code of the acquired image. While the example of the acquired image refers to the acquired vehicle image, it is understood that various types of images can be acquired. As another example, an image of an individual or an individual's face makes it possible to compute an image code for any type of image containing an image of an individual or an individual's face (e.g., for facial recognition purposes). Computing an image code based on an image of an individual or an individual's face can be utilized in security applications, such as identifying individuals in a dataset of convicted criminals.

[0038] Using image codes, a Hamming processing request can be generated to compare the image code of an acquired image with the corresponding image code of an image stored on a memory device. For example, hashing can also be applied to images stored on a memory device to obtain the image code of the corresponding image used for comparison with the image code of the acquired image. The comparison involves calculating the corresponding Hamming distance between the image code of the acquired image and each corresponding image code of the corresponding image stored on the memory device. Once calculated, a result set can be obtained (e.g., using the K-nearest neighbor algorithm) indicating the “shortest” Hamming distance, for example, the minimum value of the Hamming distance for binary embeddings. This result set indicates a set of stored images that share most of the features of the acquired image. Therefore, using the result set, various features of the acquired image can be identified as features corresponding to those obtained in the result set, or at least features obtained using one of the results in the result set. Alternatively or additionally, if the Hamming distance is calculated to be null or zero, an “exact” match can be determined because the acquired image has an image code that matches another image code of the stored image. Therefore, any information associated with the stored image (e.g., stored fields / parameters) can be inferred as features of the acquired image.

[0039] Advantageously, examples of the systems and methods described herein can be used in neural networks (e.g., deep learning neural networks) to facilitate the processing of datasets, which may involve the use of binary embeddings or image codes. The examples in this paper increase the processing speed and throughput of such datasets in neural networks because comparisons of image codes (e.g., binary embeddings) can be performed "closer" to the memory device, such as at a processing unit having a memory device, at a memory controller coupled to the memory device, or at the memory device itself. "Closer" to the memory device generally means that the systems described herein offer fewer interfaces and / or less latency compared to systems performing Hamming distance calculations in a host computer. For example, when processing is performed at the memory device itself, an IoT device can perform "edge" calculations by comparing a given image with images stored on the IoT device itself to determine if there exists a set of results matching the given image. Memory devices, which may include non-volatile memory, as described herein can be used to facilitate the processing of datasets in neural networks; thereby increasing the processing capacity of computations performed on the neural network. Therefore, for example, compared to a conventional neural network processing system, increasing the processing rate and volume of the dataset in the neural network, where the conventional neural network processing system can first retrieve images or content from a memory device to process images or content in the local cache or storage device of the associated host computing device, thereby introducing latency into the processing speed and processing rate.

[0040] Furthermore, and advantageously, examples of the systems and methods described herein can increase the precision or accuracy of neural networks because more efficient use of memory (e.g., via increased processing speed and throughput) also allows for the storage of larger datasets on memory devices, thereby increasing the size of the training dataset and / or data to be processed / stored on the memory devices. Therefore, data centers accommodating various multiple memory devices can use examples of the systems and methods described herein to increase the processing speed of datasets in neural networks. For example, a host computing device containing a user application configured to perform Hamming distance calculations can allocate processing of the calculations when a Hamming processing request is generated to be processed by a memory controller or processing unit having a memory device. When allocating Hamming processing requests, the host computing device is not limited to a bus connection to the memory device (e.g., a PCIe bus), which may have limited bandwidth or be limited to the use of local memory associated with the host computing device itself (e.g., local cache memory or local storage devices). As described herein, Hamming processing units (e.g., processors or memory controllers) can process various Hamming distance calculations via multiple buses (e.g., multiple PCIe buses) coupled to the memory devices. For example, in various embodiments, one or more memory controllers and multiple memory devices may communicate via a non-volatile memory high-speed (“NVMe”) protocol that facilitates the processing of memory access requests to the memory devices.

[0041] Figure 1 This is a schematic illustration of a system 100 arranged according to the examples described herein. System 100 includes a host computing device 102 coupled to IoT devices 114 and 118. System 100 also includes a Hamming processing unit 108 coupled to the host computing device 102 via a host bus 104. The Hamming processing unit 108 includes a host interface 110 that couples the host bus 104 between the host computing device 102 and the Hamming processing unit 108. The host interface 110 is coupled to a processor 116 that includes Hamming control logic 112. The processor 116 is coupled to a memory controller 106 via a corresponding controller bus 120. The memory controller 106 provides memory access requests (e.g., generated by the Hamming control logic 112) from the processor 116 to a memory device 124. The memory device 124 is coupled to the corresponding memory controller 106 via a corresponding memory bus 122.

[0042] IoT device 114 may be a camera or include an image capture device, and IoT device 118 may be implemented using a smartphone or include an image capture device. IoT devices 114 and 118 can capture images and provide them to host computing device 102. Host computing device 102 can obtain images from IoT devices 114 and 118 and further generate image codes based on the obtained images. For example, host computing device 102 may host a neural network that utilizes image codes such as binary embeddings to process neural network requests (e.g., requests in a deep learning environment). Host computing device 102 may utilize hashes to generate corresponding image codes for the obtained images. Hashes can be used to generate binary embeddings to represent images obtained from IoT device 114 or IoT device 118. Each bit of the binary embedding can represent an aspect, feature, or size of the image, such that multiple sizes can be classified as binary values ​​in the binary embedding. Therefore, an image can be represented as a binary embedding, and an image that shares one or more bits with the same corresponding value can be said to "match" the image. For example, a 128-byte binary embedding contains 1,024 bits (i.e., 8 bits in one byte multiplied by 128). Each of the 1,024 bits can represent an aspect of the image. Another image that shares most or most of the same bits with the 1,024 bits can be matched with the binary embedding, thereby referring to the at least part of the compared images having the same image code as a match. Therefore, there is a need for systems and methods to facilitate this matching in an efficient and fast manner. Advantageously, the systems and methods described herein (e.g., Hamming processing unit 108) efficiently allocate computation and comparison, enabling the identification of matching images in a dataset much faster than conventional systems that may rely on a local cache at host computing device 102 to store and compare such images.

[0043] One or more neural networks hosted on host computing device 102 (e.g., as part of a deep learning neural network environment) can process large datasets to obtain neural network results about the datasets. For example, the neural network on host computing device 102 can acquire images from IoT device 114, such as to identify vehicles in the images compared to other images in a large dataset of vehicle images. To perform this neural network recognition, the neural network on host computing device 102 generates a Hamming processing request to Hamming processing unit 108 to obtain image processing results about the acquired images using Hamming distance calculation. For example, host computing device 102 may include a host processor configured to execute a user application that uses Hamming distance calculation to obtain image processing results about the acquired images. During execution, the user application generates a Hamming processing request to perform Hamming distance calculation on the acquired images using a dataset (e.g., a vehicle image dataset) to obtain image processing results about the acquired images. The image processing results may be a set of results obtained from Hamming processing unit 108. Continuing with the example, the result set may contain or indicate certain vehicle images in a dataset of vehicle images, wherein the vehicle images share at least one feature of the acquired images or a match of vehicles in the acquired images, such as a comparison based on image codes, as described herein.

[0044] Memory device 124 is configured to store data containing datasets, such as image or content datasets. For example, if Hamming processing unit 108 obtains a dataset from one or more associated computing devices via host bus 104, the dataset may be stored on memory device 124. In this example, the one or more associated computing devices may be computing devices in a data center or personal computing devices coupled to Hamming processing unit 108 via a network connection. Hamming processing unit 108 is configured to store datasets (e.g., images) in memory device 124. For example, Hamming processing unit 108 may store discrete units of the dataset (e.g., images or video frames) in memory device 124. In the examples described herein, memory device 124 may be a non-volatile memory device, such as a NAND memory device. Memory device 124 may also include one or more types of memory, including, but not limited to: DRAM, SRAM, three-level cell (TLC) NAND, single-level cell (SLC) NAND, SSD, or 3D XPoint memory devices. Data stored on or to be accessed on memory device 124 is transmitted from the corresponding memory controller 106 via memory bus 122. For example, memory bus 122 may be a PCIe bus operating according to the NVMe protocol.

[0045] To obtain a set of results (e.g., a set of images of a dataset) from the memory device 124 in the Hamming processing unit 108, a host processor on the host computing device may also be configured to generate a Hamming processing request and provide the Hamming processing request to the Hamming processing unit 108 via a host bus 104, the Hamming processing request being received at a host interface 110 of the Hamming processing unit 108. For example, the host bus 104 may be a PCIe bus that can communicate processing requests between the host computing device 102 and the Hamming processing unit 108. The host interface 110 is configured to receive the Hamming processing request from the host computing device 102; for example, the host interface 110 may include a port configured to receive the Hamming processing request upon detection on the host bus 104. The host interface 110 is further configured to provide the Hamming processing request to a processor 116 including Hamming control logic 112, the Hamming control logic 112 being configured to receive the Hamming processing request and calculate multiple Hamming distances between the dataset and the obtained images.

[0046] The Hamming processing request may contain an image code associated with the acquired image, such as a hash provided by host computing device 102. To perform a comparison of the image code of the acquired image with images in a dataset (e.g., a vehicle image dataset), Hamming control logic 112 may contain control instructions that, when executed by processor 116, generate a memory access request and provide it to memory device 124. Thus, in response to the Hamming processing request, processor 116, containing Hamming control logic 112, generates one or more memory access requests (e.g., read commands) for memory device 124 to access information associated with images in the dataset. For example, the accessed information may be the image itself read from memory device 124, or the accessed information may be an image code representing an image in the dataset. In this example, when accessing the image itself, processor 116 may hash the image using the same hash applied by host computing device 102 to images acquired from IoT device 114 or IoT device 118. Although described herein as a read command, memory access requests can be of various types. For example, in an embodiment having multiple memory planes on a memory die, a memory access request may include a multi-plane read command to read multiple planes on the memory die. As another example, in an embodiment having multiple logic units (LUNs) of memory device 124, a memory access request may include a parallel LUN read command.

[0047] In an example implementation, processor 116 may include any type of microprocessor, central processing unit (CPU), application-specific integrated circuit (ASIC), digital signal processor (DSP) implemented as part of a field-programmable gate array (FPGA), system-on-chip (SoC), or other hardware to provide Hamming distance calculation as part of Hamming processing unit 108.

[0048] Once information is accessed from memory device 124, Hamming control logic 112 can be configured to control processor 116 to compare the image codes of the dataset with the image codes provided as part of a Hamming processing request. To compare the image codes of the dataset with the image codes provided as part of a Hamming processing request, Hamming control logic 112 is further configured to provide control instructions to processor 116 to calculate the corresponding Hamming distance for each image code of the dataset and the image code provided as part of the Hamming processing request. In embodiments where the image codes are binary embedded or represented, the Hamming distance can be calculated for each image code of the dataset and the image code of the obtained image, as provided in equation (1):

[0049]

[0050] Where p is the number of bits in the binary image code provided in the Hamming processing request, q is the number of bits in the corresponding image code of the image in the dataset, and n is the number of bits in the corresponding image code. As indicated, the "modulo-2" operation, i.e., modulo-2 calculation, is performed after the summation of the bits. The results from each modulo-2 operation are summed to provide the Hamming distance between the specific image code representing the image in the dataset and the image code provided as part of the Hamming processing request (e.g., the image code of an image obtained from IoT device 114 or IoT device 118).

[0051] When calculating multiple Hamming distances, the image code provided as part of the Hamming processing request is compared with each image code of the corresponding image representing the image dataset. Each Hamming distance calculation represents a comparison between an image obtained from IoT device 114 or IoT device 118 and an image in the dataset. To determine a match among the obtained images, processor 116 can obtain a result set by executing a result algorithm to include the match in the result set, as controlled by Hamming control logic 112. For example, Hamming control logic 112 can be configured to apply a result algorithm, as executed by processor 116, as a threshold comparison, such that any Hamming distance calculation that passes the threshold is included in the result set. As another example, the result algorithm can be executed by processor 116 as a null test, such that only null results of Hamming distance calculations are included in the result set, which can be monolithic. As yet another example, the result algorithm can be executed by processor 116 as a k-nearest neighbor algorithm (“k-nn” algorithm), such that several neighbors in the bit-dimensional space are selected as the “closest” image code to the obtained image based on the calculated Hamming distance calculations of the corresponding image codes representing the images in the dataset. In the example described, the image code of the obtained image is defined as "k" in the k-nn algorithm; and it represents that each image code of the corresponding image in the image dataset is classified according to a different label (e.g., the label could be an image code with a Hamming distance of two bits to the image code of the obtained image). Continuing with the example, the result set includes the most frequent labels for the image codes of the obtained images.

[0052] As an additional or alternative implementation of such a result algorithm, the result algorithm may be executed by processor 116 as a null test, such that only null results from Hamming distance calculations are included in the result set, which may be uniform. The result algorithm may additionally or alternatively utilize a set of matching criteria, such as specifying a matching tolerance, a match with the minimum error (e.g., compared to other matches), or any error criterion. For example, a thresholding result algorithm may use an upper limit of error to identify matches, such that a match is defined as one with an error below the upper limit of error. Therefore, any number or result algorithm may be applied to Hamming distance calculations to identify a result set indicating certain images in the dataset that match the obtained images.

[0053] While examples of identifying matching image codes for acquired images based on Hamming distance calculated from hashed binary embeddings have been described, it is understood that alternative hashing techniques can be used to compute image codes, such as perceptual hashing or feature hashing. This hashing can be compared using k-nn as a result algorithm or an alternative algorithm. As an example of an alternative hashing technique, the acquired images can be hashed using a Bloom filter, and image codes for the dataset can be computed. In this example, instead of applying modulo-2 computation as described herein with respect to Hamming computation, the image codes can be summed together, for example, via an AND operation, to obtain sets of hashed image codes from different distributions. Using the summed set of hashed image codes, control logic (such as Hamming control logic 112 described herein) can identify which image codes share the same set (e.g., the result set), thereby indicating a match of image codes representing the acquired images.

[0054] Once identified as a result set, Hamming control logic 112 is configured to control processor 116 to provide the result set to host computing device 102 via host bus 104. For example, the result set can be provided to host computing device 102 as a corresponding image of the dataset contained in the result set, which may be referred to as an image processing result, according to control instructions from Hamming control logic 112. Therefore, host computing device 102 can utilize the provided result set in a neural network hosted by host computing device 102.

[0055] To perform Hamming distance calculations, system 100 utilizes a Hamming processing unit 108 (including its Hamming control logic 112 and memory controller 106) to distribute the processing of memory access requests to memory devices 124. The Hamming control logic 112 is configured to generate one or more memory access requests, which are provided to memory devices 124 via memory controller 106 and corresponding memory buses 122. The Hamming control logic 112 provides memory access requests to memory controller 106 via corresponding controller buses 120. In response to receiving a memory access request from the Hamming control logic 112, memory controller 106 provides the corresponding memory access request to multiple memory devices 124. Memory controller 106 may use information in the memory access request to identify the memory device 124 that will provide the memory access request. For example, memory controller 106 may use the memory address provided in the memory access request (e.g., as header information) and a mapping table of memory addresses to identify the associated memory device 124 for the memory access request. As another example, memory controller 106 may use the memory device identification (whether logical or physical) provided in the memory access request to identify the associated memory device 124 for the memory access request. When identifying the memory device 124 to provide a memory access request based on the information in the memory access request, the memory controller issues one or more memory access requests (e.g., read commands) to the corresponding memory device 124 to access information associated with the image in the dataset (e.g., image code or the image itself).

[0056] When utilizing the corresponding controller bus 120 and memory bus 122, the Hamming processing unit 108 can access information associated with images in a dataset, whether image codes or the images themselves, more quickly using multiple buses compared to a conventional system that obtains information about the dataset via a single host bus 104 for processing at the host computing device 102. Therefore, advantageously, the system 100 can increase the processing speed and throughput of such datasets in a neural network hosted on the host computing device 102, because the comparison of image codes is performed "closer" to the memory device 124, for example, at the Hamming processing unit 108 with the memory device 124 via the controller bus 120 and the corresponding memory bus 122.

[0057] In various embodiments, memory controller 106 may be an NVMe memory controller 106 coupled to processor 116 via a corresponding PCIe bus operating according to the NVMe protocol. When operating according to the NVMe protocol, controller bus 120 may be an NVMe bus. In such embodiments, memory device 124 may be a NAND memory device coupled to NVMe memory controller 106 via a corresponding PCIe bus operating according to the NVMe protocol. Therefore, memory bus 120 may be referred to as an NVMe memory bus. Thus, compared to a conventional memory system that can access memory via a single host bus to host computing device 102, system 100 advantageously increases processing speed and throughput by having several NVMe memory buses 120 connected to the respective memory devices 124. Therefore, in embodiments where processor 116 is an FPGA, system 100 may be referred to as “accelerated” Hamming distance calculation, which has increased availability of data transfer via memory bus 122.

[0058] Figure 2 This is a schematic illustration of method 200 according to an example described herein. Example method 200 can be executed using, for example, a Hamming processing unit 108 that executes executable instructions (e.g., executed by Hamming control logic 112 at processor 116) to interact with memory device 124 via controller bus 120 and a corresponding memory bus 122. For example, the operations described in blocks 202-206 can be stored as computer-executable instructions in a computer-readable medium accessible by processor 116. In embodiments, the computer-readable medium accessible by processor 116 can be one of the memory devices 124. For example, executable instructions can be stored on one of the memory devices 124 and retrieved by memory controller 106 for execution by Hamming processing unit 108 to perform executable instructions for performing method 200. Alternatively or additionally, executable instructions can be stored in memory coupled to host computing device 102 and retrieved by processor 116 to perform executable instructions for performing method 300.

[0059] Example method 200 may begin at block 202, which initiates the execution of the method and includes the following operation: obtaining a Hamming processing request from a host computing device via a host bus at a processor configured with Hamming control logic to calculate Hamming distances between multiple images and images associated with the same image code. In an example embodiment of Hamming processing unit 108, the Hamming processing request is obtained from host computing device 102 via host bus 104, for example. For example, host bus 104 may be a PCIe bus coupling Hamming processing unit 108 to host computing device 102, such that host computing device 102 may provide information from a user application executing on the host processor to Hamming processing unit 108, the user application utilizing Hamming distance calculation and generating a Hamming processing request. For example, the Hamming processing request generated by the user application may specify that Hamming distance calculations regarding image codes of images and a dataset containing multiple images stored on memory device 124 will be performed; and thus, by means of the Hamming processing request, Hamming control logic 112 is requested to perform such calculations at Hamming processing unit 108. In the example described, the Hamming processing request may contain an image code associated with the image. The image may be an image obtained from an image capture device (e.g., IoT device 114 or IoT device 118). Therefore, in various embodiments, at block 202, Hamming control logic 112 receives the Hamming processing request.

[0060] Block 202 may be followed by block 204, such that the method further includes the operation of providing at least one memory access request to multiple memory devices to access information associated with multiple images in response to a Hamming processing request. In an example embodiment of Hamming processing unit 108, Hamming control logic 112 executes executable instructions to provide a memory access request to memory device 124. For example, in response to a Hamming processing request, Hamming control logic 112 generates one or more memory access requests (e.g., read commands) for memory device 124 to access information associated with images in a dataset having multiple images. For example, the information to be accessed may be the image itself stored on memory device 124, or the information accessed may be image codes representing images in the dataset. When providing a memory access request, Hamming control logic 112 may include information for a memory controller (e.g., memory controller 106) to identify the memory device 124 providing at least one memory access request in the at least one memory access request. For example, Hamming control logic 112 may include the memory address of an image or an image code associated with an image in a plurality of images and / or the memory device identification of an image or an image code associated with an image in each of at least one memory access request (e.g., as header information).

[0061] Box 204 may be followed by box 206, such that the method further includes the operation of comparing at least one of a plurality of images with an image code to calculate a corresponding Hamming distance among a plurality of Hamming distances. In an example embodiment of Hamming processing unit 108, Hamming control logic 112 executes executable instructions to compare at least one of a plurality of images with an image code to calculate a corresponding Hamming distance. For example, Hamming control logic 112 may execute executable instructions at processor 116 to calculate the corresponding Hamming distance between each image code of the dataset and the image code of the obtained image according to equation (1). Thus, each Hamming distance calculation represents comparing an image code with one of a plurality of images (e.g., the corresponding image of the dataset). Method 200 ends after executing box 206.

[0062] The boxes included in the described example method 200 are for illustrative purposes. In some embodiments, these boxes may be executed in a different order. In some other embodiments, various boxes may be eliminated. In still other embodiments, various boxes may be divided into additional boxes, supplemented by other boxes, or combined together into fewer boxes. Other variations of these specific boxes are contemplated, including changes in the order of the boxes, changes in the splitting or combination of box content into other boxes, etc.

[0063] Figure 3 This is a schematic illustration of method 300 according to an example described herein. Example method 300 may be executed, for example, by host computing device 102, and / or Hamming processing unit 108 may execute executable instructions to interact with memory device 124 via controller bus 120 and a corresponding memory bus 122. For example, some instructions may be executed by a host processor at host computing device 102 (e.g., blocks 302-306), while other instructions may be executed by Hamming control logic 112 at processor 116 (e.g., blocks 308-314). The operations described in blocks 302-314 may be stored as computer-executable instructions on one or more computer-readable media accessible by the host processor and / or processor 116. For example, executable instructions may be stored on one of the memory devices 124 and retrieved by memory controller 106 for execution by host computing device 102 and / or Hamming processing unit 108 for performing executable instructions for method 300. Alternatively, executable instructions may be stored in a memory coupled to the host computing device 102 and retrieved by the processor 116 for execution of executable instructions for performing method 300.

[0064] Example method 300 may begin at block 302, which initiates the execution of the method and includes the operation of acquiring an image from an IoT computing device. In an example embodiment, host computing device 102 acquires an image from IoT device 114 or IoT device 118. The IoT device may include an image capturing device, such as a camera, and may capture images to provide to host computing device 102. In an example embodiment, host computing device 102 may be configured to execute a user application on a host processor of host computing device 102, the user application being configured to acquire the image from IoT device 114 or IoT device 118. Thus, host computing device 102 may acquire the image as part of a user application that performs a request or utilizes the image.

[0065] Box 302 may be followed by box 304, such that the method further includes the operation of generating an image code based on a hash of the image. In an example implementation, host computing device 102 uses a hash to generate an image code for the acquired image. In this example, the hash generates a binary embedding, which may be referred to as an image code, to represent an image acquired from IoT device 114 or IoT device 118. As, but only as one example, the SSDH algorithm can be used to compute the image code for an image. Host computing device 102 may be configured to execute a user application on a host processor of host computing device 102, the user application being configured to hash the image and generate a corresponding image code.

[0066] Box 304 may be followed by box 306, such that the method further includes the operation of generating a Hamming processing request based on a neural network request to obtain an image processing result using multiple Hamming distances. In an example implementation, a neural network (or one or more neural networks) on host computing device 102 requests the image processing result with respect to an acquired image and a dataset (e.g., an image dataset). The neural network request may be generated as part of a deep learning neural network environment to match the acquired image with an image in the dataset or at least one image in the dataset that shares one or more features / labels / aspects with the acquired image. The neural network request may include a request for the image processing result with respect to the image and the dataset, thereby generating a Hamming processing request to calculate a Hamming distance relative to the location of the image in the stored dataset (e.g., memory device 124). The Hamming processing request includes an image code such that a Hamming distance is calculated on the dataset relative to the image code. The Hamming processing request may be provided by host computing device 102 to a memory system, memory controller, or memory device configured to execute the Hamming processing request, for example, to interact with memory device 124 storing the dataset. In an example implementation, a user application on host computing device 102 is configured to generate a Hamming processing request and provide it to Hamming processing unit 108 for calculating multiple Hamming distances relative to image codes.

[0067] Box 308 may follow box 306, such that the method further includes the operation of obtaining a Hamming processing request at the processor via the host bus to calculate Hamming distances. In an example embodiment, Hamming control logic 112 at processor 116 obtains the Hamming processing request via host bus 104 to calculate one or more Hamming distances relative to image codes provided as part of the Hamming processing request. In an example embodiment, box 308 may be performed similarly to box 202 of method 200. Thus, in various embodiments, at box 308, Hamming control logic 112 obtains the Hamming processing request.

[0068] Block 310 may follow block 308, such that the method further includes the operation of: providing at least one memory access request to multiple memory devices to access information associated with multiple images in response to a Hamming processing request. In an example embodiment, Hamming control logic 112 at processor 116 executes instructions to provide a memory access request to memory device 124. In an example embodiment, block 310 may be executed similarly to block 204 of method 200. Thus, as described above, in response to a Hamming processing request, Hamming control logic 112 generates one or more memory access requests (e.g., read commands) for memory device 124 to access information associated with images in a dataset having multiple images.

[0069] Block 310 may be followed by block 312, such that the method further includes the operation of obtaining multiple images from multiple memory devices. In an example embodiment, Hamming control logic 112 at processor 116 obtains multiple images from a local cache or local memory associated with processor 116 for processing of the multiple images, such as calculating Hamming distance, thereby comparing image codes with each of the multiple images. For example, the local memory or local cache associated with processor 116 may be the static memory of Hamming processing unit 108, such as SRAM or DRAM. In an example embodiment of obtaining multiple images, memory device 124 may only store images of the dataset, rather than corresponding image codes. Therefore, multiple images are obtained from a local cache or memory directed by Hamming control logic 112 to hash the multiple images using the same hash applied to the obtained images of IoT device 114 or IoT device 118 to generate image codes associated with said images. Therefore, Hamming control logic 112 obtains multiple images from memory device 124, and can also hash the multiple images to generate a corresponding image code for each of the multiple images.

[0070] Box 310 may be followed by box 314, such that the method further includes the operation of comparing at least one of the plurality of images with image codes to compute a corresponding Hamming distance among a plurality of Hamming distances. In an example implementation, box 314 may be performed similarly to box 206 of method 200. For example, Hamming control logic 112 may execute instructions at processor 116 to compute the corresponding Hamming distance between each image code of the dataset and the image code of the acquired image according to equation (1). Method 300 may end after box 314 is completed.

[0071] The boxes included in the described example method 300 are for illustrative purposes. In some embodiments, these boxes may be executed in a different order. In some other embodiments, various boxes may be eliminated. In still other embodiments, various boxes may be divided into additional boxes, supplemented by other boxes, or combined together into fewer boxes. Other variations of these specific boxes are contemplated, including changes in the order of the boxes, changes in the splitting or combination of box content into other boxes, etc.

[0072] Figure 4 This is a schematic illustration of a system 400 arranged according to the examples described herein. System 400 includes a host computing device 102 coupled to IoT devices 114 and 118. System 400 also includes a memory controller 402 coupled to the host computing device 102 via a host bus 104. The memory controller 402 includes a host interface 408 that couples the host bus 104 between the host computing device 102 and the memory controller 402. The host interface 408 is coupled to a processor 412 that includes Hamming control logic 410. For example, the host interface 408 and the processor 412 may also be coupled to a cache 404 via an internal memory controller bus. The processor 412 is coupled to a memory device 124 via a memory interface 406 and a corresponding memory bus 416. The memory interface 406 is also coupled to the cache 404, for example, via an internal memory controller bus. The cache 404 is coupled to an error correction unit 414, which is configured to perform error correction on data transmitted from the cache 404. Alternatively, it can be understood that the same numbered elements in system 100 and system 400, such as host computing device 102, host bus 104, IoT device 114, IoT device 118, and memory device 124, can be configured in a similar manner. For example, as previously described, host bus 104 may be a component that can be connected to host computing device 102 and... Figure 4 The memory controller 402 instead of Figure 1The PCIe bus communicates Hamming processing requests between the Hamming processing units 108. As another example, the memory device 124 is configured to store data containing a dataset, such as a dataset obtained by the memory controller 402 from one or more associated computing devices via the host bus 104. Furthermore, as used... Figure 4 As indicated by the same numbered element, memory device 124 may include one or more types of memory, including but not limited to: DRAM, SRAM, TLC NAND, SLC NAND, SSD, or 3D XPoint memory devices.

[0073] Similar to, for example, about Figure 1 As described, host computing device 102 can host neural networks that utilize image codes, such as binary embedded image codes, to process neural network requests (e.g., requests in a deep learning environment). Furthermore, host computing device 102 can utilize hashing to generate corresponding image codes for acquired images. Advantageously, when using memory device 124 and the corresponding memory bus 416, memory controller 402 efficiently allocates the computation and comparison of the corresponding image codes, enabling faster identification of matching images in the dataset (e.g., matching of image codes of acquired images) compared to conventional systems that rely on local caches at host computing device 102 for storing and comparing such images.

[0074] Continuing with the example of an image acquired from IoT device 114, a neural network on host computing device 102 generates a Hamming processing request to memory controller 402 to obtain an image processing result for the acquired image using Hamming distance calculation. Host computing device 102 may include a host processor configured to execute a user application that uses Hamming distance calculation to obtain the image processing result for the acquired image. During execution, the user application generates the Hamming processing request to perform Hamming distance calculation on the acquired image using a dataset (e.g., a vehicle image dataset) to obtain the image processing result for the acquired image. Host computing device 102 provides the Hamming processing request to host interface 408 via host bus 104. Upon receipt by host interface 408, host interface 408 is configured to provide the image code associated with the Hamming processing request to cache 404 for storage of the image code. As described herein, memory controller 402 utilizes cache 404 with the stored acquired image code to calculate and compare the Hamming distance.

[0075] To recap, the Hamming processing request contains an image code associated with the acquired image, such as one hashed by host computing device 102. To perform a comparison of the image code of the acquired image with images in a dataset (e.g., a vehicle image dataset), the image stored on memory device 124 of the dataset will also be hashed for comparison with the image code stored at cache 404. In some instances, the image code of the dataset itself may be stored on memory device 124, in which case the computation and comparison can be performed by Hamming control logic 410 without hashing. In some embodiments, cache 404 may be directly coupled to a storage device, such as an SRAM or DRAM storage device, which is part of host computing device 102, and the image code to be directly stored is obtained from the storage device. For example, a Hamming processing request provided to host interface 408 may include a memory access command provided to the cache to access a storage device on host computing device 102 to obtain the image code associated with the Hamming processing request. In various implementations, cache 404 may be a dynamic memory device, such as DRAM, and configured to interact with processor 412. For example, cache 404 may be a data cache that includes (e.g., as a multi-level cache) or corresponds to L1, L2, L3, L4, or any other cache level as will be understood by those skilled in the art.

[0076] To obtain the image code for calculating and comparing the image code stored at cache 404, Hamming control logic 410 may include control instructions that, when executed by processor 412, generate memory access requests and provide them to memory device 124 via memory interface 406 and memory bus 416. For example, Hamming control logic 410 may be configured to control the issuance of one or more memory access requests (e.g., read commands) to the corresponding memory device 124 to access information (e.g., image code or the image itself) associated with an image in the dataset of memory controller 402. In this example, memory controller 402 identifies memory device 124 to provide information in the Hamming processing request, and processor 412 uses the identification information to generate one or more memory access requests. Therefore, in response to the Hamming processing request and as executed by processor 412, Hamming control logic 410 generates one or more memory access requests (e.g., read commands) for memory device 124 coupled to memory controller 402 to access information associated with an image in the dataset. For example, the accessed information may be the image itself read from memory device 124, or the accessed information may be image codes representing images in a dataset. Memory interface 406, which interacts with processor 412 via an internal memory controller bus, is configured to provide one or more memory access requests to memory device 124 via a corresponding memory bus 416. Therefore, memory bus 416, which couples memory device 124 to memory controller 402, provides the stored information (e.g., image codes and / or datasets of images) for calculating and comparing Hamming distances with the image codes of the acquired images.

[0077] In an example implementation, processor 412 may include any type of microprocessor, central processing unit (CPU), application-specific integrated circuit (ASIC), digital signal processor (DSP) implemented as part of a field-programmable gate array (FPGA), system-on-chip (SoC), or other hardware to provide Hamming distance calculation as part of memory controller 402.

[0078] In response to one or more memory access requests, memory device 124 provides access to the requested information, such as information associated with images in the dataset, such as image codes or the images themselves. When access is available at memory device 124, memory interface 406 is further configured to provide the requested information (whether image codes or images) to cache 404 for storage via memory bus 416 and an internal memory controller bus between memory interface 406 and cache 404. Thus, cache 404 obtains the requested information (whether image codes or images) via memory device 124 and its corresponding memory bus 416.

[0079] In an example implementation, when accessing the image itself, the processor 412 may hash the image using the same hash applied by the host computing device 102 to images obtained from IoT device 114 or IoT device 118. Continuing with the example, the memory interface 406 may provide the image itself to the processor 412 for hashing; and once hashed, the processor 412 may provide the image code of the retrieved image back to the cache 404 via the internal memory controller bus. In this example, the internal memory controller bus between the processor 412 and the cache 404 may be an AXI bus.

[0080] In some implementations, cache 404 may be configured to provide information (e.g., images or image codes) obtained from memory device 124 to error correction unit 414 for error correction of the obtained information, and subsequently receive error-corrected information from error correction unit 414. Error correction unit 414 is configured to perform error correction on data or information obtained from memory device 124. For example, error correction unit 414 may be configured to perform error correction on data based on the desired bit error rate (BER) of the operation of memory device 124. For example, error correction unit 414 may include a low-density parity correction unit configured to perform error correction on data based on a low-density parity check code. When using error correction unit 414, memory controller 402 may correct for errors that may occur in data during memory retrieval or storage at memory device 124. Such implementations may be used depending on whether the desired BER is specified by host computing device 102 or by a user executing a user application at host computing device 102. In various embodiments, as described herein, the error correction unit 414 may not perform error correction on the data obtained from the memory device 124, for example, because the obtained data includes binary embeddings as image codes. Since the host computing device 102 obtains the image processing results as a set of neural network results indicating the closest match, some bit differences in the neural network result sets may not result in any difference in the results, for example, with respect to the same set of neural network results obtained using the error correction unit 414. For example, depending on the length of the image codes used in the Hamming distance comparison, the corresponding Hamming distance calculation in such systems may not vary significantly to obtain different sets of neural network results. Therefore, optional embodiments using the error correction unit 414 can be utilized when specifying the desired BER of the memory device 124.

[0081] Once the information from memory device 124 is stored in cache 404, Hamming control logic 410 can be configured to control processor 412 to compare the image codes of the dataset with the image codes provided as part of the Hamming processing request. To compare the image codes of the dataset with the image codes provided as part of the Hamming processing request, Hamming control logic 410 is further configured to provide control instructions to processor 412 to calculate the corresponding Hamming distance for each image code of the dataset and the corresponding image code provided as part of the Hamming processing request. For example, processor 412 can use the image codes stored at cache 404 and the corresponding image codes associated with the information retrieved from memory device 124, which are also stored at cache 404, to calculate the corresponding Hamming distance. In embodiments where the image codes are binary embedded or represented, the Hamming distance can be calculated for each image code of the dataset and the image code of the obtained image, as provided in equation (1) described above. Thus, when calculating multiple Hamming distances, the image codes stored in cache 404, provided as part of the Hamming processing request, are compared with each image code of the corresponding image representing the image dataset.

[0082] Similar to the operation of Hamming control logic 112, Hamming control logic 410 can also be configured to apply a result algorithm, such as the k-nearest neighbor algorithm executed by processor 412, to identify a result set of images in the dataset that match the acquired images, as described herein. As an additional or alternative implementation of such a result algorithm, the result algorithm can be executed by processor 116 as a null test, such that only null results from Hamming distance calculations are included in the result set, which may be uniform. The result algorithm may additionally or alternatively utilize a set of matching criteria, such as specifying a matching tolerance, a match with the minimum error (e.g., compared to other matches), or any error criterion. For example, a threshold result algorithm may use an upper limit of error to identify matches, such that a match is defined as one with an error below the upper limit of error. Also similar to the operation of Hamming control logic 112, Hamming control logic 410 is configured to control processor 412 to provide the result set to host computing device 102 via host bus 104. For example, the result set can be provided to the host computing device 102 according to the control instructions of Hamming control logic 112, as the corresponding image of the dataset contained in the result set, again referred to as the image processing result.

[0083] While examples of identifying matching image codes for acquired images based on Hamming distance calculated from hashed binary embeddings have been described, it is understood that alternative hashing techniques can be used to compute image codes, such as perceptual hashing or feature hashing. This hashing can be performed using k-nn as a result algorithm or alternative algorithm to compare the results of the computed image codes. As an example of an alternative hashing technique, a Bloom filter can be used to hash the acquired images and compute image codes for the dataset. In this example, instead of applying modulo-2 computation as described herein with respect to Hamming computation, the image codes can be summed together, for example, via an AND operation, to obtain a set of hashed image codes from different distributions. Using the summed set of hashed image codes, control logic (such as Hamming control logic 410 described herein) can identify which image codes share the same set (e.g., the result set), thereby indicating a match of image codes representing the acquired images.

[0084] When utilizing cache 404 and memory bus 416, memory controller 402 can access information associated with images in a dataset, whether image codes or the images themselves, much faster than conventional systems that obtain information about the dataset via a single host bus 104 for processing at host computing device 102. Therefore, system 100 advantageously increases the processing speed and throughput of such datasets in neural networks hosted on host computing device 102 because the comparison of image codes is performed "closer" to memory device 124, for example, at memory controller 402, which is coupled to memory device 124 via memory bus 416.

[0085] In various embodiments, memory controller 402 may be an NVMe memory controller coupled to host computing device 102 via host bus 104 (e.g., a PCIe bus operating according to the NVMe protocol). In such embodiments, memory device 124 may be a NAND memory device coupled to NVMe memory controller 402 via a corresponding PCIe bus operating according to the NVMe protocol. Therefore, memory bus 416 may be referred to as an NVMe memory bus. Thus, compared to conventional memory systems that can access memory via a single host bus to host computing device 102, system 400 advantageously increases processing speed and throughput by connecting several NVMe memory buses 416 to the respective memory devices 124. Therefore, system 400 may be referred to as “accelerated” Hamming distance computation, which has increased availability of data transfer via memory bus 416.

[0086] Figure 5This is a schematic illustration of method 500 according to an example described herein. Example method 500 can be executed using, for example, a host computing device 102, and / or a memory controller 402 can execute executable instructions to interact with memory device 124 via memory bus 416. For example, some instructions can be executed by a host processor at host computing device 102 (e.g., blocks 502-506), while other instructions can be executed by Hamming control logic 410 at processor 412 (e.g., blocks 508-516). The operations described in blocks 502-516 can be stored as computer-executable instructions on one or more computer-readable media accessible by the host processor and / or processor 412. For example, executable instructions can be stored on one of the memory devices 124 and retrieved by memory controller 402. In this example, host computing device 102 and / or itself, and memory controller 402 can execute executable instructions for performing method 300. Alternatively or concurrently, executable instructions may be stored on memory coupled to host computing device 102 and retrieved by processor 412 for execution of executable instructions for performing method 500. The blocks included in the described example method 500 are for illustrative purposes. In some embodiments, these blocks may be executed in a different order. In some other embodiments, various blocks may be eliminated. For example, in some embodiments, block 514 may be an optional block when performing method 500. In yet another embodiment, various blocks may be divided into additional blocks, supplemented by other blocks, or combined together into fewer blocks. Other variations of these specific blocks are contemplated, including changes in the order of blocks, changes in the splitting or combination of block contents into other blocks, etc.

[0087] Example method 500 may begin at block 502, which initiates the execution of the method and includes instructions to obtain an image from an IoT computing device. Block 502 may be executed in the same manner as described with respect to block 302 of method 300. As described, in an example implementation, host computing device 102 may be configured to execute a user application on a host processor of host computing device 102, the user application being configured to obtain an image from IoT device 114 or IoT device 118. Thus, host computing device 102 may obtain the image as part of a user application that executes a request or utilizes the image.

[0088] Box 502 may be followed by box 504, such that the method further includes the operation of generating an image code based on a hash of the image. Box 504 may be performed in the same manner as described with respect to box 304 of method 300. As described, in an example embodiment, host computing device 102 uses a hash to generate an image code for the acquired image.

[0089] Box 504 may be followed by box 506, such that the method further includes the operation of generating a Hamming processing request based on a neural network request to obtain an image processing result using multiple Hamming distances. Box 506 may be performed in the same manner as described with respect to box 306 of method 300. As described, in an example embodiment, a neural network (or one or more neural networks) on host computing device 102 requests an image processing result with respect to an obtained image and dataset (e.g., an image dataset). Therefore, the Hamming processing request may be provided by host computing device 102 to a memory controller 402 configured to execute the Hamming processing request, for example, to interact with a memory device 124 storing the dataset. In an example embodiment, a user application on host computing device 102 is configured to generate a Hamming processing request and provide it to memory controller 402 for calculating multiple Hamming distances relative to image codes.

[0090] Block 508 may follow block 506, such that the method further includes the operation of obtaining the image code of the Hamming processing request via the host bus at a cache of the memory controller to calculate the Hamming distance. In an example embodiment, Hamming control logic 410 at processor 412 obtains the image code of the Hamming processing request via host bus 104 of memory controller 402 to calculate one or more Hamming distances. The image code may be provided to cache 404 by processor 412 or host interface 408 via internal memory controller bus of memory controller 402, such that cache 404 obtains the image code for storage. Upon obtaining a Hamming processing request, cache 404 may store the image code associated with the Hamming processing request at cache 404. Thus, in various embodiments, at block 508, cache 404 obtains the Hamming processing request to store the image code associated with the Hamming processing request.

[0091] Box 510 may follow box 508, such that the method further includes the operation of providing at least one memory access request to multiple memory devices to access information associated with multiple images in response to a Hamming processing request. In an example implementation, box 510 may be performed similarly to box 204 of method 200. In this example, memory controller 402 identifies memory device 124 to provide information in the Hamming processing request, and processor 412 uses the identification information to generate one or more memory access requests. Thus, in response to the Hamming processing request and if executed by processor 412, Hamming control logic 410 generates one or more memory access requests (e.g., read commands) for memory device 124 coupled to memory controller 402 to access information associated with images in the dataset.

[0092] Box 510 may be followed by box 512, such that the method further includes the operation of obtaining multiple images from multiple memory devices at a cache. In an example embodiment, cache 404 obtains multiple images for processing of the multiple images, such as calculating Hamming distance, thereby comparing the image code stored at cache 404 with each of the multiple images. In an example embodiment of obtaining multiple images, memory device 124 may only store images of the dataset rather than corresponding image codes. Thus, multiple images are obtained at cache 404, directed by Hamming control logic 410, to hash the multiple images using the same hash applied to the obtained images of IoT device 114 or IoT device 118 to generate image codes associated with said images. Thus, Hamming control logic 410 obtains multiple images from memory device 124 and may also hash the multiple images to generate a corresponding image code for each of the multiple images.

[0093] Block 514 may follow block 512, such that the method further includes performing error correction on the acquired plurality of images. In the example described, cache 404 provides the acquired plurality of images (e.g., obtained from memory device 124) to error correction unit 414. Error correction unit 414 performs error correction on the acquired information and subsequently provides the plurality of images back to the same storage location in cache 404 as the one where the error correction was performed. As described above, in some embodiments of method 500, block 514 may be an optional step.

[0094] Box 516 may follow box 512, such that the method further includes the operation of comparing at least one of a plurality of images with image codes at a memory controller to compute a corresponding Hamming distance among a plurality of Hamming distances. In an example implementation, box 516 may be performed similarly to box 206 of method 200. For example, Hamming control logic 410 may execute executable instructions at processor 412 to compute, according to equation (1), the corresponding Hamming distance between each image code of the dataset obtained at cache 404 and the image code of the obtained image, which is also stored at cache 404. Method 500 may end after box 516 is completed.

[0095] Figure 6This is a schematic illustration of a memory system 600 arranged according to the examples described herein. System 600 includes a host computing device 602 coupled to IoT devices 604 and 608. System 600 also includes a Hamming memory device 610 coupled to the host computing device 602 via a host bus 606. Hamming memory device 610 includes a processor 614 containing Hamming logic 612 coupled to the host computing device 602 via the host bus 606. Processor 614 is also coupled to an input / output (I / O) control unit 618 via an I / O bus 616 and to control logic 622 via a control bus 620. Control logic 622 interacts with other elements of Hamming memory device 610, including a memory array 624 via a data register 626 and a cache register 628. I / O control unit 618 also interacts with memory array 624, for example, when certain control pins are enabled at control logic 622. For example, Figure 6 Certain control pins depicted (e.g., Eni, CE#, CLE, ALE, WE#, RE#, WP#) may receive control signals that enable operation of the I / O control unit 618 corresponding to the functionality of the control pins. In various embodiments, the Hamming memory device 610 may be a NAND memory device, and the memory array 624 may be a NAND flash memory array. In addition to the numbered elements of the Hamming memory device 610, as those skilled in the art will appreciate, other features of the memory device (e.g., a NAND memory device) may be included in the Hamming memory device 610, such as various voltages (e.g., Vpp, Vrefq, Vcc, Vss, Vccq, and / or Vssq) that may be used for the elements of the Hamming memory device 610. For example, as depicted, an address register may interact with the I / O control unit 618 to store and / or provide memory addresses for access at the memory array 624 using row decoders and column decoders coupled to the memory array 624. As also depicted, the status register can interact with the I / O control unit 618 and control logic 622 to store and / or provide the status of certain memory elements in the Hamming memory device 610. Furthermore, the command register can interact with the I / O control unit 618 and control logic 622 to store commands from the I / O control unit 618 and / or provide said commands to the control logic 622.

[0096] Alternatively, it can be understood that similar named elements in system 100 and system 600 can be configured in a similar manner, for example: host computing device 102 and host computing device 602, host bus 104 and host bus 606, IoT device 114 and IoT device 604, IoT device 118 and IoT device 608, and Hamming control logic 112 and Hamming logic 612. For example, as described similarly with respect to host bus 104, host bus 606 can be configured to connect host computing device 602 and... Figure 6 Hamming memory device 610 instead of Figure 1 The PCIe bus transmits Hamming processing commands between the Hamming processing units 108. Therefore, similar to... Figure 1 As described, the host computing device 602 can host a neural network that uses image codes, such as binary embedded image codes, to process neural network requests. Furthermore, the host computing device 602 can utilize hashing to generate corresponding image codes for the acquired images. Advantageously, when processing Hamming processing commands on the Hamming memory device 610, the processing rate and throughput of the dataset (e.g., image code dataset) in the neural network of system 600 are increased, for example, compared to a conventional neural network processing system that can first retrieve images or content from a memory device to process the images or content at a local cache or storage device of the host computing device 602, thereby introducing latency into the processing speed and throughput.

[0097] In an example where an image is acquired from IoT device 604, a neural network on host computing device 602 generates Hamming processing commands to Hamming memory device 610 to obtain an image processing result for the acquired image using Hamming distance calculation. The generated Hamming processing commands contain a hashed image code of the acquired image. (The last sentence appears to be incomplete and possibly refers to a different context.) Figure 1 As described in host computing device 102, host computing device 602 may also utilize hashing to generate image codes for the acquired image (e.g., binary embedding). Continuing with the example, host computing device 602 may include a host processor configured to execute a user application that uses Hamming distance calculation to obtain image processing results regarding the image codes. During execution, the user application generates Hamming processing commands to perform Hamming distance calculations on the hashed image codes to obtain image processing results regarding the acquired image. Host computing device 602 provides the Hamming processing commands to processor 614 via host bus 606. Alternatively or additionally, in various embodiments, host computing device 602 may include a memory controller 402 that provides Hamming processing requests as Hamming processing commands to Hamming memory device 610 via a memory bus (e.g., memory bus 416).

[0098] When a Hamming processing command is received at processor 614, Hamming logic 612 can recognize the Hamming processing command to provide the image code associated with the Hamming processing command to I / O control unit 618 and to provide one or more control signals indicating the Hamming processing command to control logic 622. The one or more control signals provided to Hamming logic 612 may include control signals provided to control logic pins (e.g., control logic pins: WE#, CLE, and / or ALE) of control logic 622 via control bus 620. Therefore, control logic 622 can obtain one or more control signals from Hamming logic 612 based on the Hamming processing command. In an example, obtaining a control signal at the Command Latch Enable (CLE) pin of control logic 622 can indicate to Hamming memory device 610 that a Hamming processing command has been received. Therefore, control logic 622 can activate the gate of a transistor that controls the R / B# pin output indicating the ready / busy state of memory array 624. For example, when a control signal is received at the CLE pin, control logic 622 can provide a gate signal to the transistor, setting the transistor's drain (e.g., if it's an n-type transistor) high, indicating a busy state at memory array 624. Therefore, in response to one or more control signals indicating Hamming processing commands, control logic 622 can provide internal control signals to control various memory access circuits to perform memory access operations (e.g., read, write, program). For example, the internal control signals may include one or more memory access requests to perform memory access operations. A memory access request may contain the memory address of the cell to be accessed during the memory access operation.

[0099] Various memory access circuitry is used during such memory access operations and may typically include circuitry such as row and column decoders, charge pump circuitry, signal line drivers, I / O control unit 618, data register 626, and cache register 628. Memory cells in the memory array 624 can be accessed using various signal lines (e.g., global word line (GWL), local word line (LWL), and bit line (BL)). Memory cells may be non-volatile memory cells, such as NAND or NOR flash cells, phase-change memory cells, or may typically be any type of memory cell. Memory cells may be single-level cells configured to store one data bit of data. Memory cells may also be multi-level cells configured to store more than one data bit of data.

[0100] Additionally, upon receiving a Hamming processing command at processor 614, and in some embodiments, simultaneously, Hamming logic 612 may also recognize the image code within the Hamming processing command to provide the image code to I / O control unit 618. For example, the image code may be provided to the DQ pin of I / O control unit 618 via I / O bus 616. In this example, I / O control unit 618 may obtain the image code from the DQ pin upon confirming one or more control signals at control logic 622. For example, a combination of control signals may be provided to control logic 622, causing I / O control unit 618 to obtain the image code to provide it to cache register 628. For example, control logic 622 may receive additional control signals instructing the image code to be written to a data register of a memory device, and thus I / O control unit 618 provides the image code to cache register 628. In the example described, after confirming one or more control signals at control logic 622 to obtain the image code from the DQ pin, additional control signals can be obtained at the write enable (WE#) pin of control logic 622. When the WE# pin is activated, a certain voltage (e.g., Vrefq) can be provided to cache register 628 to enable the writing of the image code to cache register 628 by I / O control unit 618.

[0101] Typically, commands, address information, and write data are provided to the memory array 624 as a set of sequential I / O operations transmitted via various buses coupled to the I / O control unit 618 and the memory array 624. A DQS signal (e.g., at the DQS pin) can be used to provide timing information for transferring data to or from the memory array 624. An address register coupled to the I / O control unit 618 can be provided with address information to be temporarily stored by the I / O control circuitry. The I / O control unit 618 can also be coupled to a status register storing status bits. For example, the I / O control circuitry, the I / O control unit 618, can provide status bits stored in the status register in response to a read status command provided to the memory array 624. The status bits can have corresponding values ​​to indicate the status conditions of the memory array 624 and various aspects of its operation.

[0102] As described, in response to receiving one or more control signals indicating Hamming processing commands, control logic 622 provides one or more memory access requests containing memory addresses of image codes (e.g., image code datasets) stored at memory array 624 to various memory access circuits to perform memory access operations. In the example described, Hamming logic 612 generates multiple memory addresses to be read based on the Hamming processing commands. The multiple memory addresses are provided to control logic 622 as data to be read, for example via a command register such as that parsed by I / O control unit 618 via I / O bus 616. Control logic 622 generates one or more memory access requests based on the multiple memory addresses obtained from the command register.

[0103] Hamming logic 612 may also provide one or more control signals to control logic 622 to calculate a corresponding Hamming distance between the image code of the Hamming processing command and each of the plurality of stored image codes at memory array 624. In calculating the plurality of Hamming distances, the image code provided as part of the Hamming processing command is compared with each image code representing, for example, a corresponding image or image code dataset. In embodiments where the image code is a binary embedding or representation, the Hamming distance may be calculated for each stored image code and the image code of the Hamming processing command, as provided in equation (1) above. For example, each memory access request including the memory address of the stored image code is provided to the row and column decoders of Hamming memory device 610 for reading memory array 624. In this example, each memory address is read into data register 626 of Hamming memory device 610 to obtain the plurality of stored image codes on data register 626.

[0104] To calculate each corresponding Hamming distance, control logic 622 controls data register 626 to calculate each corresponding Hamming distance at data register 626 using the image code associated with the Hamming processing command stored at cache register 628 and a plurality of stored image codes at data register 626. In the example described, at data register 626, as indicated by equation (1), a "modulo-2" operation (i.e., modulo-2 calculation) is performed after adding the image code and the bits of each corresponding stored image code. The results from each modulo-2 operation are summed to provide the Hamming distance representing the specific stored image code of the image dataset and the image code provided as part of the Hamming processing command. The calculated Hamming distance is stored in cache register 628. Hamming logic 612 provides additional control signals to control logic 622 such that the calculated Hamming distance is provided to processor 614 via I / O bus 616 for application of the result algorithm.

[0105] To determine a match in the acquired images, processor 614 can obtain a result set by executing a result algorithm to include the match in the result set, as controlled by Hamming logic 612. For example, Hamming logic 612 can be configured to apply a result algorithm, as executed by processor 614, as a threshold comparison, such that any Hamming distance calculation that passes the threshold is included in the result set. As another example, the result algorithm can be executed by processor 614 as a null test, such that only null results from Hamming distance calculations are included in the result set, which can be monolithic. As yet another example, the result algorithm can be executed by processor 614 as a k-nearest neighbor algorithm, such that several neighbors in the bit-dimensional space are selected as the image codes closest to the acquired images based on the calculated Hamming distance calculations of the corresponding image codes representing the images in the dataset. Therefore, any number or result algorithm can be applied to Hamming distance calculations to identify a result set indicating certain images in the dataset that match the acquired images.

[0106] While examples of identifying matching image codes for acquired images based on Hamming distance calculated from hashed binary embeddings have been described, it is understood that alternative hashing techniques, such as perceptual hashing or feature hashing, can be used to compute image codes. This hashing can be performed using k-nn as a result algorithm or alternative algorithm to compare the results of the computed image codes. As an example of an alternative hashing technique, a Bloom filter can be used to hash the acquired images and compute image codes for the dataset. In this example, instead of applying modulo-2 computation as described herein with respect to Hamming computation, the image codes can be summed together, for example, via an AND operation, to obtain sets of hashed image codes from different distributions. Using the summed set of hashed image codes, control logic (such as Hamming control logic 612 described herein) can identify which image codes share the same set (e.g., the result set), thereby indicating a match of image codes representing the acquired images.

[0107] Once identified as a result set, Hamming logic 612 is configured to control processor 614 to provide the result set to host computing device 602 via host bus 606. For example, the result set can be provided to host computing device 602 according to control instructions from Hamming logic 612, as a corresponding stored image code included in the result set, which may be referred to as the image processing result of Hamming memory device 610. Therefore, host computing device 602 can utilize the result set provided from Hamming memory device 610 in a neural network hosted by host computing device 602.

[0108] Figure 7This is a schematic illustration of method 700 according to an example described herein. Example method 700 can be performed using, for example, a Hamming memory device 610, which executes executable instructions (e.g., executed by Hamming logic 612 at processor 614) to interact with memory array 624. For example, the operations described in blocks 602-606 can be stored as computer-executable instructions in a computer-readable medium accessible by processor 614. The operations described in blocks 702-706 can be stored as computer-executable instructions in one or more computer-readable media accessible by processor 614. For example, executable instructions can be stored on a memory coupled to host 602 and retrieved by processor 614 to execute executable instructions for performing method 700.

[0109] Example method 700 may begin at block 702, which initiates the execution of the method and includes the following operation: obtaining image codes associated with Hamming processing commands at the I / O control unit of the memory device. In an example embodiment of the Hamming memory device 610, the Hamming processing commands may be provided by the host computing device 602 to the processor 614 via a host bus 606. For example, the host bus 606 may be a PCIe bus coupling the Hamming memory device 610 to the host computing device 602, such that the host computing device 602 may provide information from a user application executing on the host processor to the Hamming memory device 610, the user application utilizing Hamming distance calculations and generating Hamming processing commands. For example, the Hamming processing commands generated by the user application may specify that Hamming distance calculations will be performed on image codes of an image and a dataset of image codes containing multiple image codes stored in the memory array 624; and thus, by means of the Hamming processing commands, Hamming logic 612 is requested to perform such calculations at the Hamming memory device 610. In the example described, the Hamming processing command may include an image code associated with the image. The image may be an image acquired from an image capture device (e.g., IoT device 604 or IoT device 608). When the Hamming processing command is received at processor 614, Hamming logic 612 may recognize the Hamming processing command to provide the image code associated with the Hamming processing command to I / O control unit 618. Thus, in various embodiments, at block 702, I / O control unit 618 receives the image code associated with the Hamming processing command.

[0110] Block 702 may be followed by block 704, such that the method further includes the operation of: providing at least one memory access request for a plurality of memory addresses to obtain a plurality of stored image codes in response to a Hamming processing command. In an example embodiment of the Hamming memory device 610, in response to receiving one or more control signals indicating a Hamming processing command, control logic 622 provides one or more memory access requests containing memory addresses of stored image codes (e.g., image code datasets) at memory array 624 to various memory access circuits to perform memory access operations. In this example, Hamming logic 612 generates a plurality of memory addresses to be read based on the Hamming processing command. The plurality of memory addresses are provided as data to control logic 622 to be read into data register 626. In this example, each memory address is read into data register 626 of the Hamming memory device 610 to obtain a plurality of stored image codes on data register 626.

[0111] Box 704 may be followed by box 706, such that the method further includes the operation of comparing each of a plurality of stored image codes with an image code associated with a Hamming processing command to calculate a corresponding Hamming distance among a plurality of Hamming distances. In an example embodiment of the Hamming memory device 610, control logic 622 controls data register 626 to calculate each corresponding Hamming distance at data register 626 using the image code associated with the Hamming processing command stored at cache register 628 and the plurality of stored image codes at data register 626. For example, Hamming control logic 622 may execute executable instructions at processor 614 to calculate the corresponding Hamming distance of each image code in the dataset and the image code of the acquired image according to equation (1). Thus, each Hamming distance calculation represents comparing an image code with one of a plurality of stored image codes. Method 700 may end after box 706 is completed.

[0112] Figure 8An example of a system 800 according to aspects of this disclosure is shown. System 800 includes a person 802 and a vehicle 804 having a person 806. System 800 also includes an IoT device 808 attached to a house 810 and communicatively coupled to the house 810, for example, via a wired connection (not depicted). The house 810, which may include a network endpoint, is coupled to a network switch 814 that couples the house 810 to a data center 820. For example, the house 810 may be coupled to the network switch 814 via a wired connection (e.g., fiber optic, Ethernet) and / or a wireless connection (e.g., Wi-Fi, Bluetooth). The IoT device 808 may also be communicatively coupled to a wireless endpoint 812 via a wireless connection. For example, the wireless connection of the IoT device 808 may be a narrowband wireless communication connection according to the Narrowband IoT (NB-IoT) 5G standard. The wireless endpoint 812 is coupled to the data center 820 via a wired connection (e.g., fiber optic cable). Data center 820 includes host computing device 816 coupled to memory system 822, which interacts with memory device 824.

[0113] IoT device 808 may be an image capture device that, for example, acquires images of person 802, vehicle 804, and / or person 806 within vehicle 804 when person 802 or vehicle 804 approaches house 810. In this example, IoT device 808 may acquire an image of person 802, for example, to compare the image of person 802 with images of individuals in a dataset of convicted criminals (e.g., a dataset of images of convicted violent offenders or sexual predators). In this example, an image of vehicle 804 may be acquired by IoT device 808 to compare the image of vehicle 804 (or a portion thereof, e.g., an image of a license plate) with a dataset of license plates (e.g., license plates associated with Active Amber Warnings (e.g., government warnings about child trafficking or abduction of children)). Thus, IoT device 808 may acquire images of individuals or vehicles surrounding or approaching house 810. House 810, having a network endpoint, may transmit the acquired images to data center 820 via network switch 814 or via a wireless connection to wireless endpoint 812.

[0114] Once the acquired image is provided to data center 820, host computing device 816 may generate a Hamming processing request, for example, while processing the acquired image on a neural network hosted by host computing device 816 and interacting with memory system 822. Memory system 822 may calculate the Hamming distance between the image code of the acquired image and the corresponding image code of an image in a related dataset stored on memory device 824. Alternatively, IoT device 808 may process the acquired image on its own memory device. In this example, IoT device 808 includes a memory device implementing Hamming memory device 610, for example, to calculate the Hamming distance between the image code of the acquired image and the corresponding image code of an image in a related dataset stored on its own memory device. Advantageously, when processing Hamming processing commands on the memory device of the IoT device 808 itself, for example compared to a conventional neural network processing system, the processing rate and processing volume of the dataset in the neural network at the IoT device 808 itself are increased, thereby introducing latency into the processing speed and processing rate.

[0115] System 800 facilitates the processing of datasets that can be performed by a neural network hosted on host computing device 816 at data center 820. In this example, memory system 822 may, for example, implement Hamming processing unit 108 using memory device 824; or memory system 822 may implement memory controller 402. In either case, this implementation increases the processing capacity for computations performed by the neural network, for example, because image code comparison can be performed "closer" to memory device 824 by implementing memory system 822 as a processing unit having memory device 824 (e.g., Hamming processing unit 108) or as a memory controller coupled to memory device 824 (e.g., memory controller 402). In this example, memory system 822 is closer to memory device 824 than a conventional system, for example, that can retrieve and store datasets from memory device 824 on host computing device 816 for processing datasets, or a conventional system that can retrieve and store images to be processed at a network endpoint of house 810 from data center 820. In an embodiment where the memory system 822 serves as the Hamming processing unit 108, image code comparison is performed at the Hamming processing unit 108 with memory device 124 via controller bus 120 and corresponding memory bus 122. For example, the memory system 822 may implement method 200 or method 300 as the Hamming processing unit 108 with memory device 824 to obtain a result set regarding the image acquired by the IoT device 808. As another example, the memory system 822 may implement method 500 at the Hamming control logic 420 of the Hamming processing unit 108 with memory device 824 to obtain a result set regarding the image acquired by the IoT device 808.

[0116] Advantageously, when Hamming processing requests are allocated to memory device 824 when memory system 822 is implemented as Hamming processing unit 108 or memory controller 402, memory system 822 facilitates an increased rate of data processing by using an available memory bus coupled to memory system 822 (e.g., a memory bus operating according to the NVMe protocol).

[0117] Certain details have been set forth above to provide a full understanding of the described examples. However, it will be apparent to those skilled in the art that the examples can be practiced without these specific details. The example configurations described herein in conjunction with the accompanying drawings are not intended to represent all implementable examples or all examples within the scope of the claims. As may be used herein, the terms “exemplary” and “example” mean “serving as an example, illustration, or description” and are not necessarily “preferred” or “advantageous over other examples.” Specific details are included in the detailed description for the purpose of providing an understanding of the described techniques. However, these techniques can be practiced without these specific details. In some examples, well-known structures and apparatuses are shown in block diagram form to avoid obscuring the concepts of the described examples.

[0118] The information and signals described herein can be represented using any of a variety of different techniques and methods. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the foregoing description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or light particles, or any combination thereof.

[0119] The techniques described herein can be used in a variety of wireless communication systems, including multiple access cellular communication systems, and can employ Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), or Single Carrier Frequency Division Multiple Access (SC-FDMA), or any combination of such techniques. Some of these techniques have been used in or related to standardized wireless communication protocols of organizations such as the 3rd Generation Partnership Project (3GPP), 3rd Generation Partnership Project 2 (3GPP2), and IEEE. These wireless standards include Ultra Mobile Broadband (UMB), Universal Mobile Telecommunications System (UMTS), Long Term Evolution (LTE), LTE-Advanced (LTE-A), LTE-APro, New Radio (NR), IEEE 802.11 (WiFi), and IEEE 802.16 (WiMAX), among others.

[0120] The term "5G" or "5G communication system" can refer to a system operating according to standardized protocols developed or discussed, for example, after LTE version 13 or 14 or WiMAX 802.16e-2005 of its respective sponsoring organization. The features described herein can be adopted in systems configured according to other generations of wireless communication systems (including systems configured according to the standards described above).

[0121] The various illustrative blocks and modules described herein in conjunction with this disclosure may be implemented or performed using a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors incorporating a DSP core, or any other such configuration).

[0122] The functions described herein may be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions may be stored as one or more instructions or code on or transmitted over a computer-readable medium. Computer-readable media includes both non-transitory computer storage media and communication media, with communication media including any media that facilitates the transfer of a computer program from one place to another. Non-transitory storage media may be any available media accessible by a general-purpose or special-purpose computer. By way of example, and not limitation, non-transitory computer-readable media may include RAM, ROM, electrically erasable programmable read-only memory (EEPROM), or optical disc storage, magnetic disk storage, or other magnetic storage devices, or any other non-transitory media that can be used to carry or store desired program code components in the form of instructions or data structures and accessible by a general-purpose or special-purpose computer or a general-purpose or special-purpose processor.

[0123] Furthermore, any connection is appropriately referred to as computer-readable media. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then those coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. Combinations of the above are also included within the scope of computer-readable media.

[0124] Other examples and embodiments are within the scope of this disclosure and the appended claims. For example, due to the nature of software, the functions described above can be implemented using software executed by a processor, hardware, firmware, hardwired, or any combination thereof. Features implementing the functions can also be physically located in various locations, including being distributed such that different parts of the functions are implemented in different physical locations.

[0125] Furthermore, as used herein (included in the claims), "or" as used in a list of items (e.g., a list of items beginning with phrases such as "at least one of" or "one or more of") indicates an inclusive list, such that a list of at least one of, for example, A, B, or C means A or B or C or AB or AC or BC or ABC (i.e., A and B and C). Furthermore, as used herein, the phrase "based on" should not be construed as referring to a closed set of conditions. For example, without departing from the scope of this disclosure, an exemplary step described as "based on condition A" may be based on both condition A and condition B. In other words, as used herein, the phrase "based on" should be interpreted in the same manner as the phrase "at least partially based on".

[0126] As will be understood from the foregoing, although specific examples have been described herein for illustrative purposes, various modifications may be made while maintaining the scope of the claimed art. The descriptions herein are provided to enable those skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the scope of this disclosure. Therefore, this disclosure is not limited to the examples and designs described herein, but is endowed with the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method comprising: The image code associated with the Hamming processing command is obtained at the I / O control unit of the memory device; In response to the Hamming processing command, at least one memory access request to a plurality of memory addresses is provided to obtain a plurality of stored image codes; and Each of the plurality of stored image codes is compared with the image code associated with the Hamming processing command to calculate the corresponding Hamming distance among a plurality of Hamming distances.

2. The method according to claim 1, further comprising: A control signal instructing the Hamming processing command is obtained at the control logic of the memory device.

3. The method of claim 2, wherein obtaining the control signal instructing the Hamming processing command comprises: The control signal indicating the Hamming processing command is obtained at the command latch enable (CLE) pin of the control logic.

4. The method of claim 2, further comprising: Additional control signals are obtained at the control logic of the memory device to instruct the image code to be written to the data register of the memory device.

5. The method of claim 4, wherein obtaining the additional control signal at the control logic of the memory device instructing the image code to be written to the cache register of the memory device comprises: The additional control signal is obtained at the write enable (WE#) pin of the control logic.

6. The method of claim 2, further comprising: In response to the control signal indicating the Hamming processing command, the ready / busy (R / B) pin of the control logic is confirmed.

7. The method according to claim 1, wherein, In response to the Hamming processing command, providing at least one memory access request to the plurality of memory addresses to obtain the plurality of stored image codes includes: The plurality of memory addresses to be read are generated based on the Hamming processing command; The generated memory addresses are provided to the control logic via the command register; and At the control logic, at least one memory access request for the plurality of memory addresses is generated based on the plurality of memory addresses generated in the command register.

8. The method according to claim 7, wherein, In response to the Hamming processing command, providing at least one memory access request to the plurality of memory addresses to obtain the plurality of stored image codes includes providing the at least one memory access request to the row and column decoders of the memory device for reading the memory array of the memory device.

9. The method of claim 8, further comprising: The plurality of memory addresses are read into the data register of the memory device at the memory array to obtain the plurality of stored image codes on the data register of the memory device.

10. The method of claim 9, wherein comparing each of the plurality of stored image codes with the image code associated with the Hamming processing command to calculate the corresponding Hamming distance among the plurality of Hamming distances comprises: The corresponding Hamming distance among the plurality of Hamming distances is calculated at the data register based on each of the plurality of stored image codes and the image code associated with the Hamming processing command.

11. The method of claim 10, wherein calculating the corresponding Hamming distance among the plurality of Hamming distances comprises: For each corresponding stored image code among the plurality of stored image codes: The corresponding bit of the image code associated with the Hamming processing command is added to the bit of each stored image code to produce the corresponding first bit result; Perform a modulo-2 operation on each corresponding bit result to produce the corresponding second bit result; and The corresponding second-order results are summed to produce the corresponding Hamming distance among the plurality of Hamming distances.

12. The method of claim 1, wherein the plurality of stored image codes are stored in the NAND memory of the memory device.

13. The method of claim 1, further comprising: Hamming processing commands are generated at the host computing device based on the user application that performs Hamming distance calculations.

14. A memory comprising: processor; A memory array configured to store multiple stored image codes representing corresponding images among a plurality of images; and A non-transitory computer-readable medium storing executable instructions that, when executed by the processor, cause the memory to perform operations including: Image codes associated with Hamming processing commands are obtained at the I / O control unit of the memory; In response to the Hamming processing command, at least one memory access request, including multiple memory addresses, is provided to the memory array; and At the data register of the memory array, each of the plurality of stored image codes is compared with the image code associated with the Hamming processing command to generate a corresponding Hamming distance among a plurality of Hamming distances.

15. The memory of claim 14, wherein the operation further comprises: Based on the multiple Hamming distances, an image processing result is generated that includes a result set indicating certain images among the multiple images; and The image processing results are provided to a host computing device for use by a neural network hosted on the host computing device.

16. The memory of claim 14, further comprising: Obtain an image different from the plurality of images from an Internet of Things (IoT) computing device; and The image code associated with the Hamming processing command is generated based on the image using a hash algorithm.

17. The memory of claim 16, wherein the IoT computing device corresponds to at least one of a camera, a smartphone device, or an image capture device.

18. An apparatus comprising: A host computing device configured to acquire an image from an IoT computing device and provide a Hamming processing request based on the image; The memory includes: A memory array configured to store multiple stored image codes representing corresponding images among a plurality of images; Processor; and At least one non-transitory computer-readable medium encoded with executable instructions, which, when executed by the processor, cause the device to perform operations including: The image code in the Hamming processing command is obtained at the I / O control unit of the memory based on the Hamming processing request; In response to the Hamming processing command, at least one memory access request, including multiple memory addresses, is provided to the memory array; and At the data register of the memory array, each of the plurality of stored image codes is compared with the image code associated with the Hamming processing command to generate a corresponding Hamming distance among a plurality of Hamming distances.

19. The apparatus of claim 18, wherein the host computing device includes a memory controller configured to provide the Hamming processing request to the processor.

20. The apparatus of claim 19, wherein the operation further comprises: The Hamming processing request is obtained from the memory controller via the NVMe memory bus; and The Hamming processing command generates the image code to be provided to the I / O control unit of the memory based on the Hamming processing request.

Images