Processing circuit and template matching method

Abstract

The invention provides a processing circuit and a template matching method. The processing circuit comprises an image source device, an adjusting device, a resistive memory and a control circuit. The image source device is used for providing a target image. The adjusting device adjusts the scale of a template image according to setting information so as to generate a sample image. The resistive memory includes a storage region and an operational circuit. The storage area stores a target image. The operation circuit calculates the sample image and the target image to generate a plurality of operation results. The control circuit finds out a matching position matched with the sample image in the target image according to the operation result. According to the processing circuit and the template matching method provided by the invention, the efficiency of template matching operation can be improved, and the power consumption of the processing circuit can be reduced.

Description

Technical Field

[0001] The present invention relates to a processing circuit, and more particularly to a processing circuit for performing template matching operations. Background Art

[0002] With the progress of technology, the types and functions of electronic devices are increasing. Most electronic devices have a memory for storing data. Generally, the data in the memory is read and loaded into a computing unit (independent of the memory) for operation. However, the movement and transmission of data increase the power consumption of the electronic device and add additional processing time. Summary of the Invention

[0003] An embodiment of the present invention provides a processing circuit, including an image source device, an adjustment device, a resistive memory, and a control circuit. The image source device is used to provide a target image. The adjustment device adjusts the scale of a template image according to a set of information to generate a sample image. The resistive memory includes a storage area and an arithmetic circuit. The storage area stores the target image. The arithmetic circuit calculates the sample image and the target image to generate a plurality of operation results. The control circuit finds a matching position in the target image that matches the sample image according to the operation results.

[0004] The present invention further provides a template matching method applicable to a resistive memory. The template matching method of the present invention includes: adjusting the scale of a template image according to a set of information to generate a sample image; storing a target image in a storage area of the resistive memory; dividing the target image to generate a plurality of sub-images; calculating the sample image and the sub-images to generate a plurality of operation results; and finding a matching position in the target image that matches the sample image according to the operation results.

[0005] The template matching method of the present invention can be executed by the processing circuit of the present invention, which is hardware or firmware capable of executing specific functions, or can be included in a recording medium in the form of program code and combined with specific hardware to execute. When the program code is loaded and executed by an electronic device, a processor, a computer, or a machine, the electronic device, the processor, the computer, or the machine becomes a processing circuit for implementing the present invention. Brief Description of the Drawings

[0006] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative efforts.

[0007] Figure 1 Schematic diagram of the processing circuit of the present invention.

[0008] Figure 2 Schematic diagram of dividing the storage area of the resistive memory of the present invention.

[0009] Figure 3 Another schematic diagram of dividing the storage area of the resistive memory of the present invention.

[0010] Figure 4 A possible schematic diagram of the resistive memory of the present invention.

[0011] Figure 5 Schematic diagram of the template matching method of the present invention.

[0012] Description of reference numerals:

[0013] 100: Processing circuit

[0014] 110: Image source device

[0015] 120: Adjustment device

[0016] 130: Resistive memory

[0017] 140: Control circuit

[0018] 150: Image storage device

[0019] IM_TP: Template image

[0020] ST: Setting information

[0021] IM_SP~IM_SP”: Sample images

[0022] IM_TR: Target image

[0023] 131: Storage area

[0024] R1~R7, 211~214, 311~316: Image areas

[0025] C[1,2,3,…], C’[1,2,3,…], C”[1,2,3,…]: Operation results

[0026] 132: Arithmetic circuit

[0027] 134: Writing circuit

[0028] VE1~VE4, HO1~HO4: Transmission lines

[0029] C 11 ~C 44 : Memory cells

[0030] R 11 ~R 44 : Resistance layer

[0031] 410: Sampling and holding circuit

[0032] 420: Analog-to-digital conversion circuit

[0033] 430: Processing circuit

[0034] I0~I3: Sampling signals

[0035] S / H_1~S / H_4: Sampling and holding devices

[0036] AL0~AL3: Digital signals

[0037] 133: Digital-to-analog conversion circuit

[0038] 441~444: Digital-to-analog converters

[0039] V0~V3: Analog signals

[0040] S511~S515: Steps Detailed implementation manners

[0041] To make the objectives, features, and advantages of the present invention more obvious and understandable, specific embodiments are hereinafter given and described in detail in conjunction with the accompanying drawings. The specification of the present invention provides different embodiments to illustrate the technical features of different implementation manners of the present invention. Among them, the configuration of each element in the embodiments is for illustrative purposes only and is not intended to limit the present invention. In addition, the partial repetition of the reference numerals in the drawings of the embodiments is for the purpose of simplifying the description and does not imply the relevance between different embodiments.

[0042] Figure 1 It is a schematic diagram of the processing circuit of the present invention. As shown in the figure, the processing circuit 100 includes an image source device 110, an adjustment device 120, a resistive random access memory (ReRAM) 130, and a control circuit 140. The present invention does not limit the type of the processing circuit 100. In a possible embodiment, the processing circuit 100 is a microcontroller unit (MCU) or a microprocessor unit (MPU).

[0043] The image source device 110 is used to provide a target image IM_TR. The present invention does not limit the type of the image source device 110. In a possible embodiment, the image source device 110 includes an image sensor, such as a CMOS or a CCD. In this example, the target image IM_TR is the sensing result of the image source device 110. In another possible embodiment, the image source device 110 is a memory, such as an SRAM. In this example, the target image IM_TR is stored in the image source device 110.

[0044] In some embodiments, the processing circuit 100 further includes an image storage device 150. The image storage device 150 provides a template image IM_TP. The present invention does not limit the type of the image storage device 150. In a possible embodiment, the image storage device 150 is a memory, such as an SRAM.

[0045] The adjustment device 120 adjusts the scale of the template image IM_TP according to a setting information ST to generate a sample image IM_SP. The adjustment device 120 generates different sample images according to different setting information ST. In a possible embodiment, the adjustment device 120 further generates sample images IM_SP’ and IM_SP”. The scale of the sample image IM_SP is smaller than the scale of the sample image IM_SP’. The scale of the sample image IM_SP’ is smaller than the scale of the sample image IM_SP”. In a possible embodiment, the scale of one of the sample images IM_SP to IM_SP” is the same as the scale of the template image IM_TP. In some embodiments, the adjustment device 120 provides at least one sample image (such as at least one of IM_SP to IM_SP”) according to the setting information ST.

[0046] The resistive memory 130 stores the target image IM_TR and performs a template matching operation on at least one of the sample images IM_SP to IM_SP” and the target image IM_TR. Since the resistive memory 130 has the advantages of excellent storage density, low power consumption, and fast writing, the efficiency of the template matching operation can be improved. Furthermore, since the template matching operation is directly performed in the resistive memory 130, it is not necessary to load the target image IM_TR stored in the resistive memory 130 into an external computing unit for processing, thus avoiding data movement and transmission and reducing the power consumption of the processing circuit 100.

[0047] In a possible embodiment, the resistive memory 130 stores the target image IM_TR in a storage area 131. The storage area 131 includes image areas R1 to R7. The image areas R1 to R7 have the same size. The image areas R1 to R7 do not overlap with each other, but this is not intended to limit the present invention. In other embodiments, at least one of the image areas R1 to R7 overlaps with an adjacent image area. The resistive memory 130 calculates the images of the sample image IM_SP and each of the image areas R1 to R7 to generate a plurality of operation results C[1, 2, 3, …].

[0048] The control circuit 140 determines, based on the operation results C[1, 2, 3, …], whether there is an image in the target image IM_TR that best matches the sample image IM_SP. For example, if each operation value of the operation results C[1, 2, 3, …] is lower than a threshold value, it indicates that there is no image in the target image IM_TR that best matches the sample image IM_SP.

[0049] In other embodiments, the resistive memory 130 also calculates the images of the sample image IM_SP’ and each of the image areas R1 to R7 to generate a plurality of operation results C’[1, 2, 3, …], and calculates the images of the sample image IM_SP” and each of the image areas R1 to R7 to generate a plurality of operation results C”[1, 2, 3, …]. The present invention does not limit the number of operation values of each operation result. The number of operation values of one of the operation results C[1, 2, 3, …], C’[1, 2, 3, …], and C”[1, 2, 3, …] may be the same as or different from that of another.

[0050] The control circuit 140 determines, based on the operation results C[1, 2, 3, …], C’[1, 2, 3, …], and C”[1, 2, 3, …], whether there is an image in the target image IM_TR that best matches the sample images IM_SP to IM_SP”. In a possible embodiment, the control circuit 140 ranks the set of all operation results, and then the best-matching image can be found in the target image IM_TR, thereby achieving the positioning function. For example, based on the operation results C[1, 2, 3, …], C’[1, 2, 3, …], and C”[1, 2, 3, …], the control circuit 140 knows that the image (v) in the image area R3 is most similar to the sample image IM_SP’.

[0051] In other embodiments, the resistive memory 130 divides the storage area 131 according to the setting information ST, and calculates the images stored in the divided areas with the sample images. Figure 2A schematic diagram of dividing a storage area 131 for a resistive memory 130. In this example, the resistive memory 130 divides the storage area 131 horizontally to generate a plurality of image regions. For ease of explanation, Figure 2 only image regions 211 to 214 are shown. The image regions 211 to 214 overlap adjacent image regions. For example, image region 211 overlaps image region 212, and image region 212 overlaps image regions 211 and 213. In this example, the areas of image regions 211 to 214 are the same. In other embodiments, the number of pixels of the images stored in image regions 211 to 214 is the same as the number of pixels of the sample image IM_SP’. For example, if the number of pixels of the sample image IM_SP’ is 6x4, then the number of pixels of the image stored in each of image regions 211 to 214 will also be 6x4.

[0052] The resistive memory 130 selects image region 211 and calculates the sample image IM_SP' and the image stored in image region 211 to generate an operation result C’1. Then, the resistive memory 130 selects image region 212 and calculates the sample image IM_SP' and the image stored in image region 212 to generate an operation result C’2. Next, the resistive memory 130 selects image region 213 and calculates the sample image IM_SP' and the image stored in image region 213 to generate an operation result C’3. Then, the resistive memory 130 selects image region 214 and calculates the sample image IM_SP' and the image stored in image region 214 to generate an operation result C’4. After calculating all the image regions, the resistive memory 130 integrates all the operation results C’1 to C’4 and outputs the integrated result (such as C’[1,2,3,…]). In some embodiments, the number of operation results is the same as the number of pixels of the sample image IM_SP'. For example, assuming that the number of pixels of the sample image IM_SP' is 6x4, then the number of operation results provided by the resistive memory 130 is also 6x4.

[0053] Figure 3 Another schematic diagram of dividing a storage area 131 for a resistive memory 130. In this example, the resistive memory 130 divides the storage area 131 horizontally to generate a plurality of image regions. For ease of explanation, Figure 3 only image regions 311 to 316 are shown. The image regions 311 to 316 overlap adjacent image regions. For example, image region 311 overlaps image regions 312 to 314, and image region 313 overlaps image regions 311, 312, 314 to 316. In this example, the areas of image regions 311 to 316 are the same. Additionally, the number of pixels of the image stored in each of image regions 311 to 316 is the same as the number of pixels of the sample image IM_SP.

[0054] The resistive memory 130 selects the image region 311 and calculates the images of the sample image IM_SP and the image region 311 to generate an operation result C1. Then, the resistive memory 130 selects the image region 312 and calculates the images of the sample image IM_SP and the image region 312 to generate an operation result C101. Next, the resistive memory 130 selects the image region 313 and calculates the images of the sample image IM_SP and the image region 313 to generate an operation result C2. Then, the resistive memory 130 selects the image region 314 and calculates the images of the sample image IM_SP and the image region 314 to generate an operation result C102. Next, the resistive memory 130 selects the image region 315 and calculates the images of the sample image IM_SP and the image region 315 to generate an operation result C3. Then, the resistive memory 130 selects the image region 316 and calculates the images of the sample image IM_SP and the image region 316 to generate an operation result C103. After calculating all the image regions, the resistive memory 130 integrates all the operation results C1, C101, C2, C102, C3, C103 and outputs the integrated result (such as C[1,2,3,…]).

[0055] The present invention does not limit how the resistive memory 130 calculates the sample image and the target image. In a possible embodiment, the resistive memory 130 performs a multiplication operation on the sample image and the target image. Taking the sample image IM_SP’ as an example, assume that the size of the sample image IM_SP’ is:

[0056]

[0057] Assume that the number of pixels of the target image IM_TR is 4x4, as follows:

[0058]

[0059] After the resistive memory 130 calculates equations (1) and (2), the operation results are as follows:

[0060]

[0061] The control circuit 140 determines whether the target image IM_TR has an image that conforms to the sample image IM_SP’ according to equation (3).

[0062] Through the high-speed and low-power characteristics of the resistive memory 130, a template matching operation for image processing is performed. In the template matching operation, correlation is used as the feature metric method to measure the similarity between the sample image and the target image. Utilizing the characteristics of the resistive memory 130, fast feature extraction and correlation calculation are achieved. Image features are efficiently calculated through the resistive memory 130, and based on the operation results, the image region with the best match is determined. In some embodiments, the control circuit 140 achieves a positioning function according to the operation results of the resistive memory 130, and can also determine the flow of people and the number of people in a specific space. For example, the control circuit 140 performs inference operations and learning operations according to the operation results of the resistive memory 130 for AI (Artificial Intelligence) analysis, such as image positioning, tracking, or extracting text from pictures. Due to the fast operation speed of the resistive memory 130, the control circuit 140 can quickly perform image judgment, improving the speed of AI analysis.

[0063] Figure 4 FIG. is a possible schematic diagram of the resistive memory 130. As Figure 4 shown, the resistive memory 130 includes a storage area 131, an arithmetic circuit 132, and a write circuit 134. The write circuit 134 writes the target image IM_TR into the storage area 131. In some embodiments, the storage area 131 includes transmission lines VE1-VE4, HO1-HO4, and memory cells C 11 ~C 44 , but is not intended to limit the present invention. In other embodiments, the storage area 131 has other numbers of transmission lines and memory cells. Each of the memory cells C 11 ~C 44 is coupled between two transmission lines. For example, the memory cell C 11 is coupled between the transmission lines HO1 and VE1, and the memory cell C 12 is coupled between the transmission lines HO1 and VE2.

[0064] In a possible embodiment, each of the memory cells C 11 ~C 44 has a resistive layer for storing the gray-scale value of a pixel of the target image IM_TR. For example, the write circuit 134 converts the gray-scale values of all pixels of the target image IM_TR into corresponding voltage values, and then provides the voltage values to the memory cells C 11 ~C 44 to set the resistance values of the resistive layers of the memory cells C 11 ~C 44 . In this example, the resistance value of each resistive layer is related to the gray-scale value of the corresponding pixel. For example, the resistive layer R 11 of the memory cell C 11The resistance value is related to the gray scale value (such as the numerical value 0) of a first pixel of the target image IM_TR, and the storage unit C 12 The resistance layer R of 12 The resistance value is related to the gray scale value (such as the numerical value 205) of a second pixel of the target image IM_TR.

[0065] The operation circuit 132 calculates the sample image IM_SP’ and the target image IM_TR stored in the storage area 131 to generate an operation result C’[1, 2, 3, …]. The present invention does not limit the architecture of the operation circuit 132. In a possible embodiment, the operation circuit 132 includes a sampling and holding circuit 410, an analog-to-digital conversion circuit (ADC) 420, and a processing circuit 430. The sampling and holding circuit 410 is coupled to the transmission lines VE1 to VE4 and generates sampling signals I0 to I3.

[0066] In a possible embodiment, the sampling and holding circuit 410 includes sampling and holders S / H_1 to S / H_4. The sampling and holder S / H_1 samples and holds the signal on the transmission line VE1 to provide the sampling signal I0. The sampling and holder S / H_2 samples and holds the signal on the transmission line VE2 to provide the sampling signal I1. The sampling and holder S / H_3 samples and holds the signal on the transmission line VE3 to provide the sampling signal I2. The sampling and holder S / H_4 samples and holds the signal on the transmission line VE4 to provide the sampling signal I3.

[0067] The analog-to-digital conversion circuit 420 converts the sampling signals I0 to I3 to generate digital signals AL0 to AL3. In this embodiment, the analog-to-digital conversion circuit 420 converts the sampling signals I0 to I3 from an analog format to a digital format and uses the converted result as the digital signals AL0 to AL3. The processing circuit 430 sums the digital signals AL0 to AL3 to generate an operation result C’[1, 2, 3, …].

[0068] In other embodiments, the resistive memory 130 further includes a digital-to-analog conversion circuit 133. The digital-to-analog conversion circuit 133 converts the gray scale values of the sample image IM_SP’ and provides the converted analog signals V0 to V3 to the transmission lines HO1 to HO4. In this embodiment, the digital-to-analog conversion circuit 133 includes digital-to-analog converters (DAC) 441 to 444. The digital-to-analog converters 441 to 444 are respectively coupled to the transmission lines HO1 to HO4.

[0069] In some embodiments, the number of pixels of the target image in storage area 131 (such as 4x4) is greater than the number of pixels of the sample image IM_SP' (4). Additionally, the number of digital signals AL0 to AL3 is the same as the number of pixels of the sample image IM_SP'. The number of analog signals V0 to V3 is also the same as the number of pixels of the sample image IM_SP'.

[0070] Figure 5 FIG. is a schematic diagram of a template matching method of the present invention. The template matching method of the present invention is applicable to a resistive memory. The template matching method of the present invention can exist in the form of program code. When the program code is loaded and executed by a machine, the machine becomes a processing circuit for implementing the present invention.

[0071] First, according to a set of information, adjust the scale of a template image to generate a sample image (step S511). In a possible embodiment, step S511 also reads a memory to generate a template image, and then adjusts the scale of the template image according to a set of information.

[0072] Store a target image in a storage area of a resistive memory (step S512). In a possible embodiment, step S512 receives the output of an image sensor and stores the output of the image sensor as the target image in the storage area of the resistive memory. In some embodiments, step S512 sets the resistance values of all the resistance layers in the storage area according to all the gray-scale values of the target image. In other embodiments, step S512 also reads a memory to generate a target image. Additionally, step S512 may receive the output of an image sensor and store the output of the image sensor as a target image.

[0073] Next, divide the target image to generate a plurality of sub-images (step S513). The present invention does not limit how step S513 divides the target image. In a possible embodiment, each sub-image may or may not overlap adjacent sub-images. Additionally, the number of pixels of each sub-image may be the same as the pixel data of the sample image. In this example, the number of pixels of the target image is greater than the number of pixels of the sample image.

[0074] In other embodiments, the target image is stored in a storage area of a resistive memory. In this example, step S513 divides the storage area to generate a plurality of image areas. Each image area may or may not overlap adjacent image areas. Each image area stores a part of the target image. In a possible embodiment, the area of each image area is the same. The number of pixels of the image stored in each image area is the same as the pixel data of the sample image.

[0075] Calculate the sample image and each image to generate multiple operation results (step S514). In a possible embodiment, step S514 is to multiply the sample image by each image. Then, according to the operation results, find a matching position in the target image that matches the sample image (step S515). For Figure 1 example, the control circuit 140 knows that the image in the image region R3 is most similar to the sample image IM_SP' according to the operation results C[1, 2, 3,...], C'[1, 2, 3,...] and C''[1, 2, 3,...].

[0076] It must be understood that when an element is referred to as being "coupled" to another element, it can be directly coupled or connected to other elements, or there can be other elements in between. Conversely, when an element is "connected" to other elements, there will be no other elements in between.

[0077] The template matching method of the present invention, or a specific type or part thereof, can exist in the form of program code. The program code can be stored in a physical medium, such as a floppy disk, a CD-ROM, a hard disk, or any other machine-readable (such as computer-readable) storage medium, or it is not limited to the external form of a computer program product. Among them, when the program code is loaded and executed by a machine, such as a computer, this machine becomes a processing circuit for participating in the present invention. The program code can also be transmitted through some transmission media, such as wires or cables, optical fibers, or any transmission type. Among them, when the program code is received, loaded, and executed by a machine, such as a computer, this machine becomes a processing circuit for participating in the present invention. When executed on a general-purpose processing unit, the program code combined with the processing unit provides a unique device that operates similar to an application-specific logic circuit.

[0078] Unless otherwise defined, all terms (including technical and scientific terms) herein are within the general understanding of those skilled in the art. In addition, unless clearly stated, the definitions of terms in a general dictionary should be interpreted as being consistent with their meanings in the articles of the relevant technical field, and should not be interpreted as an ideal state or an overly formal voice. Although terms such as "first", "second", etc. can be used to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. In the claims, terms such as "first", "second", etc. are used as labels and do not intend to impose numerical requirements on their objects.

[0079] Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Those skilled in the art can make some modifications and refinements without departing from the spirit and scope of the present invention. For example, the systems, devices, or methods described in the embodiments of the present invention can be implemented in physical embodiments of hardware, software, or a combination of hardware and software. Therefore, the protection scope of the present invention shall be subject to what is defined by the claims.

Claims

1. A processing circuit, characterized in that: include: An image source device for providing a target image; an adjusting device, adjusting the scale of a template image according to a setting information, so as to generate a sample image; A resistive memory comprising: a storage area for storing the target image; and a computing circuit for computing the sample image and the target image to generate a plurality of computing results; and A control circuit finds a matching position matching the sample image in the target image according to the plurality of operation results.

2. The processing circuit according to claim 1, characterized in that The operation circuit divides the storage area into a plurality of image areas according to the setting information, and calculates the sample image and the image stored in each of the plurality of image areas to generate the plurality of operation results.

3. The processing circuit according to claim 2, characterized in that Each image region overlaps adjacent image regions.

4. The processing circuit according to claim 2, characterized in that: The number of pixels of the image stored in each image region is the same as the number of pixels of the sample image.

5. The processing circuit according to claim 2, characterized in that: The storage area includes a plurality of transmission lines and a plurality of resistance layers, and each resistance layer is coupled to two of the plurality of transmission lines.

6. The processing circuit according to claim 5, characterized in that The resistive memory comprises: A writing circuit sets the resistance values ​​of the plurality of resistance layers according to the grayscale values ​​of all pixels of the target image.

7. The processing circuit according to claim 6, characterized in that The operation circuit comprises: a sampling and holding circuit coupled to the plurality of transmission lines and generating a plurality of sampling signals; an analog-to-digital conversion circuit, converting the plurality of sampled signals to generate a plurality of digital signals; and A processing circuit sums up the multiple digital signals to generate the multiple operation results.

8. The processing circuit according to claim 7, characterized in that: Also includes: A digital-to-analog conversion circuit converts all grayscale values ​​of the sample image to generate a plurality of analog signals, and provides the plurality of analog signals to the storage area.

9. A template matching method, applicable to a resistive memory, characterized in that: include: According to a setting information, adjusting the scale of a template image to generate a sample image; storing a target image in a storage area of ​​the resistive memory; dividing the target image to generate a plurality of sub-images; Calculating the sample image and the plurality of sub-images to generate a plurality of calculation results; as well as According to the plurality of operation results, a matching position matching the sample image is found in the target image.

10. The template matching method according to claim 9, characterized in that: Also includes: The output of an image sensor is used as the target image.

Images