Image retrieval system

Abstract

To provide an image retrieval system for retrieving an image with high accuracy in a short time.SOLUTION: An image retrieval system has a processing part provided with a neural network. The neural network has a layer provided with neurons. The processing part has a function of comparing query image data with a plurality of database image data, and extracting database image data including a region having high coincidence with the query image data as extraction image data, and has a function of extracting data of a region having high coincidence with the query image data as partial image data, from the extraction image data. The layer has a function of outputting an output value corresponding to a feature amount of the image data input to the neural network. The processing part also has a function of comparing the output value when the partial image data is input with the output value when the query image data is input, with respect to the partial image data.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] One aspect of the present invention relates to an image search system and an image search method.

[0002] Note that one embodiment of the present invention is not limited to the above technical field. Examples of the semiconductor device include a semiconductor device, a display device, a light-emitting device, a power storage device, a memory device, an electronic device, a lighting device, The driving method thereof or the manufacturing method thereof can be given as an example. [Background technology]

[0003] From a database containing a large number of stored images, the system searches for images similar to the query image to be searched for. A method called template matching is used to find images that contain similar images in part. For example, in Patent Document 1, a part of a matching image and a reference image are The invention calculates the similarity between two images by performing template matching. It is written.

[0004] In addition, the use of artificial intelligence (AI) is being considered for various purposes. In particular, neural networks By using the network, it is possible to develop a computer with higher performance than the conventional von Neumann type computer. It is expected that it will be possible to realize this, and in recent years, there have been many attempts to build neural networks on electronic circuits. Various research projects are underway.

[0005] For example, Patent Document 2 describes a matrix for the convolution operation of a convolutional neural network. The amount of calculations is reduced by dividing the data into two parts, the first half and the second half, based on the threshold value. The invention discloses a method for producing the same. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2005-284731 A [Patent Document 2] JP 2018-160086 A Summary of the Invention [Problem to be solved by the invention]

[0007] As a template matching method, feature-based matching and region-based matching are used. Feature-based matching involves extracting feature points from an image and calculating the feature amount. In region-based matching, the query image and the target image are matched. The query image is compared with a template image one by one. A template image that partially contains an image similar to the above is extracted.

[0008] However, in feature-based matching, it may take a long time to extract feature points. In addition, region-based matching is particularly vulnerable to image deformation, making it difficult to perform image retrieval with high accuracy. There are cases.

[0009] Therefore, one aspect of the present invention provides an image search system capable of searching for images with high accuracy. Another object of the present invention is to provide an image search system that can perform image search in a short time. Another object of the present invention is to provide a novel image detection system. One of the objectives of the present invention is to provide a search system.

[0010] Another aspect of the present invention is to provide an image retrieval method capable of retrieving images with high accuracy. Another object of the present invention is to provide an image search method capable of performing an image search in a short time. Another object of the present invention is to provide a novel image search method. One of our goals is to provide

[0011] Note that the description of these problems does not preclude the existence of other problems. However, it is not necessary to solve all of these problems. It is possible to extract other issues from the description in the section. [Means for solving the problem]

[0012] One aspect of the present invention is a method for detecting a neural network comprising: The neural network has an intermediate layer, and the intermediate layer is configured to input the first image data to the neural network. When the first image data is input to the network, the first output value corresponding to the first image data is output. The intermediate layer, when the second image data is input to the neural network, The processing unit has a function of outputting a second output value corresponding to the second image data, and the processing unit outputs the first output The image search system has a function of comparing the first output value with the second output value.

[0013] One aspect of the present invention is a method for detecting a neural network comprising: The convolutional network includes a convolutional layer and a pooling layer, and the pooling layer is When image data is input to the neural network, The pooling layer has a function of outputting a first output value corresponding to the second image data. When the second image data is input to the network, a second output value corresponding to the second image data is output. the processing unit has a function of comparing the first output value with the second output value. It is an image search system.

[0014] One aspect of the present invention is a method for detecting a neural network comprising: The network has an intermediate layer, and the processing unit processes image data and a plurality of database image data. When the data is input to the processing unit, image data, database image data, The processing unit has a function of comparing the image data with a database including an area having a high degree of match. The intermediate layer has a function of extracting the source image data as extracted image data. When the image data is input to the neural network, a first output value corresponding to the image data is obtained. The intermediate layer has a function of outputting the extracted image data input to the neural network. when the extracted image data is not included in the image data, the processing unit has a function of outputting a second output value corresponding to the extracted image data, The image search system has a function of comparing the first output value with the second output value.

[0015] One aspect of the present invention is a method for detecting a neural network comprising: The neural network has a convolution layer and an intermediate layer, and the processing unit receives image data and a plurality of When database image data and the like are input to the processing unit, the image data and the database The processing unit has a function of comparing the source image data with the image data having a high degree of match. It has a function to extract database image data including the area as extracted image data, The ring layer is a layer that extracts image data when the image data is input to the neural network. The pooling layer has a function of outputting a first output value corresponding to the extracted image data. a second output value corresponding to the extracted image data when the extracted image data is input to the neural network; The processor has a function of comparing the first output value with the second output value. It is an image search system.

[0016] One aspect of the present invention is a method for detecting a neural network comprising: The network has an intermediate layer, and the processing unit processes image data and a plurality of database image data. When the data is input to the processing unit, image data, database image data, The processing unit has a function of comparing the image data with a database including an area having a high degree of match. The processing unit has a function of extracting the base image data as extracted image data, From the data, data in an area having a high degree of coincidence with the image data is extracted as partial image data. The intermediate layer has the function of determining the corresponding image data when the image data is input to the neural network. The intermediate layer has a function of outputting a first output value corresponding to the image data, and the intermediate layer outputs the partial image data When the partial image data is input to the neural network, the second output corresponding to the partial image data is The processing unit has a function of comparing the first output value with the second output value. It is an image retrieval system with the following functions.

[0017] One aspect of the present invention is a method for detecting a neural network comprising: The neural network has a convolution layer and a pooling layer, and the processing unit receives image data and When a plurality of database image data are input to the processing unit, the image data and the database The processing unit has a function of comparing the database image data with the image data, and the processing unit determines the degree of coincidence with the image data. The function of extracting database image data including areas with high values ​​as extracted image data is provided. The processing unit extracts data of an area having a high degree of matching with the image data from the extracted image data. The pooling layer has the function of extracting the image data as a neural network. When image data is input to the network, a first output value corresponding to the image data is output. When partial image data is input to the neural network, the pooling layer The processing unit has a function of outputting a second output value corresponding to the partial image data, and the processing unit outputs the first output value. The image search system has a function of comparing the force value with the second output value.

[0018] The number of pixel data items contained in the image data is equal to the number of pixel data items contained in the database image data. It may be the following.

[0019] The comparison of the image data with the database image data is performed by region-based matching. This is also fine.

[0020] The image data includes a plurality of pixel data, and the processing unit performs a process based on the input image data. The image processing unit 10 may have a function of generating a plurality of image data items each having a different number of pieces of pixel data to be stored.

[0021] Although neural networks have the ability to learn using database image data, good.

[0022] One aspect of the present invention is to input first image data to a neural network having an intermediate layer. By this, the first output value output from the intermediate layer is obtained and input to the neural network. By inputting second image data, a second output value output from the intermediate layer is obtained; This is an image search method in which the first output value and the second output value are compared.

[0023] One aspect of the present invention is a neural network having a convolutional layer and a pooling layer. By inputting the first image data, the first output value output from the pooling layer is obtained. By inputting the second image data to the neural network, the pooling layer A second output value output from the first output value is obtained, and the first output value and the second output value are compared. It is an image retrieval method.

[0024] In one aspect of the present invention, the image data is compared to a plurality of database image data, and the image data is Extract the database image data containing areas with high matching to the target image data as extracted image data. By inputting image data into a neural network with a hidden layer, The first output value is obtained, and the extracted image data is input to the neural network. By this, a second output value output from the intermediate layer is obtained, and the first output value and the second output This is an image search method that compares values ​​with.

[0025] In one aspect of the present invention, the image data is compared to a plurality of database image data, and the image data is Extract the database image data containing areas with high matching to the target image data as extracted image data. Then, image data is input to a neural network that has a convolution layer and a pooling layer. By inputting the first output value from the layer, the neural network obtains the first output value. By inputting the extracted image data, the second output value output from the pooling layer is obtained. The first output value and the second output value are compared.

[0026] In one aspect of the present invention, the image data is compared to a plurality of database image data, and the image data is Extract the database image data containing areas with high matching to the target image data as extracted image data. Then, from the extracted image data, data in an area having a high degree of matching with the image data is extracted as a partial image data. The image data is extracted as image data and input to a neural network with an intermediate layer. The first output value output from the intermediate layer is obtained, and the partial image is input to the neural network. By inputting image data, the second output value output from the intermediate layer is obtained, and the first output This is an image retrieval method for comparing the force value with the second output value.

[0027] In one aspect of the present invention, the image data is compared to a plurality of database image data, and the image data is Extract the database image data containing areas with high matching to the target image data as extracted image data. Then, from the extracted image data, data in an area having a high degree of matching with the image data is extracted as a partial image data. The data is extracted as a neural network with a convolutional layer and a pooling layer. By inputting image data, a first output value output from the pooling layer is obtained; By inputting partial image data into the neural network, the output from the pooling layer is and comparing the first output value with the second output value. It is law.

[0028] The number of pixel data items contained in the image data is equal to the number of pixel data items contained in the database image data. It may be the following.

[0029] The image data may be compared to database image data by region-based matching.

[0030] The image data includes a plurality of pixel data, and the pixel data is provided based on the image data. After generating a plurality of image data with different numbers of pixels, the image data is stored as a database image data. It may be compared with the

[0031] The neural network may be trained using database image data. Effect of the Invention

[0032] According to one aspect of the present invention, it is possible to provide an image search system capable of searching for images with high accuracy. Alternatively, according to one aspect of the present invention, it is possible to provide an image search system capable of performing an image search in a short time. Alternatively, according to one aspect of the present invention, a novel image search system can be provided.

[0033] Alternatively, according to one aspect of the present invention, it is possible to provide an image retrieval method capable of retrieving images with high accuracy. Alternatively, according to one aspect of the present invention, it is possible to provide an image search method that can perform image search in a short time. Alternatively, according to one aspect of the present invention, a novel image search method can be provided.

[0034] The description of these effects does not preclude the existence of other effects. However, it is not necessary to have all of these effects. However, it is possible to extract effects other than these. [Brief description of the drawings]

[0035] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an image search system. [Diagram 2] FIG. 2 is a flow chart showing an example of an image search method. [Diagram 3] 3A and 3B are diagrams showing an example of an image search method. [Figure 4] FIG. 4 is a diagram showing an example of an image search method. [Diagram 5] FIG. 5A, as well as FIGS. 5B1 and 5B2, are diagrams showing an example of an image search method. [Figure 6]6A and 6B are diagrams showing an example of an image search method. [Figure 7] 7A and 7B are diagrams showing examples of the configuration of a neural network. [Figure 8] 8A and 8B are diagrams showing an example of an image search method. [Figure 9] FIG. 9 is a diagram showing an example of an image search method. [Figure 10] FIG. 10 is a diagram illustrating an example of the configuration of a neural network. [Figure 11] FIG. 11 is a diagram showing an example of an image search method. [Figure 12] FIG. 12 is a flowchart showing an example of an image search method. [Figure 13] 13A and 13B are diagrams showing an example of an image search method. [Figure 14] FIG. 14 is a diagram showing an example of an image search method. [Figure 15] FIG. 15 is a diagram showing an example of an image search method. [Figure 16] FIG. 16 is a diagram illustrating a configuration of a neural network according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0036] The embodiment will be described in detail with reference to the drawings. However, the present invention is not limited to the following description. The present invention is not limited to the above embodiment, and various changes and modifications may be made in the form and details without departing from the spirit and scope of the present invention. It will be easily understood by those skilled in the art that the above-mentioned invention can be obtained by the following embodiments. It should not be construed as being limited to the contents described.

[0037] In the configuration of the invention described below, the same parts or parts having similar functions are referred to as the same parts. The same reference numerals are used in common among different drawings, and the repeated explanations are omitted. When referring to a function, the hatch pattern may also be the same.

[0038] (Embodiment) In this embodiment, an image search system and an image search method according to one embodiment of the present invention will be described. This will be explained using a surface.

[0039] The image search system according to one aspect of the present invention includes a processing unit and a database. A neural network is provided with neurons. It has a layer.

[0040] In this specification, the term "neural network" refers to a network that imitates the neural circuitry of living organisms and that learns by learning. This refers to a general model in which the strength of connections between neurons is determined by the neural network, thereby giving the brain problem-solving capabilities.

[0041] In this specification, when discussing neural networks, Determining the connection strength (also called weight coefficient) between neurons is called "learning". .

[0042] In the present specification, the neural network is constructed using the connection weights obtained by learning. The process of constructing a theory and deriving a new conclusion from it is called "inference."

[0043] The processing unit receives query image data representing a query image and data stored in a database. The user of the image search system according to one embodiment of the present invention inputs the database image data. The query image is input to the processing unit. Here, the number of pixels of the query image is, for example, the number of pixels of the database image The image search system according to one aspect of the present invention includes a query image and a pixel count of 100 pixels or less. It has the ability to search for database images that contain similar images.

[0044] The processing unit compares, for example, one query image data with a plurality of database image data. Specifically, first, the query image data, the database image data, By comparing the regions of each database image data using region-based matching, The degree of match with the query image data is calculated for each region. The database image data including the extracted image data is extracted as extracted image data. Extract partial image data that is data of an area that has a high degree of matching with the query image data from the do.

[0045] In this specification, one image data refers to image data corresponding to one image, for example. show.

[0046] Next, the query image data and the partial image data are fed to a neural network (NN) of the processing unit. As a result, the output value output from the layer of the neural network is The processing unit can obtain the output from the layer of the neural network. The value indicates, for example, the feature amount of image data input to a neural network.

[0047] After that, the output value when the partial image data is input is calculated for each partial image data. The output values ​​are compared with the above values ​​when the query image data is input. Calculating the similarity of the partial image data to the query image data as a numerical value. Therefore, for each extracted image data, the similarity to the query image data can be calculated. The degree can be calculated as a numerical value.

[0048] As described above, in the image search system according to one embodiment of the present invention, the layers of the neural network are By comparing the output values, the similarity is calculated with high accuracy in a short time. Therefore, for example, the query image and the database image can be compared using region-based matching alone. In the case of comparing images in a database that includes some images similar to the query image, It is possible to perform searches with high accuracy. In addition, for example, the query image and the database image can be searched by using the features. Compared to the case of comparison by base matching only, the image contains some images that are similar to the query image. This allows the database image to be searched in a short time. For example, for each database image, the query image can be calculated by A ranking of matches to the images can be performed.

[0049] <1. Image search system> FIG. 1 is a block diagram showing an example of the configuration of an image search system 10. The image retrieval system 10 shown in FIG. The system includes an input unit 11, a transmission path 12, a storage unit 15, a database 17, and an output unit 19.

[0050] [Input section 11] Image data and the like are supplied to the input unit 11 from outside the image search system 10. The image data and the like supplied to the image processing unit 1 are transmitted via a transmission line 12 to a processing unit 13, a storage unit 15, or a digital This is provided to database 17.

[0051] [Transmission path 12] The transmission path 12 has a function of transmitting image data, etc. The transmission and reception of information between the input unit 15, the database 17, and the output unit 19 is performed via the transmission line 12. It is possible to do so.

[0052] [Processing section 13] The processing unit 13 receives image data supplied from the input unit 11, the storage unit 15, the database 17, etc. The processing unit 13 has a function of performing calculations, inferences, etc. using the neural network. The neural network can be used to perform calculations, inferences, etc. The processing unit 13 can perform calculations without using a neural network. The calculation results, inference results, etc. are supplied to the memory unit 15, the database 17, the output unit 19, etc. This can be done.

[0053] The processing section 13 is preferably made of a transistor having a metal oxide in a channel forming region. Since the off-state current of the transistor is extremely low, the transistor can be used as a memory element. It is used as a switch to hold the charge (data) that has flowed into the capacitive element that functions as a switch. This makes it possible to secure a long data retention period. By using it for at least one of the registers and cache memory of 3, The processing unit 13 is operated only when the In other words, the processing unit 13 can be turned off by switching the processing unit 13 off. This makes it possible to reduce the power consumption of the image search system.

[0054] In this specification, the term "metal oxide" refers to a metal oxide in a broad sense. Metal oxides are oxide insulators and oxide conductors (including transparent oxide conductors). , oxide semiconductors (also called OS), etc. For example, when a metal oxide is used for the semiconductor layer of a transistor, the metal oxide In other words, metal oxides have amplifying, rectifying, and and switching action, the metal oxide is called a metal oxide semiconductor. Metal oxide semiconductor, or OS for short. can.

[0055] In this specification and the like, a transistor using an oxide semiconductor or a metal oxide for a channel formation region Transistors are called oxide semiconductor transistors, or OS transistors. It's called Ninjasta.

[0056] The metal oxide of the channel formation region preferably contains indium (In). When the metal oxide in the panel formation region contains indium, the OS transistor The carrier mobility (electron mobility) of the semiconductor is increased. The oxide is preferably an oxide semiconductor containing an element M. The element M is preferably aluminum. Al, Ga, or Sn, etc. Can be applied to other elements M The elements that can be used are boron (B), silicon (Si), titanium (Ti), iron (Fe), Nickel (Ni), Germanium (Ge), Yttrium (Y), Zirconium (Zr), Molybdenum (Mo), Lanthanum (La), Cerium (Ce), Neodymium (Nd), Hafnium Hf, tantalum (Ta), tungsten (W), etc. However, the element M is In some cases, a combination of the above elements may be used. The element M may be, for example, bonded with oxygen. For example, the bond energy with oxygen is higher than that with indium. In addition, the metal oxide of the channel formation region is a metal containing zinc (Zn). An oxide is preferable. Metal oxides containing zinc may tend to crystallize.

[0057] The metal oxide contained in the channel formation region is not limited to a metal oxide containing indium. The semiconductor layer is preferably made of a non-indium-containing material, such as zinc tin oxide, gallium tin oxide, etc. Metal oxides containing zinc, metal oxides containing gallium, metal oxides containing tin, etc. That's fine.

[0058] The processing unit 13 is, for example, an arithmetic circuit or a central processing unit (CPU: Central Processing Unit). It also has a cessing unit.

[0059] The processing unit 13 includes a DSP (Digital Signal Processor), a GPU, (Graphics Processing Unit) and other microprocessors. The microprocessor may be a field programmable gate array (FPGA). e Gate Array), FPAA(Field Programmable An PLD (Programmable Logic Device) such as The processing unit 13 may be realized by a processor. By interpreting and executing commands from programs, various data processing and program control are performed. The program that can be executed by the processor can be stored in the memory of the processor. The information is stored in at least one of the memory area and the memory unit 15.

[0060] The processing unit 13 may have a main memory. The main memory may be a RAM (Random Access Memory). Volatile memory such as ROM (Read Only Access Memory) and The optical fiber has at least one of a non-volatile memory such as a volatile memory (VRAM) and a non-volatile memory such as a volatile memory (VRAM).

[0061] As an example of RAM, DRAM (Dynamic Random Access Memory) mory), SRAM (Static Random Access Memory), etc. is used, and a virtual memory space is allocated and used as a working space for the processing unit 13. The operating system, application programs, and programs stored in the storage unit 15 The program modules, program data, and lookup tables are used for execution. These data, programs, and programs loaded into RAM are The RAM modules are each directly accessed and operated by the processing unit 13.

[0062] The ROM contains a BIOS (Basic Input / Output) that does not require rewriting. It can store the ROM, ... ROM, OTPROM (One Time Programmable Read Only Memory), EPROM (Erasable Programmable EPROM is a type of memory that can be read and written by ultraviolet light. UV-EPROM (Ultra-Violet) Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmability) e Read Only Memory, flash memory, etc.

[0063] [Storage section 15] The storage unit 15 has a function of storing a program executed by the processing unit 13. 15 is a calculation result and an inference result generated by the processing unit 13 and a calculation result input to the input unit 11. It may also have a function of storing image data, etc.

[0064] The storage unit 15 includes at least one of a volatile memory and a non-volatile memory. The memory unit 15 may include a volatile memory such as a DRAM or an SRAM. For example, ReRAM (Resistive Random Access Memory) ry, also known as resistive memory), PRAM (Phase change Random Access Memory) m Access Memory), FeRAM (Ferroelectric Ran dom Access Memory), MRAM (Magnetoresistive Random Access Memory, also known as magnetoresistive memory, or flash memory The storage unit 15 may have a non-volatile memory such as a hard disk drive. Hard Disc Drive (HDD) and Solid State Drive It has a recording media drive such as a Solid State Drive (SSD). It's fine.

[0065] [Database 17] The database 17 has a function of storing image data. The calculation results and inference results generated by the processing unit 13 and the image data input to the input unit 11 The storage unit 15 and the database 17 may have a function of storing the data. For example, the image search system 10 may include a storage unit 15 and a data The base 17 may have a storage unit that performs both functions.

[0066] In this specification, image data stored in a database or the like is referred to as database image data. An image corresponding to database image data is called a database image.

[0067] [Output section 19] The output unit 19 has a function of supplying information to the outside of the image search system 10. For example, The calculation results or inference results in the logic unit 13 can be supplied to the outside.

[0068] <2. Image search method-1> FIG. 2 is a flowchart showing an example of an image search method using the image search system 10. .

[0069] [Step S01] First, the processing unit 13 receives image data GD and database image data GD DB and is entered .

[0070] Image data GD is input to the input unit 11 from outside the image search system 10. The image data GD is supplied from an input unit 11 to a processing unit 13 via a transmission line 12. The image data GD is stored in the storage unit 15 or the database 17 via the transmission path 12. The data may be supplied from the storage unit 15 or the database 17 to the processing unit 13 via the transmission line 12. stomach.

[0071] Database image data GD DB The data is transmitted from the database 17 to the processing unit via the transmission line 12. 13. Or, the database image data GD DB The transmission line 12 to processing unit 13.

[0072] Here, the image data GD can be called, for example, query image data. The number of pixels of the image corresponding to D can be made equal to or smaller than the number of pixels of the database image. Therefore, the number of pixels in the query image can be equal to or less than the number of pixels in the database image. The search system 10 is a database including images similar to the image corresponding to the image data GD. That is, the image retrieval system 10 can retrieve a source image based on, for example, a query image It is possible to search for database images that contain images similar to the above.

[0073] Image data GD and database image data GD DB Drawings containing intellectual property information Here, the intellectual property information can be, for example, patent documents (publications) These include publications such as patent publications, patent bulletins, utility model bulletins, design bulletins, and papers. Not only publications published in Japan, but also publications published in countries around the world are protected by intellectual property rights. It can be used for information.

[0074] Intellectual property information is not limited to publications. For example, the user or user group of an image search system Various files such as image files that the body has independently are also stored in the image data GD or database. Image data GD DB It can be used as:

[0075] Additionally, intellectual property information may include drawings or the like that illustrate an invention, device or design. These are image data GD and database image data GD DB It is suitable for both .

[0076] The image data GD may be, for example, data corresponding to drawings that explain a pre-filed invention, device, or design. It can be used as a data.

[0077] Database image data GD DB are described, for example, in patent documents of certain applicants. Data representing drawings or drawings described in patent documents in a particular technical field It is possible to have.

[0078] By using the image search system 10, for example, patent documents and papers similar to an invention pending application can be easily searched. This makes it possible to search for prior art related to inventions before filing. By identifying and reviewing the relevant prior art, you can strengthen your invention and This can result in an invention that will become a strong patent that is difficult for the company to circumvent.

[0079] In addition, by using the image search system 10, it is possible to search for images that are similar to, for example, industrial products that have not yet been released. You can search for patent literature, papers, or industrial products. Database Image Data GD DB If the company has data corresponding to the images in its patent documents, it may You can check whether the technology related to the product has been patented in-house. Database image data GD DB Data corresponding to images described in patent documents of other companies In such cases, it is necessary to check whether pre-release industrial products infringe on the intellectual property rights of other companies. By understanding and reviewing related prior art, you can discover new inventions and improve your business. It is possible to make an invention that will become a strong patent that contributes to the development of a new product. You can also search for industrial products after they have been released.

[0080] In addition, for example, the image search system 10 can be used to search for patent documents, papers, and other documents similar to a specific patent. or industrial products. In particular, by searching based on the filing date of the patent, It is possible to easily and accurately investigate whether the patent in question contains grounds for invalidity.

[0081] [Step S02] Next, the processing unit 13 calculates the image data GD and the database image data GD DB Compare with the picture Database image data GD containing areas with high matching to the image data GD DB Extract Here, the extracted database image data GD DB Extracted image data GD Ex Tosu Image data GD and database image data GD DB Comparison with, for example, region-based This can be done by etching.

[0082] An example of the operation of step S02 will be described in detail with reference to FIGS. In S02, as shown in FIG. 3A, the image data GD is divided into n pieces of data (n is an integer equal to or greater than 1). Base image data GD DB where n is the number of The database image data GD DB The number may be equal to or less than the number of Also, n may be the number of database image data GD D B In this case, the number of databases stored in the database 17 may be larger than the number of databases. Image data GD DB In addition to the above, image data and / or image data stored in the storage unit 15 Image data input from outside the search system 10 to the processing unit 13 via the input unit 11; The image data GD is compared with the database image data GD. DB The number is less than However, the image data stored in the storage unit 15 and / or the image search system 10 may be used. The image data inputted from the input unit 11 to the processing unit 13 is compared with the image data GD. You can compare them.

[0083] When n is small, the operation of step S02 can be performed in a short time. In this case, the database image data GD includes an area having a high degree of match with the image data GD. DB It is possible to extract with high accuracy.

[0084] In this specification, in order to distinguish between multiple elements, [1], [2], etc. are written after the reference numerals. For example, in FIG. 3A, the database image data GD DB [1], Database Source image data GD DB [2] Database Image Data GD DB Write [n] and n Database image data GD DB It distinguishes between:

[0085] FIG. 3B shows image data GD and database image data GD DB and area-based matching FIG. 1 is a diagram for explaining a procedure for comparing by a group corresponding to image data GD. The number of pixels of the image to be processed is 2 x 2, and the database image data GD DB The number of pixels in the image corresponding to In other words, the image data GD has 2×2 pixel data, and the database image Data GD DB has 4x4 pixel data.

[0086] In FIG. 3B, the 2×2 pixel data included in the image data GD is denoted by pixel data d For example, image data d11, pixel data d12, pixel data d21, and pixel data d22. In the data GD, pixel data d11 corresponds to the pixel in the first row and first column, and pixel data d21 corresponds to the pixel in the first row and second column. The pixel data corresponding to the pixel in the 2nd row and the 1st column is pixel data d12. The pixel data corresponding to the pixel in the second row and second column is pixel data d21. In addition, the database image data GD DB The 4 × 4 pixel data of These are pixel data ddb11 to ddb44. For example, Data GD DB In this case, the pixel data corresponding to the pixel in the first row and first column is pixel data ddb11. , the pixel data corresponding to the pixel in the 1st row, 4th column is pixel data ddb14, the pixel data corresponding to the pixel in the 4th row, 1st column is pixel data ddb15. The corresponding pixel data is pixel data ddb41, and the pixel data corresponding to the pixel in the 4th row and 4th column is The pixel data is ddb44.

[0087] In this specification, pixel data refers to data that represents the luminance of each pixel. When data is 8 bits, the brightness of each pixel can be expressed in 256 levels. Image data is It can be said that the image includes a set of pixel data, for example, the same number of pixel data as the pixels. For example, if the number of pixels in an image is 2 x 2, the image data representing the image can be It can be said that it contains pixel data of ×2.

[0088] First, pixel data d11, pixel data d12, pixel data d21, and pixel data d22 , pixel data ddb11, pixel data ddb12, pixel data d21, and pixel data ddb22. This allows the image data GD to be compared with the database image data G D DB Among them, pixel data ddb11, pixel data ddb12, pixel data ddb21, and The degree of coincidence between the area formed by the pixel data ddb21 and the area formed by the pixel data ddb22 can be calculated. In FIG. 3B, the database image data GD DB Among the pixel data held by The pixel data to be compared with GD is indicated as a comparison data area 21 surrounded by a dotted line.

[0089] Next, the comparison data area 21 is DB For the pixel data that The pixel data is then moved by one column, and the pixel data is compared in the same manner to calculate the degree of match. pixel data d11, pixel data d12, pixel data d21, and pixel data d22, pixel data ddb12, pixel data ddb13, pixel data ddb22, and pixel data ddb2 3 and . As a result, the image data GD and the database image data GD DB of Among them, pixel data ddb12, pixel data ddb13, pixel data ddb22, and pixel data The degree of match between the region composed of the data ddb23 and the region composed of the data ddb23 can be calculated.

[0090] After that, the comparison data area 21 is DB The pixel data The pixel data is then moved by one column and the degree of match is calculated. Raw data d11, pixel data d12, pixel data d21, and pixel data d22, Data ddb13, pixel data ddb14, pixel data ddb23, and pixel data dd b24. This compares the image data GD with the database image data GD D B Among them, pixel data ddb13, pixel data ddb14, pixel data ddb23, and The degree of coincidence with the region formed from the raw data ddb24 can be calculated.

[0091] Next, the comparison data area 21 is DB For the pixel data that and move it one line, and the database image data GD DB The second row of pixel data in The pixel data of the first row, the pixel data of the second row, and the image data GD are processed in the same manner as above. This allows the database image data GD DB The second and third lines The degree of agreement between the pixel data and the image data GD can be calculated for each column in the same manner as above. do.

[0092] Then, the comparison data area 21 is DB For the pixel data that and move it one line, and the database image data GD DB Of the pixel data held by The pixel data of the 1st row and the pixel data of the 4th row, and the image data GD are processed for each column in the same manner as above. This compares the database image data GD DB The third and fourth lines The degree of coincidence between the pixel data and the image data GD can be calculated for each column in the same manner as above. Cut.

[0093] After performing the above operations, for example, the highest degree of match is selected as the database image data GD DB Picture The above is the degree of match for n database image data GD DB Noso Then, n database image data GD DBFrom the image data Database image data GD with high degree of agreement with data GD DB Extracted image data GD Ex For example, the database image data GD DB In order, the following regulations Extract number of image data GD Ex Alternatively, it may be extracted as a match with the image data GD. Database image data GD with a match rate above the specified value DB Extracted image data GD Ex As It may be extracted.

[0094] In addition, the database image data GD DB This is because the image data does not need to be extracted. Database image data GD compared with GD DB All of the above are extracted as image data GD E x This can be rephrased as follows.

[0095] FIG. 4 shows the database image data GD DB FIG. 4 is a diagram for explaining extraction of the data. Database image data GD DB [1] or database image data GD DB [3] Extract one image data Image data GD Ex This shows the case where the image is extracted as follows.

[0096] The image corresponding to the image data GD shown in FIG. 4 includes, for example, a symbol of a transistor. In addition, the database image data GD shown in FIG. DB Image corresponding to [2] contains a transistor symbol, but the database image data GD DB [1] Corresponding image and database image data GD DBThe image corresponding to [3] has a transistor In this case, the database image data GD DB [2 The degree of match of the image data GD of the database image data GD DB [1] and Database image data GD DB The degree of match is higher than that of the image data GD in [3]. Database image data GD DB [2] is extracted image data GD Ex Extract as It is possible.

[0097] In addition, image data GD and database image data GD DB The comparison and calculation of the degree of agreement are SAD (Sum of Absolute Differences), SSD (Sum of Absolute Differences) of Squared Differences), NCC(Normalized Cross Correlation), ZNCC (Zero-mean Normal ized Cross Correlation), POC (Phase-Only C This can be done by using methods such as orrelation.

[0098] In addition, in FIG. 3B, the comparison data area 21 is the database image data GD DB The pixel data In the above example, the data is moved by one column or one row at a time, but this is not a limitation of the present invention. The comparison data area 21 is stored in the database image data GD DB For the pixel data It is also possible to move the pixel data by two or more columns, or by two or more rows. For example, pixel data d1 1, pixel data d12, pixel data d21, pixel data d22, and pixel data ddb 11. Compare pixel data ddb12, pixel data ddb21, and pixel data ddb22. Immediately after that, pixel data d11, pixel data d12, pixel data d21, and pixel data d22, pixel data ddb13, pixel data ddb14, pixel data ddb23, and In this case, the pixel data d11, the pixel data ddb24, and the pixel data ddb25 may be compared. d12, pixel data d21, and pixel data d22, pixel data ddb12, pixel data No comparison is made with pixel data ddb13, pixel data ddb22, and pixel data ddb23. In addition, pixel data d11, pixel data d12, pixel data d21, and pixel data Data d22, pixel data ddb13, pixel data ddb14, and pixel data ddb23 Immediately after comparing pixel data d11, pixel data d1 2, pixel data d21, pixel data d22, pixel data ddb31, pixel data d db32, pixel data ddb41, and pixel data ddb42 may be compared.

[0099] By increasing the movement range of the comparison data area 21, the pixel data of the image data GD Data and database image data GD DB Reduce the number of comparison operations between the pixel data of This allows the database image data GD DB Image data GD The degree of match can be calculated in a short time.

[0100] In FIG. 3A, one image data GD is divided into n database image data GD DB And each However, one embodiment of the present invention is not limited to this. Based on the image data GD input to the processing unit 13, a plurality of images with different numbers of pixel data are generated. FIG. 5A shows a process for generating image data based on image data GD input to the processing unit 13. The image data GD[1], GD[2], and 5A shows a case where image data GD[3] is generated. The number of pixels of the image corresponding to D[1] and the number of pixels of the image corresponding to the image data GD[2], The number of pixels of the image corresponding to the image data GD[3] is different. The images corresponding to the image data GD[1] to GD[3] are input to the processing unit 13. It can be said that the image corresponding to the data GD is enlarged or reduced.

[0101] When generating multiple image data GD, for each of the multiple image data GD, Database image data GD DB [1] or database image data GD DB [n] This compares the database image data GD DB [1] or database Image data GD DB For each of the multiple image data GD, Then, for example, the degree of coincidence between the plurality of image data GD can be calculated. The highest match score is calculated as the database image data GD DB , input to the processing unit 13 The degree of match with the acquired image data GD can be used as the degree of match.

[0102] For example, in the case shown in FIG. 5A, image data GD[1] is stored in the database image data GD D B [1] or database image data GD DB[n], and compare the image data G D[2] is the database image data GD DB [1] or database image data GD DB [n], and compare the image data GD[3] with the database image data GD DB [ 1] to database image data GD DB [n]. This Database image data GD DB [1] or database image data GD DB [n] that For each, the degree of match for image data GD[1] and the degree of match for image data GD[2] are The degree of match and the degree of match with respect to the image data GD[3] can be calculated.

[0103] For example, the degree of match for image data GD[1], the degree of match for image data GD[2] The highest match score among the match scores and the match scores for the image data GD[3] is calculated based on the database. Image data GD DB The degree of agreement with the image data GD input to the processing unit 13 is For example, the database image data GD DB Image data of [1] GD[1] The degree of match for the image data GD[1], the degree of match for the image data GD[2], and the degree of match for the image data GD[3] The highest match score among the scores is calculated as the database image data GD DB [1] Processing section 1 3 can be regarded as the degree of match with the image data GD inputted in the

[0104] An image corresponding to the image data GD and database image data GD DB and an image corresponding to Even if the same element is shown in the data, if the size of both elements is different, Base image data GD DBdoes not contain areas with a high degree of match to the image data GD. In the case shown in FIG. 5B1, the image corresponding to the image data GD may be determined as , database image data GD DB The image corresponding to the transistor symbol and However, the same elements are shown in the image corresponding to the image data GD. Transistor symbol size and database image data GD DB The corresponding picture shown In this case, the size of the symbol of the transistor in the database image data is different. Data GD DB There is a possibility that the degree of match with the image data GD will be determined to be low.

[0105] On the other hand, in the case shown in FIG. 5B2, an image corresponding to the image data GD and a database image data Data GD DB Both the image corresponding to and the image shown show the same element: a transistor symbol. Furthermore, the size of both elements is equal. Therefore, the database image data GD DB To The processing unit 13 can determine that the image data GD includes an area having a high degree of match. Cut.

[0106] As shown in FIG. 5A, multiple image data GDs with different numbers of pixel data are generated to The size of the elements shown in the image corresponding to the image data GD can be enlarged or reduced. Therefore, the image corresponding to the image data GD input to the processing unit 13 and the database Image data GD DB where the corresponding image is shown in and the same element is shown in but at different sizes Even if the image data is not processed properly, the degree of match between the two image data can be made high. When the image data GD shown in FIG. 5B1 is input to the processing unit 13, the image data GD is By generating the image data GD shown in FIG. , database image data GD DB The degree of agreement with the image data GD is high. This allows the database image data GD DB The input to the processing unit 13 The degree of match with respect to the image data GD can be calculated with high accuracy.

[0107] [Step S03] Next, the extracted image data GD Ex From the data of the area with high degree of agreement with the image data GD, The partial image data GD part The processing unit 13 extracts the above. For example, the method shown in FIG. By this, the database image data GD DB The degree of agreement for each area of ​​the image data GD is calculated. When each of the calculations is performed, the area with the highest degree of matching is selected as the partial image data GD part Extract as Therefore, the partial image data GD part The number of pixel data that the image data GD has is It can be made equal to the number of pixel data that the image sensor has.

[0108] 6A and 6B are diagrams showing an example of the operation of step S03. Extracted image data GD Ex [1] or extracted image data GD Ex In [4], image data The areas with high agreement with GD are shown with hatching. As shown in the figure, the hatched area is extracted and the partial image data GD part [1] Part Image data GD part [4]. In FIG. 6A and FIG. 6B, the extracted image data TaGD Ex [1] or extracted image data GD Ex [4] The image data extracted from Partial image data GD part [1] Partial image data GD part [4] .

[0109] FIG. 6A shows database image data for one image data GD as shown in FIG. GD DB In this case, the partial image data GD part To The corresponding images may all have the same number of pixels.

[0110] FIG. 6B shows the result of the above processing for a plurality of image data GDs with different numbers of pixel data as shown in FIG. 5A. Database image data GD DB In this case, the partial image is compared with the Image data GD part The number of pixels of the image corresponding to the most similar image data G Therefore, the number of pixels of the image corresponding to the partial image data GD pa rt If there are multiple partial image data GD part The number of pixels in the image corresponding to the part Image data GD part In FIG. 6B, the partial image data GD pa rt [1] Partial image data GD part [4] The number of pixels in the images corresponding to each is different. This shows the case where

[0111] In addition, the partial image data GD part In this case, the extraction of the partial image data is not necessary. Data GD part Extract image data GD appropriatelyEx By replacing it with ", the following explanation applies. Alternatively, the extracted image data GD Ex The whole of the partial image data GD part For example, the pixel data included in the image data GD can be expressed as follows: The number is the extracted image data GD Ex If the number of pixel data is equal to the number of pixel data that the partial image data GD part The image retrieval method according to one aspect of the present invention can be carried out without extracting the In addition, the number of pixel data contained in the image data GD and the extracted image data GD Ex has Even if the number of pixel data is different, the partial image data GD part Do not extract In this case, for example, the image data can be used to execute the image search method according to one aspect of the present invention. Data GD and extracted image data GD Ex The pixel data included in one or both of the image data By increasing or decreasing the number, the image search method according to one aspect of the present invention can be performed. Cut.

[0112] [Step S04] Next, the image data GD and the partial image data GD part The neural network of the processing unit 13 Input to the network.

[0113] FIG. 7A shows a neural network 3 which is a neural network included in the processing unit 13. FIG. 1 is a diagram showing an example of the configuration of layer 31[1] to layer 31[ m] (m is an integer of 1 or greater).

[0114] Layers 31[1] to 31[m] each have neurons. For example, the neurons in layer 31[1] are connected to the neurons in layer 31[ 2]. Also, the neurons in layer 31[2] are connected to the The neurons are those in layer 31[1] and those in layer 31[3]. In other words, layers 31[1] to 31[m] are connected to the neurons that are connected to the A hierarchical neural network is constructed.

[0115] Image data GD and partial image data GD part is input to layer 31[1], and layer 31[1 ] outputs data corresponding to the input image data. The data is input to layer 31[2]. The layer 31[2] outputs data corresponding to the input data. The data output from layer 31[m-1] is input to layer 31[m], and the data input to layer 31[m] is In this way, the layer 31[1] is the input layer, and the layers 31[2] to The layer 31[m-1] can be an intermediate layer, and the layer 31[m] can be an output layer.

[0116] In step S04, when the image data GD is input to the neural network 30, The processing unit 13 acquires the values ​​(output values) of the data output from layers 31[1] to 31[m]. In addition, the partial image data GD part When input to neural network 30 The output values ​​output from layers 31[1] to 31[m] are, for example, GD part The processing unit 13 acquires the following information. The output value output from the neural network 30 is, for example, a characteristic of the image data input to the neural network 30. Represents a quantity.

[0117] The neural network 30 includes outputs from layers 31[1] to 31[m], for example. The force values ​​are set to represent the features of the image data input to the neural network 30. It is trained in advance. Training can be done by unsupervised learning, supervised learning, etc. In particular, unsupervised learning is preferable because it does not require training data (also called correct labels). In addition, whether learning is performed by unsupervised or supervised learning, The algorithm may be, for example, a backpropagation method.

[0118] Here, the database image data GD DB It is preferable to use: As a result, for example, the output values ​​output from layers 31[1] to 31[m] are It is possible to accurately represent the feature quantities of the image data input to the network 30. For example, the database image data GD stored in the database 17 DB All of Using the training data, the neural network 30 can be trained. For example, Database image data GD DB A part of the data is used as training data to create a neural network. 30 can perform learning. For example, the database image data GD DB In addition, memory Image data stored in the input unit 15 and image data stored in the input unit 11 from outside the image search system 10 are input. The image data input to the processing unit 13 via the Q30 can study.

[0119] As learning data, the database image data GD DB For example, Image data input from the outside of the image search system 10 to the processing unit 13 via the input unit 11 The neural network 30 can perform learning using only the data as training data.

[0120] The number of pixel data contained in the image data used for learning data and the number of pixel data contained in the neural network 30 It is preferable that the number of pixel data contained in the image data input to the input device is equal to that of the input device. When the neural network 30 performs learning, the database image data used for learning is TaGD DB It is preferable to adjust the number of pixel data by increasing or decreasing the number of pixel data as necessary. In addition, the neural network 30 is provided with image data GD or partial image data GD pa rt When inputting the image data GD or partial image data GD part If necessary, It is preferable to adjust the number of pixel data by increasing or decreasing it according to the needs. The increase in is preferably performed by padding, e.g., by zero padding. It is preferable to do so.

[0121] FIG. 8A shows the database image data GD DB Explains how to adjust the number of pixel data In the case shown in FIG. 8A, the database image data GD DB [1] or Database image data GD DB [4] has a different number of pixel data. In this case, the database image data GD DB [1] or database image data GD DB [4 ] as training data for the neural network 30, the data is Database image data GD DB[1] or database image data GD DB [4] has It is preferable to make the number of pixel data uniform.

[0122] FIG. 8B shows the partial image data GD part The adjustment of the number of pixel data items held by FIG. part is the data stored in database 17 etc. Base image data GD DB In the case shown in FIG. 8B, Minute image data GD part [1] Partial image data GD part [4] has pixel data Even in such a case, the number of pieces of data is different. Image data GD part The number of pixel data that the neural network 30 has is It is preferable to set the number of pixel data equal to the number of pixel data contained in the image data used for the first time. When image data GD is input to the global network 30, the image data GD has The number of pixel data to be acquired is determined based on the number of pixels in the image data used for learning the neural network 30. It is preferable to set it equal to the number of pieces of raw data.

[0123] Neural network 30 is a convolutional neural network (CNN). Fig. 7B shows a neural network. When CNN is applied as neural network 30, FIG. 1 is a diagram showing an example of the configuration of a neural network 30 to which CNN is applied. The neural network is called 30a.

[0124] The neural network 30a includes a convolution layer CL, a pooling layer PL, and a fully connected layer F In FIG. 7B, a neural network 30a has a convolutional layer CL and a pulley Each layer has m layers (m is an integer equal to or greater than 1) of coupling layers PL, and one fully connected layer FCL. The neural network 30a has two or more fully connected layers FCL. It may have.

[0125] The convolution layer CL has the function of performing convolution on the data input to the convolution layer CL. For example, the convolution layer CL[1] has the function of performing convolution on image data. In addition, the convolution layer CL[2] is a convolution layer that receives the data output from the pooling layer PL[1]. The convolution layer CL[m] has the function of performing convolution on the pooling layer P It has the function of performing convolution on the data output from L[m-1].

[0126] Convolution is the product of the data input to the convolution layer CL and the filter value of the weight filter. This is done by repeating sum operations. The neural network 30a extracts image features and the like corresponding to the image data input thereto.

[0127] The convolutional data is transformed by the activation function and then output to the pooling layer PL. The activation function is ReLU (Rectified Linear Uniform Array). ts) can be used. ReLU outputs "0" if the input value is negative, If the input value is "0" or more, the input value is output as is. As the number, a sigmoid function, a tanh function, etc. can also be used.

[0128] The pooling layer PL performs pooling on the data input from the convolution layer CL. Pooling is a method of dividing data into multiple regions and storing a certain amount of data in each region. The process of extracting and arranging the images in a matrix is ​​called pooling. This allows the amount of data to be reduced while retaining the extracted features. This can improve robustness against small deviations in the pooling. Large pooling, average pooling, Lp pooling, etc. can be used.

[0129] The fully connected layer FCL uses the convolutional and pooled image data to make image judgments. The fully connected layer FCL has the function of connecting all the nodes in a layer to all the nodes in the next layer. The data output from the convolution layer CL or the pooling layer PL is The data is a two-dimensional feature map, which is expanded to one dimension when input to the fully connected layer FCL. Then, the data obtained by inference using the fully connected layer FCL is output from the fully connected layer FCL. can be.

[0130] The configuration of the neural network 30a is not limited to the configuration shown in FIG. A neural network layer PL may be provided for each of a plurality of convolution layers CL. The number of pooling layers PL in the filter network 30a is less than the number of convolution layers CL. In addition, if you want to preserve the position information of the extracted features as much as possible, you can use the pooling layer PL. It is not necessary to provide this.

[0131] The neural network 30a learns the filter values ​​of the weighting filters, The weighting coefficients of the coupling layer FCL, etc. can be optimized.

[0132] Next, the convolution process is performed in the convolution layer CL, and the pooling process is performed in the pooling layer PL. An example of the pooling process will be described with reference to FIG. 9. The data input to layer CL is the input data value of 3 rows and 3 columns (input data value i11, input data input data value i12, input data value i13, input data value i21, input data value i22, input data value i23, input data value i31, input data value i32, and input data value i33) In addition, the weight filter is a filter value of 2 rows and 2 columns (filter value f11, filter value Here, for example, the convolution The input data value input to the layer CL[1] may be a grayscale value represented by pixel data. In addition, the above input data values ​​input to the convolution layer CL[2] can be The input to the convolution layer CL[m] can be the output of the convolution layer PC[1]. The data value may be the output value of the pooling layer PC[m-1].

[0133] The convolution is performed by multiplying and adding the input data value and the filter value. The filter value is In this case, the input data can be data showing a predetermined feature (called feature data). By comparing the data value with the filter value, the image input to the neural network 30a is Feature extraction can be performed on the image data.

[0134] In FIG. 9, the convolution layer CL receives input data value i11, input data value i12, and input data value By filtering the input data value i21 and the input data value i22, the convolution layer C The figure shows how to obtain the convolution value C11, which is the output value output from L. The convolution layer CL receives input data value i12, input data value i13, input data value i22, and By filtering the input data value i23, the output from the convolution layer CL is The figure shows how the convolutional value C12, which is the output value to be output, is obtained. is the input data value i21, the input data value i22, the input data value i31, and the input data value By filtering i32, the output value from the convolution layer CL is The figure shows how the convolution layer CL obtains a convolution value C21 from the input data. For input data value i22, input data value i23, input data value i32, and input data value i33, By filtering the output values ​​from the convolution layer CL, The figure shows how the convolution process shown in Figure 9 obtains the convolution value C22. It can be said that the number of bits is 1.

[0135] The convolution value C11, the convolution value C12, the convolution value C21, and the convolution value C22 are These can be obtained by the product-sum calculation shown in the following equation.

[0136] (Number 1) C11=i11·f11+i12·f12+i21·f21+i22·f22 (1)

[0137] (Number 2) C12=i12·f11+i13·f12+i22·f21+i23·f22 (2)

[0138] (Number 3) C21=i21·f11+i22·f12+i31·f21+i32·f22 (3)

[0139] (Number 4) C22=i22·f11+i23·f12+i32·f21+i33·f22 (4)

[0140] The convolutional values ​​C11, C12, C21, and C22 obtained by the convolutional layer CL are The convolution values ​​C22 are arranged in a matrix according to the addresses, and then passed through the pooling layer P L. Specifically, the convolution value C11 is placed in the first row and first column, and the convolution value C 12 is placed in the 1st row, 2nd column, convolution value C21 is placed in the 2nd row, 1st column, convolution value C The number 22 is placed in the second row and second column.

[0141] In FIG. 9, the pooling layer PL is provided with convolutional values ​​C11, C12, and C21. , and the convolution value C22 are input, and one value is pooled based on the four convolution values. For example, the convolution value C11, the convolution value C12, the convolution value P, The maximum value of the value C21 and the convolution value C22 can be set as the pooling value P. Or, the convolution value C11, the convolution value C12, the convolution value C21, and the convolution value C22 The pooling value P can be calculated by the pooling layer PL. This is the output value that is output from the

[0142] In FIG. 9, the data input to the convolution layer CL is processed by one weight filter. However, it may be processed by two or more weight filters. In this case, the neural It is possible to extract multiple features contained in the image data input to the network 30a. When data input to the convolution layer CL is processed by two or more weight filters, The process shown in Figure 9 is performed for each filter. As mentioned above, the stride is set to 1 in Figure 9. However, the stride may be two or more.

[0143] FIG. 10 illustrates a convolution layer CL and a pooling layer P FIG. 10 is a diagram showing an example of the configuration of the convolution layer CL and the pooling layer PL shown in FIG. 4 shows a case where the operation shown in FIG.

[0144] FIG. 10 shows a neuron 32. Specifically, the neuron 32 is In FIG. 10, neuron 32a, neuron 32b, and neuron 32c are shown. The output value output from the neuron 32 is written inside the neuron 32. The output value is output in the direction of the arrow. In addition, when the output value is multiplied by a weighting factor, the output value is output in the direction of the arrow. The weighting coefficients are shown near the marks. In FIG. 10, the filter values ​​f11 and f12 , filter value f21, and filter value f22 are used as weighting coefficients.

[0145] The neuron 32a is a neuron included in the layer L, which is the layer before the convolution layer CL shown in FIG. Layer L is, for example, the convolution layer CL shown in FIG. 10 as convolution layer CL[1]. If it is a convolutional layer CL[2], it can be the input layer, and if it is a pooling layer PL [1], and if the convolutional layer CL[m] is a pooling layer PL[m-1 ] can be used.

[0146] In FIG. 10, the neurons 32a[1] to 32a[ In the case shown in FIG. 10, neuron 32a[1] receives the input data value neuron 32a[1] outputs input data value i11, neuron 32a[2] outputs input data value i12, and neuron 3 Neuron 2a[3] outputs input data value i13, and neuron 32a[4] outputs input data value i2 neuron 32a[5] outputs the input data value i22, and neuron 32a[6] outputs the input data value i32. [6] outputs the input data value i23, and neuron 32a[7] outputs the input data value i31. neuron 32a[8] outputs input data value i32, and neuron 32a[9 ] outputs the input data value i33.

[0147] The neuron 32b is the neuron 32 included in the convolution layer CL shown in FIG. In the example of FIG. 1, the neurons 32b[1] to 32b[4] are This shows that.

[0148] In the case shown in FIG. 10, the neuron 32b[1] has a filter for the input data value i11. The input data value i12 is multiplied by the filter value f11, the input data value i12 is multiplied by the filter value f12, and the input data The input data value i21 multiplied by the filter value f21 and the input data value i22 multiplied by the filter value f22 The sum of these values, the convolution value C11, is input. It is output from terminal 32b[1].

[0149] In addition, the neuron 32b[2] receives the input data value i12 multiplied by the filter value f11. The input data value i13 multiplied by the filter value f12 and the input data value i22 multiplied by the filter value f13 are The input data value i23 is multiplied by the filter value f22, and the input data value i23 is multiplied by the filter value f21. The sum of these values, the convolution value C12, is then output from neuron 32b[2]. It is output from

[0150] In addition, the neuron 32b[3] receives the input data value i21 multiplied by the filter value f11. The input data value i22 multiplied by the filter value f12 and the input data value i31 multiplied by the filter value f12 are The input data value i32 is multiplied by the filter value f21, and the input data value i32 is multiplied by the filter value f22. The sum of these values, the convolution value C21, is then output from neuron 32b[3]. It is output from

[0151] Furthermore, neuron 32b[4] is set to the input data value i22 multiplied by the filter value f11. The input data value i23 multiplied by the filter value f12 and the input data value i32 multiplied by the filter value f13 are The input data value i33 is multiplied by the filter value f21, and the input data value i33 is multiplied by the filter value f22. The sum of these values, the convolution value C22, is then input to neuron 32b[4]. is output from

[0152] As shown in FIG. 10, each of neurons 32b[1] through 32b[4] , is connected to some of neurons 32a[1] to 32a[9]. ,The convolutional layer CL can be said to be a partially connected layer.

[0153] The neuron 32c is the neuron 32 included in the pooling layer PL shown in FIG. In the case shown in FIG. 10, the neuron 32c has the convolution value C11, the convolution value C12, , the convolution value C21, and the convolution value C22 are input. Then, the pooling value P is The convolution value output from the neuron 32b includes As mentioned above, the weighting coefficient is determined by the neural network learning. Therefore, the parameters used by the pooling layer PL during calculation are The data may be configured so that there are no parameters that are optimized by learning.

[0154] [Step S05] Next, when the image data GD is input to the neural network of the processing unit 13, The output value output from the layer of the neural network and the partial image data GD pa rt The output value output from the layer of the neural network when This compares the partial image data GD part Similarity of image data GD Calculate the degree as a number.

[0155] When the processing unit 13 has a neural network 30 having the configuration shown in FIG. 7A, for example, The output values ​​from layers 31[1] to 31[m] are compared. For example, The output values ​​output from layers 31[1] to 31[m] are compared. The output values ​​of only one layer may be compared, or the output values ​​of two or more layers may be compared. By comparing the output values ​​of 31, the partial image data GD part Image data GD It is possible to calculate the similarity between the two with high accuracy.

[0156] In this specification, image data GD is input to a neural network of the processing unit 13. Among the output values ​​output from the layers of the neural network when The output value to be compared is set as the output value VGD. Partial image data GD part When the neural network is input, Among the output values ​​output from the layer, the output value to be compared is called the output value VGD part Let's say That is, in step S05, the output value VGD and the output value VGDpart Compare with.

[0157] In FIG. 7A, the output value VGD and the output value VGD part Let be the output value from layer 31[m]. is.

[0158] When the processing unit 13 has the neural network 30a having the configuration shown in FIG. 7B, for example, The output value output from any one of the pooling layers PL[1] to PL[m] is For example, compare the output values ​​from the pooling layer PL[m]. You may compare the output values ​​output from any of the convolution layers PL[1] to PL[m], The output values ​​output from the fully connected layer FCL may be compared.

[0159] In addition, convolutional layers CL[1] to CL[m], pooling layers PL[1] to PL[m] The output values ​​of only one of the layers, PL[m] and FCL, may be compared. However, output values ​​from two or more layers may be compared. By comparing output values ​​from many layers, Image data GD part It is possible to calculate the similarity of the image data GD with high accuracy. can.

[0160] In FIG. 7B, the output value VGD and the output value VGD part from the pooling layer PL[m] This is the output value.

[0161] Figure 11 shows a pooling layer that outputs pooling value P of x rows and y columns (x and y are integers equal to or greater than 1). The output value from PL is the output value VGD and the output value VGD part This shows the case where: In the case shown in FIG. 11, image data GD and k pieces of partial image data (k is an integer equal to or greater than 1) are GD partand are input to the neural network 30a.

[0162] In this specification, for example, k pieces of partial image data GD part The output value VGD corresponds to part VGD part [1] to output value VGD part [k] The distinction is made as follows.

[0163] In FIG. 11, the output value VGD is the pooling value P GD [1,1] to pooling value P G D [x, y]. For example, the output value VGD part [1] is Pullin Value P1 part [1,1] to pooling value P1 part [x,y] and the output value V GD part [2] is the pooling value P2 part [1,1] to pooling value P2 par t [x,y] and the output value VGD part [k] is the pooling value Pk part [1,1 ] to pooling value Pk part Let us say that we have [x,y].

[0164] In the case shown in FIG. 11, the output value VGD is part [1] to output value VGD p art [k]. This results in k partial image data GD part of For each of them, the similarity to the image data GD is calculated. For example, the cosine similarity Alternatively, Euclidean similarity, Minkowski similarity, etc. may be calculated. From the above, a database image that includes an image similar to the image corresponding to the image data GD is created. can be searched for.

[0165] As described above, in the image search system 10, the neural network provided in the processing unit 13 By comparing the output values ​​of the layers, the similarity can be calculated with high accuracy in a short time. Therefore, for example, image data GD and database image data GD DB and the area Compared with the case where the comparison is performed by base matching alone, the image data GD is more similar to the corresponding image. It is possible to search for database images that contain the image with high accuracy. For example, image data GD and database image data GD DB and by feature-based matching only When comparing the image data GD, it is possible to select the image data GD that includes an image similar to the image data GD. The base image can be searched for in a short time. Furthermore, the similarity is calculated as a numerical value. Therefore, for example, in each database image, the image data GD A ranking of the degree of match for the images corresponding to the images can be performed.

[0166] In the image search system 10, image data GD and database image data GD DB and The regions with high matching to the image data GD are compared using region-based matching, etc. Database image data GD DB Extracted image data GD Ex It is extracted as Then, the extracted image data GD Ex Partial image data GD part Extract the image data G D and partial image data GD part is input to the neural network of the processing unit 13. In this way, the database image data GD DB By extracting the Data representing database images that do not contain images that are highly similar to the images corresponding to data GD Base image data GD DB is input to the neural network of the processing unit 13. Therefore, it is possible to suppress the occurrence of an image similar to the image corresponding to the image data GD. It is possible to search for database images that include the image in a short time with high accuracy. Database image data GD to compare with data GD DB If the number of databases is small, Source image data GD DB The above search can be performed with high accuracy and in a short time without the need for extraction. can be done.

[0167] <3. Image search method-2> In the above, the number of pixel data included in the image data GD input to the processing unit 13 is equal to the number of pixel data included in the data base. Source image data GD DB The number of pixel data pieces in the image of one embodiment of the present invention is equal to or less than the number of pixel data pieces in the image of one embodiment of the present invention. Although the image search method has been described above, the present invention is not limited to this. The number of pixel data contained in the image data GD is DB The pixels In this case, the image retrieval system 10 may, for example, It is possible to search for database images that are similar to the part.

[0168] FIG. 12 shows the number of pixel data items included in the image data GD. DB retrieval using image retrieval system 10 when the number of pixel data in the image is greater than the number of pixel data in the image retrieval system 10 1 is a flow chart illustrating an example of a method.

[0169] [Step S11] First, similarly to step S01 shown in FIG. 2, image data GD and database image data G D DB are input to the processing unit 13.

[0170] [Step S12] Next, the processing unit 13 calculates the image data GD and the database image data GD DB Compare with the picture Database image data GD that matches a portion of the image data GD DB A, Extract image Data GD Ex Image data GD and database image data GD DB With The comparison can be performed, for example, by region-based matching, as in step S02.

[0171] An example of the operation of step S12 will be described in detail with reference to FIGS. In step S12, the image data GD is converted into n database image data as shown in FIG. 13A. TaGD DB Compare with each of the above.

[0172] FIG. 13B shows image data GD and database image data GD DB and region-based match FIG. 1 is a diagram for explaining a procedure for comparing image data GD by filtering. The number of pixels of the corresponding image is 4 x 4, and the database image data GD DB The number of pixels in the image corresponding to In other words, the image data GD has 4×4 pixel data, and the database image Image data GD DB has 2x2 pixel data.

[0173] In FIG. 13B, the 4×4 pixel data included in the image data GD is For example, in the image data GD, the pixel data in the first row and first column The pixel data corresponding to the pixel in the first row and fourth column is pixel data d11. The pixel data corresponding to the pixel in row 4, column 1 of pixel data d14 is d41, and the pixel data corresponding to the pixel in row 4, column 4 is d42. The pixel data corresponding to the pixel in the th column is pixel data d44. Data GD DB The 2×2 pixel data of For example, the pixel data ddb12, pixel data ddb21, and pixel data ddb22. Source image data GD DB In this case, the pixel data corresponding to the pixel in the first row and first column is ddb11, pixel data corresponding to the pixel in row 1, column 2, ddb12, pixel data corresponding to the pixel in row 2, column 1 The pixel data corresponding to the first pixel is pixel data ddb21, and the pixel data corresponding to the second row, second column is pixel data ddb22. The raw data is pixel data ddb22.

[0174] First, pixel data ddb11, pixel data ddb12, pixel data ddb21, and pixel Data ddb22, pixel data d11, pixel data d12, pixel data d21, and pixel data This compares the raw data d22 with the database image data GD DB And, image Among the data GD, pixel data d11, pixel data d12, pixel data d21, and pixel data The degree of coincidence between the area composed of the data d22 and the area composed of the data d23 can be calculated. Among the pixel data contained in the image data GD, the database image data GD DB Compare The pixel data to be compared is shown surrounded by a dotted line as a comparison data area 21.

[0175] Next, the comparison data area 21 is moved by one column relative to the pixel data contained in the image data GD. , the pixel data is compared in the same manner, and the degree of match is calculated. Pixel data ddb12, pixel data ddb21, pixel data ddb22, and pixel data The pixel data d12, pixel data d13, pixel data d22, and pixel data d23 are compared. This allows the database image data GD DB and pixel data d of the image data GD 12, an area consisting of pixel data d13, pixel data d22, and pixel data d23 The degree of agreement between and can be calculated.

[0176] After that, the comparison data area 21 is moved by one column relative to the pixel data contained in the image data GD. The pixel data is then compared in the same manner to calculate the degree of match. 1, pixel data ddb12, pixel data ddb21, pixel data ddb22, and pixel Compare data d13, pixel data d14, pixel data d23, and pixel data d24. This allows the database image data GD DB The pixel data in the image data GD The pixel data d13, pixel data d14, pixel data d23, and pixel data d24 are included. The degree of match with the region can be calculated.

[0177] Next, the comparison data area 21 is moved by one line relative to the pixel data contained in the image data GD. Among the pixel data contained in the image data GD, the pixel data in the second row and the pixel data in the third row Data and database image data GD DB and are compared column by column in the same way as above. Therefore, the pixel data of the second and third rows of the image data GD and the database image data Data GDDB The degree of agreement with can be calculated for each column in the same manner as above.

[0178] Then, the comparison data area 21 is moved by one line relative to the pixel data contained in the image data GD. Among the pixel data contained in the image data GD, the pixel data in the third row and the pixel data in the fourth row Data and database image data GD DB and are compared column by column in the same way as above. Thus, the pixel data of the third and fourth rows of the image data GD and the database image Data GD DB The degree of agreement with can be calculated for each column in the same manner as above.

[0179] After performing the above operations, for example, the highest degree of match is selected as the database image data GD DB Picture The above is the degree of match for n database image data GD DB Noso Then, similarly to step S02, the n database image data are GD DB Among them, the database image data GD that has a high degree of matching with the image data GD is selected. DB of , extracted image data GD Ex As in step S02, the database is Image data GD DB It is not necessary to extract the above.

[0180] FIG. 14 shows the database image data GD DB FIG. 14 is a diagram for explaining extraction of the , database image data GD DB [1] or database image data GD DB [3] or Then, extract one image data GD Ex This shows the case where the image is extracted as follows.

[0181] The image corresponding to the image data GD shown in FIG. 14 includes, for example, a symbol of a transistor and a capacitor. The symbol of the element is included. Also, the database image data GD shown in FIG. DB The image corresponding to [2] contains a transistor symbol, but the database image Data GD DB [1] Images corresponding to the database image data GD DB [3] The corresponding image does not include any transistor or capacitor symbols. In this case, the database image data GD DB [2] for the image data GD The degree of match is calculated based on the database image data GD DB [1] and database image data GD D B [3] is higher than the match rate for the image data GD. TaGD DB [2] is extracted image data GD Ex It can be extracted as:

[0182] In addition, image data GD and database image data GD DB The comparison and calculation of the degree of agreement are The same method as that used in step S02 can be used. In 13B, the comparison data area is set to one column or one pixel of the pixel data of the image data GD. In this case, the comparison data area 21 is moved by one line at a time, but in the same manner as in step S02, The pixel data in the GD may be shifted by two or more columns, or by two or more rows. Furthermore, similarly to the case shown in FIG. 5A, the image data GD input to the processing unit 13 may be Based on this, multiple image data GD with different numbers of pixel data may be generated.

[0183] [Step S13] Next, the extracted image data GD Ex Data in areas with high agreement with The partial image data GD part-q For example, the processing unit 13 extracts the In this way, the database image data GD of each area of ​​the image data GD is DB Degree of agreement with When each of the above is calculated, the area with the highest degree of match is the partial image data GD part-q year Therefore, the partial image data GD part-q The number of pixel data that Image data GD Ex can be made equal to the number of pixel data that the image sensor has.

[0184] FIG. 15 is a diagram showing an example of the operation of step S13. The image data GD shown in FIG. , which corresponds to a circuit diagram with a transistor symbol in the upper left and a capacitor symbol in the lower right. The extracted image data GD Ex The image corresponding to [1] is a transistor. Extracted image data GD Ex [2] The image corresponding to this one contains the symbol for the capacitive element. Let us assume that.

[0185] In the case shown in FIG. 15, the upper left part of the image corresponding to the image data GD is the extracted image data G D Ex This is the area with the highest degree of agreement with [1]. Therefore, The data corresponding to the upper left area is the partial image data GD part-q [1] On the other hand, The lower right part of the image corresponding to the image data GD is the extracted image data GD Ex [2] This is the area with the highest degree of match. Therefore, the area corresponding to the lower right area of ​​the image data GD is Data is partial image data GD part-q [2]. In other words, one image data GD Multiple partial image data GD part-q has been extracted.

[0186] In step S13, the extracted image data GD Ex The same number of image data are stored as image data G Partial image data GD from D part-q Alternatively, the extracted image data G D Ex A smaller number of image data is divided into partial image data GD and partial image data GD. part- q For example, multiple extracted image data GD Ex The degree of agreement is high If the area of ​​the image data GD that is not the same, the image data GD is Partial image data GD extracted from GD part-q The number of pieces can be one. That is, the same partial image data GD part-q You need to extract multiple images from the image data GD. Good too.

[0187] In addition, the partial image data GD part-q In this case, it is not necessary to extract the partial image. Image data GD part-q The following explanation applies by replacing with image data GD as appropriate. Alternatively, the entire image data GD can be converted into partial image data GD. part-q Tosu For example, the number of pixel data included in the image data GD is , extracted image data GD Ex If the number of pixel data is equal to the number of pixel data in the partial image data GD part-q The image retrieval method according to one aspect of the present invention can be carried out without extracting the In addition, the number of pixel data contained in the image data GD and the extracted image data GD Ex has Even if the number of pixel data is different, the partial image data GD part-q Extraction of In this case, for example, the image search method according to the present invention can be performed without using the image search function. Image data GD and extracted image data GD Ex The pixel data included in one or both of the image data By increasing or decreasing the number of data, the image search method according to one aspect of the present invention can be performed. can be done.

[0188] [Step S14] Next, the partial image data GD part-q and extracted image data GD Ex The processing unit 13 The neural network is shown in FIG. The configuration shown in B can be used.

[0189] The operation in step S14 is to convert the image data GD into partial image data GD part-q and partial image data GD part Extract image data GD Ex etc. to be read as For details on step S04, please refer to the explanation of step S04. Image data GD Ex and partial image data GD part The partial image data GD par t-q It may also be read as:

[0190] [Step S15] Next, the extracted image data GD Ex Entered In this case, the output value output from the layer of the neural network and the partial image data GD part-q When input is given, the output from the layer of the neural network is The output value is compared with the extracted image data GD Ex Partial image data GD part-q The similarity to is calculated as a numerical value.

[0191] Specifically, the extracted image data GD Ex The extracted image data GD Ex The corresponding partial image Image data GD part-q For example, as shown in FIG. Image data GD and extracted image data GD Ex [1] is compared to the image data GD and the partial image data Data GD part-q [1] is extracted, and the image data GD and the extracted image data GD Ex [2] Compare the image data GD to the partial image data GD part-q Suppose we extract [2] In this case, for example, the extracted image data GD Ex [1] Partial image data GD part-q [1]. Also, the similarity to the extracted image data GD Ex [2] Partial image data Data GD part-q Calculate the similarity to [2].

[0192] When the processing unit 13 has the neural network 30 configured as shown in FIG. 7A, step S Similarly to 05, for example, the output value output from any one of layers 31[1] to 31[m] For example, the output values ​​from layer 31[m] are compared. ] to layer 31[m], the output value of only one layer may be compared, or the output values ​​of two or more layers may be compared. Comparisons may be made.

[0193] When the processing unit 13 has the neural network 30a having the configuration shown in FIG. 7B, step As in S05, for example, any one of the pooling layers PL[1] to PL[m] For example, compare the output values ​​from the pooling layer PL[m]. The output from any of the convolutional layers PL[1] to PL[m] is The force values ​​may be compared, or the output values ​​output from the fully connected layer FCL may be compared.

[0194] In the present specification, the output values ​​to be compared are those of the neural network of the processing unit 13. Partial image data GD part-q When you enter The output value output from the layer that the block has is the output value VGD part-q In addition, the above comparison Among the output values, the processing unit 13 outputs extracted image data GD Ex The output value output from the layer of the neural network when Force value VGD Ex That is, in step S15, the output value VGD part-q and output ValueVGD Ex Compare with.

[0195] The operation in step S15 is to appropriately adjust the output value VGD part-q Read as Eh, output value VGD part The output value VGD Ex By replacing it with "Step S0" 5. By performing step S15, the image data GD It is possible to search for database images that are similar to parts of the corresponding image.

[0196] As described above, in the image search system 10, the neural network provided in the processing unit 13 By comparing the output values ​​of the layers of the network, the extracted image data can be extracted with high accuracy in a short time. TaGD Ex Partial image data GD part-q It is possible to calculate the similarity to Therefore, for example, image data GD and database image data GD DB and region-based match Compared with the case where only the image data GD is compared, the image data GD is similar to the part of the image GD. In addition, for example, image data GD and database images can be searched with high accuracy. Database image data GD DB Compared with the case where only feature-based matching is used, It is possible to quickly search for database images similar to a part of an image corresponding to the image data GD. Furthermore, since the similarity can be calculated as a numerical value, for example, For each database image, the order of matching with the image corresponding to the image data GD is Ranking can be done.

[0197] In the image search system 10, image data GD and database image data GD DB and The data with the highest degree of match to a part of the image data GD are compared using region-based matching, etc. Database image data GD DB Extracted image data GD Ex Then, , image data GD to partial image data GD part-q Extract the partial image data GD p art-q and extracted image data GD Ex The neural network of the processing unit 13 In this way, the database image data GD DBBy extracting A database image that does not include an image that is highly similar to a part of the image corresponding to the image data GD is Represents database image data GD DB The neural network of the processing unit 13 Therefore, it is possible to prevent a part of the image corresponding to the image data GD from being input. It is possible to search for database images similar to the image with high accuracy and in a short time. Database image data GD to compare with data GD DB If the number of databases is small, Source image data GD DB The above search can be performed with high accuracy and in a short time without the need for extraction. can be done. EXAMPLES

[0198] In this embodiment, an image is input to an image search system, and data including parts similar to the image is searched. The results of searching for the base image will now be described.

[0199] In this embodiment, one image is input to the image search system. We searched for 100 database images containing the part under conditions 1 and 2. In both conditions, the images entered into the image retrieval system were circuit diagrams, and the images were extracted from the database. Circuit diagrams, graphs, etc. were prepared as images.

[0200] In condition 1, the image data GD corresponding to the image input to the image search system and the database Image data GD DB and are compared using region-based matching, and the top 10 matches are selected. 0 database images searched.

[0201] In condition 2, the method shown in FIG. 2 of the embodiment is used to search for images similar to those input to the image search system. We searched for database images that contain similar parts. Specifically, we first searched for image data GD and Database image data GD DB and are compared using region-based matching, and the highest degree of match is 400 database image data GD DB Extract image data GD Ex Extracted as Next, 400 extracted image data GD Ex For each of the image data GD, The data in the area with the highest degree of matching is the partial image data GD part Next, As shown in FIG. 16, convolutional layers CL[1] to CL[5] and pooling layer P L[1] to the pooling layer PL[5], and the fully connected layer FCL[1] to the fully connected layer FCL[ 3], the neural network is composed of image data GD and partial image data G D part Next, when image data GD is input to the neural network, In this case, the output value output from the pooling layer PL[5] is the output value VGD, and the partial image data GD part The output value output from the pooling layer PL[5] when part As the output value VGD and the output value VGD part The similarity was calculated by comparing the results. Then, the 100 partial image data GDs are sorted in order of similarity to the image data GD. par t The extracted partial image data GD part A database image containing was searched.

[0202] In condition 1, of the 100 database images retrieved, only 6 contained circuit diagrams. On the other hand, in condition 2, 30 of the 100 database images were circuit In Condition 2, the image was similar to the image entered into the image search system. It was confirmed that it was possible to search for more database images containing the desired part than in condition 1. It was. [Explanation of symbols]

[0203] 10: image search system, 11: input unit, 12: transmission path, 13: processing unit, 15: memory unit, 17: database, 19: output section, 21: comparison data area, 30: neural network 30a: neural network, 31: layer, 32: neuron, 32a: neuron Ron, 32b: Neuron, 32c: Neuron

Claims

[Claim 1] A processing unit is provided. the processing unit has a function of comparing the image data with the database image data when the image data and a plurality of database image data are input to the processing unit; the processing unit has a function of extracting, as extracted image data, the database image data including an area having a high degree of matching with the image data; The processing unit has a function of extracting data of an area having a high degree of matching with the image data from the extracted image data as partial image data, The processing unit is an image search system having a function of calculating a similarity of the partial image data with respect to the image data.

Images