Retrieval device, retrieval method, and information processing system

Abstract

To provide a highly user-friendly retrieval device, a retrieval method, and an information processing system that can retrieve image data based on three-dimensional information of a retrieval target.SOLUTION: A retrieval device 1 outputs retrieval results in response to retrieval requests, and comprises: an image data registration unit 12 which vectorizes features based on characteristics of image data for registering the same in a retrieval database 13; and an image retrieval unit 16 which vectorizes features based on characteristics extracted from three-dimensional information of a retrieval target 2 for generating a retrieval request, and retrieves image data in the retrieval database using the generated retrieval request.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a search device, a search method, and an information processing system.

Background Art

[0002] For example, in a company that has manufacturing equipment such as machine tools, it may be necessary to search for drawings similar to those shown by a customer from among the drawings it owns, and create an estimate based on the found drawings. For this reason, a device for searching for drawings similar to those shown by a customer has been proposed (Patent Document 1). Furthermore, a device has also been proposed that iconifies a component drawing area corresponding to a keyword specified by a user and arranges it in a virtual space to provide the user with a clue for searching (Patent Document 2).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the technology described in Patent Document 1, although it is possible to search for in-house drawings similar to a solid or planar drawing shown by a customer, it is not compatible with data other than drawings, so the usability is poor. In the technology described in Patent Document 2, it is necessary for the user to specify an appropriate key image, and the usability is poor.

[0005] The present invention has been made in view of the above problems, and an object thereof is to provide a search device, a search method, and an information processing system that can search for image data based on three-dimensional information of a search target and have high usability.

Means for Solving the Problems

[0006] To solve the above problems, a search device according to one aspect of the present invention is a search device that outputs search results according to a search request, and includes an image data registration unit that vectorizes a feature amount based on the features of image data and registers it in a search database, and an image search unit that vectorizes a feature amount based on the features extracted from the three-dimensional information of the object to be searched, generates a search request, and searches the search database using the generated search request.

Effects of the Invention

[0007] According to the present invention, it is possible to search for image data registered in a search database based on the three-dimensional information of the object to be searched.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Modes for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this embodiment, as an example of "image data", drawing data such as a plan view or a three-dimensional view is cited. Feature amounts based on the features of the drawing data are calculated, vectorized, and registered in a search database. The object to be searched is an object that is searching for drawing data similar to its own shape. The object is, for example, various products or product models. The product may be a prototype, a special product, or a mass-produced product. The product model is, for example, an appearance model, a miniature, a sample, etc. The product includes finished products and parts. For example, three-dimensional information of the object to be searched is acquired using a three-dimensional information acquisition device such as SfM (Structure From Motion), MVS (Multi-View Stereo), LiDAR (Light Detection And Ranging). Features are extracted from the acquired three-dimensional information, feature amounts are calculated, and the calculated feature amounts are vectorized. By searching the search database using a search request based on the three-dimensional information of the object to be searched, drawing data similar to the object to be searched is retrieved.

[0010] The device for acquiring three-dimensional information of the object (object to be searched) may be a portable device or a non-portable device installed in a facility such as a factory or a laboratory.

[0011] According to the search device of this embodiment, feature quantities based on the features of drawing data (image data) are vectorized in advance and registered in a search database. When searching for drawing data related to an object to be searched, three-dimensional information is acquired for the object to be searched itself or for a model of the object to be searched. Then, the search database is searched using a search request based on the acquired three-dimensional information. Therefore, even when there is no drawing data of the object to be searched or there is only a model of the object to be searched, it is possible to search for drawing data similar to the object to be searched. Furthermore, by associating a manufacturing device capable of manufacturing the object shown in the drawing data with the drawing data, it is possible to easily identify a manufacturing device capable of manufacturing the object to be searched, and it is also possible to calculate an estimate when manufacturing an object (product) related to the object to be searched using the identified manufacturing device.

Example

[0012] The first embodiment will be described with reference to FIGS. 1 to 7. FIG. 1 is an overall configuration diagram of a search system 1 as an example of a "search device". In this embodiment, a case where drawing data registered in a search database in advance is searched by a search request based on three-dimensional information of an object to be searched will be described. Hereinafter, although the user terminal U1 used by the user who registers the drawing data in the search system 1 and the user terminal U2 used by the user who searches for drawing data similar to the object to be searched will be described as if they were separate terminals, the user terminal U1 and the user terminal U2 may be the same. That is, the user who registers the drawing data and the user who searches for drawing data similar to the object to be searched may belong to the same organization. Hereinafter, the user terminal U1 that registers the drawing data in the search system 1 may be referred to as the registration user terminal U1, and the user terminal U2 that is used to search for the registered drawing data may be referred to as the search U2.

[0013] The search system 1 includes, for example, an image data collection unit 11, an image data registration unit 12, an index database 13, a project information database 14, a three-dimensional information reception unit 15, and an image search unit 16. The image data collection unit 11 and the image data registration unit 12 are responsible for the indexing process of the drawing data. The three-dimensional information reception unit 15 and the image search unit 16 are responsible for the search process of searching the indexed drawing data.

[0014] The image data collection unit 11 collects drawing data as "image data". The image data collection unit 11 collects drawing data as image data from the web server 111 or the file server 112. The registrant of the drawing data can send the drawing data to the image data collection unit 11 through the web server 111 using the registration user terminal U1. For example, the registrant may input image data obtained by photographing the drawing data from the web server 111 to the image data collection unit 11. The drawing data may be input to the image data collection unit 11 via a file server 112 connected to a CAD (Computer Aided Design) system (not shown), for example.

[0015] The image data registration unit 12 extracts the feature amounts of the image data collected by the image data collection unit 11 by the feature point extraction unit 121, vectorizes the extracted feature amounts by the vectorization processing unit 122, and registers them in the index database 13. The image data registration unit 12 registers the drawing data, the information of the manufacturing apparatus capable of manufacturing the object shown in the drawing data, the parts list, the project information including the cost list, etc. in the project information database 14.

[0016] The index database 13 is an example of a "search database". As described above, in the index database 13, the drawing data DW (see FIG. 3) of products (objects to be searched, objects) that can be manufactured by the manufacturing apparatus 3 (see FIG. 3) is indexed and stored based on the feature amounts. The project information database 14 stores information regarding the manufacture and sale of the products shown by the drawing data. Examples of the information regarding the manufacture and sale of the products include a parts list, a material list, a cost list, a selling price, a delivery date, and the like.

[0017] The three-dimensional information reception unit 15 receives the three-dimensional information of the object to be searched 2 as an image serving as a basis for the search. The three-dimensional information of the object to be searched 2 can be generated using an apparatus that acquires the three-dimensional information of the object. Technologies used in such an apparatus include, for example, SfM (Structure From Motion), MVS (Multi-View Stereo), LiDAR (Light Detection And Ranging), and the like.

[0018] The apparatus that acquires the three-dimensional information of the object may be built in the search user terminal U2 or may be connected to the search user terminal U2. The three-dimensional information of the object to be searched 2 can be input from the user terminal U2 to the three-dimensional information reception unit 15. Alternatively, when an apparatus for creating the three-dimensional information of the object is provided at the origin of the object to be searched 2 (for example, the end user for the user), the three-dimensional information of the object to be searched 2 is acquired at the origin of the object to be searched 2, the acquired three-dimensional information is fixed to a storage medium, brought into the search system 1, and input to the three-dimensional information reception unit 15. As in the embodiments described later, the three-dimensional information of the object to be searched 2 can also be input to the three-dimensional information reception unit 15 via the Internet.

[0019] The image search unit 16 generates a search request based on the three-dimensional information of the object to be searched 2, searches the index database 13, and extracts drawing data similar to the object to be searched 2. The image search unit 16 includes, for example, a 3D preprocessing unit 161, an image preprocessing unit 162, and a search unit 163.

[0020] The 3D preprocessing unit 161 performs preprocessing of three-dimensional information. The 3D preprocessing unit 161 includes, for example, an opaque processing unit 1611 and a multi-viewpoint processing unit 1612. Hereinafter, the opaque processing unit 1611 may be abbreviated as opaque 1611, and the multi-viewpoint processing unit 1612 may be abbreviated as multi-viewpoint 1612.

[0021] The opaque processing unit 1611 performs opaque processing so that only the surface viewed from the viewpoint among the three-dimensional information of the search target object 2 is displayed. That is, in the three-dimensional information subjected to the opaque processing, the point groups inside and on the back side cannot be seen.

[0022] The multi-viewpoint processing unit 1612 generates images when the three-dimensional information subjected to the opaque processing is viewed from a plurality of viewpoints. Examples of the processing of the opaque processing unit 1611 and examples of the processing of the multi-viewpoint processing unit 1612 will be described later.

[0023] The image preprocessing unit 162 preprocesses the image data processed by the 3D preprocessing unit 161 in order to generate a search request. The image preprocessing includes, for example, a process of extracting feature points and a process of vectorizing the extracted feature points. By extracting feature points from the three-dimensional information of the search target object 2 and performing vectorization processing on those feature points, it is possible to search for the vectorized drawing data in the index database 13.

[0024] The search unit 163 generates a search request based on the vectorized data and searches the index database 13. The search result of the drawing data is transmitted to the search user terminal U2 together with the related information read from the project information database 14.

[0025] FIG. 2 is a hardware configuration diagram of the search system 1. The search system 1 can be generated using one computer 1000. Alternatively, the search system 1 can also be generated by coordinating a plurality of computers.

[0026] The computer 1000 includes, for example, a processor 1100, a memory 1200, an auxiliary storage device 1300, a communication unit 1400, and a user interface unit 1500. In the figure, the user interface is abbreviated as UI.

[0027] By executing a computer program stored in the auxiliary storage device 1300 by the processor 1100, each function of the search system 1 is realized. The processor 1100 may include not only a CPU (Central Processing Unit) but also a GPU (Graphics Processing Unit) or an ASIC (Application Specific Integrated Circuit), etc.

[0028] The memory 1200 includes a RAM (Random Access Memory) and a ROM (Read Only Memory).

[0029] The communication unit 1400 is a device for communicating with user terminals U1, U2 and other systems (not shown) via a communication network CN. Other systems (not shown) include, for example, a design support system, a production management system, a sales management system, a quotation creation system, a document management system, etc. The type of the communication network CN is not limited. The communication network CN may be a public line or a dedicated line.

[0030] The user interface unit 1500 is a device for the user of the search system 1 (the registrant of the drawing data, the searcher of the drawing data) and the search system 1 to exchange information. The user interface unit 1500 can include an information input device and an information output device (both not shown). Examples of the information input device include a keyboard, a voice input device, a pointing device, and a touch panel. Examples of the information output device include a monitor display, a printer, and a synthesized voice output device. The user and the search system 1 may exchange information using technologies such as Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), and Extended Reality (XR).

[0031] The auxiliary storage device 1300 is a relatively large-capacity storage device such as a semiconductor memory device, a hard disk device, an optical disk device, or a magnetic tape device. The auxiliary storage device 1300 stores a predetermined computer program for realizing each function such as the image data collection unit 11, the image data registration unit 12, the three-dimensional information reception unit 15, and the image search unit 16. The auxiliary storage device 1300 can also store the index database 13 and the project information database 14. These databases 13 and 14 may be arranged in a computer (not shown).

[0032] The search system 1 can be connected to a storage medium MM. The storage medium MM is configured, for example, as a memory device, a hard disk device, an optical disk device, a magneto-optical disk device, a magnetic tape device, etc., and stores computer programs and data non-temporarily. The storage medium MM can transfer and store computer programs and data to the auxiliary storage device 1300 of the search system 1. It is also possible to transfer and store computer programs and data from the auxiliary storage device 1300 to the storage medium MM. By storing a computer program that realizes the functions of the search system 1 in the storage medium MM, connecting the storage medium MM to another computer, and installing the computer program stored in the storage medium MM on the other computer, the other computer can be made to function as the search system 1.

[0033] FIG. 3 is a conceptual diagram showing a state in which various search requests are input to the search system 1 and drawing data similar to the search target 2 is searched for.

[0034] As described above, drawing data DW of products that can be manufactured by the manufacturing apparatus 3 is registered in advance in the index database 13.

[0035] The manufacturing apparatus 3 is, for example, a processing apparatus or a production apparatus such as an injection molding machine, an NC machine tool, or an electric discharge machine. The manufacturing apparatus 3 is used, for example, to manufacture finished products such as automobiles, trains, airplanes, ships, robots, home appliances, transportation equipment, or their parts. The industry in which the manufacturing apparatus 3 is used is not limited. When not particularly distinguishing between the plurality of manufacturing apparatuses 3(1) to 3(n) shown in the figure, they are collectively referred to as the manufacturing apparatus 3.

[0036] The three-dimensional information reception unit 15 receives, as the three-dimensional information of the object to be searched 2, drawings on paper media, digitized drawings, samples such as samples and models, photographs, point cloud data, etc. The image search unit 16 extracts feature amounts from the three-dimensional information received by the three-dimensional information reception unit 15 and generates data obtained by vectorizing them. Then, the retrieved product drawing data is transmitted to the user terminal U2 which is the search source, together with the information acquired from the project information database 14. In FIG. 3, the illustration of the project information database 14 is omitted.

[0037] FIG. 4 is an explanatory diagram showing a state in which features of the object to be searched 2 are extracted from the three-dimensional information 21 of the object to be searched 2. On the upper side of FIG. 4, the point cloud data of the object to be searched 2 is shown as the three-dimensional information 21. The three-dimensional information 21 of the object to be searched 2 has a shape such as a chair, and a protrusion 211 is formed below the backrest thereof. The three-dimensional information 21 includes, for example, a number of features such as intersections. As one of those features, the protrusion 211 is exemplified.

[0038] On the lower side of FIG. 4, an opaque processed 3D image 22 is displayed. Also in the 3D image 22, a protrusion 221 is formed below the backrest.

[0039] FIG. 5 is an explanatory diagram showing a state in which the appearance of features varies depending on the viewpoint. On the lower left side of FIG. 5, a 3D image 22(3) of the object to be searched 2 viewed from the back is shown. On the lower right side of FIG. 5, a 3D image 22(1) of the object to be searched 2 viewed from the right side is shown. As shown in the back image 22(3), protrusions 221L and 221R protrude from both the left and right sides below the backrest. However, in the image 22(1) viewed from the right side, the protrusion 221R blends into the background and is difficult to distinguish. Thus, even for the same three-dimensional information 21 of the object to be searched 2, the appearance of the features varies depending on the viewing angle. Therefore, in this embodiment, the features of the object to be searched 2 are captured from a plurality of viewpoints and vectorized respectively, so that similar drawing data can be retrieved from the registered drawing data.

[0040] FIG. 6 is an explanatory diagram showing a state in which the same search object 2 is imaged from a plurality of viewpoints, the distance between each image 22 and the image data (drawing data) in the index database 13 is calculated based on the features of each image 22, and the search results are output in ascending order of the distance. Here, 3D images viewed from a total of four viewpoints, namely, the right side image 22(1), the left side image 22(2), the back image 22(3), and the front image 22(4), are shown. Furthermore, a top surface image and a bottom surface image (both not shown) can also be added.

[0041] In addition to the images 22 from predetermined viewpoints such as the front, back, top, bottom, left, and right, or instead of all or part of the images 22 from predetermined viewpoints, images 22 from free viewpoints may be adopted. A searcher operating the search user terminal U2 can also generate an image 22 from an angle at which the features of the search object 2 are most clearly shown.

[0042] In the central part of FIG. 6, the relationship between the image 22 of the search object 2 and the image data (drawing data) registered in the index database 13 that is similar to the image 22 is shown.

[0043] For example, when focusing on the right side image 22(1) of the search object 2, the image data (drawing data) DW11, DW12, DW13 similar to the right side image 22(1) are extracted. The distances indicating the similarity between the right side image 22(1) and the similar image data DW11, DW12, DW13 are d11, d12, d13.

[0044] Similarly, in the left side image 22(2) of the search object 2, the image data DW21, DW22, DW23 similar to the left side image 22(2) are extracted. The distances indicating the similarity between the left side image 22(2) and the similar image data DW21, DW22, DW23 are d21, d22, d23.

[0045] Similarly, focusing on the rear image 22(3) of the object 2 to be searched, image data DW31, DW32, and DW33 similar to the rear image 22(3) are extracted. The distances indicating the similarity between the rear image 22(3) and the similar image data DW31, DW32, and DW33 are d31, d32, and d33.

[0046] Focusing on the left side image 22(2) of the object 2 to be searched, image data DW41, DW42, and DW43 similar to the front image 22(4) are extracted. The distances indicating the similarity between the front image 22(4) and the similar image data DW41, DW42, and DW43 are d41, d42, and d43.

[0047] Search requests are generated for each of the images 22(1) to 22(4) obtained from these multiple viewpoints, and the index database 13 is searched. Then, for each of the multiple viewpoints, an image DW similar to the object 2 to be searched is extracted. The image search unit 16 extracts similar images with a short distance (similar images with a high similarity) from the results of the similar images searched simultaneously from multiple viewpoints and provides them to the search user terminal U2.

[0048] FIG. 7 is a sequence diagram showing the processing of the search system 1. First, image data registration processes S1 to S7 for registering image data (drawing data) in the index database 13 are performed. Next, image search processes S11 to S17 for searching the index database 13 based on the three-dimensional information of the object 2 to be searched are performed.

[0049] In the image data registration process, registration of image data (drawing data) is instructed from the registration user terminal U1 to the image data registration unit 12 via the image data collection unit 11 (S1). The feature point extraction unit 121 of the image data registration unit 12 extracts the feature points included in the image data and calculates the feature amounts (S2). The vectorization processing unit 122 converts the calculated feature amounts into n-dimensional vector data (S3) and generates an index (S4).

[0050] The image data registration unit 12 registers the generated index in the index database 13 (S5). The image data registration unit 12 associates the image data registered in the index database 13 with the information in the project information database 14 related to this image data (S6), and stores it in the project information database 14 (S7).

[0051] In the image search process, a search instruction is input from the search user terminal U2 to the image search unit 16 via the three-dimensional information reception unit 15 (S11). This search instruction includes the three-dimensional information of the search object 2.

[0052] The search user terminal U2 can also indicate the important features of the search object 2 to the image search unit 16. For example, the search user terminal U2 can notify the image search unit 16 that the feature is important by marking the most characteristic part of the three-dimensional information of the search object 2.

[0053] The image search unit 16 can set the priority of the search result for the search request from the viewpoint containing the feature amount with high importance among multiple viewpoints to be higher than the priority of the search result for other viewpoints, and output the search result according to the similarity and priority. The search user terminal U2 may instruct the image search unit 16 to perform a search without indicating the important features.

[0054] The 3D preprocessing unit 161 of the image search unit 16 performs 3D preprocessing on the three-dimensional information of the search object 2, namely, opaque processing and multi-viewpoint extraction processing (S12). The image preprocessing unit 162 of the image search unit 16 extracts feature points from the 3D preprocessed image, calculates the feature amounts of the extracted feature points (S13), and generates n-dimensional vector data from the calculated feature amounts (S14). Note that when generating the vector data, the image preprocessing unit 162 may set a higher priority for the feature amounts with high importance (the feature amounts of the feature points with high importance).

[0055] The search unit 163 generates a search request based on the vectorized data and searches for the image data registered in the index database 13 (S15).

[0056] When the image search unit 16 obtains the search results of the index database 13, it acquires the information related to the search results from the project information database 14 (S16). The image search unit 16 associates the search results of the index database 13 with the related information acquired from the project information database 14 and transmits the associated results to the search user terminal U2 for presentation to the user (S17).

[0057] According to the search system 1 of this embodiment configured as described above, when searching for the drawing data related to the search object 2, for the search object itself or for a model of the search object, three-dimensional information is acquired, and the index database 13 is searched using the search request based on the acquired three-dimensional information. Therefore, even when there is no drawing data of the search object 2 or when there is only a model of the search object 2, it is possible to search for image data (drawing data) similar to the search object 2.

[0058] In the search system 1 of this embodiment, a search request is generated for each image of the search object 2 viewed from a plurality of viewpoints, and the index database 13 is searched simultaneously with a plurality of search requests. Therefore, the search system 1 can capture the features (feature points) of the search object 2 and search the index database 13, and can find image data similar to the search object 2.

[0059] The search system 1 of this embodiment can easily identify the manufacturing apparatus 3 capable of manufacturing the object represented by the drawing data by associating the manufacturing apparatus capable of manufacturing the object with the drawing data of the search object 2. Therefore, the search system 1 can also calculate an estimate or the like when manufacturing an object (product) related to the search object using the identified manufacturing apparatus 3.

[0060] When the search system 1 of this embodiment inputs the three-dimensional information of the object to be searched 2 into the search system 1, important feature points can be specified. Therefore, the search system 1 can search for image data (drawing data) that is similar in important feature points. Thereby, for example, the index database 13 can be searched while emphasizing the key points in manufacturing (such as the operation method of cutting tools, the ease of mold removal, the ease of injecting molten resin, etc.), and the usability is further improved for the user who is seeking the drawing data.

Example

[0061] Example 2 will be described with reference to FIG. 8. In each of the following examples including this example, the description will focus on the differences from Example 1. FIG. 8 is an explanatory diagram of the information processing system according to Example 2.

[0062] The search system 1A of this embodiment is connected to the target embedding system 4. That is, the search system 1A is used as one function of the target embedding system 4. The target embedding system 4 is, for example, a sales support system, an estimate creation support system, a production management system, a design support system, or the like.

[0063] This embodiment configured in this way also has the same effects as those of Example 1. Since the search system 1A of this embodiment is incorporated as one function of the target embedding system 4, the convenience of the target embedding system 4 can be improved.

Example

[0064] FIG. 9 is an explanatory diagram of the search system 1B according to Example 3. The search system 1B of this embodiment further includes an external disclosure unit 17 in addition to the configuration of the search system 1 described in Example 1.

[0065] The external disclosure unit 17 has a function for disclosing the search system 1B on the communication network CN. For example, the registration user terminal U1 and the search user terminal U2 can access the search system 1B via the communication network CN, register image data (drawing data) in the search system 1B, or extract image data similar to the object to be searched 2 from the search system 1B. The search system 1B may be disclosed 24 hours a day, 365 days a year via the external disclosure unit 17. Therefore, the convenience for the user is improved.

[0066] Furthermore, in the search system 1B of this embodiment, the material providing unit 181 and the estimate unit 182 are connected to the project information database 14. The material providing unit 181 provides materials such as catalogs, explanatory videos, and specifications of the manufacturing apparatus 3 to the project information database 14. The estimate unit 182 creates or supports the creation of an estimate when the manufacturing apparatus 3 manufactures the object shown in the drawing data.

[0067] This embodiment configured as described above also has the same operational effects as those of the first embodiment. Since the search system 1B of this embodiment is disclosed on the communication network CN through the external disclosure unit 17, the usability for the user is improved. Furthermore, in this embodiment, it is possible to provide the material data and the estimate of the manufacturing apparatus 3 related to the drawing data to the search user terminal U2, and the convenience of the user who is the searcher can be improved.

[0068] Note that the present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Also, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. Also, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration. Furthermore, the embodiments can be appropriately combined as long as they are not clearly contradictory.

[0069] In this embodiment, an apparatus or method expressed as follows is clearly described as being implementable.

[0070] (Expression 1) A search device that outputs search results according to a search request, comprising: an image data registration unit that vectorizes a feature amount based on the features of image data and registers it in a search database; and an image search unit that vectorizes a feature amount based on the features extracted from the three-dimensional information of the object to be searched, generates a search request, and searches the search database using the generated search request.

[0071] (Expression 2) The search device according to Expression 1, wherein the image search unit creates, for each of a plurality of viewpoints, a search request corresponding to an image obtained by viewing the three-dimensional information of the object to be searched from the plurality of viewpoints, and searches the search database using the search requests for each of the plurality of viewpoints.

[0072] (Expression 3) The search device according to Expression 1 or 2, wherein the image search unit performs an opaque process on the three-dimensional information of the object to be searched, creates, for each of a plurality of viewpoints, a search request corresponding to an image obtained by viewing the opaque-processed three-dimensional information of the object to be searched from the plurality of viewpoints, and searches the search database using the search requests for each of the plurality of viewpoints.

[0073] (Expression 4) The search device according to any one of Expressions 1-3, wherein the image search unit outputs the search results according to the similarity between the search results for each of the plurality of search requests and the three-dimensional information of the object to be searched.

[0074] (Expression 5) The search device according to any one of Expressions 1-4, wherein the image search unit sets the priority of the search results for a search request from a viewpoint including a feature amount with high importance among the plurality of viewpoints to be higher than the priority of the search results for other viewpoints, and outputs the search results according to the similarity and the priority.

[0075] (Expression 6) The search device according to any one of Expressions 1-5, wherein the three-dimensional information of the object to be searched is generated using a three-dimensional information acquisition method for acquiring three-dimensional information.

[0076] (Expression 7) The search target is the search device according to any one of Expressions 1-6, which is a physical sample of an object.

[0077] (Expression 8) The image data registered in the search database is the search device according to any one of Expressions 1-7, which is two-dimensional image data.

[0078] (Expression 9) The image data registered in the search database is the search device according to any one of Expressions 1-8, in which the object related to the image data is associated with a manufacturing device capable of manufacturing the object.

[0079] (Expression 10) A search method for causing a search device to output a search result according to a search request, the method including: an image data registration step of vectorizing a feature amount based on a feature of image data and registering the vectorized feature amount in a search database; and an image search step of vectorizing a feature amount based on a feature extracted from three-dimensional information of a search target to generate a search request, and searching the search database using the generated search request.

[0080] (Expression 11) An information processing system including the search device according to any one of Expressions 1-9, the information processing system outputting information related to production or sale of the search target based on a search result output by the search device.

Explanation of Signs

[0081] 1, 1A, 1B: Search system, 2: Search target, 3: Manufacturing device, 4: Embedded destination system, 11: Image data collection unit, 12: Image data registration unit, 13: Index database, 14: Project information database, 15: Three-dimensional information reception unit, 16: Image search unit, U1: Registration user terminal, U2: Search user terminal

Claims

1. A search device that outputs search results according to a search request, comprising: an image data registration unit that vectorizes a feature amount based on the features of image data and registers it in a search database; an image search unit that vectorizes a feature amount based on the features extracted from the three-dimensional information of the object to be searched, generates a search request, and searches the search database using the generated search request; A search device comprising the above.

2. The image search unit creates a search request corresponding to an image of the three-dimensional information of the object to be searched from a plurality of viewpoints, and searches the search database using the search requests for each of the plurality of viewpoints. The search device according to Claim 1.

3. The image search unit performs an opaque process on the three-dimensional information of the object to be searched, creates a search request corresponding to an image of the opaque-processed three-dimensional information of the object to be searched from a plurality of viewpoints, and searches the search database using the search requests for each of the plurality of viewpoints. The search device according to Claim 2.

4. The image search unit outputs the search results according to the similarity between the search results for each of the plurality of search requests and the three-dimensional information of the object to be searched. The search device according to Claim 3.

5. The image search unit sets the priority of the search results for the search request from the viewpoint including the feature amount with high importance among the plurality of viewpoints to be higher than the priority of the search results for other viewpoints, and outputs the search results according to the similarity and the priority. The search device according to Claim 4.

6. The three-dimensional information of the object to be searched is generated using a three-dimensional information acquisition method for acquiring three-dimensional information. The search device according to Claim 5.

7. The object to be searched is a physical sample of an object. The search device according to Claim 6.

8. The image data registered in the search database is two-dimensional image data. The search device according to Claim 7.

9. The image data registered in the search database is associated with a manufacturing device capable of manufacturing an object related to the image data. The search device according to Claim 8.

10. A search method for causing a search device to output search results according to a search request, comprising: in the search device, an image data registration step of vectorizing a feature amount based on the features of image data and registering it in a search database; An image search step of generating a search request by vectorizing a feature amount based on features extracted from three-dimensional information of a search target object, and searching the search database using the generated search request; A search method for causing the above to be executed.

11. An information processing system including the search device according to any one of Claims 1 to 9, outputting information regarding production or sales of the search target object based on a search result output by the search device The information processing system.

Images