Information linking device, image processing device, information linking method, image processing method, and program

Abstract

To provide an information association device and method and the like for a retrieval object, which can easily associate identification information and position information on the retrieval object.SOLUTION: An information association device 10 comprises: a starting position information acquisition section that acquires first starting position information; a movement information acquisition section that acquires first user movement information; an image information acquisition section that acquires image information on a retrieval object; an identification information extraction section that extracts identification information on the retrieval object from the image information on the retrieval object ; a position information extraction section that extracts position information on a first user when acquiring the image information on the retrieval object on the basis of the first starting position information and the first user movement information; and an output section that associates the identification information on the retrieval object with the position information on the first user when acquiring the image information on the retrieval object and outputs as position information on the retrieval object. The first starting position information includes information on a first starting position that is a starting point of the position information on the first user, and the first user movement information includes information on the movement amount of the first user from the first starting position.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information linking device, an image processing device, an information linking method, an image processing method, and a program. [Background technology]

[0002] BACKGROUND ART There has been reported a technology that uses AR (Augmented Reality) technology to provide information about products desired by users in stores such as supermarkets (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-192091 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there is a problem with registering the correspondence between product information and display location for products on store shelves, for example, due to the fast update cycle of location information, the huge variety of products, etc. Furthermore, this problem is not limited to products in retail stores such as supermarkets, electronics retailers, and convenience stores, but is also true for, for example, goods in warehouses, packages at shipping company terminals, office documents, etc.

[0005] Therefore, an object of the present invention is to provide an information linking device for search objects that can easily link the identification information of the search objects with the location information of the search objects. [Means for solving the problem]

[0006] In order to achieve the above object, the information linking device for search objects of the present invention includes a starting position information acquisition unit, a movement information acquisition unit, an image information acquisition unit, an identification information extraction unit, a position information extraction unit, and an output unit, the origin position information acquisition unit acquires first origin position information; The first origin location information includes information on a first origin location that is a origin of location information of the first user, the movement information acquisition unit acquires first user movement information; the first user movement information includes information on a movement amount of the first user from the first origin position; the image information acquisition unit acquires image information of a search target; the identification information extraction unit extracts identification information of the search object from image information of the search object; the location information extraction unit extracts location information of the first user when image information of the search target object is acquired based on the first origin location information and the first user movement information; The output unit is a device that links the identification information of the search object with the location information of the first user when image information of the search object was acquired, and outputs the linked information as location information of the search object.

[0007] The image processing device of the present invention includes a search object information linking unit, an imaging information acquisition unit, an object position information acquisition unit, a virtual information generation unit, a map information management unit, a terminal information acquisition unit, an identification unit, and an image information output unit, the search object information linking unit is the search object information linking device of the present invention, the imaging information acquisition unit acquires a captured image captured in a specific area by a second user, the captured image including second origin position information; the second origin location information includes information on a second origin location that is a origin of location information of the second user; the object position information acquisition unit acquires the position information of the search object output by the search object information linking unit, the virtual information generation unit generates a virtual image relating to the specific area; the map information management unit manages location information of the search object by linking it to a map of the specific area; the terminal information acquisition unit acquires location information and orientation information of the terminal based on the second origin location information; the image information output unit outputs a composite image in which the virtual image is superimposed on the captured image; the virtual image generated by the virtual information generation unit includes a virtual image for each of the search objects in the specific area, the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects; The image information output unit is a device that outputs to the terminal an image in which the position image is superimposed at a position corresponding to the search target object present in the captured image.

[0008] The method for linking information on search objects of the present invention includes a starting position information acquisition step, a movement information acquisition step, an image information acquisition step, an identification information extraction step, a position information extraction step, and an output step, The origin position information acquisition step acquires first origin position information, The first origin location information includes information on a first origin location that is a origin of location information of a first user, The movement information acquisition step acquires first user movement information, the first user movement information includes information on a movement amount of the first user from the first origin position; the image information acquisition step acquires image information of a search target; the identification information extraction step extracts identification information of the search object from image information of the search object; the position information extraction step extracts position information of the first user when image information of the search object is acquired based on the first origin position information and the first user movement information; The output step is a method of linking the identification information of the search object with location information of the first user when image information of the search object was acquired, and outputting the linked information as location information of the search object.

[0009] The image processing method of the present invention includes a search object information linking step, an image capture information acquisition step, an object position information acquisition step, a virtual information generation step, a map information management step, a terminal information acquisition step, a specification step, and an image information output step, The information linking step of the search object is the information linking method of the search object of the present invention, the imaging information acquisition step acquires a captured image captured in a specific area by a second user and including second origin position information; the second origin location information includes information on a second origin location that is a origin of location information of the second user; the object position information acquisition step acquires the position information of the search object output by the search object information linking step, the virtual information generating step generates a virtual image relating to the specific area; the map information management step links and manages position information of the search object on a map of the specific area; the terminal information acquisition step acquires location information and orientation information of the terminal based on the second origin location information; the image information output step outputs a composite image in which the virtual image is superimposed on the captured image, the virtual image generated in the virtual information generating step includes a virtual image for each of the search objects in the specific area, the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects; The image information output step is a method of outputting to the terminal an image in which the position image is superimposed at a position corresponding to the search target in the captured image, for the search target that exists in the captured image.

[0010] A first program of the present invention is a program for causing a computer to execute a starting point position information acquisition procedure, a movement information acquisition procedure, an image information acquisition procedure, an identification information extraction procedure, a position information extraction procedure, and an output procedure, The origin location information acquisition step includes acquiring first origin location information; The first origin location information includes information on a first origin location that is a origin of location information of the first user, The movement information acquisition step includes acquiring first user movement information; the first user movement information includes information on a movement amount of the first user from the first origin position; The image information acquisition step acquires image information of a search target, the identification information extraction step extracts identification information of the search object from image information of the search object; the location information extraction step extracts location information of the first user when image information of the search target object is acquired based on the first origin location information and the first user movement information; The output step links the identification information of the search object with location information of the first user when image information of the search object was acquired, and outputs the linked information as location information of the search object.

[0011] A second program of the present invention is a program for causing a computer to execute a search object information linking procedure, an image capture information acquisition procedure, an object position information acquisition procedure, a virtual information generation procedure, a map information management procedure, a terminal information acquisition procedure, an identification procedure, and an image information output procedure, the information linking procedure for the search object is executed by the first program of the present invention, The imaging information acquisition step includes acquiring a captured image captured in a specific area by a second user and including second origin position information; the second origin location information includes information on a second origin location that is a origin of location information of the second user; the object position information acquisition step acquires position information of the search object output by the search object information linking step, The virtual information generation step generates a virtual image relating to the specific area; the map information management step includes managing location information of the object to be searched for by linking it to a map of the specific area; the terminal information acquisition step acquires location information and orientation information of the terminal based on the second origin location information; the image information output step includes outputting a composite image in which the virtual image is superimposed on the captured image; the virtual image generated in the virtual information generation step includes a virtual image for each of the search objects in the specific area; the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects; The image information output step includes outputting to the terminal an image in which the position image is superimposed at a position corresponding to the search target object present in the captured image. [Effects of the Invention]

[0012] According to the present invention, it is possible to easily link the identification information of a search object with the location information of the search object. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an information associating device according to the first embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of the hardware configuration of the information associating device according to the first embodiment. [Figure 3] FIG. 3 is a flowchart illustrating an example of processing in the information associating device according to the first embodiment. [Figure 4] FIG. 4 is a block diagram showing an example of the configuration of an information associating device according to the second embodiment. [Figure 5] FIG. 5 is a flowchart showing an example of processing in the information associating device according to the second embodiment. [Figure 6] FIG. 6 is a block diagram showing an example of the configuration of an image processing system including the image processing apparatus according to the third embodiment. [Figure 7] FIG. 7 is a block diagram showing an example of the hardware configuration of an image processing apparatus according to the third embodiment. [Figure 8] FIG. 8 is a block diagram showing an example of the hardware configuration of the image display terminal according to the third embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of processing in the image processing apparatus according to the third embodiment. [Figure 10]FIG. 10 is a block diagram showing an example of the configuration of an image processing system including the image processing apparatus according to the fourth embodiment. [Figure 11] FIG. 11 is a flowchart showing an example of processing in the image processing apparatus according to the fourth embodiment. [Figure 12] FIG. 12 is a schematic diagram showing an example of an image output from the image processing apparatus according to the fourth embodiment to a user terminal, displayed on the display of the user terminal. DETAILED DESCRIPTION OF THE INVENTION

[0014] In the present invention, the "search object" is not particularly limited as long as it is possible to link location information and identification information. Specific examples of the search object include merchandise in a retail store, goods in a warehouse, goods transported by a shipping company, documents in an office, servers in a server room, library materials, etc.

[0015] In the present invention, a "first user" is, for example, a user who links the location information and identification information of a search object using the information linking device of the present invention. Specific examples of the first user include a person who manages the search object, such as an employee of a retail store, warehouse, or shipping company; a server room manager; or a library staff member.

[0016] In the present invention, a "second user" refers to, for example, a user who displays an image processed by the image processing device of the present invention on a terminal. Specific examples of the second user include a customer visiting a retail store, an employee of a warehouse or a shipping company, an administrator of a server room, a user visiting a library, or any other person who accesses a search object to which information is linked.

[0017] Next, embodiments of the present invention will be described with reference to the drawings. The present invention is not limited to the following embodiments. In the following drawings, the same parts are denoted by the same reference numerals. Furthermore, the descriptions of the embodiments can be mutually incorporated unless otherwise specified, and the configurations of the embodiments can be combined unless otherwise specified.

[0018] [Embodiment 1] Fig. 1 is a block diagram showing an example of the configuration of an information linking device 10 according to this embodiment. As shown in Fig. 1, the device 10 includes a starting point position information acquisition unit 11, a movement information acquisition unit 12, an image information acquisition unit 13, an identification information extraction unit 14, a position information extraction unit 15, and an output unit 16. The device 10 may also include a storage unit (not shown).

[0019] The device 10 may be, for example, a single device including the above-described units, or a device in which the units can be connected via a communication network. The device 10 can also be connected to an external device (described later) via the communication network. The communication network is not particularly limited and any known network can be used, for example, a wired or wireless network. Examples of the communication network include the Internet, the World Wide Web (WWW), a telephone line, a Local Area Network (LAN), a Storage Area Network (SAN), a Delay Tolerant Networking (DTN), a Low Power Wide Area Network (LPWA), and a Local 5G (L5G). Examples of wireless communication include Wi-Fi (registered trademark), Bluetooth (registered trademark), Local 5G, and LPWA. Examples of the wireless communication include direct communication between devices (Ad Hoc communication), infrastructure communication, and indirect communication via an access point. The device 10 may be incorporated into a server as a system. Furthermore, the present device 10 may be, for example, a personal computer (PC, e.g., desktop or notebook type) on which the program of the present invention is installed, a smartphone, a tablet terminal, or the like. The present device 10 may be, for example, a terminal-type device that performs all processing within the terminal owned by the first user, or a server-type device that acquires and processes information acquired by the terminal owned by the first user. Furthermore, the present device 10 may be in the form of cloud computing or edge computing, for example, in which at least one of the above-mentioned units is located on a server and the other units are located on a terminal.

[0020] 2 shows a block diagram of the hardware configuration of the device 10. The device 10 includes, for example, a central processing unit (CPU, GPU, etc.) 101, a memory 102, a bus 103, a storage device 104, an input device 105, a display device 106, and a communication device 107. The components of the device 10 are connected to each other via the bus 103 and their respective interfaces (I / F).

[0021] The central processing unit 101 is responsible for overall control of the device 10. In the device 10, the central processing unit 101 executes, for example, the program of the present invention and other programs, and also reads and writes various types of information. Specifically, for example, the central processing unit 101 functions as a starting point position information acquisition unit 11, a movement information acquisition unit 12, an image information acquisition unit 13, an identification information extraction unit 14, a position information extraction unit 15, and an output unit 16.

[0022] The bus 103 can also be connected to, for example, an external device. Examples of the external device include an external storage device (such as an external database), a printer, an external input device, an external display device, and an external imaging device. The device 10 can be connected to an external network (the communication line network) by, for example, a communication device 107 connected to the bus 103, and can also be connected to other devices via the external network.

[0023] The memory 102 may be, for example, a main memory (primary storage device). When the central processing unit 101 performs processing, the memory 102 reads various operating programs, such as the program of the present invention, stored in the storage device 104 (described later), and the central processing unit 101 receives data from the memory 102 and executes the programs. The main memory may be, for example, a RAM (random access memory). Alternatively, the memory 102 may be, for example, a ROM (read only memory).

[0024] The storage device 104 is also referred to as an auxiliary storage device, for example, in contrast to the main memory (primary storage device). As described above, the storage device 104 stores an operating program including the program of the present invention. The storage device 104 may be, for example, a combination of a recording medium and a drive for reading and writing data from and to the recording medium. The recording medium is not particularly limited and may be, for example, an internal or external type, such as a hard disk (HD), CD-ROM, CD-R, CD-RW, MO, DVD, flash memory, or memory card. The storage device 104 may be, for example, a hard disk drive (HDD) or a solid-state drive (SSD) in which the recording medium and drive are integrated. When the device 10 includes the storage unit, the storage device 104 functions as the storage unit, for example.

[0025] In the present device 10, the memory 102 and the storage device 104 can also store various information such as log information, information acquired from an external database (not shown) or an external device, information generated by the present device 10, and information used when the present device 10 executes processing. Note that at least a portion of the information may be stored, for example, in an external server other than the memory 102 and the storage device 104, or may be stored in a distributed manner across multiple terminals using blockchain technology or the like.

[0026] The device 10 further includes, for example, an input device 105 and a display 106. Examples of the input device 105 include pointing devices such as a touch panel, track pad, and mouse; a keyboard; imaging means such as a camera and scanner; a card reader such as an IC card reader and a magnetic card reader; and audio input means such as a microphone. Examples of the display 106 include display devices such as an LED display and a liquid crystal display. In the first embodiment, the input device 105 and the display 106 are configured separately, but the input device 105 and the display 106 may be configured as an integrated device, such as a touch panel display. The device 10 may also include, for example, an imaging device such as a camera as the input device 105, and the central processing unit 101 may control the imaging device to acquire an image. In this case, for example, the imaging device functions as the image acquisition unit 13.

[0027] Next, an example of the image processing method of this embodiment will be described based on the flowchart of FIG. 3. The image processing method of this embodiment is implemented as follows, for example, using the information linking device 10 of FIG. 1 or FIG. 2. Note that the image processing method of this embodiment is not limited to use with the information linking device 10 of FIG. 1 or FIG. 2. In the following examples, an example will be described in which this device 10 acquires information from a terminal carried by a first user via a communication network and performs processing based on the acquired various information. However, the present invention is not limited to this. For example, the terminal carried by the first user may be this device 10, and this terminal (this device 10) may process information acquired directly. The terminal is the terminal of the first user and is not particularly limited, and may be, for example, a personal computer, a smartphone, a mobile phone, a wearable terminal such as glasses or a wristwatch, or the like.

[0028] First, the origin position information acquisition unit 11 acquires first origin position information (S1, origin position information acquisition step). The first origin position information includes information on a first origin position that is the origin of the first user position information. The first origin position information is, for example, information that identifies the position and orientation (direction) of the first user's terminal using a position code or a direction mark arranged within a specific area. The position code may also serve as the direction mark, for example, and the position code and the direction mark are also referred to as an origin marker image. The origin marker image is, for example, a symbol image that can identify the origin position. Specific examples include a QR code (registered trademark), an AR marker, a barcode, and a Chameleon Code (registered trademark). In this case, for example, an origin marker image is captured by an image acquisition unit of a terminal carried by the first user, the image information acquisition unit 13 acquires the origin marker image from the terminal, and the origin position information acquisition unit 11 acquires the first origin position information from the origin marker image.

[0029] Next, the movement information acquisition unit 12 acquires first user movement information (S2, movement information acquisition step). The first user movement information includes information on the amount of movement of the first user from the first starting position. The information on the amount of movement of the first user can be calculated using information acquired by a position sensor such as a gyro sensor, an acceleration sensor, or a geomagnetic sensor in the terminal of the first user, for example.

[0030] Next, image information of the search object is acquired by the image information acquisition unit 13 (S3, image information acquisition step). Specifically, an image of the search object is captured by the image acquisition means of the terminal of the first user, and the image information acquisition unit 13 acquires the captured image of the search object.

[0031] Next, the identification information extraction unit 14 extracts the identification information of the search object from the image information of the search object (S4, identification information extraction step). The identification information is not particularly limited as long as it is information that can identify the search object, and examples of the identification information include product identification codes such as JAN code, EAN code, UPC code, ITF (Interleaved Two of Five), CODE39, Code128, and NW-7; image information of product packaging; and the like.

[0032] Next, the position information extraction unit 15 extracts the position information of the first user when the image information of the search object was acquired based on the first origin position information and the first user movement information (S5, position information extraction step). Specifically, for example, the position information extraction unit 15 can extract the position information of the first user by calculating the position when the terminal acquired the first origin position image, i.e., the amount of movement from the first origin position to when the image information of the search object was acquired, based on the first user movement information.

[0033] Then, the output unit 16 links the identification information of the search object with the location information of the first user when the image information of the search object was acquired, and outputs it as location information of the search object (S6, output step), thereby ending the process. For example, if the device 10 includes the storage unit, the location information of the search object can be stored in the storage unit. Also, for example, the output unit 16 may output the location information of the search object to an external database via the communication network, and the external database may store the location information of the search object.

[0034] In the information linking method of this embodiment, S4 and S5 are performed in this order, but the information linking method of the present invention is not limited to this order, and S4 and S5 may be performed in the order of, for example, S5 and S4, or S4 and S5 may be performed in parallel.

[0035] According to the information associating device 10 of this embodiment, for example, based on the first origin position information and the first user movement information, the position information of the first user at the time of acquiring the image information of the search object can be associated with the identification information of the search object, and the position information of the search object can be output. Therefore, according to the information associating device 10 of this embodiment, for example, the identification information of the search object can be easily associated with the position information of the search object.

[0036] [Embodiment 2] This embodiment is similar to the information linking device 10 of embodiment 1, except that it further includes an attribute information acquisition unit in addition to the configuration of the information linking device 10 of embodiment 1, and the description therefor can be cited. The information linking device of this embodiment further includes, for example, an attribute information acquisition unit, which acquires attribute information of the search object based on identification information of the search object, and the output unit links the attribute information of the search object to location information of the search object and outputs the linked attribute information. Therefore, according to this embodiment, it is possible to link the attribute information of the search object in addition to the identification information of the search object. Therefore, according to the information linking device of this embodiment, it is possible to link more detailed information of the search object to the location information of the search object.

[0037] Fig. 4 is a block diagram showing an example of the configuration of an information linking device 10A of this embodiment. As shown in Fig. 4, the information linking device 10A further includes an attribute information acquisition unit 17 in addition to the configuration of the report linking device 10 of embodiment 1. The hardware configuration of the information linking device 10A is the same as that of the report linking device 10 of Fig. 2, except that the central processing unit 101 includes the configuration of the information linking device 10A of Fig. 4 instead of the configuration of the report linking device 10 of Fig. 1.

[0038] An example of processing in a support system including information associating device 10A and support terminal 2 according to this embodiment will be described with reference to the flowchart of Fig. 5. Fig. 5 is a flowchart showing an example of processing (S1 to S5, S11, S12) of information associating device 10A.

[0039] First, steps S1 to S5 are carried out in the same manner as steps S1 to S5 in the processing of the information associating device 10 of the first embodiment.

[0040] Next, the attribute information acquisition unit 17 acquires attribute information of the search object based on the identification information of the search object (S17, attribute information acquisition step). The attribute information is not particularly limited, and may be, for example, detailed information about the product to which the search object belongs. Specific examples of the attribute information include, for example, if the search object is a supermarket product, information such as the production date, expiry date, best-before date, ingredients, and price.

[0041] Then, the output unit 16 outputs the location information of the search object in association with the attribute information (S12, output step), and the process ends.

[0042] In the information linking method of this embodiment, S11 is performed after S5, but the information linking method of the present invention is not limited to this order, and S11 may be performed, for example, in a process downstream of S4 and upstream of S12. Therefore, in the information linking method of this embodiment, S11 and S5 may be performed in the order of, for example, S5 and S11, or S11 and S5 may be performed in parallel.

[0043] [Embodiment 3] This embodiment is an example of an image processing system including an image processing device of the present invention and an image display terminal.

[0044] An image processing system according to this embodiment will be described with reference to FIG. 6. FIG. 6 is a block diagram showing an example of the configuration of an image processing system 200 including an image processing device 20 and an image display terminal 30 according to this embodiment. Although not shown, the image processing system 200 may include, for example, a plurality of image display terminals 30. As shown in FIG. 6, the image processing device 20 can be connected to the image display terminal 30 via a communication network 40. The image processing system 200 may also include, for example, a terminal of the first user. In this case, the image processing device 20 can be connected to the terminal of the first user via, for example, the communication network 40. Although not shown, the image processing device 20 can also be connected to an external terminal of a system administrator via, for example, the communication network 40, and the system administrator may manage the image processing device 20 from the external terminal.

[0045] The image processing device of this embodiment will be described with reference to Fig. 6. As shown in Fig. 6, this device 20 includes a search object information linking unit 10, an image information acquisition unit 21, an object position information acquisition unit 22, a virtual information generation unit 23, a map information management unit 24, a terminal information acquisition unit 25, and an image information output unit 26. Although not shown, this device 20 may include, for example, a storage unit.

[0046] The device 20 may be, for example, a single device including the above-described units, or a device in which the units can be connected via a communication network. The device 20 can also be connected to an external device (described later) via a communication network 40. The communication network 40 is not particularly limited and may be a known network, for example, wired or wireless. Examples of the communication network 40 include the Internet, the World Wide Web (WWW), a telephone line, a Local Area Network (LAN), a Storage Area Network (SAN), a Delay Tolerant Networking (DTN), a Low Power Wide Area Network (LPWA), and a Local 5G (L5G). Examples of wireless communication include Wi-Fi (registered trademark), Bluetooth (registered trademark), Local 5G, and LPWA. Examples of the wireless communication include direct communication between devices (Ad Hoc communication), infrastructure communication, and indirect communication via an access point. The device 20 may be incorporated into a server as a system. The device 20 may be, for example, a personal computer (PC, for example, a desktop or notebook type) on which the program of the present invention is installed, a smartphone, a tablet terminal, etc. Furthermore, the device 20 may be in the form of cloud computing or edge computing, for example, in which at least one of the units is located on a server and the other units are located on a terminal.

[0047] Examples of the image display terminal 30 include a PC; a mobile terminal such as a mobile phone, smartphone, or tablet terminal; a smart watch, smart glasses, or a wearable terminal. The image display terminal 30 may include, for example, an imaging means such as a camera or scanner, an audio input means such as an IC (integrated circuit) card reader, or a microphone. The image display terminal 30 is, for example, a terminal of a second user.

[0048] 7 shows a block diagram of the hardware configuration of the device 20. The device 20 includes, for example, a central processing unit (CPU, GPU, etc.) 201, a memory 202, a bus 203, a storage device 204, an input device 205, a display device 206, and a communication device 207. The components of the device 20 are connected to each other via the bus 203 and their respective interfaces (I / F).

[0049] The central processing unit 201 is responsible for overall control of the device 20. In the device 20, the central processing unit 201 executes, for example, the program of the present invention and other programs, and also reads and writes various information. Specifically, for example, the central processing unit 201 functions as a search object information linking unit 10, an image information acquisition unit 21, an object position information acquisition unit 22, a virtual information generation unit 23, a map information management unit 24, a terminal information acquisition unit 25, and an image information output unit 26.

[0050] The bus 203 can also be connected to, for example, an external device. Examples of the external device include an external storage device (such as an external database), a printer, an external input device, an external display device, and an external imaging device. The device 20 can be connected to an external network (the communication line network) by, for example, a communication device 207 connected to the bus 203, and can also be connected to other devices via the external network.

[0051] The memory 202 may be, for example, a main memory (primary storage device). When the central processing unit 201 performs processing, the memory 202 reads various operating programs, such as the program of the present invention, stored in the storage device 204 (described later), and the central processing unit 201 receives data from the memory 202 and executes the programs. The main memory may be, for example, a RAM (random access memory). Alternatively, the memory 202 may be, for example, a ROM (read only memory).

[0052] The storage device 204 is also referred to as an auxiliary storage device, for example, in contrast to the main memory (primary storage device). As described above, the storage device 204 stores an operating program including the program of the present invention. The storage device 204 may be, for example, a combination of a recording medium and a drive for reading and writing data from and to the recording medium. The recording medium is not particularly limited and may be, for example, an internal or external type, such as a hard disk (HD), CD-ROM, CD-R, CD-RW, MO, DVD, flash memory, or memory card. The storage device 204 may be, for example, a hard disk drive (HDD) or a solid-state drive (SSD) in which the recording medium and drive are integrated. When the device 20 includes, for example, the storage unit, the storage device 204 functions as the storage unit.

[0053] In the present device 20, the memory 202 and the storage device 204 can also store various information such as log information, information acquired from an external database (not shown) or an external device, information generated by the present device 20, and information used when the present device 20 executes processing. Note that at least a portion of the information may be stored, for example, in an external server other than the memory 202 and the storage device 204, or may be stored in a distributed manner across multiple terminals using blockchain technology or the like.

[0054] The device 20 further includes, for example, an input device 205 and a display 206. Examples of the input device 205 include pointing devices such as a touch panel, track pad, and mouse; a keyboard; imaging means such as a camera and scanner; card readers such as an IC card reader and a magnetic card reader; and audio input means such as a microphone. Examples of the display 206 include display devices such as an LED display and a liquid crystal display. In the first embodiment, the input device 205 and the display 206 are configured separately, but the input device 205 and the display 206 may be configured as an integrated unit, such as a touch panel display.

[0055] Fig. 8 illustrates a block diagram of the hardware configuration of the image display terminal 30. As shown in Fig. 8, the image display terminal 30 includes, for example, a CPU 301, a memory 302, a bus 303, a storage device 304, an input device (input unit) 305, an imaging unit 31, a communication device (communication unit) 32, and a display (display unit) 33. The components of the image display terminal 30 are connected via the bus 303 and their respective interfaces (I / F). The description of the components of the image display terminal 30 can be cited from the description of the components of the image processing device 20.

[0056] Next, an example of the image processing method of this embodiment will be described with reference to the flowchart of Fig. 9. The image processing method of this embodiment is implemented as follows, for example, using the image processing system apparatus 200 of Fig. 6. Note that the image processing method of this embodiment is not limited to use of the image processing system apparatus 200 of Fig. 6. Fig. 9 is a flowchart showing an example of the processing (S21 to S27) of the image processing system apparatus 200.

[0057] First, the information linking unit 10 for the search object links the identification information of the search object with the location information of the search object (S21, information linking step for the search object). S21 can be implemented, for example, by implementing S1 to S6, or S1 to S5, S11, and S12, in the same manner as S1 to S6 in the first embodiment, or S1 to S5, S11, and S12 in the second embodiment.

[0058] Next, the imaging information acquisition unit 21 acquires an image captured in a specific area and including second origin location information (S22, imaging information acquisition step). The specific area is not particularly limited and may be indoors or outdoors. Examples of indoor locations include retail stores such as supermarkets and electronics retailers; warehouses; shipping company terminals; server rooms; offices; libraries; etc. The captured image may be, for example, a still image or a video. The second origin location information includes information on a second origin location that is the origin of the second user's location information. The second origin location information is similar to, for example, the first origin location information, except that it is information on the second origin location that is the origin of the second user's location information, and the description therefor can be used. The captured image can be acquired, for example, by acquiring the captured image captured by the imaging unit 31 of the second user's terminal (image display terminal 30) via a communication network 40.

[0059] Next, the object position information acquisition unit 22 acquires the position information of the search object output by the search object information linking unit 10 (S23, object position information acquisition step).

[0060] Next, the virtual information generation unit 23 generates a virtual image related to the specific area (S24, virtual information generation step). The generated virtual image includes, for example, a virtual image of the search object in the specific area. The virtual image of the search object is, for example, a virtual image of the search object, in which position information of the search object and a position image of the search object are linked to each other. Generating the virtual image may mean, for example, reading out the virtual image stored in advance in the memory 202 or the storage device 204, or may mean acquiring the virtual image from an external database via the communication network. Furthermore, the virtual information generation unit 23 may, for example, acquire information related to the specific area (also referred to as specific area information) and generate the virtual image based on the specific area information. The virtual image of the search object is also referred to as, for example, a "pin" or "pin image" of the search object.

[0061] Next, the map information management unit 24 links the location information of the search object to a map of the specific area and manages it (S25, map information management step). The map may be, for example, a map that represents the specific area in two dimensions, or a map that represents the specific area in three dimensions.

[0062] Next, the terminal information acquisition unit 25 acquires the position information and orientation information of the terminal based on the second origin position information (S26, terminal information acquisition step). The terminal information acquisition unit 26 may acquire the position information and orientation information of the terminal, for example, in association with the captured image. The orientation information may also be, for example, the imaging direction of the imaging device of the terminal.

[0063] Then, the image information output unit 26 outputs a composite image in which the virtual image is superimposed on the captured image (S27, image information output step), and the process ends (END). The output may be performed, for example, via the communication line network. Specifically, the image information output unit 26 superimposes the position image on the search object present in the captured image at a position corresponding to the search object in the captured image.

[0064] Furthermore, the virtual image for each object generated by the virtual information generation unit 23 may further include a detailed information image of the search object. The detailed information image includes, for example, attribute information of the search object. In this case, the position image of the search object and the detailed information image are linked to each other for each search object. Then, for example, in the step (S27), the image information output unit 26 further provides a window section in the captured image and superimposes the detailed information image linked to the position image of the search object present in the captured image on the window section. The window section can display, for example, a plurality of the detailed information images on a screen that allows transitions between them. The detailed information image is, for example, an image with a hierarchical structure of one or more layers. When the detailed information image is an image with a hierarchical structure of one or more layers, the window section can display, for example, the detailed information image on a screen that allows transitions between images for each layer. Specifically, the detailed information image is, for example, an image based on detailed information of the search object (for example, attribute information of the search object), and one detailed information image may be linked to one position image of the object, or two detailed information images may be linked to one position image of the object.

[0065] According to this embodiment, a virtual image of the search object can be displayed using the position information of the search object linked by the information linking device of the present invention.

[0066] [Embodiment 4] This embodiment is similar to the image display system device 200 of Embodiment 3, except that the image processing device 20 further includes a selection information acquisition unit and the image display terminal 30 further includes an input unit, and the description thereof can be used. The image processing device of this embodiment includes, for example, a selection information acquisition unit, which acquires selection information for selecting the search objects, and the image information output unit selects the search objects present in the captured image based on the selection information and outputs an image with the position image superimposed at a position corresponding to the selected search object to the terminal. Furthermore, the image display terminal of this embodiment includes, for example, an input unit, which is capable of inputting the selection information, and the communication unit is capable of transmitting the selection information to the image processing device of the present invention and receiving an image with the position image superimposed at a position corresponding to the selected search object, and the display unit displays the image with the position image superimposed at a position corresponding to the selected search object. Therefore, according to this embodiment, search objects can be selected based on the selection information, and an image can be output in which the position image is superimposed at a position corresponding to the selected search object. Therefore, according to this embodiment, for example, the second user can select only any search object on the virtual image, so that the desired search object can be easily found. The image processing system device of this embodiment can also be called, for example, an object search system device.

[0067] FIG. 10 is a block diagram showing an example of the configuration of an image processing system apparatus 200A including an image processing apparatus 20A and an image display terminal 30A according to this embodiment. As shown in FIG. 10, the image processing apparatus 20A further includes a selection information acquisition unit 27 in addition to the configuration of the image processing apparatus 20 according to the third embodiment. The hardware configuration of the image processing apparatus 20A is the same as that of the image processing apparatus 20 shown in FIG. 7, except that the central processing unit 201 has the configuration of the image processing apparatus 20A shown in FIG. 10 instead of the configuration of the image processing apparatus 20 shown in FIG. 6. Also, as shown in FIG. 10, the image display terminal 30A further includes an input unit 34 in addition to the configuration of the image display terminal 30 according to the third embodiment. The hardware configuration of the image display terminal 30A is the same as that of the image display terminal 30 shown in FIG. 8, except that the input device 305 functions as the input unit 34.

[0068] Next, an example of the image processing method of this embodiment will be described with reference to the flowchart of Fig. 11. The image processing method of this embodiment is implemented as follows, for example, using the image processing system apparatus 200A of Fig. 10. Note that the image processing method of this embodiment is not limited to use of the image processing system apparatus 200A of Fig. 10. Fig. 11 is a flowchart showing an example of the processing (S21 to S27, S31) of the image processing system apparatus 200A.

[0069] First, steps S21 to S26 are carried out in the same manner as steps S21 to S26 in the processing of the image processing system apparatus 200 of the third embodiment.

[0070] Next, for example, the second user inputs selection information through the input unit 34 of the image display terminal 30A, and the communication unit 32 of the image display terminal 30A transmits the selection information to the image processing system device 200A via the communication network 40. The selection information is, for example, information for selecting the search objects, and any information can be set. A specific example of the selection information is attribute information of the search objects. Specifically, for example, if the search objects are supermarket products, information such as production date, expiration date, best-before date, ingredients, and price can be input as the selection information. Specific examples of the selection information include information on ingredients of the search object (e.g., grains and sesame such as peanuts, wheat, buckwheat, soybeans, sesame, barley, etc.; meat, eggs, and seafood such as eggs, milk, beef, pork, chicken, salmon, mackerel, shrimp, crab, squid, octopus, fish eggs, abalone, salmon roe, gelatin, etc.); and price information (e.g., ¥0-100, ¥101-500, ¥500 and above). The selection information may be, for example, information for excluding search objects that include the attribute information from display, or information for displaying search objects that include the attribute information. Furthermore, the selection information may include a plurality of pieces of information. In this case, for example, selection information that brings together multiple attribute information, such as excluding foods containing specific allergens, selecting only Halal or vegan products, or selecting only products in a specific price range, may be displayed on the display unit 33 of the image display terminal 30A, and the second user may input the selection information by selecting these.

[0071] Then, the selection information acquisition unit 27 acquires the selection information (S31, selection information acquisition step).

[0072] Next, the image information output unit 26 selects the search object present in the captured image based on the selection information, and outputs an image with the position image superimposed at a position corresponding to the selected search object to the terminal (S27, image information output process), and ends processing by the image processing device 200A (END).

[0073] Thereafter, the communication unit 32 of the image display terminal 30A receives an image in which the position image is superimposed at a position corresponding to the selected search object, and the display unit 33 displays the image in which the position image is superimposed at a position corresponding to the selected search object. Fig. 12 shows an example of an image displayed on the image display terminal 30A.

[0074] 12, a virtual image in which a position image 331 is superimposed at a position corresponding to a search object (a product in the supermarket) included in a captured image of the interior of a supermarket is displayed on the display unit 33 of the image display terminal 30A as a specific area. When the position image 331 is selected, a detailed information image about the selected product is superimposed and displayed in a window portion 332 displayed on the display unit 33. By inputting the selection information, the second user can display on the display unit 33 only position images 331 corresponding to search objects having desired attributes, thereby easily finding search objects having desired attributes.

[0075] [Embodiment 5] The first program of this embodiment is a program for causing a computer to execute each step of the information linking method described above. Specifically, the program of this embodiment is a program for causing a computer to execute a starting point position information acquisition procedure, a movement information acquisition procedure, an image information acquisition procedure, an identification information extraction procedure, a position information extraction procedure, and an output procedure.

[0076] The origin location information acquisition step includes acquiring first origin location information; The first origin location information includes information on a first origin location that is a origin of location information of the first user, The movement information acquisition step includes acquiring first user movement information; the first user movement information includes information on a movement amount of the first user from the first origin position; the image information acquisition step acquires image information of a search target; the identification information extraction step extracts identification information of the search object from image information of the search object; the location information extraction step extracts location information of the first user when image information of the search target object is acquired based on the first origin location information and the first user movement information; The output step links the identification information of the search object with location information of the first user when image information of the search object was acquired, and outputs the linked information as location information of the search object.

[0077] The program of this embodiment can also be said to be a program that causes a computer to function as a starting point position information acquisition procedure, a movement information acquisition procedure, an image information acquisition procedure, an identification information extraction procedure, a position information extraction procedure, and an output procedure.

[0078] The program of this embodiment can cite the descriptions of the information linking device and information linking method of the present invention. For example, the "procedure" in each of the steps can be read as "processing." The program of this embodiment may be recorded on a computer-readable recording medium. The recording medium is, for example, a non-transitory computer-readable storage medium. The recording medium is not particularly limited, and examples thereof include random access memory (RAM), read-only memory (ROM), hard disk (HD), optical disk, and floppy disk (FD).

[0079] [Embodiment 6] The second program of this embodiment is a program for causing a computer to execute each step of the image processing method described above. Specifically, the program of this embodiment causes a computer to execute a search object information linking procedure, an image information acquisition procedure, an object position information acquisition procedure, a virtual information generation procedure, a map information management procedure, a terminal information acquisition procedure, an identification procedure, and an image information output procedure.

[0080] the information linking procedure for the search object is executed by the first program of the present invention, The imaging information acquisition step includes acquiring a captured image captured in a specific area by a second user and including second origin position information; the second origin location information includes information on a second origin location that is a origin of location information of the second user; The virtual information generation step generates a virtual image relating to the specific area; the object position information acquisition step acquires position information of the search object output by the search object information linking step, the map information management step includes managing location information of the object to be searched for by linking it to a map of the specific area; the terminal information acquisition step acquires location information and orientation information of the terminal based on the second origin location information; the image information output step includes outputting a composite image in which the virtual image is superimposed on the captured image; the virtual image generated in the virtual information generation step includes a virtual image for each of the search objects in the specific area; the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects; The image information output step includes outputting to the terminal an image in which the position image is superimposed at a position corresponding to the search target object present in the captured image.

[0081] In addition, the second program of this embodiment can also be said to be a program that causes a computer to function as a search object information linking procedure, an image information acquisition procedure, an object position information acquisition procedure, a virtual information generation procedure, a map information management procedure, a terminal information acquisition procedure, an identification procedure, and an image information output procedure.

[0082] The program of this embodiment can be implemented by invoking the descriptions of the image processing device and image processing method of the present invention. For each of the steps, for example, "step" can be read as "processing." The program of this embodiment may also be recorded on a computer-readable recording medium. The recording medium is, for example, a non-transitory computer-readable storage medium. The recording medium is not particularly limited, and examples thereof include random access memory (RAM), read-only memory (ROM), hard disk (HD), optical disk, and floppy disk (FD).

[0083] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0084] <Additional Notes> Some or all of the above embodiments can be described as, but not limited to, the following supplementary notes. (Appendix 1) The system includes a starting point position information acquisition unit, a movement information acquisition unit, an image information acquisition unit, an identification information extraction unit, a position information extraction unit, and an output unit, the origin position information acquisition unit acquires first origin position information; The first origin location information includes information on a first origin location that is a origin of location information of the first user, the movement information acquisition unit acquires first user movement information; the first user movement information includes information on a movement amount of the first user from the first origin position; the image information acquisition unit acquires image information of a search target; the identification information extraction unit extracts identification information of the search object from image information of the search object; the location information extraction unit extracts location information of the first user when image information of the search target object is acquired based on the first origin location information and the first user movement information; The output unit links the identification information of the search object with the location information of the first user when image information of the search object was acquired, and outputs it as the location information of the search object. (Appendix 2) the image information acquisition unit acquires an origin marker image; The information linking device according to claim 1, wherein the origin position information acquisition unit acquires the first origin position information from the origin marker image. (Appendix 3) An attribute information acquisition unit is included, the attribute information acquisition unit acquires attribute information of the search object based on identification information of the search object; 3. The information linking device according to claim 1, wherein the output unit links attribute information of the search object to the location information of the search object and outputs the linked information. (Appendix 4) The system includes a search object information linking unit, an imaging information acquisition unit, an object position information acquisition unit, a virtual information generation unit, a map information management unit, a terminal information acquisition unit, an identification unit, and an image information output unit, the information linking unit for the search object is the information linking device for the search object described in any one of Supplementary Notes 1 to 3, the imaging information acquisition unit acquires a captured image captured in a specific area by a second user, the captured image including second origin position information; the second origin location information includes information on a second origin location that is a origin of location information of the second user; the object position information acquisition unit acquires the position information of the search object output by the search object information linking unit, the virtual information generation unit generates a virtual image relating to the specific area; the map information management unit manages location information of the search object by linking it to a map of the specific area; the terminal information acquisition unit acquires location information and orientation information of the terminal based on the second origin location information; the image information output unit outputs a composite image in which the virtual image is superimposed on the captured image; the virtual image generated by the virtual information generation unit includes a virtual image for each of the search objects in the specific area, the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects; The image information output unit outputs to the terminal an image in which the position image is superimposed at a position corresponding to the search target object present in the captured image. (Appendix 5) the virtual image for each of the search objects further includes a detailed information image for the search object, and the position image and the detailed information image are linked to each other for each of the search objects; The image processing device described in Appendix 4, wherein the image information output unit provides a window portion within the captured image and superimposes the detailed information image linked to the position image of the search object present in the captured image on the window portion. (Appendix 6) A selection information acquisition unit is provided, the selection information acquisition unit acquires selection information for selecting the search objects; The image processing device according to claim 4 or 5, wherein the image information output unit selects the search object present in the captured image based on the selection information, and outputs to the terminal an image in which the position image is superimposed at a position corresponding to the selected search object. (Appendix 7) It includes an imaging unit, a communication unit, and a display unit, the imaging unit captures an image including starting point position information in a specific area; the communication unit is capable of communicating with the image processing device according to any one of Supplementary Notes 4 to 6, capable of transmitting the captured image to the image processing device, and capable of receiving an image in which the position image is superimposed at a position corresponding to the search object in the captured image; The display unit displays an image on which the position image is superimposed. (Appendix 8) an input section, the input unit is capable of inputting the selection information, the communication unit is capable of transmitting the selection information to the image processing device described in Supplementary Note 6, and is capable of receiving an image in which the position image is superimposed at a position corresponding to the selected search object; 8. The image display terminal according to claim 7, wherein the display unit displays an image in which the position image is superimposed at a position corresponding to the selected search object. (Appendix 9) A server and a terminal, the server and the terminal are capable of communicating with each other, the server is an image processing device according to any one of Supplementary Notes 4 to 6, 9. An image processing system apparatus, wherein the terminal is an image display terminal according to claim 7 or 8. (Appendix 10) The method includes a starting point position information acquisition step, a movement information acquisition step, an image information acquisition step, an identification information extraction step, a position information extraction step, and an output step, The origin position information acquisition step acquires first origin position information, The first origin location information includes information on a first origin location that is a origin of location information of the first user, The movement information acquisition step acquires first user movement information, the first user movement information includes information on a movement amount of the first user from the first origin position; the image information acquisition step acquires image information of a search target; the identification information extraction step extracts identification information of the search object from image information of the search object; the position information extraction step extracts position information of the first user when image information of the search object is acquired based on the first origin position information and the first user movement information; The output process links the identification information of the search object with the location information of the first user when the image information of the search object was acquired, and outputs the linked information as the location information of the search object. (Appendix 11) The image information acquiring step includes acquiring an origin marker image, The information linking method according to claim 10, wherein the origin position information acquisition step acquires the first origin position information from the origin marker image. (Appendix 12) An attribute information acquisition step is included, the attribute information acquisition step acquires attribute information of the search object based on identification information of the search object; 12. The information linking method according to claim 10, wherein the output step links attribute information of the search object to the location information of the search object and outputs the linked information. (Appendix 13) The method includes a search object information linking step, an imaging information acquisition step, an object position information acquisition step, a virtual information generation step, a map information management step, a terminal information acquisition step, a specification step, and an image information output step, The information linking step of the search object is a method for linking information of a search object according to any one of Supplementary Notes 10 to 12, the imaging information acquisition step acquires a captured image captured in a specific area by a second user and including second origin position information; the second origin location information includes information on a second origin location that is a origin of location information of the second user; the object position information acquisition step acquires the position information of the search object output by the search object information linking step, the virtual information generating step generates a virtual image relating to the specific area; the map information management step links and manages position information of the search object on a map of the specific area; the terminal information acquisition step acquires location information and orientation information of the terminal based on the second origin location information; the image information output step outputs a composite image in which the virtual image is superimposed on the captured image, the virtual image generated in the virtual information generating step includes a virtual image for each of the search objects in the specific area, the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects; The image processing method includes outputting to the terminal an image in which the position image is superimposed at a position corresponding to the search target in the captured image, for the search target present in the captured image, in the image information output step. (Appendix 14) the virtual image for each of the search objects further includes a detailed information image for the search object, and the position image and the detailed information image are linked to each other for each of the search objects; The image processing method described in Appendix 13, wherein the image information output process provides a window process within the captured image and superimposes the detailed information image linked to a position image of the search target object present in the captured image onto the window process. (Appendix 15) A selection information acquisition step is provided, the selection information acquisition step acquires selection information for selecting the search objects; The image processing method according to claim 13 or 14, wherein the image information output step selects the search object present in the captured image based on the selection information, and outputs to the terminal an image in which the position image is superimposed at a position corresponding to the selected search object. (Appendix 16) A program for causing a computer to execute a starting point position information acquisition procedure, a movement information acquisition procedure, an image information acquisition procedure, an identification information extraction procedure, a position information extraction procedure, and an output procedure, The origin location information acquisition step includes acquiring first origin location information; The first origin location information includes information on a first origin location that is a origin of location information of the first user, The movement information acquisition step includes acquiring first user movement information; the first user movement information includes information on a movement amount of the first user from the first origin position; the image information acquisition step acquires image information of a search target; the identification information extraction step extracts identification information of the search object from image information of the search object; the location information extraction step extracts location information of the first user when image information of the search target object is acquired based on the first origin location information and the first user movement information; The program is characterized in that the output procedure links the identification information of the search object with the location information of the first user when image information of the search object was acquired, and outputs it as location information of the search object. (Appendix 17) The image information acquisition step includes acquiring an origin marker image; 17. The program according to claim 16, wherein the step of acquiring the origin position information acquires the first origin position information from the origin marker image. (Appendix 18) A program for causing a computer to further execute an attribute information acquisition procedure, the attribute information acquisition step acquires attribute information of the search object based on identification information of the search object; 18. The program according to claim 16, wherein the output step links attribute information of the search object to location information of the search object and outputs the linked information. (Appendix 19) A program for causing a computer to execute a search object information linking procedure, an image capture information acquisition procedure, an object position information acquisition procedure, a virtual information generation procedure, a map information management procedure, a terminal information acquisition procedure, a specification procedure, and an image information output procedure, The information linking procedure for the search object is executed by a program described in any one of Supplementary Notes 16 to 17, The imaging information acquisition step includes acquiring a captured image captured in a specific area by a second user and including second origin position information; the second origin location information includes information on a second origin location that is a origin of location information of the second user; the object position information acquisition step acquires position information of the search object output by the search object information linking step, The virtual information generation step generates a virtual image relating to the specific area; the map information management step includes managing location information of the object to be searched for by linking it to a map of the specific area; the terminal information acquisition step acquires location information and orientation information of the terminal based on the second origin location information; the image information output step includes outputting a composite image in which the virtual image is superimposed on the captured image; the virtual image generated in the virtual information generation step includes a virtual image for each of the search objects in the specific area; the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects; The image information output step is a program that outputs to the terminal an image in which the position image is superimposed at a position corresponding to the search target in the captured image, for the search target that exists in the captured image. (Appendix 20) the virtual image for each of the search objects further includes a detailed information image for the search object, and the position image and the detailed information image are linked to each other for each of the search objects; The program described in Appendix 19, wherein the image information output procedure provides a window procedure within the captured image and superimposes the detailed information image linked to the position image of the search target object present in the captured image onto the window procedure. (Appendix 21) A program for causing a computer to further execute a selection information acquisition procedure, the selection information acquisition step acquires selection information for selecting the search objects; The program described in Appendix 19 or 20, wherein the image information output procedure selects the search object present in the captured image based on the selection information, and outputs to the terminal an image in which the position image is superimposed at a position corresponding to the selected search object. (Appendix 22) A computer-readable recording medium having recorded thereon a program according to any one of appendices 16 to 21. [Industrial Applicability]

[0085] According to the present invention, it is possible to easily link the identification information of a search object with the location information of the search object. Therefore, according to the present invention, it is possible to easily build a database in which the type of search object and its location information are associated with each other, and it is possible to efficiently search for the search object. Therefore, it is particularly useful in various fields where the management of goods is required, for example. [Explanation of symbols]

[0086] 10, 10A Information linking device, information linking unit 11 Origin position information acquisition unit 12 Mobile Queen Acquisition Department 13 Image information acquisition unit 14 Identification information extraction unit 15 Location information extraction section 16 Output section 17 Attribute information extraction part 101 Central Processing Unit 102 memory 103 Bus 104 Storage device 105 Input Device 106 Display device 107 Communication Devices 200, 200A Image Processing System 20, 20A Image processing device 21 Imaging information acquisition unit 22 Object position information acquisition unit 23 Virtual Information Generation Unit 24 Map Information Management Department 25 Terminal information acquisition unit 26 Image information output unit 27 Selection information acquisition unit 201 Central Processing Unit 202 memory 203 Bus 204 Storage device 205 Input Device 206 Display device 207 Communication Devices 30, 30A Image display terminal 31 Imaging unit 32 Communication unit (communication device) 33 Display unit (display) 34 Input section 301 Central Processing Unit 302 memory 303 Bus 304 Storage device 305 Input section

Claims

1. The system includes a starting point position information acquisition unit, a movement information acquisition unit, an image information acquisition unit, an identification information extraction unit, a position information extraction unit, and an output unit, the image information acquisition unit acquires an origin marker image; the origin position information acquisition unit acquires first origin position information from the origin marker image; the first origin location information includes information on a first origin location that is a origin of location information of a first user; The movement information acquisition unit acquires first user movement information, the first user movement information includes information on a movement amount of the first user from the first origin position; the image information acquisition unit acquires image information of a search target; the identification information extraction unit extracts identification information of the search object from image information of the search object; the location information extraction unit extracts the location information of the first user by calculating a movement amount of the first user up to a point when image information of the search target object was acquired, based on the first origin location information and the first user movement information; The output unit links the identification information of the search object with the location information of the first user when the image information of the search object was acquired, and outputs it as the location information of the search object.

2. An attribute information acquisition unit is included, the attribute information acquisition unit acquires attribute information of the search object based on identification information of the search object; The information linking device according to claim 1 , wherein the output unit links attribute information of the search object to the location information of the search object and outputs the linked information.

3. The system includes a search object information linking unit, an imaging information acquisition unit, an object position information acquisition unit, a virtual information generation unit, a map information management unit, a terminal information acquisition unit, an identification unit, and an image information output unit, The information linking unit for the search object is an information linking device for the search object according to claim 1 or 2, the imaging information acquisition unit acquires a captured image captured in a specific area by a second user, the captured image including second origin position information; the second origin location information includes information on a second origin location that is a origin of location information of a second user; the object position information acquisition unit acquires the position information of the search object output by the search object information linking unit, the virtual information generation unit generates a virtual image relating to the specific area; the map information management unit manages location information of the search object by linking it to a map of the specific area; the terminal information acquisition unit acquires position information of the terminal of the second user and orientation information of the terminal of the second user based on the second origin position information; the image information output unit outputs a composite image in which the virtual image is superimposed on the captured image; the virtual image generated by the virtual information generation unit includes a virtual image for each of the search objects in the specific area, the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects; The image information output unit outputs to the second user's terminal an image in which the position image is superimposed at a position corresponding to the search target object present in the captured image.

4. the virtual image for each of the search objects further includes a detailed information image for the search object, and the position image and the detailed information image are linked to each other for each of the search objects; The image processing device according to claim 3 , wherein the image information output unit provides a window portion within the captured image, and superimposes the detailed information image linked to a position image of the search object present in the captured image on the window portion.

5. A selection information acquisition unit is provided, the selection information acquisition unit acquires selection information for selecting the search objects; 5. The image processing device according to claim 3, wherein the image information output unit selects the search object present in the captured image based on the selection information, and outputs an image in which the position image is superimposed at a position corresponding to the selected search object to the terminal of the second user.

6. A method of obtaining first image information, a starting point position information obtaining step, a movement information obtaining step, a second image information obtaining step, an identification information extracting step, a position information extracting step, and an outputting step, The first image information acquisition step includes acquiring an origin marker image; the origin position information acquiring step acquires first origin position information from the origin marker image; the first origin location information includes information on a first origin location that is a origin of location information of a first user; The movement information acquisition step acquires first user movement information, the first user movement information includes information on a movement amount of the first user from the first origin position; the second image information acquisition step acquires image information of a search target object; the identification information extraction step extracts identification information of the search object from image information of the search object; The position information extraction step extracts the position information of the first user by calculating a movement amount of the first user up to a time when image information of the search target object was acquired based on the first origin position information and the first user movement information; The output step links the identification information of the search object with location information of the first user at the time when image information of the search object was acquired, and outputs the linked information as location information of the search object. A method for linking information on search objects, comprising:

7. The method includes a search object information linking step, an imaging information acquisition step, an object position information acquisition step, a virtual information generation step, a map information management step, a terminal information acquisition step, a specification step, and an image information output step, The information linking step of the search object is a method for linking information of the search object according to claim 6, The image information acquisition step acquires a captured image captured in a specific area by a second user, the captured image including second origin position information; the second origin location information includes information on a second origin location that is a origin of location information of a second user; the object position information acquisition step acquires the position information of the search object output by the search object information linking step, the virtual information generating step generates a virtual image relating to the specific area; the map information management step links and manages position information of the search object on a map of the specific area; the terminal information acquisition step acquires position information of the terminal of the second user and orientation information of the terminal of the second user based on the second origin position information; the image information output step outputs a composite image in which the virtual image is superimposed on the captured image, the virtual image generated in the virtual information generating step includes a virtual image for each of the search objects in the specific area, the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects; The image information output process is an image processing method in which an image in which the position image is superimposed at a position corresponding to the search target in the captured image is output to the second user's terminal.

8. A program for causing a computer to execute a first image information acquisition procedure, a starting point position information acquisition procedure, a movement information acquisition procedure, a second image information acquisition procedure, an identification information extraction procedure, a position information extraction procedure, and an output procedure, The first image information acquisition step includes acquiring an origin marker image; the origin position information acquisition step includes acquiring first origin position information from the origin marker image; the first origin location information includes information on a first origin location that is a origin of location information of a first user; The movement information acquisition step includes acquiring first user movement information; the first user movement information includes information on a movement amount of the first user from the first origin position; the second image information acquisition step acquires image information of a search target; the identification information extraction step extracts identification information of the search object from image information of the search object; the location information extraction step extracts location information of the first user by calculating a movement amount of the first user up to a time when image information of the search target object was acquired based on the first origin location information and the first user movement information; The program is characterized in that the output procedure links identification information of the search object with location information of the first user when image information of the search object was acquired, and outputs the linked information as location information of the search object.

9. A program for causing a computer to execute a search object information linking procedure, an image capture information acquisition procedure, an object position information acquisition procedure, a virtual information generation procedure, a map information management procedure, a terminal information acquisition procedure, a specification procedure, and an image information output procedure, The information linking procedure for the search object is executed by a program according to claim 8, The image information acquisition step includes acquiring a captured image captured in a specific area by a second user and including second origin position information; the second origin location information includes information on a second origin location that is a origin of location information of a second user; the object position information acquisition step acquires position information of the search object output by the search object information linking step, The virtual information generation step generates a virtual image relating to the specific area; the map information management step includes managing location information of the object to be searched for by linking it to a map of the specific area; the terminal information acquisition step includes acquiring position information of the terminal of the second user and orientation information of the terminal of the second user based on the second origin position information; the image information output step includes outputting a composite image in which the virtual image is superimposed on the captured image; the virtual image generated in the virtual information generation step includes a virtual image for each of the search objects in the specific area; the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects; The image information output procedure is a program that outputs to the second user's terminal an image in which the position image is superimposed at a position corresponding to the search target in the captured image, for the search target present in the captured image.

Images