Information processing device, information processing method, and program

Abstract

To provide an information processing device capable of appropriately acquiring code identifiers for each of multiple codes contained in a captured image.SOLUTION: An information processing device 1 includes: a reception unit 11 configured to receive a capture instruction from a user; an image acquisition unit 12 configured to acquire a captured image, which is a still image of multiple codes, upon receiving the capture instruction; an identifier acquisition unit 14 configured to acquire code identifiers identifying each of the multiple codes contained in the captured image; a generation unit 15 configured to generate acquisition status information for each captured image indicating the acquisition status of the code identifiers acquired by the identifier acquisition unit 14; an acquisition status output unit 16 configured to output the acquisition status information generated by the generation unit 15; and an identifier output unit 17 configured to output multiple code identifiers acquired from one or more captured images acquired by the image acquisition unit 12.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device and the like that acquires a code identifier for identifying a code using a photographed image of the code. [Background technology]

[0002] Conventionally, a captured image of a code attached to an object has been used to search for the object (see, for example, Patent Document 1). When performing processing using a captured image of such a code, a code identifier that identifies the code is usually obtained using the captured image of the code. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-159856 Summary of the Invention [Problem to be solved by the invention]

[0004] However, depending on the circumstances under which the code is photographed, it may not be possible to properly obtain the code identifier of the code included in the photographed image. If the code identifier is not obtained, for example, it may become impossible to accurately perform processes such as inventory taking using the photographed image of the code.

[0005] The present invention has been made to solve the above-mentioned problem, and aims to provide an information processing device etc. that can appropriately acquire a code identifier for each of multiple codes contained in a captured image. [Means for solving the problem]

[0006] In order to achieve the above object, an information processing device according to one aspect of the present invention includes a reception unit that receives a shooting instruction from a user, an image acquisition unit that acquires a captured image, which is a still image of multiple codes, when the shooting instruction is accepted by the reception unit, an identifier acquisition unit that acquires a code identifier that identifies the code for each of the multiple codes included in the captured image, a generation unit that generates acquisition status information for each captured image that indicates the acquisition status of the code identifier by the identifier acquisition unit, an acquisition status output unit that outputs the acquisition status information generated by the generation unit, and an identifier output unit that outputs multiple code identifiers acquired from one or more captured images acquired by the image acquisition unit. With this configuration, the user can know from the output acquisition status information whether the code identifiers were properly acquired from each code included in the photographed image. Therefore, if the code identifiers could not be acquired for some codes, the user can prevent the code identifiers from being missed by acquiring the photographed image again.

[0007] In addition, in an information processing device according to one aspect of the present invention, the image acquisition unit acquires two or more captured images, and the multiple codes that are the subject of the capture are all different codes, the identifier acquisition unit integrates the code identifiers acquired from the two or more captured images acquired by the image acquisition unit, and the identifier output unit outputs the multiple code identifiers integrated by the identifier acquisition unit. With this configuration, it is possible to output the acquired code identifiers without duplication.

[0008] In addition, in an information processing device according to one aspect of the present invention, the image acquisition unit acquires two or more captured images, and further includes an image synthesis unit that acquires a composite image by synthesizing the two or more captured images acquired by the image acquisition unit based on codes contained in the captured images, the identifier acquisition unit acquires a code identifier for each of the multiple codes contained in the composite image, and the identifier output unit may output the multiple code identifiers acquired from the composite image acquired by the image synthesis unit. With this configuration, even if the codes of the photographed subject contain the same code, by using the composite image, it is possible to obtain one code identifier for one code that exists in real space.

[0009] In the information processing device according to an aspect of the present invention, the generation unit may generate acquisition status information that is an image indicating, in the captured image, the position of the code for which the code identifier has been acquired. With this configuration, if there is a code whose position is not indicated among the codes included in the image that is the acquisition status information, it is possible to know that a code identifier has not been acquired for that code.

[0010] In addition, in an information processing device according to one aspect of the present invention, the generation unit may generate acquisition status information which is an image that displays a graphic indicating that a code identifier has been acquired at the position of the code for which the code identifier has been acquired. With this configuration, it is possible to know whether or not a code identifier has been acquired for a code depending on whether or not a graphic is displayed at the position of the code included in the image, which is the acquisition status information.

[0011] In addition, in an information processing device according to one aspect of the present invention, the generation unit may have access to multiple correspondence information that associates a code identifier with a code type corresponding to an object to which the code identified by the code identifier is assigned, and may generate acquisition status information that is an image that displays a figure displayed at the position of a code for which a code identifier has been acquired in a manner that allows it to be distinguished according to the code type associated with the code identifier by the correspondence information. With this configuration, for example, the object to which the code is assigned can be known by the graphic displayed at the position of the code for which the code identifier has been acquired. Therefore, for example, when it is necessary to photograph both the code assigned to the transportable object and the code assigned to the environment in which the object is placed, it becomes easy to determine whether both codes are included by referring to the image that is the acquisition status information.

[0012] In addition, in an information processing device according to one aspect of the present invention, the type of code may include a type indicating whether the object to which the code is assigned is a transportable object or an environment in which the transportable object is placed. With this configuration, by referring to the image that is the acquisition status information, it becomes possible to easily distinguish between, for example, a code given to a transportable object and a code given to the environment.

[0013] In the information processing device according to an aspect of the present invention, the type of code may include a type according to the affiliation of the object to which the code is assigned. With this configuration, by referring to the image that is the acquisition status information, it becomes possible to easily know, for example, the affiliation of the object to which the code is assigned.

[0014] In addition, in an information processing device according to one aspect of the present invention, the generation unit may identify each of the multiple codes contained in the captured image, and when the results of identifying the multiple codes and the results of acquiring the code identifier by the identifier acquisition unit indicate that there are codes for which the code identifier could not be acquired, generate acquisition status information indicating that there are codes for which the code identifier could not be acquired. With this configuration, the user can easily know from the acquisition status information whether there is a code for which a code identifier could not be acquired.

[0015] In addition, in an information processing device according to one aspect of the present invention, the identifier output unit may output, together with the multiple code identifiers, one or more captured images used in acquiring the multiple code identifiers and acquisition status information corresponding to the one or more captured images. With this configuration, for example, it is possible to check whether the code identifier has been properly acquired based on the output photographed image and acquisition status information.

[0016] Furthermore, an information processing method according to one aspect of the present invention is an information processing method that is processed using a reception unit, an image acquisition unit, an identifier acquisition unit, a generation unit, an acquisition status output unit, and an identifier output unit, and includes the steps of: the reception unit receiving a shooting instruction from a user; the image acquisition unit acquiring a captured image, which is a still image of multiple codes when the shooting instruction is accepted; the identifier acquisition unit acquiring a code identifier that identifies the code for each of the multiple codes included in the captured image; the generation unit generating acquisition status information that indicates the acquisition status of the code identifier for each captured image; the acquisition status output unit outputting the generated acquisition status information; and the identifier output unit outputting the multiple code identifiers acquired from the one or more acquired captured images. [Effects of the Invention]

[0017] According to an information processing device or the like according to an aspect of the present invention, it becomes possible to appropriately acquire a code identifier for each of a plurality of codes included in a captured image. [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 1 is a block diagram showing a configuration of an information processing device according to an embodiment of the present invention; [Figure 2] A flowchart showing the operation of the information processing device according to the embodiment. [Figure 3] FIG. 10 is a diagram showing an example of a chord and a graphic displayed at the chord position according to the embodiment; [Figure 4] FIG. 10 is a diagram showing an example of an object to which a code is assigned in the embodiment. [Figure 5] FIG. 10 shows an example of a code identifier and its position in the embodiment. [Figure 6] FIG. 10 is a diagram showing an example of display of acquisition status information in the embodiment; [Figure 7] FIG. 10 is a diagram showing an example of display of acquisition status information in the embodiment; [Figure 8]FIG. 10 is a block diagram showing another example of the configuration of the information processing device according to the embodiment. [Figure 9] FIG. 10 is a diagram showing an example of an object to which a code is assigned in the embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a composite image according to the embodiment; [Figure 11] FIG. 10 is a diagram showing an example of correspondence information according to the embodiment; [Figure 12] FIG. 10 is a diagram showing an example of display of acquisition status information in the embodiment; [Figure 13] FIG. 2 shows an example of the configuration of a computer system according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0019] An information processing device and an information processing method according to the present invention will be described below using embodiments. In the following embodiments, components and steps with the same reference numerals are the same or equivalent, and repeated description may be omitted. The information processing device according to this embodiment generates and outputs acquisition status information indicating the acquisition status of code identifiers for multiple codes included in a captured image, thereby enabling re-capturing when a captured image contains a code for which a code identifier has not been acquired.

[0020] 1 is a block diagram showing the configuration of an information processing device 1 according to this embodiment. The information processing device 1 according to this embodiment includes a reception unit 11, an image acquisition unit 12, a storage unit 13, an identifier acquisition unit 14, a generation unit 15, an acquisition status output unit 16, and an identifier output unit 17. The information processing device 1 may be, for example, a portable information processing terminal with a photographing function, such as a smartphone, a tablet terminal, or a PDA (Personal Digital Assistant). In this embodiment, the case where the information processing device 1 is a smartphone will be mainly described.

[0021] The reception unit 11 receives an instruction to take a photograph from a user. This instruction to take a photograph indicates the timing of taking a photograph. For example, the reception unit 11 may receive an instruction to take a photograph in response to the user tapping a photographing button displayed on a touch panel, or may receive an instruction to take a photograph in response to the user pressing a physical button for photographing. Note that the reception unit 11 may receive an instruction other than an instruction to take a photograph. For example, the reception unit 11 may receive an instruction to take a photograph again, an instruction to ensure that no code identifiers have been omitted from the photographed images, or an instruction to output code identifiers acquired from one or more photographed images.

[0022] The reception unit 11 may receive information input from an input device (for example, a touch panel or a physical button). The reception unit 11 may or may not include a device for receiving the information (for example, an input device). The reception unit 11 may be realized by hardware, or may be realized by software such as a driver that drives a predetermined device.

[0023] When the reception unit 11 receives an instruction to take a photograph, the image acquisition unit 12 acquires a photographed image, which is a still image of a plurality of codes. The image acquisition unit 12 may acquire, for example, one photographed image, or may acquire two or more photographed images. In this embodiment, a case in which two or more photographed images are acquired by the image acquisition unit 12 will be mainly described. The image acquisition unit 12 may be, for example, an optical device such as a camera that captures images, or may acquire images photographed by an optical device such as a camera. In this embodiment, a case in which the image acquisition unit 12 is a camera will be mainly described. The photographed image may be, for example, a color image, or a black and white or grayscale image. If the code has a color other than black and white, it is preferable that the photographed image be a color image.

[0024] The image acquisition unit 12 acquires the captured image when the instruction to capture the image is accepted by the acceptance unit 11. Therefore, at that time, it is preferable that the optical axis of the camera that acquires the captured image is directed toward the multiple codes by the user. Furthermore, the image acquisition unit 12 may store the acquired captured image in the storage unit 13, for example.

[0025] When the image acquisition unit 12 acquires multiple captured images, the multiple captured images may be, for example, the result of capturing multiple codes of the capture target in separate images. In this case, for example, the multiple captured images may or may not include overlapping portions. In the former case, for example, a single code may be included in two or more captured images. Furthermore, when capturing multiple codes in separate images, it is preferable that the capture be performed so that each of the multiple codes of the capture target is included in at least one of the captured images.

[0026] The code may be, for example, a color code such as Chameleon Code (registered trademark), a QR Code (registered trademark), a two-dimensional code such as an AR marker, a barcode, or other codes. The code may be, for example, a code composed of a predetermined number of two-dimensional codes. As an example, the code may be composed of a plurality of AR markers. The number of two-dimensional codes constituting the code may be, for example, two, three, or four or more. The AR marker may be, for example, an ArUco marker. Furthermore, it is preferable that the code be capable of simultaneously reading a plurality of codes so that code identifiers corresponding to the plurality of codes included in the captured image can be acquired collectively. In this embodiment, a case where the code is composed of two square AR markers will be mainly described. The two AR markers may be arranged with one side facing each other, as in the code 5 shown in FIG. 3(a).

[0027] The code to be photographed may be attached to the object, for example. The object may be, for example, a product, a storage object stored in a warehouse, or other object. The object may be, for example, a box-shaped object such as a cardboard box, a container, a foldable container, or a food box, or may be an object such as a bag, a bottle, or a can, or other object. In this embodiment, a case where the object is a box-shaped product will be mainly described. As an example, the image acquisition unit 12 may acquire captured images of codes attached to multiple product boxes stacked in a warehouse, for example. The object to which the code is attached may be, for example, a transportable object such as the box-shaped object described above. The object to which the code is attached may also be, for example, an environment in which a transportable object is placed. In this case, the code may be attached to an object that is fixed to the environment, such as a shelf, a floor, a wall, or a plate hanging from the ceiling, or an object whose position relative to the environment does not easily change.

[0028] The method by which a code is assigned to an object is not important. For example, the code may be attached to the object, printed on the surface of the object, a tag displaying the code may be attached to the object, or other methods may be used to assign the code to the object. It is preferable that the code is assigned to the object in a manner that makes it easy to read. For example, it is preferable that when the part of the object to which the code is assigned is photographed, the code can be read in the photographed image.

[0029] The multiple codes to be photographed may be, for example, all different codes, or may include overlapping codes. In this embodiment, the former case will be mainly described, and the latter case will be described later. Note that different codes may mean that the code identifiers that identify the codes are different.

[0030] The storage unit 13 stores, for example, photographed images and the like. The storage unit 13 may also store information such as a code identifier acquired from the photographed images. The storage unit 13 is preferably realized by a non-volatile recording medium, but may also be realized by a volatile recording medium. The recording medium may be, for example, a semiconductor memory or a magnetic disk.

[0031] The identifier acquisition unit 14 acquires a code identifier for identifying each of a plurality of codes included in the captured image. The identifier acquisition unit 14 may also acquire the position of the code along with the code identifier. The code position may be, for example, the position of the code in the captured image. The identifier acquisition unit 14 may acquire, for example, a pair of a code identifier and a position for each of a plurality of codes included in the captured image. The code position may be indicated by coordinate values ​​in a coordinate system of the captured image. The code position may be, for example, a representative position of the code or a position indicating the outline of the code. The representative position may be a predetermined position in the code, such as the center of the code, the center of gravity of the code, or the upper left vertex of the code. The identifier acquisition unit 14 may, for example, detect the code in the captured image and acquire the code identifier by reading the code in the detected code area. The code identifier may be any identifier that can identify the code. The code identifier may be, for example, the information indicated by the code (i.e., information obtained by reading the code) itself, or information managed in association with the information obtained by reading the code. In this embodiment, the former case will be mainly described.

[0032] When acquiring code identifiers using multiple captured images, it is preferable that the identifier acquiring unit 14 acquires one code identifier for each code present in the real space. Therefore, the identifier acquiring unit 14 may, for example, combine code identifiers acquired from two or more captured images acquired by the image acquiring unit 12. For example, the identifier acquiring unit 14 may combine the acquired code identifiers by performing unique processing on the code identifiers acquired from the multiple captured images. The unique processing may be processing to eliminate duplication. When multiple codes of the captured object are all different codes, combining the code identifiers in this manner makes it possible to acquire unique code identifiers. For example, each time a code identifier is acquired from one captured image, the identifier acquiring unit 14 may combine the previously acquired code identifier with the newly acquired code identifier. Alternatively, when the receiving unit 11 receives an instruction to output multiple code identifiers acquired from two or more captured images, the identifier acquiring unit 14 may combine the previously acquired code identifiers.

[0033] The generation unit 15 generates, for each captured image, acquisition status information indicating the acquisition status of the code identifier by the identifier acquisition unit 14. The acquisition status information may be, for example, information that enables knowledge of whether or not any code identifier has been omitted from acquisition for a plurality of codes included in the captured image.

[0034] For example, the generation unit 15 may generate acquisition status information that is an image showing the position of a code whose code identifier has been acquired in the captured image. For example, the generation unit 15 may generate acquisition status information that is an image that displays a graphic indicating that the code identifier has been acquired at the position of the code whose code identifier has been acquired in the captured image. This graphic may be a predetermined graphic, such as a circular graphic or a rectangular graphic. Displaying a graphic at the position of a code whose code identifier has been acquired may also mean displaying a graphic that shows the correspondence between the code and the graphic. For example, the graphic may be displayed superimposed on the code, near the code, or surrounding the code. The graphic 8 indicating that the code identifier has been acquired may be, for example, a frame-shaped graphic surrounding the code 5 as shown in FIG. 3(b), a graphic displayed on the code 5 as shown in FIG. 3(c), a graphic displayed adjacent to the code 5 as shown in FIG. 3(d), or another graphic. The generation unit 15 may, for example, receive the position of the code whose code identifier has been acquired from the identifier acquisition unit 14, and generate acquisition status information that is an image that displays a predetermined graphic at the received position in the captured image. In this case, among the codes included in the image that is the acquisition status information, a code that does not display a graphic indicating that the code identifier has been acquired is a code whose code identifier has not been acquired.

[0035] Furthermore, the generation unit 15 may generate acquisition status information that is an image in which the color of a code displayed at a position of a code whose code identifier has been acquired in the captured image is displayed in a predetermined color (for example, a color different from the color of the code existing in real space) indicating that the code identifier has been acquired. For example, if the original color of the code is black, the acquisition status information may be generated that is an image in which the color of the code displayed at a position of a code whose code identifier has been acquired is changed to red. In this case, among the codes included in the image that is the acquisition status information, codes that are not displayed in a color indicating that the code identifier has been acquired are codes whose code identifier has not been acquired.

[0036] As another example, the generation unit 15 may identify each of a plurality of codes included in the captured image, and when the identification results of the plurality of codes and the acquisition results of code identifiers for the captured image by the identifier acquisition unit 14 indicate that there is a code for which a code identifier could not be acquired, generate acquisition status information indicating that there is a code for which a code identifier could not be acquired. Note that, for example, when the identification results of the plurality of codes and the acquisition results of code identifiers indicate that there is no code for which a code identifier could not be acquired, the generation unit 15 may generate acquisition status information indicating that there is no code for which a code identifier could not be acquired.

[0037] The generation unit 15 may identify multiple codes included in the captured image by, for example, template matching, or may identify the codes using a learning model such as a neural network for detecting the codes. The identification of codes by the generation unit 15 may be performed in the same manner as the detection of codes in the captured image by the identifier acquisition unit 14, or may be performed using looser criteria than the detection of codes by the identifier acquisition unit 14. In the latter case, the generation unit 15 may also be able to identify codes not detected by the identifier acquisition unit 14.

[0038] Furthermore, the result of identifying multiple codes may be, for example, the number of codes identified in the captured image by the generation unit 15. In this case, when the number of codes identified in the captured image is smaller than the number of code identifiers acquired from the captured image, the generation unit 15 may generate acquisition status information indicating that there are codes for which code identifiers could not be acquired, and when the number of codes identified in the captured image is equal to the number of code identifiers acquired from the captured image, the generation unit 15 may generate acquisition status information indicating that there are no codes for which code identifiers could not be acquired.

[0039] Furthermore, the identification result of the multiple codes may be, for example, the positions of the codes identified by the generation unit 15 in the captured image. In this case, the identifier acquisition unit 14 may also acquire a pair of a code identifier and the position of each code included in the captured image. Then, when some of the code positions identified in the captured image have not been acquired by the identifier acquisition unit 14, the generation unit 15 may generate acquisition status information indicating that there are codes whose code identifiers could not be acquired. When all of the code positions identified in the captured image have been acquired by the identifier acquisition unit 14, the generation unit 15 may generate acquisition status information indicating that there are no codes whose code identifiers could not be acquired. Note that, for example, when the distance between the position of the code identified in the captured image and the position of the code acquired by the identifier acquisition unit 14 is smaller than a predetermined threshold, the generation unit 15 may determine that the position of the identified code has been acquired by the identifier acquisition unit 14.

[0040] The acquisition status output unit 16 outputs the acquisition status information generated by the generation unit 15. It is preferable that the generation unit 15 generates the acquisition status information and the acquisition status output unit 16 outputs the acquisition status information each time a captured image is acquired. This output may be, for example, a display on a display device (e.g., a liquid crystal display or an organic EL display), transmitted to a predetermined device via a communication line, printed by a printer, output as audio through a speaker, stored in a recording medium, or passed to another component. In any case, it is preferable that the acquisition status information is ultimately presented to a user operating the information processing device 1. It is preferable that the acquisition status output unit 16 include a device for outputting the information (e.g., a display device or a speaker). The acquisition status output unit 16 may or may not include a device for outputting the information (e.g., a display device or a speaker). The acquisition status output unit 16 may be implemented by hardware, or may be implemented by software such as a driver for driving such a device.

[0041] For example, if a user photographs multiple codes and the acquisition status information output for a certain captured image indicates that a code identifier has been missed, the user may delete the captured image and photograph the multiple codes again, or if the acquisition status information indicates that no code identifier has been missed, the user may photograph the multiple codes again.

[0042] The identifier output unit 17 outputs a plurality of code identifiers acquired from one or more captured images acquired by the image acquisition unit 12. For example, when an instruction to output a plurality of code identifiers acquired from one or more captured images is received by the reception unit 11, the identifier output unit 17 may output the plurality of code identifiers. Typically, the identifier output unit 17 outputs the plurality of code identifiers acquired from one or more captured images after a series of acquisitions of the one or more captured images is completed. Furthermore, for example, when the identifier acquisition unit 14 integrates a plurality of code identifiers, the identifier output unit 17 may output the plurality of code identifiers integrated by the identifier acquisition unit 14. The output code identifiers may be used, for example, for inventory.

[0043] The identifier output unit 17 may output, together with the code identifiers, one or more captured images used in acquiring the code identifiers and acquisition status information corresponding to the one or more captured images. The acquisition status information corresponding to the captured image may be acquisition status information indicating the acquisition status of the code identifiers from the captured image. In this case, the output captured image and acquisition status information may be used, for example, as evidence that no code identifier has been missed from the output code identifiers.

[0044] Here, this output may be, for example, a display on a display device (e.g., a liquid crystal display or an organic EL display), a transmission to a predetermined device via a communication line, a printing by a printer, storage on a recording medium, or a transfer to another component. The identifier output unit 17 may or may not include a device that performs the output (e.g., a display device or a communication device). The identifier output unit 17 may be realized by hardware, or may be realized by software such as a driver that drives such a device.

[0045] Next, the operation of the information processing device 1 will be described with reference to the flowchart of Fig. 2. This flowchart will mainly describe the case where two or more captured images are acquired. (Step S101) The reception unit 11 determines whether or not an instruction to take a photograph has been received. If an instruction to take a photograph has been received, the process proceeds to step S102, and if not, the process proceeds to step S108.

[0046] (Step S102) The image acquisition unit 12 acquires a captured image. The captured image may be stored in the storage unit 13, for example.

[0047] (Step S103) The identifier acquisition unit 14 acquires a code identifier from the photographed image acquired in step S102. The identifier acquisition unit 14 may acquire, for example, a pair of a code identifier and a code position. The acquired code identifier may be stored in the storage unit 13 in association with the photographed image.

[0048] (Step S104) The generation unit 15 generates acquisition status information for the photographed image acquired in step S102. The generated acquisition status information may be stored in the storage unit 13, for example, in association with the photographed image.

[0049] (Step S105) The acquisition status output unit 16 outputs the acquisition status information generated in step S104.

[0050] (Step S106) Image acquisition unit 12 determines whether or not to take another photograph. If taking another photograph, the process proceeds to step S107; if not, the process returns to step S101. Note that image acquisition unit 12 may determine to take another photograph when reception unit 11 receives a request to take another photograph, or may determine not to take another photograph when reception unit 11 receives a request that there are no problems with the photograph or that another photograph will not be taken.

[0051] (Step S107) The image acquisition unit 12 deletes the latest captured image and the code identifier and acquisition status information corresponding to the captured image. This deletion may be, for example, deletion of the captured image stored in the storage unit 13. Then, the process returns to step S101.

[0052] (Step S108) The identifier output unit 17 determines whether to output the code identifier. If the code identifier is to be output, the process proceeds to step S109; if not, the process returns to step S101. Note that the identifier output unit 17 may determine to output the code identifier when, for example, the receiving unit 11 receives an instruction to output the code identifier.

[0053] (Step S109) The identifier acquisition unit 14 integrates the multiple code identifiers acquired from the two or more captured images.

[0054] (Step S110) The identifier output unit 17 outputs the code identifier integrated in step S109. The identifier output unit 17 may output the captured image and acquisition status information together with the code identifier. Then, the process returns to step S101. After outputting the code identifier, the identifier output unit 17 may delete the output code identifier, the captured image used to acquire the code identifier, and the acquisition status information corresponding to the captured image from the storage unit 13.

[0055] The order of the steps in the flowchart of Fig. 2 is an example, and the order of the steps may be changed as long as the same results are obtained. Also, in the flowchart of Fig. 2, the processing may be terminated by power-off or an interrupt to terminate the processing.

[0056] Next, the operation of the information processing device 1 according to this embodiment will be described using a specific example. In this specific example, a case will be described in which multiple box-shaped products 3 with codes 5 attached to their sides, as shown in FIG. 4, are photographed by the information processing device 1, which is a smartphone. In this specific example, it is assumed that the codes 5 are composed of two AR markers arranged side by side, as shown in FIG. 4. In this specific example, it is assumed that the codes 5 to be photographed are all different codes.

[0057] When a user selects the capture button 1a while pointing the information processing device 1, which is a smartphone, at a product 3 as shown in FIG. 4, the reception unit 11 receives a capture instruction and transfers the capture instruction to the image acquisition unit 12 (step S101). The image acquisition unit 12 then acquires captured images of multiple codes present in the optical axis direction of the camera at that time, stores the acquired first captured image in the storage unit 13, and transfers the first captured image to the identifier acquisition unit 14 (step S102). Upon receiving the first captured image, the identifier acquisition unit 14 acquires a pair of a code identifier and a position of the code in the captured image for each of the multiple codes included in the captured image, as shown in FIG. 5, transfers each position of the code to the generation unit 15, and stores the acquired code identifier in association with the first captured image stored in the storage unit 13 (step S103). In this specific example, the position of the code is assumed to be the position of the center of gravity of the code.

[0058] Upon receiving the code positions, the generation unit 15 reads out the most recent captured image stored in the storage unit 13, i.e., the first captured image, and generates acquisition status information, which is an image in which rectangular shapes surrounding the code are displayed at each of the received positions in the captured image, and passes the information to the acquisition status output unit 16 (step S104). The acquisition status output unit 16 displays the received acquisition status information on the touch panel of the information processing device 1 (step S105). As a result, the display shown in FIG. 6 is produced.

[0059] In Fig. 6, no rectangular shape is displayed for the code in the lower right, indicating that the code identifier for that code has not been acquired, so the user selects the "Retake" button in the display in Fig. 6. Then, the acceptance unit 11 accepts the instruction to take another photograph and passes it to the image acquisition unit 12 (step S106). Upon receiving the instruction to take another photograph, the image acquisition unit 12 deletes the first photographed image and the code identifier and acquisition status information stored in association with it from the storage unit 13 (step S107).

[0060] Thereafter, when the user selects the capture button 1a, the reception unit 11 receives a capture instruction in response, and the image acquisition unit 12 acquires the second captured image (steps S101 and S102). Then, from the second captured image, the identifier acquisition unit 14 acquires a pair of a code identifier and a position (step S103). Furthermore, using the position, acquisition status information is generated and displayed as shown in FIG. 7 (steps S104 and S105). In FIG. 7, rectangular shapes are displayed for all codes, so the user determines that code identifiers have been appropriately acquired for all codes and selects the "OK" button. In this manner, the third and subsequent captured images are acquired (step S106).

[0061] When all the codes to be photographed, i.e., the codes 5 assigned to all the products 3 shown in FIG. 4, have been photographed, the user inputs an instruction to the information processing device 1 to output the acquired code identifiers. The instruction is received by the reception unit 11 and passed to the identifier acquisition unit 14 and the identifier output unit 17 (step S108). Upon receiving the instruction to output the code identifiers, the identifier acquisition unit 14 integrates the code identifiers associated with the multiple photographed images stored in the storage unit 13 and passes the integrated code identifier to the identifier output unit 17 (step S109). Upon receiving the integrated code identifiers, the identifier output unit 17 transmits the integrated code identifiers, the photographed images stored in the storage unit 13, and the acquisition status information stored in association with the photographed images to a predetermined address (step S110). The identifier output unit 17 also deletes the transmitted photographed images, the code identifiers stored in association with the photographed images, and the acquisition status information from the storage unit 13. In this way, the series of processes of photographing a code, acquiring a code identifier of the photographed code, and transmitting the code identifier is completed. The transmitted code identifier may be used for, for example, inventory.

[0062] As described above, the information processing device 1 according to the present embodiment can output acquisition status information indicating the acquisition status of code identifiers from captured images. Therefore, the output acquisition status information allows the user to know whether code identifiers were properly acquired from each code included in the captured image. If code identifiers could not be acquired for some codes, the user can re-acquire the captured image to prevent missing code identifiers. Furthermore, by using still images rather than moving images as the captured images from which code identifiers are to be acquired, it becomes easy to determine which captured images need to be re-captured if a code identifier could not be properly acquired for a certain captured image. Furthermore, by outputting the captured image and acquisition status information together with the acquired code identifiers, the user can later confirm whether the output code identifiers were properly acquired from the captured images.

[0063] In this embodiment, the case where there is no overlap in the codes of the photographed objects has been mainly described, but there may be overlap in the codes of the photographed objects. In other words, the codes of the photographed objects do not all have to be different codes. In this case, since it is not possible to perform unique processing when obtaining and outputting the code identifier of the output object from multiple photographed images, it is also possible to obtain a composite image by combining multiple photographed images, and obtain the code identifier from the composite image. This processing will be described below. When combining photographed images, the information processing device 1 may further include an image combining unit 18, as shown in FIG. 8.

[0064] The image synthesis unit 18 synthesizes two or more captured images acquired by the image acquisition unit 12 using a code contained in the captured images as a reference to obtain a composite image. Synthesizing using a code as a reference means that, when a certain code is contained in multiple captured images, the multiple captured images are synthesized so that the position of the code is the same in the composite image. Whether a code contained in a first captured image is the same as a code contained in a second captured image can be determined by whether the code identifiers of the codes are the same. Therefore, more accurate image synthesis is possible compared to synthesizing multiple captured images using only the shape of the subject contained in the captured images. In particular, as shown in FIG. 4, when the subject has a repetitive monotonous shape, synthesizing multiple captured images using only the shape of the subject may result in an incorrect synthesis. However, synthesizing multiple captured images using a code as a reference can avoid such an incorrect synthesis and obtain an accurate composite image corresponding to the subject. When synthesizing two captured images using a code contained in the captured images as a reference, it is preferable that the two captured images contain an image of the same code that exists in real space. The image synthesis unit 18 may, for example, store the synthesized image in the storage unit 13.

[0065] The code used as a reference when combining multiple captured images may be a code assigned to a transportable object such as a product, or may be a code different from the code assigned to the transportable object. The latter case will be described below. The code assigned to the transportable object may be, for example, a code used for processing such as inventory. Furthermore, a code different from the code assigned to the transportable object and used as a reference for combination may also be referred to as an "image combining code." It is preferable that all image combining codes be different. The image combining code may be located in a location where the position of the code does not change in real space, such as a shelf, wall, pillar, or floor. For example, as shown in FIG. 9 , an image combining code 6 may be located on a shelf 7 storing multiple products 3. Note that, for example, a location code used to acquire the location of the product 3 may be used as the image combining code. The image combining code may or may not be the subject of acquisition of a code identifier by the identifier acquisition unit 14. When the code identifier of the image compositing code is also acquired, for example, acquisition status information for the image compositing code may also be generated, or may not be generated. Also, the code identifier of the image compositing code may also be output, or may not be output.

[0066] The image synthesis unit 18 may synthesize at least some of the captured images by enlarging or reducing them so that the sizes of the subjects included in the two or more captured images are the same. The enlarging or reducing of the captured images may be performed by changing the size (number of pixels) of the captured images. To synthesize the sizes of the subjects included in the two or more captured images, the image synthesis unit 18 may synthesize at least one of the captured images by enlarging or reducing it, for example, as follows: The image synthesis unit 18 can identify at least one corresponding point between the first and second captured images by using the position of a code included in a first captured image and the position of the same code included in a second captured image. Then, the image synthesis unit 18 may enlarge or reduce at least one of the images so that the size of the shape around the corresponding point (e.g., a contour extracted by contour extraction, a segment extracted by image segmentation, etc.) is the same. Furthermore, if multiple codes included in the first captured image are also included in the second captured image, the image synthesis unit 18 may enlarge or reduce at least one of the images so that the distances between the multiple codes are equal. Furthermore, if the actual size of the code displayed on the product or the like is the same, the image composition unit 18 may enlarge or reduce at least one of the captured images so that the size of the code included in the multiple captured images is the same. From the viewpoint of making it possible to read the code included in the composite image, enlargement, which increases the number of pixels, is more preferable than reduction, which decreases the number of pixels. Therefore, the image composition unit 18 may enlarge and combine at least some of the captured images so that the size of the subject included in two or more captured images is the same. Furthermore, if the size of the subject included in two or more captured images is the same, it is not necessary to enlarge or reduce the captured images when combining them. Furthermore, if it is assumed that the outline of the product or shelf included in the captured images is rectangular, the image composition unit 18 may perform keystone correction (correction of trapezoidal distortion) so that the outline becomes rectangular before combining the images. Furthermore, the image composition unit 18 may also combine the multiple captured images after matching the exposure, color tone, etc. of the multiple captured images.

[0067] Furthermore, when combining the first to third captured images, the image combining unit 18 may, for example, combine the first and second captured images using a first code that serves as a reference for combination, and then combine the second and third captured images using a second code that serves as a reference for combination, thereby combining the first to third captured images. The first and second codes may, for example, be image combination codes. Furthermore, for example, the first code may be included only in the first and second captured images, and the second code may be included only in the second and third captured images. In a similar manner, three or more captured images may be combined. As an example, the image combining unit 18 may combine the captured images M1 to M6 to obtain a composite image M7 shown in FIG. 10.

[0068] When a composite image is acquired by the image composition unit 18, the identifier acquisition unit 14 may also acquire a code identifier for each of a plurality of codes included in the composite image. For example, the identifier acquisition unit 14 may acquire a code identifier from a composite image M7 shown in FIG. 10 . Then, the identifier output unit 17 may output a plurality of code identifiers acquired by the identifier acquisition unit 14 from the composite image acquired by the image composition unit 18. Even in this case, since the composite image is composed of two or more captured images acquired by the image acquisition unit 12, the identifier output unit 17 outputs a plurality of code identifiers acquired from two or more captured images acquired by the image acquisition unit 12. Note that when an image composition code is included in the composite image, for example, the code identifier may not be acquired or output for the image composition code, or the code identifier may be acquired or output.

[0069] The image composition unit 18 may compose multiple captured images without using an image composition code. In this case, for example, the multiple captured images may be composed using two or more codes included in the captured images and the positional relationship of the codes. As an example, if the first and second codes included in the first captured image and their positional relationship are the same as the first and second codes included in the second captured image and their positional relationship, the first and second captured images may be composed using the first and second codes as a basis for composition.

[0070] As explained above, even when images are combined, acquisition status information is generated and output for each captured image. By appropriately re-capturing the captured images in accordance with the output of the acquisition status information, it is possible to prevent the omission of code identifiers from being acquired in the multiple captured images to be combined, and as a result, it is possible to prevent the omission of code identifiers from being acquired in the combined image.

[0071] Although the above description has been given mainly on the case where two or more captured images are combined when there is overlap in the codes of the photographed objects, this is not the only option. Even when there is no overlap in the codes of the photographed objects, two or more captured images may be combined and the code identifier for output may be obtained from the combined image.

[0072] Furthermore, in the present embodiment, when there is overlap among multiple codes of a photographed object, a composite image obtained by combining multiple photographed images is used to obtain a code identifier for output. However, this is not necessarily the case. When there is overlap among multiple codes of a photographed object, the code identifier for output may be obtained by integrating multiple code identifiers using the code positions as well. In this case, for example, each photographed image may include a reference code. The reference code is a code that indicates a positional reference. For example, the position of the reference code may serve as the reference position for the positions of other codes. As an example, each reference code may be associated with its position, for example, coordinate values ​​in a predetermined coordinate system. Then, the position of the reference code may be used to obtain the position of a code that is in a predetermined positional relationship with the reference code.

[0073] For example, the predetermined coordinate system, an xy Cartesian coordinate system, may be set so that the x-axis direction is horizontal and the y-axis direction is vertical. Furthermore, the unit length in the xy Cartesian coordinate system may be determined, for example, according to the length of the reference cord or the lengths of other cords. For example, the longitudinal length of the reference cord may be the unit length. In this way, even if a captured image contains only one reference cord, the positions of other cords relative to the reference cord can be determined using the unit length. Furthermore, if a single captured image acquired by the image acquisition unit 12 contains two or more reference cords, the unit length may be determined using the two or more reference cords. This unit length may be calculated, for example, by dividing the distance between two reference cords on the captured image by the distance between two positions in the predetermined coordinate system corresponding to the two reference cords. Furthermore, the captured image may be captured so that the left-right direction of the captured image is horizontal, i.e., the x-axis direction in the xy Cartesian coordinate system, and the up-down direction is vertical, i.e., the y-axis direction in the xy Cartesian coordinate system. In this way, in one captured image, the position of a code relative to the position of a reference code can be determined using a unit length, and the position (coordinate value) of the reference code in the xy orthogonal coordinate system can be used to determine the position (coordinate value) of the code. The reference code may be the same as other codes except that it is used as a reference for the code position. It is preferable that the position of the reference code does not change in real space. Therefore, it is preferable that the reference code be placed on a structure such as a shelf, wall, pillar, or floor.

[0074] In this case, the identifier acquisition unit 14 may acquire, from each captured image, a pair of a code identifier and a code position identified by the code identifier. The identifier acquisition unit 14 may then integrate the acquired pairs using the code position. In this integration, the identifier acquisition unit 14 may integrate the pairs so that one pair remains for each code existing in real space. For this reason, the identifier acquisition unit 14 may perform unique processing on the pairs of code identifiers and positions acquired from the multiple captured images. In this unique processing, positions separated by a distance shorter than a predetermined threshold may be considered to be the same position. The threshold is preferably set to a value such that two positions separated by a distance shorter than the threshold are considered to be substantially the same position. The identifier acquisition unit 14 may then output each of the code identifiers included in the integrated set.

[0075] Furthermore, in this embodiment, when generating acquisition status information that is an image displaying a graphic indicating that a code identifier has been acquired at the position of a code for which the code identifier has been acquired, the generation unit 15 may generate acquisition status information that is an image displaying the graphic in a manner that distinguishes between the code types. In this case, for example, the generation unit 15 may be able to access multiple pieces of correspondence information that associate the code identifier with the code type corresponding to the object to which the code identified by the code identifier is assigned. The multiple pieces of correspondence information may be stored, for example, in the storage unit 13 accessible to the generation unit 15, or in a server or the like external to the information processing device 1 that is accessible to the generation unit 15. Then, the generation unit 15 may generate acquisition status information that is an image displaying a graphic displayed at the position of a code for which the code identifier has been acquired in a manner that distinguishes between the code types associated with the code identifier by the correspondence information.

[0076] The code type may include, for example, a type indicating whether the object to which the code is assigned is a transportable object or an environment in which the transportable object is placed. In this case, the code type may include, for example, an object code assigned to the transportable object and a position code assigned to the environment, such as a shelf, floor, or wall. The position code may be, for example, a code indicating the location where the transportable object is placed. In this case, for example, a captured image may be acquired so as to include at least one position code. This is because doing so makes it possible to identify the location of the object to which the object code is assigned. As an example, location information corresponding to the position code may be acquired by using information that associates a code identifier that identifies the position code with location information indicating the location of the object to which the position code is assigned. For example, the location code used to acquire the location of the product 3 described above may be a position code. Furthermore, the position code may be used, for example, as the image synthesis code 6 or the reference code described above.

[0077] FIG. 11 is a diagram illustrating an example of correspondence information. In FIG. 11, the correspondence information includes a code identifier and a code type identified by the code identifier. Note that the code identifier and the code type included in one piece of correspondence information may correspond to each other. For example, the code type identified by the code identifier "G001" may be an "object code" assigned to a transportable object. Note that, although FIG. 11 illustrates a case in which one code identifier and one type are associated by one piece of correspondence information, this is not the only possible case. One piece of correspondence information may associate multiple code identifiers with one type. For example, the correspondence information may be information that associates a code identifier "G0**" with the code type "object code." Note that "*" may be any number from 0 to 9. In this case, the code types corresponding to the code identifiers "G000" to "G099" are "object codes."

[0078] Furthermore, displaying a graphic indicating that a code identifier has been acquired in a manner that allows it to be distinguished for each code type may involve, for example, changing the color of the graphic for each code type, changing the shape of the graphic for each code type, changing the size of the graphic for each code type, changing the display position of the graphic for each code type, or changing other attributes of the graphic for each code type. The generation unit 15 may generate an acquisition status information image that displays a graphic displayed at the position of the code identifier acquired according to the display attribute associated with the code type corresponding to the code identifier, using information that associates the code type with the display attribute (e.g., color, etc.). When the color of the graphic is changed for each code type, for example, the graphic displayed for the object code may be black, and the graphic displayed for the position code may be red. When the shape of the graphic is changed for each code type, for example, the graphic displayed for the object code may be a rectangular graphic surrounding the object code, and the graphic displayed for the position code may be a circular graphic surrounding the position code. When the size of the graphic is changed for each code type, for example, the graphic displayed for the object code may be larger than the graphic displayed for the position code. Furthermore, when the position at which the graphic is displayed is changed for each type of code, as an example, the graphic displayed for the object code may be a graphic displayed surrounding the object code, and the graphic displayed for the position code may be a graphic displayed on the position code.

[0079] As a specific example, the image of acquisition status information output for a product 3 assigned an object code 5 and placed on a shelf 7 assigned a location code 9 may be as shown in FIG. 12. In the image of acquisition status information shown in FIG. 12, for example, for an object code 5 assigned to a product 3 that is a transportable object, a black frame-shaped graphic 8a may be displayed indicating that the code has been acquired, and for a location code 9 assigned to a shelf 7, a red frame-shaped graphic 8b may be displayed indicating that the code has been acquired. Note that in FIG. 12, the black frame-shaped graphic 8a is shown with a solid line, and the red frame-shaped graphic 8b is shown with a dashed line. As described above, these graphics 8a and 8b are displayed according to the type of code associated with the code identifier. By displaying the graphics 8a and 8b in this manner, for example, by capturing an image of the output acquisition status information in such a way that each code 5, 9 is surrounded by a frame-shaped graphic 8a, 8b and that at least one red frame-shaped graphic 8b is included in the image, it is possible to easily confirm that a code identifier has been acquired for each code 5, 9 included in the captured image and that at least one position code 9 is included in the captured image. Note that, when both a code identifier corresponding to an object code and a code identifier corresponding to a position code are acquired from a captured image, for example, the code identifier corresponding to the object code and the code identifier corresponding to the position code acquired from one captured image may be output in association with each other, or the code identifier corresponding to the object code and the position information associated with the code identifier corresponding to the position code acquired from one captured image may be output in association with each other. This allows, for example, the correspondence between the code identifier corresponding to the object code and the code identifier and position information corresponding to the position code to be known, thereby making it possible to know the location of the object to which the object code was assigned. Note that the location information corresponding to the location code may be different for each location code, or the same location information may be associated with location codes that exist in the same location.In the latter case, for example, the location information corresponding to multiple location codes assigned to the same shelf may be the same. Specifically, when multiple image synthesis codes 6 assigned to the shelf 7 shown in Fig. 9 are location codes, the location information corresponding to the multiple location codes assigned to the shelf 7 may all be the same. In such a case, for example, the code identifiers may be integrated for each piece of location information.

[0080] Here, the description has been given mainly on the case where the code type includes a type indicating whether the object to which the code is assigned is a transportable object or an environment in which the object is placed, i.e., the case where the code includes an object code and a location code. However, it goes without saying that the code type is not limited to these. The code type may also include, for example, a type according to the affiliation of the object to which the code is assigned. In this case, the code may include a code assigned to an object of Company A, a code assigned to an object of Company B, a code assigned to an object of Company C, etc. Acquisition status information may be generated as an image that displays a graphic displayed at the position of a code for which a code identifier has been acquired, in a manner that distinguishes the affiliation of the object corresponding to the code identifier. For example, the graphic displayed for a code assigned to an object of Company A may be black, the graphic displayed for a code assigned to an object of Company B may be red, and the graphic displayed for a code assigned to an object of Company C may be green. In this way, by displaying the graphic in a color according to the affiliation, the user can easily understand the affiliation of each object by referring to the acquisition status information. As another example, the graphic displayed for the code assigned to an object belonging to Company A may be a string indicating "Company A," the graphic displayed for the code assigned to an object belonging to Company B may be a string indicating "Company B," and the graphic displayed for the code assigned to an object belonging to Company C may be a string indicating "Company C." In this case, the user can learn the affiliation of each object from the graphic itself.

[0081] Although the above description has been focused on a case where acquisition status information is generated as an image that displays a graphic at the position of a code whose code identifier has been acquired in a manner that distinguishes the code type associated with the code identifier, this is not necessarily the case. For example, acquisition status information may be generated as an image that displays a code displayed at the position of a code whose code identifier has been acquired in a captured image in a manner that distinguishes the code type associated with the code identifier. For example, each code may be displayed in a different color depending on its type. In this case, for example, if the original color of the code is black, the code whose code identifier has been acquired may be displayed in a color other than black (e.g., red, blue, yellow, etc.) depending on the code type associated with the code identifier. This makes it possible to know whether a code identifier has been acquired and the type of code corresponding to the acquired code identifier based on the color of each code in the image of the acquisition status information. Furthermore, for example, the code itself may be distinguishable by its type. As an example, the code itself may have a color that corresponds to the code type. For example, the object code may be a black code and the position code may be a red code. This allows the user to understand the type of code based on the code color. In this case, the graphic displayed at the position of the chord for which the chord identifier has been acquired does not need to be displayed in a manner that allows distinction between the chord types.

[0082] Furthermore, in the present embodiment, the case where two or more captured images are acquired has been mainly described, but this is not necessarily the case. Only one captured image may be acquired. In this case, integration of the acquired code identifiers, synthesis of the two or more captured images, etc. may not be performed.

[0083] Furthermore, in the above embodiments, each process or function may be realized by centralized processing by a single device or a single system, or may be realized by distributed processing by multiple devices or multiple systems.

[0084] Furthermore, in the above embodiments, the transfer of information between components may be performed, for example, by one component outputting information and the other component receiving information if the two components transferring the information are physically different, or by moving from a processing phase corresponding to one component to a processing phase corresponding to the other component if the two components transferring the information are physically the same.

[0085] Furthermore, in the above-described embodiments, information related to the processing performed by each component, such as information accepted, acquired, selected, generated, transmitted, or received by each component, and information such as thresholds, formulas, and addresses used in processing by each component, may be temporarily or long-term stored in a recording medium (not shown), even if not explicitly stated in the above description. Furthermore, the storage of information in the recording medium (not shown) may be performed by each component or a storage unit (not shown). Furthermore, the reading of information from the recording medium (not shown) may be performed by each component or a reading unit (not shown).

[0086] Furthermore, in the above-described embodiments, if the information used by each component, such as thresholds, addresses, and various setting values ​​used by each component in processing, may be changed by the user, the user may or may not be able to change the information as appropriate, even if not explicitly stated in the above description. If the information is changeable by the user, the change may be realized, for example, by a receiving unit (not shown) that receives a change instruction from the user and a changing unit (not shown) that changes the information in accordance with the change instruction. The change instruction may be received by the receiving unit (not shown), for example, from an input device, by receiving information transmitted via a communication line, or by receiving information read from a predetermined recording medium.

[0087] Furthermore, in the above embodiment, when two or more components included in the information processing device 1 have a communication device, an input device, etc., the two or more components may physically have a single device, or may have separate devices.

[0088] Furthermore, in the above-described embodiments, each component may be configured by dedicated hardware, or a component that can be realized by software may be realized by executing a program. For example, each component may be realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. During execution, the program execution unit may execute the program while accessing a storage unit or recording medium. Note that the software for realizing the information processing device in the above-described embodiments is the following program. That is, this program may cause a computer to function as: a reception unit that receives a photographing instruction from a user; an image acquisition unit that, when the reception unit receives the photographing instruction, acquires a photographed image, which is a still image of multiple codes; an identifier acquisition unit that acquires code identifiers that identify the multiple codes included in the photographed image; a generation unit that generates, for each photographed image, acquisition status information indicating the acquisition status of the code identifiers by the identifier acquisition unit; an acquisition status output unit that outputs the acquisition status information generated by the generation unit; and an identifier output unit that outputs the multiple code identifiers acquired from one or more photographed images acquired by the image acquisition unit.

[0089] Note that the functions realized by the program do not include functions that can only be realized by hardware, such as a modem or interface card in an acquisition unit that acquires information or an output unit that outputs information.

[0090] This program may be executed by being downloaded from a server or the like, or by being read from a predetermined recording medium (for example, an optical disk such as a CD-ROM, a magnetic disk, or a semiconductor memory). This program may also be used as a program constituting a program product. This program may be executed by a single computer or multiple computers. In other words, it may perform centralized processing or distributed processing.

[0091] 13 is a diagram showing an example of a computer system 900 that executes the above program to realize the information processing device 1 according to the above embodiment. The above embodiment can be realized by computer hardware and a computer program executed thereon.

[0092] 13, computer system 900 includes an MPU (Micro Processing Unit) 911, a ROM 912 such as a flash memory in which programs such as a boot-up program, application programs, system programs, and data are stored, a RAM 913 connected to MPU 911 for temporarily storing instructions of application programs and providing temporary storage space, a touch panel 914, a wireless communication module 915, and a bus 916 for interconnecting MPU 911, ROM 912, etc. Note that a wired communication module may be provided instead of wireless communication module 915. Also, instead of touch panel 914, a display and input devices such as a mouse and a keyboard may be provided.

[0093] A program that causes the computer system 900 to execute the functions of the information processing device 1 according to the above embodiment may be stored in the ROM 912 via the wireless communication module 915. The program is loaded into the RAM 913 when executed. The program may also be loaded directly from the network.

[0094] The program does not necessarily include an operating system (OS) or a third-party program that causes the computer system 900 to execute the functions of the information processing device 1 according to the above embodiment. The program may include only instructions that call appropriate functions or modules in a controlled manner to achieve the desired results. How the computer system 900 operates is well known, and a detailed description thereof will be omitted.

[0095] Furthermore, the above-described embodiments are merely examples for specifically implementing the present invention, and are not intended to limit the technical scope of the present invention. The technical scope of the present invention is defined by the claims, not by the description of the embodiments, and is intended to include modifications within the literal scope of the claims and within the scope of equivalent meanings. [Explanation of symbols]

[0096] 1. Information processing equipment 11 Reception 12 Image acquisition unit 13 Storage section 14 Identifier acquisition unit 15 Generation part 16 Acquisition status output section 17 Identifier output section 18 Image synthesis unit

Claims

1. a reception unit that receives a photographing instruction from a user; an image acquisition unit that acquires a captured image, which is a still image of a plurality of codes, when the instruction to capture the image is accepted by the accepting unit; an identifier acquisition unit that acquires a code identifier that identifies each of a plurality of codes included in the captured image; a generating unit that generates acquisition status information indicating an acquisition status of the code identifier by the identifier acquiring unit for each captured image; an acquisition status output unit that outputs the acquisition status information generated by the generation unit; an identifier output unit that outputs a plurality of code identifiers acquired from one or more captured images acquired by the image acquisition unit.

2. the image acquisition unit acquires two or more captured images, The multiple codes being photographed are all different codes, the identifier acquisition unit integrates code identifiers acquired from two or more photographed images acquired by the image acquisition unit, The information processing apparatus according to claim 1 , wherein the identifier output unit outputs the plurality of code identifiers integrated by the identifier acquisition unit.

3. the image acquisition unit acquires two or more captured images, an image synthesis unit for synthesizing two or more captured images acquired by the image acquisition unit based on a code included in the captured images, and acquiring a synthesized image; the identifier acquisition unit acquires a code identifier for each of a plurality of codes included in the composite image; The information processing apparatus according to claim 1 , wherein the identifier output unit outputs a plurality of code identifiers obtained from the composite image obtained by the image composition unit.

4. The information processing apparatus according to claim 1 , wherein the generating unit generates acquisition status information that is an image showing a position of a code for which a code identifier has been acquired in a captured image.

5. The information processing apparatus according to claim 4 , wherein the generating unit generates the acquisition status information as an image displaying a graphic indicating that the code identifier has been acquired at a position of the code where the code identifier has been acquired.

6. 6. The information processing device according to claim 5, wherein the generation unit is capable of accessing a plurality of correspondence information that associates a code identifier with a code type corresponding to an object to which the code identified by the code identifier is assigned, and generates acquisition status information that is an image that displays a figure displayed at the position of a code for which a code identifier has been acquired in a manner that allows it to be distinguished for each code type associated with the code identifier by the correspondence information.

7. The information processing apparatus according to claim 6 , wherein the type of code includes a type indicating whether the object to which the code is assigned is a transportable object or an environment in which the transportable object is placed.

8. The information processing device according to claim 6, wherein the types of codes include types according to the affiliation of the object to which the code is assigned.

9. 4. An information processing device according to claim 1, wherein the generation unit identifies each of a plurality of codes contained in the captured image, and when the identification result of the plurality of codes and the acquisition result of the code identifier by the identifier acquisition unit indicate that there is a code for which a code identifier could not be acquired, generates acquisition status information indicating that there is a code for which a code identifier could not be acquired.

10. 4. The information processing device according to claim 1, wherein the identifier output unit outputs, together with the plurality of code identifiers, one or more captured images used in acquiring the plurality of code identifiers, and acquisition status information corresponding to the one or more captured images.

11. An information processing method performed using a reception unit, an image acquisition unit, an identifier acquisition unit, a generation unit, an acquisition status output unit, and an identifier output unit, the receiving unit receiving an instruction to take a photograph from a user; a step in which the image acquisition unit acquires a captured image, which is a still image of a plurality of codes, when an instruction to capture an image is received; the identifier acquisition unit acquiring a code identifier for identifying each of a plurality of codes included in the captured image; generating acquisition status information indicating an acquisition status of a code identifier for each captured image by the generating unit; a step of outputting the generated acquisition status information by the acquisition status output unit; and a step in which the identifier output unit outputs a plurality of code identifiers acquired from the one or more acquired photographic images.

12. Computer, a reception unit that receives instructions for photographing from a user; an image acquisition unit that acquires a captured image, which is a still image of a plurality of codes, when the instruction to capture the image is accepted by the accepting unit; an identifier acquisition unit that acquires a code identifier that identifies each of a plurality of codes included in the captured image; a generation unit that generates acquisition status information indicating an acquisition status of the code identifier by the identifier acquisition unit for each captured image; an acquisition status output unit that outputs the acquisition status information generated by the generation unit; a program for causing the device to function as an identifier output unit that outputs a plurality of code identifiers acquired from one or more photographed images acquired by the image acquisition unit;

Images