Information processing apparatus, method for processing information, and program

Abstract

To allow a user to properly grasp the position of an item.SOLUTION: An information processing apparatus includes: an acquisition unit for acquiring images captured at time points by a sensor terminal; a management unit for managing an estimated position of a first article on the basis of a first image captured at a first time point when a marker of the first article, which is a subject of the first image, is not captured in a second image captured at a second time point later than the first time point due to being shielded by a second article; and an output unit for outputting information indicating the position of the first article managed by the management unit.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 discloses a control device capable of efficiently detecting a target object even when the target object is obstructed by another object. In Patent Document 1, measurement data acquired from a sensor is used to recognize a target object present in a three-dimensional environment. Furthermore, three-dimensional environment information is generated by integrating multiple pieces of measurement data. Then, using the three-dimensional environment information, the sensor is moved to a position where an image can be captured in which the size of the area obstructed by at least one first object is increased. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-185664 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the technology described in Patent Document 1, for example, there may be cases where the position of an article (object) cannot be detected appropriately.

[0005] In view of the above-mentioned problems, an object of the present disclosure is to provide a technology that can more appropriately grasp the position of an item. [Means for solving the problem]

[0006] In a first aspect of the present disclosure, an information processing device is provided that has an acquisition unit that acquires images captured by a sensor terminal at each time point, a management unit that manages the estimated position of a first item based on the first image when a marker of the first item that is the subject of a first image captured at a first time point is blocked by a second item and not captured in a second image captured at a second time point that is later than the first time point, and an output unit that outputs information indicating the position of the first item managed by the management unit.

[0007] In addition, a second aspect of the present disclosure provides an information processing method that acquires images taken at each time point by a sensor terminal, and if a marker of a first item that is the subject of a first image taken at a first time point is blocked by a second item and not photographed in a second image taken at a second time point later than the first time point, manages the estimated position of the first item based on the first image, and outputs information indicating the position of the first item being managed.

[0008] In addition, a third aspect of the present disclosure provides a program that causes a computer to execute a process of acquiring images taken at each time point by a sensor terminal, and if a marker of a first item that is the subject of a first image taken at a first time point is blocked by a second item and not photographed in a second image taken at a second time point later than the first time point, managing the estimated position of the first item based on the first image and outputting information indicating the position of the first item being managed. [Effects of the Invention]

[0009] According to one aspect, the location of an item can be more accurately determined. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing apparatus according to an embodiment. [Figure 2] 1 is a diagram illustrating an example of the configuration of an information processing system according to an embodiment. [Figure 3]FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing apparatus according to an embodiment. [Figure 4] 10 is a flowchart illustrating an example of processing by the information processing apparatus according to the embodiment. [Figure 5] FIG. 2 is a diagram showing an example of information stored in an item position DB (database) according to the embodiment. [Figure 6] FIG. 4 is a diagram showing an example of information stored in an item content DB according to the embodiment. [Figure 7] FIG. 10 is a diagram illustrating an example of a display screen of a user terminal according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] The principles of the present disclosure will be described with reference to some exemplary embodiments. It should be understood that these embodiments are set forth for illustrative purposes only, to aid those skilled in the art in understanding and practicing the present disclosure, without implying any limitation on the scope of the disclosure. The disclosure described herein may be implemented in various ways other than those described below.

[0012] In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that each drawing is merely an example for describing one or more embodiments. Each drawing may not relate to only one particular embodiment, but may also relate to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessarily required to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0014] (Embodiment 1) <Configuration> The configuration of an information processing device 10 according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of the information processing device 10 according to an embodiment. The information processing device 10 has an acquisition unit 11, a management unit 12, and an output unit 13. Each of these units may be realized by cooperation between one or more programs installed in the information processing device 10 and hardware such as a processor and memory of the information processing device 10.

[0015] The acquisition unit 11 acquires images captured by a sensor terminal at each time point. The management unit 12 manages the estimated position of the first item based on the first image when a marker of the first item, which is the subject of a first image captured at a first time point, is obscured by a second item and not captured in a second image captured at a second time point after the first time point. The output unit 13 outputs information indicating the position of the first item managed by the management unit 12.

[0016] (Embodiment 2) Next, the configuration of the information processing system 1 according to the embodiment will be described with reference to FIG. <System configuration> FIG. 2 is a diagram showing an example of the configuration of an information processing system 1 according to an embodiment. In the example of FIG. 2, the information processing system 1 includes an information processing device 10. The information processing system 1 also includes a sensor terminal 20. The information processing system 1 also includes a user terminal 30A and a user terminal 30B (hereinafter, when there is no need to distinguish between them, they will also be simply referred to as "user terminals 30"). The information processing system 1 also includes items 40A-C, etc. (hereinafter, when there is no need to distinguish between them, they will also be simply referred to as "items 40"), and markers 401A-C, etc. (hereinafter, when there is no need to distinguish between them, they will also be simply referred to as "markers 401") attached to the items 40A-C, etc., respectively.

[0017] 2, the information processing devices 10, the sensor terminals 20, and the user terminals 30 are connected to each other so as to be able to communicate with each other via a network N. Note that the numbers of the information processing devices 10, the sensor terminals 20, the user terminals 30, the articles 40, etc. are not limited to those shown in the example of FIG.

[0018] Examples of the network N include, for example, the Internet, a mobile communication system, a wireless LAN (Local Area Network), a LAN, a bus, etc. Examples of the mobile communication system include, for example, a fifth generation mobile communication system (5G), a sixth generation mobile communication system (6G, Beyond 5G), a fourth generation mobile communication system (4G), a third generation mobile communication system (3G), etc.

[0019] The information processing device 10 is, for example, a device such as a server, a cloud, or a personal computer. The information processing device 10 manages information about each item 40 using, for example, a sensor terminal 20, and notifies the user terminal 30 of the information about each item 40.

[0020] The sensor terminal 20 has an imaging device (for example, a digital camera or an RGB camera) that captures images. The sensor terminal 20 may also have a three-dimensional sensor. In this case, the three-dimensional sensor of the sensor terminal 20 may be, for example, a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), a ToF camera (Time of Flight Camera), or a stereo camera. The three-dimensional data measured by the three-dimensional sensor may be, for example, point cloud data, which is a collection of points having three-dimensional coordinates.

[0021] The camera and the three-dimensional sensor of the sensor terminal 20 may be provided in the same housing or may be separate. The sensor terminal 20 may be fixed to a ceiling surface or the like. The sensor terminal 20 may also be mounted on a moving body (e.g., an automatic transport robot, a forklift) that moves the item 40.

[0022] The user terminal 30 may be, for example, a terminal carried by a user (for example, a delivery person), such as a head-mounted display (smart glasses), a personal computer, a smartphone, or a tablet. The user terminal 30 displays a display screen based on information received from the information processing device 10, for example.

[0023] The item 40 is, for example, an item whose location, etc., is managed by the information processing device 10. The item 40 may be, for example, a container such as a delivery vehicle (e.g., a truck), or luggage, materials, work in progress, finished products, waste, etc. present in a factory or warehouse. The item 40 may also be, for example, an animal, a person, a robot, etc.

[0024] The marker 401 may include at least one of a two-dimensional code (e.g., a QR code (registered trademark)), an ArUco marker, a barcode, and a chameleon code. The ArUco marker may be a binary square marker that can be used for camera posture estimation. For example, a marker 401 printed on paper or the like may be attached to each item 40 in advance.

[0025] <Hardware configuration> Fig. 3 is a diagram showing an example of the hardware configuration of an information processing device 10 according to an embodiment. In the example of Fig. 3, the information processing device 10 (computer 100) includes a processor 101, a memory 102, and a communication interface 103. These components may be connected via a bus or the like. The memory 102 stores at least a part of a program 104. The communication interface 103 includes an interface required for communication with other network elements.

[0026] When the program 104 is executed by the processor 101, memory 102, and the like in cooperation with each other, the computer 100 performs at least some of the processing of the embodiments of the present disclosure. The memory 102 may be of any type. As a non-limiting example, the memory 102 may be a non-transitory computer-readable storage medium. The memory 102 may also be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory, and removable memory. Although only one memory 102 is shown in the computer 100, several physically different memory modules may exist in the computer 100. The processor 101 may be of any type. The processor 101 may include one or more of a general-purpose computer, a special-purpose computer, a microprocessor, a digital signal processor (DSP), and a processor based on a multi-core processor architecture, as a non-limiting example. The computer 100 may have multiple processors, such as application-specific integrated circuit chips that are time-slaved to a clock that synchronizes the main processor.

[0027] Embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device.

[0028] The present disclosure also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, that execute on a target real or virtual processor or device to perform the processes or methods of the present disclosure. Program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or divided among program modules as desired in various embodiments. The machine-executable instructions of the program modules may be executed in local or distributed devices. In a distributed device, the program modules may be located in both local and remote storage media.

[0029] The program code for executing the methods of the present disclosure may be written in any combination of one or more programming languages. The program code may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus. When the program code is executed by the processor or controller, the functions / acts in the flowcharts and / or implementing block diagrams are performed. The program code may be executed entirely on the machine, partly on the machine, as a standalone software package, partly on the machine and partly on a remote machine, or entirely on a remote machine or server.

[0030] The program can be stored and provided to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible recording media. Examples of non-transitory computer-readable media include magnetic recording media, magneto-optical recording media, optical disk media, and semiconductor memory. Magnetic recording media include, for example, flexible disks, magnetic tapes, and hard disk drives. Magneto-optical recording media include, for example, magneto-optical disks. Optical disk media include, for example, Blu-ray discs, CD (Compact Disc)-ROMs (Read Only Memory), CD-Rs (Recordable), and CD-RWs (Rewritable). Semiconductor memory includes, for example, solid-state drives, mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory). The program may also be provided to a computer by various types of temporary computer-readable media. Examples of temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire or an optical fiber, or via a wireless communication path.

[0031] <Processing> Next, an example of processing of the information processing device 10 according to the embodiment will be described with reference to Figs. 4 to 7. Fig. 4 is a flowchart showing an example of processing of the information processing device 10 according to the embodiment. Fig. 5 is a diagram showing an example of information stored in an item location DB (database) 501 according to the embodiment. Fig. 6 is a diagram showing an example of information stored in an item content DB 601 according to the embodiment. Fig. 7 is a diagram showing an example of a display screen of the user terminal 30 according to the embodiment. Note that the processing of Fig. 4 may be executed at various times, for example, periodically.

[0032] In step S101, the acquisition unit 11 acquires an image captured by the sensor terminal 20 at a specific time point (for example, the current time point).

[0033] Next, the management unit 12 detects the three-dimensional position of each item 40 and the code included in the marker 401 based on the acquired image (step S102). Here, the management unit 12 may calculate the direction and distance of each item 40 as seen from the sensor terminal 20, for example, based on the tilt (three-dimensional angle) and size of each item 40 in the image captured by the sensor terminal 20. In this case, information such as the direction and angle of view of the sensor terminal 20 may be set in the information processing device 10 by an operator (administrator) or the like when the sensor terminal 20 is installed, etc.

[0034] In addition, for example, if the sensor terminal 20 has a three-dimensional sensor, the management unit 12 may calculate the orientation and distance of each item 40 as seen from the sensor terminal 20 based on information measured by the three-dimensional sensor, and calculate the three-dimensional position of each item 40.

[0035] Next, the management unit 12 manages the position and the like of each item 40 (step S103). Here, the management unit 12 may record the position and the like of each item 40 in the item position DB 501, for example.

[0036] In the example of FIG. 5, the item position DB 501 stores the three-dimensional position of the item 40, the size and shape of the item 40, and the last photographed time, in association with the item ID (marker ID). The item ID is identification information of the item 40. The item ID may be, for example, information read from a marker 401 attached to the item 40. The three-dimensional position may be, for example, the three-dimensional position of the item 40 relative to the sensor terminal 20. The size of the item 40 may be, for example, the lengths of each of the three sides of the item 40. The last photographed time is the time when the item 40 was last photographed by the sensor terminal 20. This allows, for example, a delivery person or the like to know the time when the marker of a first item was blocked by a second item, making the position of the first item inaccurate.

[0037] When a marker of a first item that is the subject of a first image captured at a first time point (last captured time) is blocked by a second item in a second image captured at a second time point after the first time point and is not captured, the management unit 12 may manage the estimated position of the first item based on the first image. This makes it possible to grasp the position of the specific item 40, for example, when loading a specific item 40 that is a parcel into a container before delivery, even if the specific item 40 is blocked by other parcels and is no longer captured in an image.

[0038] In this case, the management unit 12 may determine that the marker is blocked by the second item, for example, when the three-dimensional position of the marker is behind the three-dimensional model of the second item as viewed from the sensor terminal 20. Then, for example, when a specific item 40 that was photographed in a previous image is no longer photographed in an image at a certain point in time, the management unit 12 may manage the position of the specific item 40 at the time when it was last photographed (last photographing time) as the three-dimensional position of the item 40.

[0039] Furthermore, the management unit 12 may determine that the first item is absent if the marker is not photographed in a third image photographed at a third time point that is later than the second time point and the marker is not obscured by the second item. This makes it possible to know that the first item is no longer present, for example, even if the second item is removed and the first item is carried out by a delivery person or the like when the marker is not photographed by the sensor terminal 20.

[0040] Furthermore, the management unit 12 may refer to the item location DB 501 and the item content DB 601, for example, to estimate the location of the item 40 whose marker 401 has not been photographed by the sensor terminal 20.

[0041] In the example of FIG. 6, the item content DB 601 stores a list of item IDs for each item 40 loaded into a container or the like. The item content DB 601 also stores item information associated with the item ID. The item information is information related to the item 40. The item information may include, for example, information indicating the identification information of the item 40, the size and shape of the item 40, the contents of the item 40, the weight of the item 40, the address and name of the destination of the item 40, and the address and name of the sender of the item 40. At least a portion of the information stored in the item content DB 601 may be entered automatically or manually at a business office, for example. At least a portion of the information stored in the item content DB 601 may also be entered by a delivery person from the user terminal 30, for example.

[0042] For example, the management unit 12 may determine that, among the item IDs registered in the item content DB 601, items 40 with one or more item IDs that are not recorded in the item position DB 501 are items 40 whose markers 401 have not been photographed by the sensor terminal 20.

[0043] The management unit 12 may then detect the size and shape of the first item from the first image captured at a first time point (e.g., the last captured time). The management unit 12 may then acquire, for example, the size and shape of an item 40 registered in the item content DB 601, for which the marker 401 is not captured in a second image captured at a second time point that is later than the first time point. The management unit 12 may then compare the size and shape of the first item with the size and shape of the item 40 to estimate the item ID of the object. In this case, the management unit 12 may determine that the two are the same when, for example, the degree of discrepancy between the two is equal to or less than a threshold. The management unit 12 may then record (update) the estimated item ID and the three-dimensional position of the item 40 detected based on the second image in the item position DB 501. This allows the position of the item 40 to be appropriately updated even if the position of the item 40, which is a package, is moved without capturing an image of the marker 401 due to, for example, the absence of the delivery recipient or reloading of the package during collection or delivery. In this case, the management unit 12 may, for example, compare the size, shape, and three-dimensional position of the object detected from the image with the size, shape, and three-dimensional position of the item 40 registered in the item content DB 601, and update the three-dimensional position of the item 40 recorded in the item position DB 501.

[0044] Next, the output unit 13 outputs information indicating the location of each item 40 managed by the management unit 12 (step S104). Here, the output unit 13 may, for example, display information indicating the location of the item 40 specified by a delivery person or the like using the user terminal 30. Furthermore, the output unit 13 may, for example, cause the user terminal 30 to display information indicating the location of the item 40 within a specific range (for example, a radius of 100 meters) from the current location of the user terminal 30 to the delivery address recorded in the item content DB 601. This allows, for example, the delivery person to quickly determine the location of the item 40 to be delivered when stopping the delivery vehicle near the delivery address. In this case, the current location of the delivery vehicle may be obtained using, for example, a satellite positioning system such as a GPS (Global Positioning System).

[0045] Furthermore, the output unit 13 may refer to the item position DB 501 and the item content DB 601, for example, and generate (create) data in which data on each item 40 is associated with the area of ​​each subject in the image, etc.

[0046] 7, images of an article 40B and the like are displayed on a display screen 701 of the user terminal 30. The output unit 13 also displays a display object 722 of information relating to the article 40B in association with the article 40B. The output unit 13 also displays a display object 721 of information relating to the article 40A in association with the article 40A whose marker 401A is currently hidden by the article 40B. The information relating to each article 40 may be acquired from the article content DB 601.

[0047] In the example of FIG. 7, the output unit 13 displays information about the article 40A based on the identification information of the marker 401A and information indicating the time when the marker 401A was last photographed, superimposed on a current image (an example of a "second image") using a display object 721. Also, a display object 721A indicating that the marker 401A has not been photographed at the current time is displayed in association with the article 40A. Also, in the example of FIG. 7, the output unit 13 displays, on the display screen 701, a display object 722 for information about the article 40B in which the marker 401B is exposed to the outside as seen from the sensor terminal 20, with a solid border. On the other hand, a display object 721 for information about the article 40A in which at least a part of the marker 401A is hidden in the shadow (obscured by the article 40B) as seen from the sensor terminal 20, with a dashed border.

[0048] <Other> For example, at the beginning of the day, delivery personnel load each package into a truck container, taking into account the order in which the packages will be delivered that day. However, there are cases where the recipient is not at home, or additional packages need to be reloaded due to collection. As a result, the initially assumed package locations gradually become distorted, making it difficult to determine the location of the next package to be picked up. This increases the burden on delivery personnel and reduces delivery time efficiency.

[0049] According to the present disclosure, for example, it is possible to estimate the positions of many packages loaded in a truck container, allowing delivery personnel to easily know where the package they want to deliver is currently located, thereby reducing the burden on delivery personnel and improving delivery time efficiency.

[0050] <Modification> The information processing device 10 may be a device contained in a single housing, but the information processing device 10 of the present disclosure is not limited to this. Each unit of the information processing device 10 may be realized, for example, by cloud computing configured with one or more computers. Furthermore, the information processing device 10 and at least one of the sensor terminal 20 and the user terminal 30 may be housed in the same housing and configured as an integrated information processing device. Furthermore, at least part of the processing of each functional unit of the information processing device 10 may be executed by at least one of the sensor terminal 20 and the user terminal 30. Such information processing devices 10 are also included in examples of the "information processing device" of the present disclosure.

[0051] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0052] Some or all of the above embodiments may also be described as, but are not limited to, the following appendices. Note that some or all of the elements (e.g., configurations and functions) described in each appendix dependent on appendix 1 may also be dependent on independent appendices in other categories in a similar dependency relationship. Some or all of the elements described in any appendix may be applied to various hardware, software, recording means for recording software, systems, and methods. (Appendix 1) an acquisition unit that acquires images captured by the sensor terminal at each time point; a management unit that manages an estimated position of a first item based on a first image when a marker of the first item, which is a subject of a first image captured at a first time point, is blocked by a second item and not captured in a second image captured at a second time point after the first time point; an output unit that outputs information indicating the location of the first item managed by the management unit; An information processing device having the above. (Appendix 2) the acquisition unit acquires three-dimensional data obtained by measuring the first item at the first time point using a three-dimensional sensor; the management unit manages the three-dimensional position of the first item based on the three-dimensional data. 10. The information processing device according to claim 1. (Appendix 3) The management unit determines that the first item is not present when the marker is not photographed in a third image photographed at a third time point that is later than the second time point and the marker is not obstructed by the second item. 3. The information processing device according to claim 1 or 2. (Appendix 4) the management unit records the identification information of the marker, the position of the first item, and the time when the marker was last photographed in association with each other; 3. The information processing device according to claim 1 or 2. (Appendix 5) the output unit displays, superimposed on the second image, information about the first item based on the identification information of the marker, a position of the first item, and information indicating the time when the marker was last photographed. 3. The information processing device according to claim 1 or 2. (Appendix 6) the output unit causes the user terminal to display information indicating the location of the first item within a specific range from the current location of the user terminal to the delivery destination of the first item. 3. The information processing device according to claim 1 or 2. (Appendix 7) the management unit compares the size and shape of the first item detected from the first image with the size and shape of an object in the second image for which the marker is not photographed, and estimates identification information of the marker of the object. 3. The information processing device according to claim 1 or 2. (Appendix 8) The marker includes at least one of a two-dimensional code, an ArUco marker, a barcode, and a chameleon code. 3. The information processing device according to claim 1 or 2. (Appendix 9) The sensor terminal acquires images taken at each point in time, When a marker of a first item that is a subject of a first image captured at a first time point is blocked by a second item and not captured in a second image captured at a second time point that is later than the first time point, managing an estimated position of the first item based on the first image; outputting information indicating the location of the first item being managed; Information processing methods. (Appendix 10) The sensor terminal acquires images taken at each point in time, When a marker of a first item that is a subject of a first image captured at a first time point is blocked by a second item and not captured in a second image captured at a second time point that is later than the first time point, managing an estimated position of the first item based on the first image; outputting information indicating the location of the first item being managed; A program that causes a computer to perform a process. [Explanation of symbols]

[0053] 1. Information Processing Systems 10. Information processing equipment 11 Acquisition Department 12 Management Department 13 Output section 20 Sensor terminals 30 User terminals 40 Goods 401 Marker

Claims

1. an acquisition unit that acquires images captured by the sensor terminal at each time point; a management unit that manages an estimated position of a first item based on a first image when a marker of the first item, which is a subject of a first image captured at a first time point, is blocked by a second item and not captured in a second image captured at a second time point after the first time point; an output unit that outputs information indicating the location of the first item managed by the management unit; An information processing device having the above.

2. the acquisition unit acquires three-dimensional data obtained by measuring the first item at the first time point using a three-dimensional sensor; the management unit manages the three-dimensional position of the first item based on the three-dimensional data. The information processing device according to claim 1 .

3. the management unit determines that the first item is not present when the marker is not photographed in a third image photographed at a third time point that is later than the second time point and the marker is not obstructed by the second item.

3. The information processing device according to claim 1.

4. the management unit records the identification information of the marker, the position of the first article, and the time when the marker was last photographed in association with each other; 3. The information processing device according to claim 1.

5. the output unit displays, superimposed on the second image, information about the first article based on the identification information of the marker, a position of the first article, and information indicating the time when the marker was last photographed.

3. The information processing device according to claim 1.

6. the output unit causes the user terminal to display information indicating the location of the first item within a specific range from the current location of the user terminal to the delivery destination of the first item.

3. The information processing device according to claim 1.

7. the management unit compares the size and shape of the first article detected from the first image with the size and shape of an object in the second image for which the marker is not photographed, and estimates identification information of the marker of the object.

3. The information processing device according to claim 1.

8. The marker includes at least one of a two-dimensional code, an ArUco marker, a barcode, and a chameleon code.

3. The information processing device according to claim 1.

9. The sensor terminal acquires images taken at each point in time, When a marker of a first item that is a subject of a first image captured at a first time point is blocked by a second item and not captured in a second image captured at a second time point that is later than the first time point, managing an estimated position of the first item based on the first image; outputting information indicating the location of the first item being managed; Information processing methods.

10. The sensor terminal acquires images taken at each point in time, When a marker of a first item that is a subject of a first image captured at a first time point is blocked by a second item and not captured in a second image captured at a second time point that is later than the first time point, managing an estimated position of the first item based on the first image; outputting information indicating the location of the first item being managed; A program that causes a computer to perform a process.

Images