Information processing device, information processing method, and program

Abstract

To provide a technology for specifying date / time and a place at which a package accident occurs.SOLUTION: An information processing device 10 includes an acquisition part 12, a detection part 14, and a determination part 16. The acquisition part 12 acquires an image including a package generated in a work place in which, for example, at least one of work of loading packages or transportation means and work of unloading the packages from the transportation means. The detection means 14 detects identification information of an object package by processing the image. The determination part 16 determines whether an accident occurs in the object package by using a detection result of the identification information of the object package and information on the accident of the package.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a management system, an information processing method, and a program. [Background technology]

[0002] When an accident such as damage occurs during the transportation of a package, it is necessary to identify the location and timing of the incident.

[0003] Patent Document 1 describes a method for determining whether or not a load loaded on a transport vehicle is being transported stably, based on measurements taken by a sensor provided on the transport vehicle.

[0004] Patent Document 2 describes a system in which, when the door for loading or unloading luggage is opened, a camera mounted on a vehicle such as a truck captures images of luggage being loaded or unloaded. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2020-158092 [Patent Document 2] Japanese Patent Application Laid-Open No. 2006-217554 Summary of the Invention [Problem to be solved by the invention]

[0006] The technology of Patent Document 1 does not allow for the date, time, or location of an accident to be determined after the fact. For example, with the method of Patent Document 1, if it is determined that the luggage is not being transported stably during transportation, a notification is issued and measures such as reloading are taken. Furthermore, the technology of Patent Document 2 does not allow for the objective identification of the date, time, or location of an accident. For example, with the technology of Patent Document 2, the system user must visually search for the lost luggage from the extracted video and determine the date, time, and location of the loss, as well as the cause of the loss.

[0007] The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a technology for identifying the date, time, and location of a luggage accident. [Means for solving the problem]

[0008] An information processing device according to one aspect of the present invention includes: an acquisition means for acquiring an image including the package; a detection means for detecting identification information of the target baggage by processing the image; The system further includes a determination means for determining whether or not an accident has occurred to the target package using the result of detection of the identification information of the target package and information related to the package accident.

[0009] An information processing method according to one aspect of the present invention includes: 1. A computer-implemented information processing method, comprising: Get an image containing the luggage, processing the image to detect identification information of the target package; Using the detection result of the identification information of the target package and information related to the package accident, it is determined whether or not an accident has occurred with the target package.

[0010] A program according to one aspect of the present invention includes: The above information processing method is executed by a computer. [Effects of the Invention]

[0011] According to one aspect of the present invention, a technique is provided for identifying the date, time, and location of a luggage accident. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram illustrating a functional configuration of an information processing device according to a first embodiment. [Figure 2] FIG. 1 is a diagram illustrating a computer for realizing an information processing device. [Figure 3] FIG. 2 is a diagram illustrating an example of a functional configuration of a management system according to the first embodiment. [Figure 4] FIG. 1 is a diagram illustrating an example of a usage environment of a management system according to a first embodiment. [Figure 5] FIG. 1 is a diagram illustrating an example of a usage environment of a management system according to a first embodiment. [Figure 6] FIG. 2 is a block diagram illustrating a functional configuration of an information acquisition device. [Figure 7] FIG. 2 is a diagram illustrating an example of the configuration of data stored in a storage unit. [Figure 8] FIG. 10 is a diagram illustrating an example of the configuration of threshold information. [Figure 9] FIG. 2 is a diagram illustrating an example of the configuration of video data stored in a storage unit. [Figure 10] 4 is a flowchart illustrating the flow of processing executed by the information processing apparatus according to the first embodiment. [Figure 11] 4 is a flowchart illustrating in detail the flow of processing executed by the information processing device according to the first embodiment. [Figure 12] FIG. 10 is a diagram illustrating an example of a usage environment of a management system according to a second embodiment. [Figure 13] FIG. 10 is a diagram illustrating an example of the configuration of data stored in a storage unit according to the second embodiment. [Figure 14] 10 is a flowchart illustrating the flow of processing executed by an information processing apparatus according to a second embodiment. [Figure 15] FIG. 10 is a diagram illustrating threshold information according to the second embodiment. [Figure 16] FIG. 10 is a diagram illustrating an example of a usage environment of a management system according to a third embodiment. [Figure 17] FIG. 11 is a diagram illustrating an example of the configuration of data stored in a storage unit according to the third embodiment. [Figure 18] FIG. 11 is a diagram illustrating in detail an example of the configuration of video data and second measurement data managed in the third embodiment. [Figure 19] 10 is a flowchart illustrating the flow of processing executed by an information processing apparatus according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, like components are designated by like reference numerals, and their description will be omitted where appropriate.

[0014] (First embodiment) FIG. 1 is a block diagram illustrating the functional configuration of an information processing device 10 according to the first embodiment. The information processing device 10 according to this embodiment includes an acquisition unit 12, a detection unit 14, and a determination unit 16. The acquisition unit 12 acquires an image including luggage. The detection unit 14 detects identification information of the target luggage by processing the image. The determination unit 16 determines whether an accident has occurred with the target luggage using the detection result of the identification information of the target luggage and information related to the luggage accident. This will be explained in detail below.

[0015] As a package is transported from its origin to its destination, it passes through multiple stopover points, such as collection and delivery centers and relay stations. The package is then transported between these stopover points by trucks, aircraft, or other transportation means. If the package is damaged during transport, it is necessary to identify when and where the damage occurred during the transport process. For example, identifying the location and date and time when the damage occurred, i.e., the location and date and time when the package accident occurred, can lead to identifying the cause of the accident and preventing recurrence.

[0016] According to the information processing device 10 of this embodiment, the location and date / time where the target package was located are identified by detecting the identification information of the target package from images captured during transportation. The information processing device 10 then generates information related to the package accident and determines whether or not an accident occurred at the location and date / time. This makes it possible to identify the date, time, and location of the package accident after the fact.

[0017] The type of luggage is not particularly limited. For example, luggage may be consigned luggage handled by a carrier, or products or raw materials handled by a manufacturer or distributor. Each luggage is assigned identification information (hereinafter also referred to as luggage ID) that identifies the individual luggage. The luggage ID is a number or character string unique to each luggage. The luggage ID is assigned to the luggage in the form of, for example, a barcode, a two-dimensional code, a color code, or an RFID tag. In other words, the luggage ID is assigned to the luggage in a form that can be decoded using an image of the luggage ID assigned to the luggage.

[0018] The transportation means for transporting the luggage is not particularly limited. Examples of transportation means include vehicles such as trucks and delivery vehicles, trains, and aircraft. Examples of work locations where at least one of the work of loading one or more luggage onto the transportation means and the work of unloading one or more luggage onto the transportation means include collection locations, delivery locations, collection and delivery centers, relay stations, etc.

[0019] The information processing device 10 accepts input of a parcel ID by a user. The information processing device 10 designates the parcel corresponding to the input parcel ID as the target parcel. As will be described below, the information processing device 10 identifies and outputs the location, date, and time when an accident involving the target parcel is highly likely to have occurred.

[0020] A plurality of pieces of video data are stored in advance in the memory unit 18 accessible from the acquisition unit 12 (see FIG. 7). The acquisition unit 12 reads and acquires the video data from the memory unit 18. Each piece of video data is composed of a plurality of images arranged in chronological order. That is, the acquisition unit 12 according to this embodiment acquires a plurality of images. The frame rate of the video data is not particularly limited. At least a portion of each piece of video data includes luggage. That is, at least a portion of the plurality of images acquired by the acquisition unit 12 includes luggage. The plurality of images acquired by the acquisition unit 12 may include images that do not include luggage.

[0021] The storage unit 18 may be provided inside the information processing device 10 or may be provided outside the information processing device 10. Furthermore, the video data may be held in any of the information acquisition devices 20 described below, and the acquisition unit 12 may acquire the video data from any of the information acquisition devices 20. The connection between the acquisition unit 12 and the storage unit 18 may be wired or wireless.

[0022] Each piece of video data (i.e., multiple images) may be video data taken at any location during the transportation process. Each video may be video taken, for example, at a pickup location, a delivery location, or a stopover location. In particular, each video is preferably video taken at a work location where at least one of the work of loading cargo onto a transportation means and the work of unloading cargo from a transportation means is carried out. This is because accidents are likely to occur during such work of loading and unloading cargo. Each piece of video data is linked to the location and date and time at which the video data was obtained.

[0023] The acquisition unit 12 selects and acquires video data that is highly likely to include the target baggage from among the plurality of video data. The method of selecting video data to be acquired by the acquisition unit 12 will be described in detail later.

[0024] The detection unit 14 detects the baggage ID (e.g., a barcode) of the target baggage from the video data acquired by the acquisition unit 12. When the detection unit 14 detects the baggage ID of the target baggage, it identifies the baggage with the baggage ID as the target baggage. The detection unit 14 then marks the shape, characteristics, etc. of the target baggage and tracks the target baggage within the video. The detection unit 14 can detect the baggage ID of the target baggage and track the target baggage by performing existing image recognition processing on the video data. This detection and tracking may also use baggage characteristics included in the baggage information described below. Tracking may extend to images before and after the image in which the baggage ID of the target baggage is detected within the video. Tracking may be performed within a range in which a scene with a possibility of an accident is captured, such as immediately before or immediately after the baggage is unloaded from the transportation means, or immediately before another baggage is loaded on top of the target baggage. By detecting the baggage ID from the video data, the occurrence of a baggage accident can be identified without attaching sensors to all the baggage.

[0025] The determination unit 16 according to this embodiment generates information related to an accident by analyzing video data (i.e., multiple images). As will be described later in the second embodiment, etc., there may be multiple methods for generating information related to an accident that is used to determine whether an accident has occurred with the target baggage. That is, the determination unit 16 generates at least a portion of the information related to one or more accidents that is used for the determination by analyzing video data.

[0026] The determination unit 16 analyzes multiple images to calculate, for example, the pressure applied to the target luggage as information related to the accident involving the target luggage. The pressure applied to the luggage can be calculated using the impact force and contact area when the target luggage collides with another object. The determination unit 16 may identify a period in the video during which an accident may have occurred, and then analyze the video data for that period to generate information related to the accident. For example, the determination unit 16 can identify a period during which an accident may have occurred based on changes in the shape or vibration of the target luggage in the video. Existing image processing technology can be used to generate information related to the accident. By calculating the pressure applied to the luggage from the video data, the load applied to the luggage can be understood and the accident can be accurately identified.

[0027] The impact force can be calculated, for example, as follows. In the video, a package is moved from one position to another by a person's hand, a conveyor, being dropped, or the like. At the end of the movement, the package comes into contact with another object (for example, another package, the floor, a forklift, a pallet, the platform of a transport means, a stopper on a conveyor, etc.). The impact force at this contact is calculated using the speed of the package at the moment of contact with the other object (hereinafter referred to as instantaneous speed) and the time from when the package comes into contact with the other object until the package stops (hereinafter referred to as time to stop). The determination unit 16 analyzes the video to calculate the instantaneous speed of the package and the time to stop. The determination unit 16 then calculates the impact force using the instantaneous speed and the time to stop.

[0028] Specifically, the instantaneous velocity can be calculated based on dimensional indicators in the video (e.g., known dimensions such as the height, width, and depth of the target object, other objects, forklifts, pallets, etc.). For example, the determination unit 16 measures the time it takes from the start of movement until contact with another object (hereinafter referred to as travel time). Furthermore, the determination unit 16 determines the travel distance during that time based on the dimensional indicators. The determination unit 16 then calculates the instantaneous velocity using the travel distance and travel time in accordance with the law of uniformly accelerated motion. The velocity of the target object at the start of movement can be predetermined, e.g., 0 m / s. Furthermore, the determination unit 16 determines the time when the target object contacts another object and the time when the target object stops due to the contact by analyzing the video. The determination unit 16 then calculates the difference between the two times as the time until the object stops. The determination unit 16 calculates the impact force by dividing the instantaneous velocity by the time it takes to stop.

[0029] Furthermore, the determination unit 16 can calculate the contact area when the target item comes into contact with a forklift, a pallet, another item, or the like, based on, for example, dimensional indicators in the video.

[0030] However, the method of calculating the pressure applied to the luggage and the method of calculating the impact force and contact area are not limited to these.

[0031] When generating information about an accident, the determination unit 16 associates the date, time, and location with the information about the accident. Specifically, the determination unit 16 identifies the date and time when the impact on the target baggage occurred based on the date and time associated with the video data that is the source of the information about the accident, and associates this date and time with the information about the accident. The determination unit 16 also associates the location associated with the video data that is the source of the information about the accident with the information about the accident.

[0032] Next, the determination unit 16 compares the generated information about the accident with a threshold value to determine whether an accident has occurred with the target package. The threshold value is a value that indicates the possibility of package damage. The threshold value may be a fixed value, or may be determined for each property of the package, such as the material, contents, size, and weight of the package. Threshold value information indicating the threshold value is stored in advance in the memory unit 18, and the determination unit 16 uses the threshold value read from the memory unit 18 for determination. The threshold value will be described in detail later.

[0033] Specifically, the determination unit 16 determines that an accident has occurred with the target package if the information about the accident is equal to or greater than a threshold. If the determination unit 16 determines that an accident has occurred with the target package, it identifies the date, time, and location associated with the information about the accident used in the determination as the date, time, and location at which the accident occurred with the target package. The identified date, time, and location are output, for example, by being displayed on a display or the like provided in the information processing device 10.

[0034] If the information about the accident is less than the threshold, the determination unit 16 determines that no accident occurred on the target package for the information about the accident. If it is determined that no accident occurred on the target package for all the information about the accident, the information processing device 10 may output a message indicating that no accident occurred, or may output the date, time, and location where the accident most likely occurred. The latter will be described in detail later.

[0035] <Hardware configuration of information processing device> The hardware configuration of the information processing device 10 will be described below. Each functional component of the information processing device 10 may be realized by hardware that realizes the functional component (e.g., a hardwired electronic circuit, etc.), or may be realized by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it, etc.). Below, a case where each functional component of the information processing device 10 is realized by a combination of hardware and software will be further described.

[0036] 2 is a diagram illustrating a computer 1000 for realizing the information processing device 10. The computer 1000 is any computer. For example, the computer 1000 is an SoC (System On Chip), a Personal Computer (PC), a server machine, a tablet terminal, or a smartphone. The computer 1000 may be a dedicated computer designed to realize the information processing device 10, or may be a general-purpose computer.

[0037] The computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output interface 1100, and a network interface 1120. The bus 1020 is a data transmission path through which the processor 1040, the memory 1060, the storage device 1080, the input / output interface 1100, and the network interface 1120 transmit and receive data to and from each other. However, the method of interconnecting the processor 1040 and other components is not limited to bus connection. The processor 1040 may be any of various processors, such as a central processing unit (CPU), a graphics processing unit (GPU), or a field-programmable gate array (FPGA). The memory 1060 is a main storage device implemented using a random access memory (RAM) or the like. The storage device 1080 is an auxiliary storage device implemented using a hard disk, a solid state drive (SSD), a memory card, a read-only memory (ROM), or the like.

[0038] The input / output interface 1100 is an interface for connecting the computer 1000 to an input / output device. For example, the input / output interface 1100 is connected to an input device such as a keyboard and an output device such as a display device.

[0039] The network interface 1120 is an interface for connecting the computer 1000 to a network. This communication network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The network interface 1120 may be connected to the network wirelessly or by wire.

[0040] The storage device 1080 stores program modules that realize each functional component of the information processing device 10. The processor 1040 reads each of these program modules into the memory 1060 and executes them to realize the function corresponding to each program module. In addition, when the storage unit 18 is provided inside the information processing device 10, the storage unit 18 is, for example, the storage device 1080.

[0041] <Management system> FIG. 3 is a diagram illustrating an example of the functional configuration of a management system 70 according to this embodiment. The management system 70 according to this embodiment includes an information processing device 10 and an information acquisition device 20. The information acquisition device 20 is a device that acquires images (video data) from an imaging means. If there is a package whose package ID is not included in any of the images in the video, the information acquisition device 20 notifies that the image capture is abnormal. The video data acquired by the information acquisition device 20 is stored in the memory unit 18. The video data stored in the memory unit 18 is then acquired by the acquisition unit 12 of the information processing device 10. This will be explained in detail below.

[0042] 4 and 5 are diagrams illustrating an example of a usage environment of the management system 70 according to this embodiment. The management system 70 can include a plurality of information acquisition devices 20. Each information acquisition device 20 is provided at each relay station or collection and delivery center. The information acquisition device 20 may also be provided on a transportation means 60. For example, each information acquisition device 20 is associated with a location ID corresponding to its installation location. Data such as video data acquired by the information acquisition device 20 is stored in the storage unit 18 in a state where it is associated with the location ID of the information acquisition device 20.

[0043] 4, the management system 70 includes three information acquisition devices 20: information acquisition device 20a, information acquisition device 20b, and information acquisition device 20c. There is no limit to the number of information acquisition devices 20 included in the management system 70. In the example of Fig. 4, the information processing device 10, information acquisition device 20a, information acquisition device 20b, information acquisition device 20c, package management server 30, and user terminal 40 are connected to each other via a communication network.

[0044] The package management server 30 is a server of a system that manages the total delivery of packages from their shipping point to their destination, such as a server for a Transport Management System (TMS). The package management server 30 manages tracking information, which associates information about the location and time of each package with the package ID of the package. The location and time of each package are recorded in the package management server 30 by inspecting the package using a barcode reader or the like to read the package ID attached to the package at the time of collection, unloading, loading, etc. The tracking information includes a list of the date and time and location of the package inspection. The package management server 30 also stores package information for each package. The package information includes the package ID, item name (contents), weight, size, shape, type, etc. of the package. The package management server 30 and the information processing device 10 may be integrated into one device. The information processing device 10 may also receive the package ID of the target package input to the package management server 30. In addition, the information processing device 10 may output the accident identification result to the luggage management server 30.

[0045] The user terminal 40 is a terminal device that allows a user of the management system 70 to access the information processing device 10, the information acquisition device 20, the baggage management server 30, etc. via a network. The information processing device 10 can accept input of the baggage ID of the target baggage performed directly to the information processing device 10, as well as input of the baggage ID of the target baggage performed via the user terminal 40. Furthermore, the information processing device 10 can output the results of identifying the occurrence of an accident to a display provided on the information processing device 10, as well as to a display on the user terminal 40.

[0046] 5, an information processing device 10, an information acquisition device 20, a package management server 30, and an identification terminal 50 are connected to one another via a communication network. The identification terminal 50 is a terminal used for inspecting packages, and is, for example, a barcode reader.

[0047] For example, when a transportation means 60 arrives at a collection and delivery center, work to unload packages 100 from the transportation means 60 is performed at a work site. Identification information 101 is attached to each package 100. An imaging means 200 is provided at the work site, and video of the package 100 being unloaded from the transportation means 60 is captured. The imaging means 200 is, for example, a camera. The imaging means 200 is communicably connected to the information acquisition device 20, and the information acquisition device 20 acquires video data from the imaging means 200. When unloading the package 100 from the transportation means 60, a worker positions the package 100 so that the identification information 101 is captured by the imaging means 200. Alternatively, the worker may move the imaging means 200 to a position where the identification information 101 is captured. Furthermore, multiple imaging means 200 may be provided to increase the likelihood that the identification information 101 will be captured by the imaging means 200. Mirrors or the like may be placed around the area where the luggage 100 is placed to reduce the blind spot of the imaging means 200.

[0048] Furthermore, at the collection and delivery center, inspection work is performed on the packages 100. That is, by reading the identification information 101 of the package 100 with the identification terminal 50, the reading time and reading location (location ID indicating the reading location) are linked to the package ID and transmitted to the package management server 30. The package management server 30 stores the received information.

[0049] The inspection work may be performed in the same location as the work of unloading the luggage 100, or may be performed in a different location. If the inspection work is performed in a different location from the work of unloading the luggage 100, the imaging means 200 and the information acquisition device 20 may be provided in the location where the inspection work is performed to acquire video data, or the imaging means 200 and the information acquisition device 20 may not be provided in the location where the inspection work is performed. Alternatively, the information acquisition device 20 may be provided in a location where it is expected that there is a high possibility of an accident occurring.

[0050] Each of the information acquisition device 20, the package management server 30, the user terminal 40, and the identification terminal 50 can be realized by a computer similar to that shown in FIG.

[0051] <Information acquisition device> 6 is a block diagram illustrating an example of the functional configuration of information acquisition device 20. In the example of this figure, information acquisition device 20 includes an image acquisition unit 22, an image determination unit 23, a first storage unit 24, a second storage unit 25, a processing unit 26, and an output unit 27. The process of information acquisition device 20 acquiring video data will be described in detail with reference to this figure.

[0052] The image acquisition unit 22 acquires video data from the imaging means 200 and temporarily stores it in the first storage unit 24. The image determination unit 23 determines whether the video data stored in the first storage unit 24 contains (or captures) the identification information 101. The image determination unit 23 performs this determination for each predetermined determination period. That is, the video data is divided into one or more determination periods, and the image determination unit 23 determines whether the identification information 101 is included in each determination period. The determination period is not particularly limited, but may be, for example, a predetermined length. The determination period may be a period during which the target package can be tracked by the information processing device 10. Alternatively, the determination period may be determined based on a condition such as "until another package is placed on top of the package." The image determination unit 23 detects each package in the video data and determines whether the identification information 101 is included in each determination period of the video data. The detection of the package and the identification information 101 can be performed using existing image recognition technology. If the identification information 101 is not included in the video data within the determination period, the output unit 27 outputs a notification indicating that the image capture is inappropriate. The notification may be a message urging the worker to position the image capturing means 200 so that the identification information 101 is captured on the image capturing means 200. This allows the worker unloading the luggage to retake an image of the luggage 100. As a result, video data from which the identification information 101 can be detected can be acquired later in the information processing device 10. Note that the video data stored in the storage unit 18 does not need to be associated with a luggage ID.

[0053] There is no particular limitation on the method of outputting the notification from output unit 27. For example, the output method may be a voice message, an alarm sound, or a display that can be seen by the worker.

[0054] If the image determination unit 23 determines that the video data stored in the first storage unit 24 includes identification information 101, the video data stored in the first storage unit 24 is stored in the second storage unit 25 in a predetermined unit. The video data stored in the second storage unit 25 is deleted from the first storage unit 24. Here, the predetermined unit may be, for example, "each time the amount of cargo that can be stacked on a pallet or a forklift is reached," or "each time all cargo stored in one transport means 60 is unloaded," or it may be a fixed predetermined time period. It is sufficient that identification information 101 of one or more cargoes exists in one predetermined unit.

[0055] The processing unit 26 associates information about the location where the information acquisition device 20 is located with the video data stored in the second storage unit 25. The location information is, for example, the name of a collection and delivery center or identification information (location ID) of the transportation means 60. If there is information in advance that links the ID of the information acquisition device 20 with the collection and delivery center or the transportation means 60, the location ID may be the identification information of the information acquisition device 20. In addition, the video data is associated with the date and time it was filmed.

[0056] The video data associated with the location ID is output from the information acquisition device 20. The output video data is stored in the storage unit 18. When the storage unit 18 is included in the information processing device 10, the video data is transmitted from the information acquisition device 20 to the information processing device 10. The timing of transmitting data from the information acquisition device 20 to the information processing device 10 is not particularly limited. For example, data transmission may be performed at a predetermined cycle, at a predetermined time, or dynamically when communication congestion on the information processing device 10 is alleviated. Data transmission may also be triggered by the information acquisition device 20 accepting a data transmission request from the information processing device 10.

[0057] The information processing device 10 may be a device independent of the imaging means 200, or the information processing device 10 may be integrated with the imaging means 200.

[0058] FIG. 7 is a diagram illustrating an example of the configuration of data stored in the memory unit 18. The memory unit 18 stores video data acquired by the information acquisition device 20. The memory unit 18 also stores threshold information in advance. The memory unit 18 stores baggage information. The baggage information may be stored in the memory unit 18 in advance, or may be acquired from the baggage management server 30 when the baggage ID of the target baggage is input to the information processing device 10. In other words, the information processing device 10 may acquire baggage information corresponding to the baggage ID of the target baggage from the baggage management server 30 and store it in the memory unit 18.

[0059] FIG. 8 is a diagram illustrating the configuration of threshold information. The threshold information associates the property of the package with a threshold. In the example shown in this figure, a threshold is set for each combination of the package's mass and material or contents. For example, if the package is made of plastic and weighs 2.0 kg, it indicates that there is a high possibility that the package will be damaged if a pressure of 40 Pa or more is applied.

[0060] When comparing information about an accident with a threshold, the determination unit 16 of the information processing device 10 identifies the threshold based on the luggage information about the target luggage. Specifically, the determination unit 16 extracts luggage properties (e.g., material, contents, and mass) from the luggage information about the target luggage stored in the storage unit 18. Then, the determination unit 16 obtains a threshold corresponding to the extracted properties from the threshold information. In this way, it is possible to identify and determine a threshold at which an accident is likely to occur depending on the luggage properties. Note that the threshold information is not limited to the example of FIG. 8. For example, in the threshold information, the threshold may be defined for each product or for another mass unit. Furthermore, the unit and value of the threshold are determined depending on the type of information about the accident.

[0061] 9 is a diagram illustrating an example of the configuration of video data stored in storage unit 18. In storage unit 18, video data is stored so that it can be extracted using the shooting location and shooting date and time. The video data is organized and stored in chronological order for each location based on the shooting location and shooting date and time. The shooting date and time is, for example, the timestamp of the file when the video data was generated.

[0062] The acquisition unit 12 of the information processing device 10 can select video data that is likely to include the target package from multiple video data stored in the storage unit 18, for example, as follows. When the acquisition unit 12 acquires tracking information for the target package, it is determined that collection and delivery center L1100 is one of the unloading locations. Therefore, the acquisition unit 12 references all video data managed at collection and delivery center L1100 in the storage unit 18. However, if the details of the unloading location can be identified from the tracking information, it is also possible to reference only the video data managed at the work location within the collection and delivery center. For example, if the tracking information contains information indicating that the package was inspected at inspection location L1103 and the unloading location corresponding to inspection location L1103 at collection and delivery center L1100 is uniquely determined, it is sufficient to reference only the video data managed at work location L1101.

[0063] <Information processing method> FIG. 10 is a flowchart illustrating the flow of processing executed by the information processing device 10 according to this embodiment. The information processing method according to this embodiment is executed by a computer. The information processing method according to this embodiment includes steps S11, S12, and S13. In step S11, the acquisition unit 12 acquires an image including a package. In step S12, the detection unit 14 processes the image to detect identification information of the target package. In step S13, the determination unit 16 determines whether an accident has occurred with the target package using the detection result of the identification information of the target package and information related to the package accident.

[0064] FIG. 11 is a flowchart illustrating in detail the flow of processing executed by the information processing device 10 according to this embodiment.

[0065] For example, when an accident occurs with a delivered package and the user of the information processing device 10 wants to identify the location and date and time of the accident, the user inputs the package ID attached to the package into the information processing device 10. The package ID may be input into the information processing device 10 by reading a barcode or the like with a reader, or by typing a character string or the like written on a label or the like into the information processing device 10 with a keyboard or the like. The package's shipping slip number or individual number may also be input. These numbers are linked to the package ID in the package information held in the package management server 30, and the information processing device 10 can obtain package information and tracking information for the target package from the package management server 30 based on the shipping slip number or individual number.

[0066] In S101, when the information processing device 10 receives input of the package ID of the target package, it accesses the package management server 30 and acquires tracking information corresponding to the package ID. Then, the acquisition unit 12 identifies a location included in the tracking information as a candidate location. S102 to S114 constitute loop processing A, which is repeated until processing is completed for all candidate locations. In S103, the acquisition unit 12 searches for video data of a certain candidate location i from multiple video data stored in the storage unit 18. If video data for candidate location i does not exist (No in S103), processing for candidate location i ends (S114).

[0067] If video data for the candidate location i exists (Yes in S103), the acquisition unit 12 extracts and acquires candidate video data from the video data for the candidate location i in S104. Candidate video data can be extracted as follows: First, the acquisition unit 12 specifies a search range related to date and time. If the video data was filmed at a work location where luggage is unloaded from the transportation means 60, an example of the search range is from the date and time when the target luggage is estimated to have been unloaded from the transportation means 60 at that location to the date and time when the target luggage is inspected. Here, the date and time when the target luggage is estimated to have been unloaded from the transportation means 60 at that location is a date and time that is a predetermined time before the date and time when the target luggage is inspected. The predetermined time can be determined, for example, based on the average time required from unloading to inspection. The date and time of inspection is identified from tracking information. If the video data was filmed at a work location where luggage is loaded onto the transportation means 60, an example of the search range is from the date and time when the target luggage is inspected to the date and time when the target luggage is estimated to have been loaded onto the transportation means 60. Here, the date and time when the target cargo is estimated to have been loaded onto the transportation means 60 is the date and time when a predetermined time has elapsed since the date and time when the target cargo was inspected. The predetermined time can be determined, for example, based on the average time taken from inspection to loading.

[0068] The acquisition unit 12 refers to the shooting date and time associated with each video data of the candidate location i, and extracts video data whose shooting range overlaps with at least a part of the identified search range as candidate video data. Next, the detection unit 14 performs loop process B. S105 to S113 constitute loop process B, which is repeated until processing is completed for all candidate video data of a certain candidate location. In S106, the detection unit 14 detects the baggage ID of the target baggage in certain candidate video data j. If the baggage ID of the target baggage is not detected in the candidate video data j (No in S106), the processing for the candidate video data j ends (S113). If the baggage ID of the target baggage is detected in the candidate video data j (Yes in S106), the determination unit 16 generates information about the accident of the target baggage by analyzing the candidate video data j as described above (S107). Furthermore, the determination unit 16 identifies the threshold to be used for determination as described above using the baggage information of the target baggage and the threshold information. Then, it determines whether the calculated information about the accident is equal to or greater than the threshold (S108). If the information about the accident is equal to or greater than the threshold value (Yes in S108), the determination unit 16 temporarily stores the location and date and time associated with the information about the accident in the storage unit 18 (S112), and the processing for the candidate video data j ends (S113). Note that if loop process A and loop process B are repeated and multiple accident occurrences are identified, multiple locations and dates and times will be stored in the storage unit 18.

[0069] If the information about the accident is less than the threshold value (No in S108), it is confirmed whether the location and date and time when the accident occurred, i.e., the location and date and time when the information about the accident was equal to or greater than the threshold value, are already stored in the storage unit 18 (S109).If at least one location and date and time when the accident occurred is already stored in the storage unit 18 (Yes in S109), no further locations and dates and times are stored, and processing for the candidate video data j ends (S113).

[0070] If no location and date / time at which an accident has occurred are stored in the memory unit 18 (No in S109), the determination unit 16 performs the following process to identify at least the location and date / time at which the accident is most likely to have occurred. That is, the determination unit 16 identifies the location and date / time associated with the information on the accident that is less than the threshold value and closest to the threshold value among the generated information on the accident.

[0071] Specifically, in S110, the determination unit 16 determines whether the information about the accident generated in that loop is closest to the threshold value among the information about accidents that are less than the threshold value that have been generated up to that point. If the information about the accident is closest to the threshold value (Yes in S110), the determination unit 16 stores the information about the accident and the location and date and time associated with the information about the accident in the storage unit 18 (S111). If the information about the accident that is less than the threshold value and the location and date and time associated with the information about the accident have already been stored in the storage unit 18 in the previous processing, the determination unit 16 updates that information. Then, the processing for the candidate video data j ends (S113). Note that if information about an accident that is greater than the threshold value is generated in subsequent processing and the location and date and time associated with that information are stored, the information about the accident that is less than the threshold value will no longer be needed and will be deleted.

[0072] On the other hand, if the information about the accident is not closest to the threshold value (No in S110), the location and date and time are not stored in the storage unit 18, and the process for the candidate video data j ends (S113).

[0073] When processing for all candidate video data and all candidate locations is completed in this manner, the information processing device 10 outputs the location and date and time stored in the storage unit 18 (S115). If there is information about an accident exceeding the threshold, this location and date and time are the location and date and time identified as the accident occurred. Multiple locations and dates and times identified as the accident occurred may be output. On the other hand, if there is no information about an accident exceeding the threshold, this location and date and time are the location and date and time at which the accident most likely occurred. Note that if there is no information about an accident exceeding the threshold, a notification to that effect may also be output. Furthermore, in addition to the location and date and time, the information processing device 10 may output the video captured at that location and date and time.

[0074] In this way, the user can ascertain the location and date and time when an accident may have occurred with the target package. Furthermore, based on the location and date and time of the accident, the user can identify the worker who was involved in the accident with the package.

[0075] The information processing device 10 and management system 70 according to this embodiment can also be applied to automated deliveries in which no worker is substantially involved.

[0076] <Modification> A condition such as "upside down" may be required for a piece of luggage. In this case, the determination unit 16 may determine whether an accident has occurred with the target piece of luggage based on the condition required for the piece of luggage. For example, assume that the condition "upside down" is required for the target piece of luggage. The luggage information further includes information indicating the condition required for the piece of luggage, and the determination unit 16 can identify the condition required for the piece of luggage from the luggage information. In addition, the threshold information associates the condition required for the piece of luggage with a threshold indicating the tilt angle. For example, the threshold information indicates a threshold when the "upside down" condition is required and a threshold when the "upside down" condition is not required. The threshold when the "upside down" condition is required is set to be smaller than the threshold when the "upside down" condition is not required. The determination unit 16 identifies the threshold based on the condition required for the piece of luggage.

[0077] The determination unit 16 analyzes the video data in which the identification information 101 of the target baggage is detected, and calculates the tilt angle of the target baggage as information related to the accident. The calculation of the tilt angle of the target baggage can be performed using existing image processing technology. Then, the determination unit 16 determines that an accident has occurred with the target baggage if the calculated tilt angle is equal to or greater than a threshold value. On the other hand, the determination unit 16 determines that an accident has not occurred with the target baggage if the calculated tilt angle is less than the threshold value.

[0078] Next, the operation and effect of this embodiment will be described. According to the information processing device 10 of this embodiment, by acquiring and managing video data of a package during delivery, if an accident occurs to the package, the location and date and time of the accident can be identified after the fact.

[0079] (Second embodiment) 12 is a diagram illustrating an example of a usage environment of a management system 70 according to the second embodiment. The information processing device 10, the information acquisition device 20, and the management system 70 according to this embodiment are similar to the information processing device 10, the information acquisition device 20, and the management system 70 according to the first embodiment, respectively, except for the points described below.

[0080] In this embodiment, the acquisition unit 12 of the information processing device 10 further acquires first measurement data, which is data from a sensor 220 installed in a location where video data was generated, such as a work site. The determination unit 16 identifies the target date and time using the detection result of the identification information of the target package. In addition, the determination unit 16 generates at least a portion of information regarding one or more accidents involving the target package using the target date and time and the first measurement data. This will be explained in detail below.

[0081] The information acquisition device 20 according to this embodiment further acquires first measurement data obtained using the sensor 220 and stores it in the storage unit 18 together with the video data. The information processing device 10 uses the first measurement data to generate information related to an accident involving the target baggage and determines whether an accident has occurred. This allows for more accurate identification of baggage accidents. The information processing device 10 may generate only the first measurement data as information related to the accident, or may analyze the video data as described in the first embodiment to generate further information related to the accident and use it for the determination.

[0082] In this embodiment, a sensor 220 is also installed where the imaging unit 200 is installed. The sensor 220 measures information about the environment surrounding the luggage. However, sensors 220 do not necessarily have to be installed in all locations where the imaging unit 200 is installed. The sensor 220 measures, for example, at least one of sound, temperature, and humidity. The determination unit 16 generates at least one of sound pressure, temperature, and humidity as at least part of information about one or more accidents involving the target luggage based on the target date and time and the first measurement data. Vibration and sound are related to impacts that occur on the luggage. Temperature and humidity are also related to deterioration accidents involving luggage that is sensitive to high temperatures or high humidity, or luggage that must be managed at specific temperatures and humidity levels. Therefore, luggage accidents can be identified more accurately using the first measurement data obtained from the sensor 220. The sensor 220 is communicably connected to the information acquisition device 20.

[0083] FIG. 13 is a diagram illustrating an example of the configuration of data stored in the storage unit 18 according to this embodiment. A method for the information acquisition device 20 according to this embodiment to acquire first measurement data will be described below. The sensor 220 measures, for example, the sound generated during unloading. In this case, the first measurement data obtained by the sensor 220 is sound data. The first measurement data generated by the sensor 220 is stored in the second storage unit 25 of the information acquisition device 20. The processing unit 26 of the information acquisition device 20 then associates a date and time with the first measurement data stored in the second storage unit 25. The first measurement data and video data are associated via the date and time associated with the data. The shooting date and time associated with the video data may be a timestamp as described in the first embodiment, but it is preferable for the processing unit 26 to associate a new date and time with the video data. Note that if the sensor 220 has a date and time acquisition function and this date and time is synchronized with the date and time of the same function of the imaging means 200, the processing unit 26 does not need to associate a date and time with the first measurement data. The first measurement data is associated with information indicating the location where it was acquired, similar to the video data.

[0084] In this way, the video data and the first measurement data are transmitted to the information processing device 10 in a mutually associated state and stored in the storage unit 18. The first measurement data is managed by location and in chronological order in the storage unit 18, just like the video data. In this way, information about the environment surrounding the luggage can be collected in accordance with the video data.

[0085] 14 is a flowchart illustrating the flow of processing executed by the information processing device 10 according to this embodiment. The processing executed by the information processing device 10 according to this embodiment is the same as the processing executed by the information processing device 10 according to the first embodiment, except for S207.

[0086] When the information processing device 10 receives the package ID of the target package, the information processing device 10 performs loop processing A in the same manner as described in the first embodiment to search for candidate locations. The information processing device 10 also performs loop processing B in the same manner as described in the first embodiment to search for candidate video data. However, if the package ID of the target package is detected in step S106 (YES in S106), the determination unit 16 identifies the target date and time and the target location based on the detection result, and acquires first measurement data corresponding to the target date and time and the target location from the storage unit 18 (S207). Here, the target date and time is the date and time associated with the video data in which the package ID was detected. The target date and time can be said to be the date and time when the target package, identified based on the video data, was present at the location (e.g., the unloading work location). The target date and time is defined, for example, as the period during which the target package was present at the location. The determination unit 16 may acquire, from the storage unit 18, first measurement data whose measurement date and time at least partially overlap with the target date and time. The target location is the location associated with the video data in which the package ID of the target package was detected.

[0087] When the determination unit 16 acquires the first measurement data, it extracts data at the target date and time from the acquired first measurement data to generate information related to the accident involving the target baggage. Then, using the generated information related to the accident, it performs subsequent processing in the same manner as in the first embodiment. Note that in S207, the determination unit 16 may further perform necessary processing on the first measurement data to generate information related to the accident. For example, if the first measurement data is sound waveform data, the determination unit 16 may calculate the maximum sound pressure from the extracted waveform data and use the calculated sound pressure as information related to the accident.

[0088] FIG. 15 is a diagram illustrating threshold information according to this embodiment. In the example shown in this figure, a threshold sound pressure is set for each combination of the mass of the package and the material or contents. Using this threshold information, the determination unit 16 can identify the threshold to be used for determining information related to the accident that indicates sound pressure. For example, a threshold may be further set for each piece of cushioning material included in the package. In this case, information about the cushioning material is further included in the package information.

[0089] As described above, the determination unit 16 may further analyze the video data to generate information on an accident such as the pressure applied to the luggage. In that case, the determination may be made by combining the information on the accident generated from the first measurement data and the information on the accident generated by analyzing the video data. For example, when the determination unit 16 determines whether an accident has occurred using both pressure and sound pressure, it may be specified that an accident has occurred when at least one of the pressure and sound pressure is equal to or greater than the threshold value, or it may be specified that an accident has occurred only when both are equal to or greater than the threshold value. Also, when a plurality of first measurement data are acquired by a plurality of sensors 220, the determination may be made by combining them. Note that the threshold value is determined for each type of information on the accident.

[0090] Although an example of using sound pressure as information on an accident has been described above, the information on an accident is not limited to sound pressure. For example, for luggage, there may be cases where conditions such as refrigeration or freezing are required. In this case, the determination unit 16 may determine whether an accident has occurred with respect to the target luggage based on the conditions required for the luggage. For example, assume that the target luggage is required to have the condition of "requiring refrigeration". The luggage information further includes information indicating the conditions required for the luggage, and the determination unit 16 can specify the conditions required for the luggage from the luggage information. Also, the threshold value information is associated with the conditions required for the luggage and the threshold value indicating the temperature. For example, the threshold value information shows the threshold value T1 when the condition of "requiring freezing" is required, the threshold value T2 when the condition of "requiring refrigeration" is required, and the threshold value T3 when neither the condition of "requiring freezing" nor the condition of "requiring refrigeration" is required. Here, T1 < T2 < T3. The determination unit 16 specifies the threshold value based on the conditions required for the luggage.

[0091] The sensor 220 is a temperature sensor, and the determination unit 16 calculates, for example, a maximum temperature or an average temperature as information related to the accident based on the first measurement data. If the calculated maximum temperature or average temperature is equal to or greater than a threshold, the determination unit 16 determines that an accident has occurred with the target package. On the other hand, if the calculated maximum temperature is less than the threshold, the determination unit 16 determines that no accident has occurred with the target package.

[0092] Next, the operation and effect of this embodiment will be described. In this embodiment, the same operation and effect as in the first embodiment can be obtained. In addition, the first measurement data obtained by the sensor 220 can be used to more accurately identify the occurrence of a luggage accident.

[0093] (Third embodiment) 16 is a diagram illustrating an example of a usage environment of a management system 70 according to the third embodiment. The information processing device 10, the information acquisition device 20, and the management system 70 according to this embodiment are similar to the information processing device 10, the information acquisition device 20, and the management system 70 according to the first embodiment, respectively, except for the points described below.

[0094] In this embodiment, the acquisition unit 12 further acquires second measurement data, which is measurement data for each of a plurality of positions within the transportation means 60, obtained by a sensor 260 provided on the transportation means 60 that transports the luggage 100. The determination unit 16 identifies a target position, which is the position of the target luggage within the transportation means 60, using the detection result of the identification information of the target luggage. Then, the determination unit 16 generates at least a portion of information regarding one or more accidents involving the target luggage, using the second measurement data corresponding to the target position. This will be described in detail below.

[0095] In this embodiment, the imaging means 200 is, for example, a 3D camera that can obtain three-dimensional information. The imaging means 200 is provided to capture images of the inside of a luggage loading section 610 (for example, a container) in the transport means 60. The imaging means 200 may be installed at a collection and delivery center or may be installed in the transport means 60. From the video data obtained by the imaging means 200, the position where each luggage is placed in the loading section 610 can be detected. If depth and the like can be measured by image analysis or the like, the imaging means 200 may be a 2D camera.

[0096] Furthermore, a sensor 260 is provided within the loading section 610 of the transportation means 60. The sensor 260 is, for example, a sensor that measures at least one of vibration, sound, temperature, and humidity. The loading section 610 may be provided with multiple sensors. The loading section 610 may also be provided with multiple types of sensors (vibration sensors, sound sensors, temperature sensors, humidity sensors, etc.). Examples of vibration sensors include acceleration sensors and displacement sensors. The sensor 260 provided in the loading section 610 makes it possible to identify any luggage accidents during transportation. The determination unit 16 generates, as information related to the accident, for example, at least one of acceleration, sound pressure, temperature, and humidity, based on the second measurement data corresponding to the target position.

[0097] The imaging means 200 and the sensor 260 can communicate with the information acquisition device 20. In the example shown in the figure, the information acquisition device 20 is provided in the transportation means 60, but the information acquisition device 20 may also be provided in a collection and delivery center or the like. Also, for example, video data from the imaging means 200 may be acquired by the information acquisition device 20 at the collection and delivery center, and the second measurement data from the sensor 260 may be acquired by the information acquisition device 20 provided in the transportation means 60.

[0098] In this embodiment, the information acquisition device 20 links the identification information of the transportation means 60 and the identification information of the loading or unloading work location as location information to the video data obtained by the imaging means 200. By searching such video data, the information processing device 10 can identify which transportation means 60 the cargo is loaded onto and at which position in the loading section 610 of that transportation means 60 the cargo is loaded.

[0099] The second measurement data obtained by the sensor 260 during transportation by the transportation means 60 is acquired by the information acquisition device 20 and stored in a state linked to the measurement date and time. Furthermore, in the information acquisition device 20, the second measurement data is linked to the identification information of the transportation means 60 from which the second measurement data was obtained. The process of linking the second measurement data to the date and time may be performed by the sensor 260 or the information acquisition device 20.

[0100] The second measurement data linked to the identification information of the transportation means 60 and the date and time is transmitted from the information acquisition device 20 to the information processing device 10 and stored in the storage unit 18. The timing of transmitting the data from the information acquisition device 20 to the information processing device 10 is not particularly limited. The data may be transmitted from the information acquisition device 20 to the information processing device 10, for example, when transportation by the transportation means 60 is completed, or may be transmitted during transportation. In this way, information for identifying the occurrence of a cargo accident during transportation by the transportation means 60 can be collected.

[0101] FIG. 17 is a diagram illustrating an example of the configuration of data stored in the memory unit 18 according to this embodiment. FIG. 18 is a diagram illustrating in detail the configuration of the video data and second measurement data managed in this embodiment. In the examples of FIGS. 17 and 18, the second measurement data is temperature data and acceleration data. In this embodiment, the video data and second measurement data acquired by the information acquisition device 20 installed in the transportation means 60 are managed for each identification information of the transportation means 60. The video data is linked to the location and date and time of loading or unloading. Furthermore, the second measurement data is linked to the date and time. Therefore, the information processing device 10 can manage the video data and second measurement data related to the transportation means 60 in chronological order by referencing the operation information of the transportation means 60 stored in the baggage management server 30.

[0102] For example, suppose that the date and time of second measurement data during a certain transport for truck T0001 is 15:00:00 on February 1st. Here, from the operation information of transport means 60 stored in package management server 30, it is known that the departure date and time from work location L1101 is 14:10:00 on February 1st, and the arrival date and time at work location L1000 is 15:30:00 on February 1st. Therefore, it is known that the second measurement data is information during transport from work location L1101 to work location L1000.

[0103] 19 is a flowchart illustrating the flow of processing executed by the information processing device 10 according to this embodiment. The processing executed by the information processing device 10 according to this embodiment is the same as the processing executed by the information processing device 10 according to the first embodiment, except for the points described below.

[0104] In S300, the information processing device 10 receives the package ID of the target package and acquires tracking information for the target package. Then, it extracts all transportation means 60 that have transported the target package as candidate transportation means. Then, the information processing device 10 performs loop processing C for all candidate transportation means. The loop processing is from S301 to S310.

[0105] Data is searched for a candidate transportation means k as follows. First, work locations that candidate transportation means k has passed through are extracted as candidate locations. Then, loop process A is performed until all candidate locations have been searched. Loop process A consists of S302 to S114. In the search for a candidate location i, first, video data at the candidate location i (work location) and second measurement data during transportation from candidate location i to the next work location are searched from the storage unit 18 (S303). If at least one of the data does not exist (No in S303), the process for candidate location i is terminated (S114). If both data exist (Yes in S303), candidate video data is extracted from the video data of candidate location i in the same manner as described in the first embodiment (S104). For example, in the example of FIG. 18, candidate transportation means k is truck T0001 and candidate location i is work location L1101. In this example, it can be said that both video data of work location L1101 and second measurement data from work location L1101 to work location L1000 exist.

[0106] Next, loop process B is performed for all extracted candidate video data. Loop process B is from S105 to S113. The process for a certain candidate video data j is as follows. The information processing device 10 detects the baggage ID of the target baggage in the candidate video data j in the same manner as described in the first embodiment (S106). If the baggage ID of the target baggage is not detected in the candidate video data j (No in S106), the process for the candidate video data j ends (S113).

[0107] If the package ID of the target package is detected in the candidate video data j (Yes in S106), in S307, the determination unit 16 tracks the target package in the candidate video data j and identifies the location where the target package is placed in the loading unit 610 as the target location. Then, the determination unit 16 acquires second measurement data from the storage unit 18 along the route from the candidate location i to the next work location and extracts the second measurement data at the target location. Here, the target location and the measurement location of the second measurement data do not need to be exactly the same. For example, if the sensor 260 is a thermal camera or the like, it can extract the temperature at any location within the loading unit 610. On the other hand, if the sensor 260 is a sensor that can only measure a portion of the temperature within the loading unit 610, it is sufficient to extract the temperature at the location closest to the target location. Alternatively, a predicted temperature value at the target location may be calculated from the second measurement data at two or more locations, taking into account factors such as the distance to the target location. If the material of the target package or the material of the packaging material can be identified from the package information, the predicted temperature value may be calculated by further taking into account their thermal conductivity. In this way, the determination unit 16 generates the temperature at the target position as information related to the accident.

[0108] The information processing device 10 performs the subsequent processing in the same manner as in the first embodiment, using the temperature at the target location as information related to the accident. In this way, the user can grasp the location and date and time when the accident involving the target package may have occurred. Furthermore, based on the location and date and time of the accident, the driver of the transportation means 60 in which the package was involved can be identified.

[0109] Although the above describes a case where the target position is identified using video data of loading, the information processing device 10 may also identify the target position using video data of unloading. In this case, in S303, the video data of the candidate location i (work location) and the second measurement data during transportation from the work location immediately preceding the candidate location i to the candidate location i are searched for in the memory unit 18. In addition, in S307, the determination unit 16 obtains the second measurement data from the work location immediately preceding the candidate location i from the memory unit 18 and extracts the second measurement data at the target position.

[0110] Furthermore, the information processing device 10 may combine the method according to this embodiment with at least one of the methods according to the first and second embodiments. The methods according to the first and second embodiments can identify cargo accidents at work sites, and the method according to the third embodiment can identify cargo accidents during transportation.

[0111] Next, the operation and effect of this embodiment will be described. In this embodiment, the same operation and effect as in the first embodiment can be obtained. In addition, it is possible to know after the fact from the video data where the target cargo was placed on the transportation means, and to identify cargo accidents during transportation.

[0112] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various other configurations may be adopted. For example, in the sequence diagrams and flowcharts used in the above description, multiple steps (processes) are described in order, but the order of execution of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the drawings may be changed to the extent that the content is not affected. Furthermore, the above-described embodiments may be combined to the extent that the content is not contradictory.

[0113] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various other configurations can also be adopted.

[0114] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. Below, examples of reference forms are added. 1-1. A means for acquiring an image including a package; a detection means for detecting identification information of the target baggage by processing the image; and a determination means for determining whether an accident has occurred to the target package using the result of detecting the identification information of the target package and information related to the package accident. Information processing device. 1-2. In the information processing device described in 1-1, The determining means generates at least a portion of information regarding one or more accidents of the target baggage by analyzing the plurality of images. Information processing device. 1-3. In the information processing device described in 1-2, The determining means analyzes the plurality of images to identify the pressure applied to the target baggage as at least a part of the information regarding the one or more accidents of the target baggage. Information processing device. 1-4. In the information processing device according to any one of 1-1 to 1-3, the acquisition means further acquires first measurement data, which is data from a sensor installed at a location where the image was generated; The determination means Identifying the target date and time using the detection result of the identification information of the target package; generating at least a portion of information regarding the one or more accidents of the subject package using the subject date and time and the first measurement data; Information processing device. 1-5. In the information processing device according to 1-4, The determination means generates at least one of sound pressure, temperature, and humidity as at least a part of information regarding the one or more accidents of the target baggage based on the target date and time and the first measurement data. Information processing device. 1-6. In the information processing device according to any one of 1-1 to 1-5, The acquisition means further acquires second measurement data, which is measurement data for each of a plurality of positions within the transportation means, obtained by a sensor provided on the transportation means that transports the luggage; The determination means Using the detection result of the identification information of the target baggage, a target location is identified, which is the location of the target baggage within the transportation means; Using the second measurement data corresponding to the target location, generate at least a portion of information related to the one or more incidents of the target load. Information processing device. 1-7. In the information processing device according to 1-6, The determination means generates at least one of acceleration, sound pressure, temperature, and humidity as at least a part of information about the one or more accidents based on the second measurement data corresponding to the target position. Information processing device. 1-8. In the information processing device according to any one of 1-1 to 1-5, The determination means determines whether an accident has occurred to the target baggage by comparing the generated information about the accident with a threshold value. Information processing device. 1-9. In the information processing device according to 1-8, The determination means identifies the threshold value based on package information related to the target package. Information processing device. 1-10. In the information processing device according to any one of 1-1 to 1-9, When the determination means determines that an accident has occurred with the target package, it identifies the date and time and place associated with the information about the accident used for the determination as the date and time and place when the accident occurred with the target package. Information processing device. 2-1 An information processing device according to any one of 1-1. to 1-10.; an information acquisition device for acquiring the image; The information acquisition device notifies the user that the images are not normal when there is a package whose identification information is not included in any of the images. Management system.

[0115] 3-1. A computer-implemented information processing method, Get an image containing the luggage, processing the image to detect identification information of the target package; Using the detection result of the identification information of the target baggage and information about the baggage accident, it is determined whether or not an accident has occurred with the target baggage. Information processing methods. 3-2. In the information processing method described in 3-1, Analyzing a plurality of the images to generate at least a portion of information regarding one or more incidents with the target luggage. Information processing methods. 3-3. In the information processing method described in 3-2, Analyzing the plurality of images to identify pressure applied to the target package as at least a portion of information related to the one or more incidents with the target package. Information processing methods. 3-4. In the information processing method according to any one of 3-1 to 3-3, Further, first measurement data is acquired, which is data from a sensor installed at the location where the image was generated; Identifying the target date and time using the detection result of the identification information of the target package; generating at least a portion of information regarding the one or more accidents of the subject package using the subject date and time and the first measurement data; Information processing methods. 3-5. In the information processing method described in 3-4, generating at least one of sound pressure, temperature, and humidity as at least part of information regarding the one or more accidents of the target baggage based on the target date and time and the first measurement data; Information processing methods. 3-6. In the information processing method according to any one of 3-1 to 3-5, Further acquiring second measurement data, which is measurement data for each of a plurality of positions within the transportation means, obtained by a sensor provided on the transportation means that transports the luggage; Using the detection result of the identification information of the target baggage, a target location is identified, which is the location of the target baggage within the transportation means; Using the second measurement data corresponding to the target location, generate at least a portion of information related to the one or more incidents of the target load. Information processing methods. 3-7. In the information processing method described in 3-6., generating at least one of acceleration, sound pressure, temperature, and humidity as at least a portion of information related to the one or more accidents based on the second measurement data corresponding to the target location; Information processing methods. 3-8. In the information processing method according to any one of 3-1 to 3-5, The generated information about the accident is compared with a threshold value to determine whether an accident has occurred with the target baggage. Information processing methods. 3-9. In the information processing method described in 3-8., Identifying the threshold value based on package information related to the package of interest Information processing methods. 3-10. In the information processing method according to any one of 3-1 to 3-9, If it is determined that an accident has occurred with the target package, the date and time and place associated with the information on the accident used for the determination are identified as the date and time and place where the accident occurred with the target package. Information processing methods. 4-1. A program that causes a computer to execute the information processing method described in any one of 3-1 to 3-10. [Explanation of symbols]

[0116] 10. Information processing equipment 12 Acquisition Department 14 Detector 16 Judgment section 18 Memory section 20 Information acquisition device 22 Image acquisition unit 23 Image judgment unit 24 1st memory section 25 2nd memory section 26 Processing section 27 Output section 30 Baggage management server 40 User terminals 50 Identification Terminal 60 Means of transport 70 Management Systems 100 luggage 101 Identification Information 200 Imaging means 220 Sensors 260 Sensors 610 Loading section 1000 calculator

Claims

1. an acquisition means for acquiring an image including the luggage, the image being associated with the location and date of the image; a detection means for detecting, upon receiving a user input of identification information of a target package to be tracked, the identification information attached to the target package from the images by processing the plurality of images; a determination means for calculating the pressure applied to the target baggage by analyzing the plurality of images in which the identification information of the target baggage is detected by the detection means, and for determining whether an accident has occurred to the target baggage by comparing the calculated pressure with a threshold value; an output means for identifying the photographing location and the photographing date and time associated with the image in which it is determined that an accident has occurred with the target baggage, and outputting the identified location and date and time in which the accident has occurred; Equipped with Information processing device.

2. an acquisition means for acquiring an image including the luggage, the image being associated with the location and date of the image; a detection means for detecting, upon receiving a user input of identification information of a target package to be tracked, the identification information attached to the target package from the images by processing the plurality of images; a determination means for determining whether an accident has occurred to the target package using the result of detecting the identification information of the target package and information about the package accident; an output means for outputting the location and date and time when it is determined that an accident has occurred to the target baggage; The determination means identifies the photographing location and the photographing date and time associated with the image in which the identification information of the target baggage was detected by the detection means as the target location and target date and time at which the target baggage was present, The acquisition means further acquires first measurement data corresponding to the target date and time, the first measurement data being data from a sensor installed at the target location; The determination means determines whether an accident has occurred to the target baggage by comparing the acquired first measurement data with a threshold value; the output means outputs the target location and the target date and time corresponding to the first measurement data at which it is determined that an accident has occurred with the target baggage as the location and the date and time at which it is determined that the accident has occurred; Information processing device.

3. an acquisition means for acquiring an image of the interior of a loading section of a transport means, the image being associated with the transport means transporting the luggage and the date and time of the image; a detection means for detecting, upon receiving a user input of identification information of a target package to be tracked, the identification information attached to the target package from the images by processing the plurality of images; a determination means for determining whether an accident has occurred to the target package using the result of detecting the identification information of the target package and information about the package accident; an output means for outputting the location and date and time at which the accident was determined to have occurred; The determination means Identifying the transportation means and the photographing date and time associated with the image in which the identification information of the target baggage was detected by the detection means; Detecting the position where the target cargo is placed within the loading section of the identified transportation means from the image and identifying it as a target position; the acquiring means further acquires, as second measurement data, measurement data corresponding to the target position and the photographing date and time from among measurement data for each of a plurality of positions within the transportation means obtained by a sensor provided on the transportation means; the determination means determines whether an accident has occurred to the target baggage by comparing the second measurement data corresponding to the target position with a threshold value; the output means outputs the target position and the photographing date and time corresponding to the second measurement data in which it is determined that an accident has occurred with the target baggage as the location and the date and time in which it is determined that the accident has occurred. Information processing device.

4. 4. The information processing device according to claim 1, For each of the packages, at least one of the mass, contents, and material of the package is stored as package information related to the package in association with the identification information of the package; The threshold value is stored in association with at least one of the mass, contents, and material of the luggage; The determination means identifies the luggage information related to the target luggage corresponding to the identification information of the target luggage, and identifies the threshold value corresponding to at least one of the mass, the content, and the material of the luggage information as the threshold value of the target luggage. Information processing device.

5. 1. A computer-implemented information processing method, comprising: Obtaining an image containing the luggage, associated with the location and date of the image, When receiving a user input of identification information of a target package to be tracked, the system processes the plurality of images to detect the identification information attached to the target package from the images; By analyzing the plurality of images in which the identification information of the target baggage is detected, a pressure applied to the target baggage is calculated, and by comparing the pressure with a threshold value, it is determined whether an accident has occurred to the target baggage; Identifying the photographing location and the photographing date and time associated with the image in which it is determined that an accident has occurred with the target baggage, and outputting the identified location and date and time in which the accident has occurred. Information processing methods.

6. 1. A computer-implemented information processing method, comprising: Obtaining an image containing the luggage, associated with the location and date of the image, When receiving a user input of identification information of a target package to be tracked, the system processes the plurality of images to detect the identification information attached to the target package from the images; Using the detection result of the identification information of the target baggage and information related to the baggage accident, determine whether or not an accident has occurred to the target baggage; When determining whether an accident has occurred with the target baggage, the photographing location and the photographing date and time associated with the image in which the identification information of the target baggage is detected are identified as the target location and target date and time at which the target baggage was present; Further acquiring first measurement data corresponding to the target date and time, the first measurement data being data from a sensor installed at the target location; determining whether an accident has occurred to the target baggage by comparing the acquired first measurement data with a threshold value; outputting the target location and the target date and time corresponding to the first measurement data at which it is determined that an accident has occurred with the target baggage as the location and date and time at which it is determined that an accident has occurred with the target baggage; Information processing methods.

7. 1. A computer-implemented information processing method, comprising: Acquire an image of the inside of a loading section of a transportation means for transporting the luggage, the image being associated with the transportation means for transporting the luggage and the date and time of the image acquisition; When receiving a user input of identification information of a target package to be tracked, the system processes the plurality of images to detect the identification information attached to the target package from the images; Using the detection result of the identification information of the target baggage and information related to the baggage accident, determine whether or not an accident has occurred to the target baggage; When determining whether an accident has occurred to the target baggage, Identifying the transportation means and the date and time of the image associated with the image from which the identification information of the target package was detected; Detecting the position where the target cargo is placed within the loading section of the identified transportation means from the image and identifying it as a target position; further acquiring, as second measurement data, measurement data corresponding to the target position and the photographing date and time from among measurement data for each of a plurality of positions within the transportation means obtained by a sensor provided on the transportation means; determining whether an accident has occurred to the target baggage by comparing the second measurement data corresponding to the target location with a threshold value; outputting the target position and the photographing date and time corresponding to the second measurement data for which it has been determined that an accident has occurred with the target baggage as the location and date and time for which it has been determined that the accident has occurred; Information processing methods.

8. A program that causes a computer to execute the information processing method according to any one of claims 5 to 7.

Images