Information processing device and program

Abstract

To provide an information processing device capable of detecting that a person has lost an article without attaching identification means such as an IC tag to the article.SOLUTION: A server device (information processing device) comprises: an image acquisition unit which acquires an image shot by a camera (imaging device); a person detection unit which detects a person from among the images acquired by the image acquisition unit; an object detection unit (abnormality detection unit) which detects an object separated from the person detected by the person detection unit; a distance calculation unit which calculates the distance between the person detected by the person detection unit and the object detected by the object detection unit; and a lost article determination unit which determines the object to be a lost article when the distance calculated by the distance calculation unit is a threshold value or more over a prescribed time or more.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] Embodiments of the present invention relate to an information processing apparatus and a program.

Background Art

[0002] Conventionally, there has been known a forgetting / losing / missing article notification system that attaches an IC tag to an article one possesses and notifies a notification device one possesses when a signal from the IC tag is detected when the article is lost (for example, Patent Document 1).

[0003] In such a forgetting / losing / missing article notification system, it has been troublesome because it is necessary to attach an IC tag to each of the articles one possesses. Also, when an article without an attached IC tag is lost, it cannot be notified. Therefore, there has been a demand for an information processing apparatus that can more easily detect that an article has been lost.

Summary of the Invention

Problems to be Solved by the Invention

[0004] The problem to be solved by the present invention is to provide an information processing apparatus and a program that can detect that an article has been lost without attaching an identification means such as an IC tag to the article.

Means for Solving the Problems

[0005] The information processing apparatus according to the embodiment includes an image acquisition unit, a person detection unit, an abnormality detection unit, a distance calculation unit, and a lost article determination unit. The image acquisition unit acquires an image captured by an imaging device. The person detection unit detects a person from the image acquired by the image acquisition unit. The abnormality detection unit detects an object separated from the person detected by the person detection unit. The distance calculation unit calculates the distance between the person detected by the person detection unit and the object detected by the abnormality detection unit.The lost article determination unit The object detected by the anomaly detection unit is not an object that a person had picked up before that point in time, and furthermore,If the distance calculated by the distance calculation unit remains above a threshold for a predetermined period of time or longer, the object is determined to be a lost item. [Brief explanation of the drawing]

[0006] [Figure 1] Figure 1 is a block diagram showing an example of a schematic configuration of the lost item detection system according to the embodiment. [Figure 2] Figure 2 is a hardware block diagram showing an example of the hardware configuration of the server device included in the lost item detection system of the embodiment. [Figure 3] Figure 3 shows an example of the flow of the lost item detection process performed by the server device. [Figure 4] Figure 4 shows an example of the data structure of image data stored by a server device. [Figure 5] Figure 5 shows an example of the data structure of person data stored by a server device. [Figure 6] Figure 6 shows an example of the data structure of object data stored by a server device. [Figure 7] Figure 7 shows an example of the data structure of lost and found data stored by a server device. [Figure 8] Figure 8 is a functional block diagram showing an example of the functional configuration of the server device included in the lost item detection system of the embodiment. [Figure 9] Figure 9 is a flowchart showing an example of the lost item detection process performed by the server device of the embodiment. [Figure 10] Figure 10 is a flowchart showing an example of the lost item return process performed by the server device of the embodiment. [Modes for carrying out the invention]

[0007] An embodiment in which the information processing device of the present invention is applied to a lost item detection system 10 will be described with reference to the drawings. The lost item detection system 10 is installed, for example, inside a store and detects that a customer has lost an item inside the store based on image data that monitors the store's interior.

[0008] (Outline configuration of the lost item detection system) The schematic configuration of the lost item detection system 10 will be explained using Figure 1. Figure 1 is a block diagram showing an example of the schematic configuration of the lost item detection system according to the embodiment.

[0009] The lost item detection system 10 comprises a server device 12, a camera 14, and a mobile terminal 16.

[0010] The server device 12 receives surveillance images I(t) (see Figure 3) captured by the camera 14 in chronological order. The server device 12 then performs image processing on the received surveillance images I(t) to detect the lost item and the person who lost it. The specific detection method will be described in detail later (see Figure 3). Note that the server device 12 is an example of an information processing device in this disclosure.

[0011] Camera 14 is installed in a store, for example, at least one unit, to capture images of the store interior in chronological order. Ideally, multiple cameras 14 should be installed to capture images of the store interior without blind spots. The installation location of the cameras 14 is not limited to inside the store; they may also be installed outside the store. Camera 14 is an example of an imaging device in this disclosure. Camera 14 and server device 12 are connected by a LAN (Local Area Network) 13 installed inside the store, and images captured by camera 14 are transmitted to server device 12. Camera 14 and server device 12 may be connected wirelessly.

[0012] The mobile terminal 16 is held by a store employee and receives notification information when the server device 12 detects a lost item. The mobile terminal 16 also notifies the employee that it has received the notification information. The mobile terminal 16 may be, for example, a smartphone or a tablet.

[0013] (Server hardware configuration) The hardware configuration of the server device 12 will be explained using Figure 2. Figure 2 is a hardware block diagram showing an example of the hardware configuration of the server device included in the lost item detection system of this embodiment.

[0014] The server device 12 includes a control unit 21 for controlling various parts of the server device 12. The control unit 21 includes a CPU (Central Processing Unit) 22, a ROM (Read Only Memory) 23, and a RAM (Random Access Memory) 24. The CPU 22 connects to the ROM 23 and the RAM 24 via an internal bus 41, including an address bus and a data bus. The CPU 22 loads various programs stored in the ROM 23 and the memory unit 25 into the RAM 24. The CPU 22 controls the server device 12 by operating according to the various programs loaded into the RAM 24. In other words, the control unit 21 has the configuration of a typical computer.

[0015] The control unit 21 connects to the storage unit 25, the display device 42, the operation device 43, the camera controller 44, and the communication interface 45 via the internal bus 41.

[0016] The storage unit 25 is a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). Alternatively, the storage unit 25 may be a non-volatile memory such as flash memory that retains stored information even when the power is turned off. The storage unit 25 stores the control program 26, image data 27, person data 28, object data 29, and lost item data 30.

[0017] The control program 26 is a program that controls the overall operation of the server device 12.

[0018] Note that the control program 26 may be provided by being pre - incorporated in the ROM 23. Also, the control program 26 may be a file in an installable or executable format for the control unit 21, and may be configured to be recorded and provided on a computer - readable recording medium such as a CD - ROM, a flexible disk (FD), a CD - R, a DVD (Digital Versatile Disc), etc. Further, the control program 26 may be configured to be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the control program 26 may be configured to be provided or distributed via a network such as the Internet.

[0019] The image data 27 stores the surveillance image I(t) (see Figure 3) captured by the camera 14. Details of the data structure of the image data 27 will be described later in detail (see Figure 4).

[0020] The person data 28 stores a person image P(t) (see Figure 3) indicating a person detected from the surveillance image I(t). Details of the data structure of the person data 28 will be described later in detail (see Figure 5).

[0021] The object data 29 stores an object image O(t) (see Figure 3) indicating an object that has separated from the person image P(t). Details of the data structure of the object data 29 will be described later in detail (see Figure 6).

[0022] The lost - item data 30 stores information related to lost items detected by the server device 12. Details of the data structure of the lost - item data 30 will be described later in detail (see Figure 7).

[0023] The display device 42 is an output device that displays the image information and text information generated by the server device 12. The display device 42 is, for example, a liquid - crystal monitor or an organic EL monitor, etc.

[0024] The operating device 43 is an input device used by the operator of the server device 12 to input various operation instructions to the server device 12. The operating device 43 may be, for example, a touch panel or a keyboard.

[0025] The camera controller 44 is an interface device for the server device 12 to acquire the surveillance image I(t) captured by the camera 14.

[0026] The communication interface 45 is an interface device that controls communication between the server device 12 and the mobile terminal 16.

[0027] (Flow of the lost item detection process) Figure 3 illustrates the flow of the lost item detection process performed by the server device 12. Figure 3 is a diagram showing an example of the flow of the lost item detection process performed by the server device.

[0028] To simplify the explanation, let's take the example of a situation where one camera 14 is monitoring people inside the store. In this case, let's assume that the server device 12 has acquired four monitoring images I(ta), I(ta+Δt), I(ta+2Δt), and I(ta+3Δt) from camera 14, as shown in Figure 3.

[0029] The server device 12 performs a person detection process to detect people from each of the surveillance images. The person detection process can be performed using a known deep learning-based skeleton detection method. Specifically, for example, a technique called pose estimation, which detects the skeleton data of a person, can be utilized. Through the person detection process, a person is detected from a series of surveillance images I(t), and their position is determined. The position of the person is represented by the coordinates Pa(t) of the upper left and Pb(t) of the lower right of the rectangular region that contains the person or skeleton. Through this person detection process, the person images P(ta), P(ta+Δt), P(ta+2Δt), and P(ta+3Δt) shown in Figure 3 are obtained.

[0030] Furthermore, the server device 12 determines whether the person detected in the monitoring image I(t) is the same person and performs person tracking processing to track the location of that person. Person tracking processing can be achieved, for example, by performing image classification using deep learning. Specifically, at least one feature of the person is extracted by using, for example, a convolutional layer of a CNN (Convolutional Neural Network) as a feature extractor. Then, by comparing the feature quantities extracted from different images using the nearest neighbor method or the like, it is possible to determine whether or not they are the same person.

[0031] Next, the server device 12 performs object detection processing to detect whether there are any objects separating from the tracked person. Object detection processing can be performed using known deep learning-based motion recognition. Specifically, the server device 12 generates a network by machine learning video footage of a person dropping an item and video footage of a person performing other actions. By inputting video footage of the same person being tracked into the network thus generated, it is possible to recognize when an object has separated from the person. Such object detection processing can be implemented, for example, using Slow Fast, one of the action detection methods. Through object detection processing, an object is detected from a series of person images P(t), and its position is determined. The position of the object is represented by the coordinates Oa(t) at the top left and Ob(t) at the bottom right of the rectangular region encompassing the object. Through such object detection processing, the object images O(ta+2Δt) and O(ta+3Δt) shown in Figure 3 are obtained. Furthermore, since an object that has separated from a person generally does not change its position over time, the coordinates Oa(ta+2Δt) and Ob(ta+2Δt) in the object image O(ta+2Δt) are equal to the coordinates Oa(ta+3Δt) and Ob(ta+3Δt) in the object image O(ta+3Δt).

[0032] Furthermore, in stores, it is common to see customers return items they have picked up to the shelves. This behavior is not the same as losing an item, so it needs to be distinguished. Therefore, it would be beneficial to incorporate machine learning into the behavior of returning items to shelves after picking them up, so that it can be recognized as different from lost items.

[0033] Furthermore, the server device 12 calculates the distance between the detected object and the person from which the object departed. The distance between the object and the person is, for example, the distance d(t) between the rectangular region encompassing the object and the rectangular region encompassing the person, as shown in Figure 3. In the example in Figure 3, the distance d(t) at time t=ta+3Δt is greater than the distance d(t) at time t=ta+2Δt.

[0034] The server device 12 determines that an object is a lost item if the distance d(t) calculated in this way is greater than or equal to a threshold for a predetermined period of time.

[0035] (Data structure of image data) The data structure of image data 27 will be explained using Figure 4. Figure 4 is a diagram showing an example of the data structure of image data stored by a server device.

[0036] As shown in Figure 4, the image data 27 stores a camera ID that uniquely identifies the camera 14, and associates the surveillance image I(t) taken by the camera 14 with that camera ID with the associated information.

[0037] The supplementary information includes the installation location of camera 14 with the corresponding camera ID, the observation direction of camera 14, the field of view, the date of capture of the surveillance image I(t), the time of capture, and the frame number.

[0038] (Data structure of person data) Figure 5 illustrates the data structure of the person data 28. Figure 5 shows an example of the data structure of person data stored by the server device.

[0039] Person data 28 stores the person ID that identifies the person detected by the process described in Figure 3, the person image P(t) that shows the detection result of the person, the coordinates Pa(t) and Pb(t) that represent the person's position in the person image P(t), and associated information.

[0040] The supplementary information includes the camera ID that captured the surveillance image I(t) in which the person image P(t) was detected, the date the surveillance image I(t) was captured, the time of capture, and the frame number.

[0041] (Data structure of object data) The data structure of object data 29 will be explained using Figure 6. Figure 6 is a diagram showing an example of the data structure of object data stored by the server device.

[0042] The object data 29 stores the object ID that identifies the object detected by the process described in Figure 3, the object image O(t) showing the detection result of the object, the object positions Oa(t) and Ob(t) in the object image O(t), and associated information.

[0043] The accompanying information includes a person ID indicating the source of the object's detachment, the camera ID that captured the surveillance image I(t) in which the detached object was detected from the person with that person ID, the date the surveillance image I(t) in which the detached object was detected was taken, the time it was taken, and the frame number.

[0044] (Data structure of lost and found data) Figure 7 illustrates the data structure of the lost and found data 30. Figure 7 shows an example of the data structure of lost and found data stored by the server device.

[0045] The lost item data 30 stores, in association with the following: a lost item ID that identifies the lost item detected by the process described in Figure 3; an object image O(t) showing the detection result of the object determined to be lost; the object positions Oa(t) and Ob(t) in the object image O(t); a person image P(t) showing the detection result of the person determined to have lost the item; the coordinates Pa(t) and Pb(t) representing the person's position in the person image P(t); the distance d(t) between the object and the person; and any accompanying information.

[0046] The supplementary information includes a person ID that identifies the person who lost the item, an object ID that identifies the object corresponding to the lost item, a camera ID that identifies the camera 14 that captured the surveillance image I(t) in which the person with the person ID who lost the item was detected, and the date, time, and frame number of the surveillance image I(t) that is used to determine that the item is lost.

[0047] As described above, the image data 27, person data 28, object data 29, and lost item data 30 are all linked to each other via the camera ID, person ID, and object ID. Therefore, the surveillance image I(t) in which a person was detected, the surveillance image I(t) in which an object was detected, and the surveillance image I(t) in which a lost item was determined to be present can be easily referenced.

[0048] (Server device functional configuration) The functional configuration of the server device 12 will be explained using Figure 8. Figure 8 is a functional block diagram showing an example of the functional configuration of the server device included in the lost item detection system of this embodiment.

[0049] The control unit 21 of the server device 12 loads the control program 26 into the RAM 24 and operates it, thereby realizing the following functional units as shown in Figure 8: the image acquisition unit 51, the person detection unit 52, the object detection unit 53, the distance calculation unit 54, the lost item determination unit 55, the memory control unit 56, the notification control unit 57, the image comparison unit 58, the display control unit 59, the operation control unit 60, and the communication control unit 61. Some or all of these functions may be realized by dedicated hardware.

[0050] The image acquisition unit 51 acquires surveillance images I(t) captured by the camera 14 (imaging device) installed in the store.

[0051] The person detection unit 52 detects a person from the surveillance image I(t) acquired by the image acquisition unit 51. The person detection unit 52 also tracks the same person as the one previously detected in surveillance images taken at a different time than the surveillance image I(t).

[0052] The object detection unit 53 detects objects that have separated from the person detected by the person detection unit 52. The object detection unit 53 is an example of an anomaly detection unit in this disclosure.

[0053] The distance calculation unit 54 calculates the distance d(t) between the person detected by the person detection unit 52 and the object detected by the object detection unit 53.

[0054] The lost item determination unit 55 determines that an object detected by the object detection unit 53 is a lost item if the distance d(t) calculated by the distance calculation unit 54 is greater than or equal to a threshold for a predetermined period of time or longer.

[0055] The memory control unit 56 associates the image representing the person detected by the person detection unit 52, the image representing the object that has separated from the person, and the position of the object, and stores them in the memory unit 25 (storage device).

[0056] The notification control unit 57 issues a notification on the condition that the lost item determination unit 55 determines that a lost item exists. More specifically, the notification control unit 57 transmits information indicating that a lost item exists and the location of the lost item to the mobile terminal 16, on the condition that the lost item determination unit 55 determines that a lost item exists. When the mobile terminal 16 receives information regarding the lost item from the server device 12, it notifies the mobile terminal 16 that a lost item exists through image display, audio output, etc. The mobile terminal 16 also displays the location of the lost item. Note that the notification control unit 57 is an example of a notification unit in this disclosure.

[0057] The image comparison unit 58 determines whether the person who lost the item is the same person by comparing the image of the person who lost the item with the person image P(t) related to the lost item when the owner of the item appears.

[0058] The display control unit 59 generates display information, such as image data, to be displayed on the display device 42 connected to the server device 12. The display control unit 59 also displays the generated display information on the display device 42.

[0059] The operation control unit 60 acquires operator operation information for the operation device 43 connected to the server device 12. The operation control unit 60 also passes the acquired operation information to the control unit 21.

[0060] The communication control unit 61 controls communication between the server device 12 and the mobile terminal 16.

[0061] (Flow of the lost item detection process performed by the server device) Figure 9 illustrates the flow of the lost item detection process performed by the server device 12. Figure 9 is a flowchart showing an example of the lost item detection process performed by the server device of this embodiment.

[0062] The image acquisition unit 51 acquires a surveillance image I(t) from the camera 14 (step S11).

[0063] The memory control unit 56 stores the acquired monitoring image I(t) as image data 27 in the memory unit 25 (step S12).

[0064] The person detection unit 52 performs person detection processing on the surveillance image I(t) and determines whether a person has been detected (step S13). If it is determined that a person has been detected (step S13: Yes), the process proceeds to step S14. On the other hand, if it is not determined that a person has been detected (step S13: No), the process returns to step S11.

[0065] In step S13, if it is determined that a person has been detected, the person detection unit 52 identifies the person's position (coordinates Pa(t), Pb(t)) (step S14).

[0066] The memory control unit 56 causes the person image P(t), which includes the person detection result and location, to be stored in the memory unit 25 as person data 28 (step S15).

[0067] The object detection unit 53 performs object detection processing to detect the separation of an object from the detected person and determines whether the separation of the object has been detected (step S16). If it is determined that the separation of the object has been detected (step S16: Yes), the process proceeds to step S17. On the other hand, if it is not determined that the separation of the object has been detected (step S16: No), the process returns to step S11.

[0068] In step S16, if it is determined that the object has left the system, the memory control unit 56 causes the memory unit 25 to store the object image O(t), which includes the object detection result and position, as object data 29 (step S17).

[0069] The image acquisition unit 51 acquires a surveillance image I(t) from the camera 14 (step S18).

[0070] The person detection unit 52 tracks the person that was detected first from the latest surveillance image I(t) (step S19).

[0071] The distance calculation unit 54 calculates the distance d(t) between the person and the object that has moved away from the person (step S20).

[0072] The lost item detection unit 55 determines whether the distance d(t) is greater than or equal to a threshold for a predetermined period of time or longer (step S21). If it is determined that the distance d(t) is greater than or equal to a threshold for a predetermined period of time or longer (step S21: Yes), the process proceeds to step S22. On the other hand, if it is not determined that the distance d(t) is greater than or equal to a threshold for a predetermined period of time or longer (step S21: No), the process proceeds to step S24.

[0073] In step S21, if it is determined that the distance d(t) is above a threshold for a predetermined period of time or longer, the lost item determination unit 55 determines that the object under consideration is a lost item belonging to the person from whom the object was released. The memory control unit 56 then stores the object image O(t), which includes the detection result and position of the object under consideration, and the person image P(t), which includes the detection result and position of the person from whom the object was released, in the memory unit 25 as lost item data 30 (step S22).

[0074] The notification control unit 57 notifies the mobile terminal 16 that a lost item has been found (step S23). After that, the server device 12 completes the process shown in Figure 9.

[0075] On the other hand, in step S21, if it is determined that the distance d(t) is not above a threshold for a predetermined time or longer, the image acquisition unit 51 acquires a monitoring image I(t) from the camera 14 (step S24).

[0076] The person detection unit 52 determines whether it can track the same person in the surveillance image I(t) (step S25). If it is determined that the same person can be tracked (step S25: Yes), the process returns to step S20. On the other hand, if it is not determined that the same person can be tracked (step S25: No), the server device 12 terminates the process shown in Figure 9.

[0077] (The process of returning lost items performed by the server device) Figure 10 illustrates the flow of the lost item return process performed by the server device 12. Figure 10 is a flowchart showing an example of the lost item return process performed by the server device of this embodiment.

[0078] The image acquisition unit 51 acquires an image of the declarant (step S31).

[0079] The image comparison unit 58 determines whether the person reporting the loss and the owner of the lost item are the same person (step S32). The operator of the server device 12 identifies the lost item based on the reporter's report. Then, from the lost item data 30 related to the identified lost item, the operator obtains the image P(t) of the person who owns the lost item. The image comparison unit 58 then compares the obtained image P(t) with the image of the reporter obtained in step S31. If it is determined that the reporter and the owner of the lost item are the same person (step S32: Yes), the process proceeds to step S33. On the other hand, if it is not determined that the reporter and the owner of the lost item are the same person (step S32: No), the server device 12 terminates the process shown in Figure 10.

[0080] In step S32, if it is determined that the person reporting the loss and the owner of the lost item are the same person, the operation control unit 60 determines whether information indicating that the return of the lost item has been completed has been entered (step S33). If it is determined that information indicating that the return of the lost item has been completed has been entered (step S33: Yes), the process proceeds to step S34. On the other hand, if it is not determined that information indicating that the return of the lost item has been completed has been entered (step S33: No), the process repeats step S33.

[0081] In step S33, if it is determined that information indicating the return of the lost item has been entered, the memory control unit 56 deletes the data related to the returned lost item from the lost item data 30 (step S34). At this time, the memory control unit 56 may also delete the data related to the returned lost item and the owner of the lost item from the image data 27, person data 28, and object data 29. After that, the server device 12 terminates the process shown in Figure 10.

[0082] (Effects of the embodiment) As described above, the server device 12 (information processing device) of this embodiment includes an image acquisition unit 51 that acquires a surveillance image I(t) captured by a camera 14 (imaging device), a person detection unit 52 that detects a person from the image acquired by the image acquisition unit 51, an object detection unit 53 (anomaly detection unit) that detects an object that has separated from the person detected by the person detection unit 52, a distance calculation unit 54 that calculates the distance d(t) between the person detected by the person detection unit 52 and the object detected by the object detection unit 53, and a lost item determination unit 55 that determines that an object is lost if the distance d(t) calculated by the distance calculation unit 54 is above a threshold for a predetermined time or longer.Therefore, it is possible to detect that an item has been lost without attaching an identification means such as an IC tag to the item.

[0083] Furthermore, the server device 12 (information processing device) of this embodiment further includes a storage control unit 56 that associates a person image P(t) representing a person detected by the person detection unit 52, an object image O(t) representing an object that has separated from the person, and the positions Oa(t) and Ob(t) of the object, and stores them in the storage unit 25 (storage device). Therefore, it is possible to easily and reliably determine whether an object that has separated from a person is a lost item.

[0084] Furthermore, the server device 12 (information processing device) of this embodiment further includes a notification control unit 57 (notification unit) that issues a notification on the condition that the lost item determination unit 55 determines that a lost item has been found. Therefore, it is possible to immediately notify that a lost item has been detected.

[0085] Furthermore, in the server device 12 (information processing device) of this embodiment, the memory control unit 56 deletes information related to lost items from the memory unit 25 (storage device) on the condition that information indicating that the lost item has been returned to its owner has been input. Therefore, the contents of the storage device can be managed without any effort.

[0086] Furthermore, the server device 12 (information processing device) of this embodiment is further equipped with an image comparison unit 58 that compares an image representing a person detected by the person detection unit 52 with an image of the person who declared themselves to be the owner of the lost item. The lost item determination unit 55 determines that the person who declared themselves to be the owner is the owner on the condition that the images of the people compared by the image comparison unit 58 match. Therefore, it is possible to easily and reliably determine whether the person who declared themselves to be the owner is the correct owner.

[0087] Although embodiments of the present invention have been described above, these embodiments are illustrative and are not intended to limit the scope of the invention. This novel embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of symbols]

[0088] 10. Lost and Found Detection System 12. Server device (information processing device) 13 LAN 14. Camera (imaging device) 16 Mobile devices 21 Control Unit 25 Storage unit (storage device) 27 Image data 28. Person Data 29 Object Data 30 Lost and Found Data 51 Image acquisition unit 52 Person Detection Unit 53 Object detection unit (anomaly detection unit) 54 Distance Calculation Unit 55 Lost Item Detection Unit 56 Memory Control Unit 57. Notification Control Unit (Notification Department) 58 Image Comparison Section 59 Display Control Unit 60 Operation Control Unit 61 Communication Control Unit d(t) Distance I(t) surveillance image O(t) object portrait P(t) Portrait [Advanced Technology Documents] [Chartered documents]

[0089]

Patent Document 1

Claims

1. An image acquisition unit that acquires images captured by the imaging device, The image acquisition unit detects a person from the images acquired by the aforementioned image acquisition unit, An anomaly detection unit that detects an object that has separated from the person detected by the person detection unit, A distance calculation unit calculates the distance between the person detected by the person detection unit and the object detected by the anomaly detection unit. A lost item determination unit determines that an object detected by the anomaly detection unit is a lost item if it is not an object that the person had picked up before that time, and the distance calculated by the distance calculation unit is above a threshold for a predetermined period of time or longer. An information processing device equipped with the following features.

2. The system further includes a storage control unit that associates an image representing a person detected by the person detection unit with an image representing an object that has separated from the person, and the position of the object, and stores these associated images in a storage device. The information processing apparatus according to claim 1.

3. The system further includes a notification unit that issues a notification on the condition that the aforementioned lost item detection unit determines that a lost item exists. The information processing apparatus according to claim 1 or claim 2.

4. The memory control unit, On the condition that information indicating that the lost item has been returned to its owner is entered, the information relating to the lost item is deleted from the storage device. The information processing apparatus according to claim 2.

5. The system further includes an image comparison unit that compares an image representing a person detected by the person detection unit with an image of the person who declared themselves to be the owner of the lost item. The lost item determination unit determines that the person who made the claim is the owner of the lost item, on the condition that the images of the person compared by the image comparison unit match. The information processing apparatus according to claim 1 or claim 2.

6. Computers, An image acquisition unit that acquires images captured by the imaging device, The image acquisition unit detects a person from the images acquired by the aforementioned image acquisition unit, An anomaly detection unit that detects an object that has separated from the person detected by the person detection unit, A distance calculation unit calculates the distance between the person detected by the person detection unit and the object detected by the anomaly detection unit. A lost item determination unit determines that an object detected by the anomaly detection unit is a lost item if it is not an object that the person had picked up before that time, and the distance calculated by the distance calculation unit is above a threshold for a predetermined period of time or longer. A program that makes something work.

Images