Information processing device, terminal device, information processing system, information processing method, and computer-readable recording medium

Abstract

An information processing device according to the present invention comprises a tracking unit, a classification unit, and an output control unit. The tracking unit tracks each among a plurality of detected people. The classification unit classifies each among the plurality of detected people. The output control unit performs, on the basis of tracked positions obtained by using the tracking means and the classification result obtained by using the classification means, notification control using a notification pattern for the plurality of detected people.

Description

Information processing device, terminal device, information processing system, information processing method, and computer-readable recording medium

[0001] The present invention relates to an information processing device, a terminal device, an information processing system, an information processing method, and a computer-readable recording medium.

[0002] When entering a venue such as a movie theater or concert hall, an attendant at the entrance usually checks the ticket or the visitor holds the barcode of the purchased ticket up to a scanner to confirm whether or not the person is allowed to enter. Since movie theaters, concert halls, and other venues tend to attract large numbers of people, a smoother entry process is desirable.

[0003] Patent Document 1 discloses a technology that detects a person within a monitored area as a detected person, identifies the person as an authenticated person using an authentication image obtained by photographing the monitored area with an authentication camera, and generates correspondence data that corresponds to the authenticated person.

[0004] Japanese Patent Application Laid-Open No. 2022-154335

[0005] Although the technology disclosed in Cited Document 1 is capable of authenticating a person who is moving, it is not capable of issuing a notification for guiding the person according to the classification result of whether the person is authenticated or not. Therefore, it is not possible to realize a notification for controlling smoother movement of people during the entrance procedures at places where many people gather.

[0006] An object of the present disclosure is to provide an information processing device, a terminal device, an information processing system, an information processing method, and a computer-readable recording medium that solves the above-mentioned problems.

[0007] An information processing device according to one aspect of the present disclosure includes a tracking means for tracking each of a plurality of detected persons, a classification means for classifying each of the detected persons, and an output control means for controlling notifications using a notification pattern for the detected persons based on the tracking position by the tracking means and the classification result by the classification means.

[0008] A terminal device according to one aspect of the present disclosure includes a photographing device, a display device, and a display processing means that detects a light notification pattern irradiated onto a person detected by an information processing device from an image acquired from the photographing device based on a classification result of the person, and controls the display device to display an image in which the person irradiated with the detected light notification pattern is highlighted based on the detected light notification pattern.

[0009] An information processing system according to an aspect of the present disclosure includes an information processing device according to an aspect of the present disclosure and the terminal device according to the above-described aspect of the present disclosure.

[0010] An information processing method according to one aspect of the present disclosure includes, by a computer, tracking each of a plurality of detected persons, classifying each of the detected persons, and controlling notifications using a notification pattern for the detected persons based on the tracking position obtained by the tracking and the classification result obtained by the classification.

[0011] A computer-readable recording medium according to one aspect of the present disclosure records a program that causes a computer to track each of a plurality of detected persons, classify each of the detected persons, and control notifications using notification patterns for the detected persons based on the tracking positions obtained by the tracking and the classification results obtained by the classification.

[0012] 1 is a diagram illustrating a functional configuration of an information processing system according to an embodiment of the present disclosure. FIG. 1 is a diagram illustrating an example of the configuration of notification pattern information. FIG. 2 is a flowchart illustrating the operation of an information processing device. FIG. 3 is a flowchart illustrating the operation of a terminal device. FIG. 4 is a diagram illustrating an example of a notification based on a notification pattern by an information processing device according to a first embodiment. FIG. 5 is a diagram illustrating an example of a notification based on another notification pattern by an information processing device according to the first embodiment. FIG. 6 is a diagram illustrating an example of a notification based on a notification pattern by an information processing device according to a second embodiment. FIG. 7 is a diagram illustrating an example of a notification based on a notification pattern by an information processing device according to a third embodiment. FIG. 8 is a diagram illustrating an example of a notification based on a notification pattern by an information processing device according to a fourth embodiment. FIG. 9 is a diagram illustrating an example of a notification based on a notification pattern by an information processing device according to a fifth embodiment. FIG. 10 is a diagram illustrating an example of a notification based on a notification pattern by an information processing device according to an eighth embodiment. FIG. 11 is a diagram illustrating an example of a notification based on a notification pattern by an information processing device according to a ninth embodiment. FIG. 12 is a diagram illustrating an example of a notification based on a notification pattern by an information processing device according to a tenth embodiment. FIG. 13 is a diagram illustrating an example of a notification based on a notification pattern by an information processing device according to an eleventh embodiment. FIG. 14 is a diagram illustrating an example of a notification based on a notification pattern by an information processing device according to a twelfth embodiment. FIG. 15 is a diagram illustrating an example of a configuration of an information processing device according to an embodiment of the present disclosure. 1 is a diagram illustrating an example of a configuration of a terminal device according to an embodiment of the present disclosure. 2 is a block diagram illustrating an example of a hardware configuration of an information processing device or a terminal device.

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all drawings, the same or corresponding components are designated by the same reference numerals, and common descriptions will be omitted.

[0014] Fig. 1 is a diagram showing a functional configuration of an information processing system according to an embodiment of the present disclosure. As shown in Fig. 1, the information processing system includes an information processing device 10. The information processing system may further include a terminal device 20. The information processing system also includes an image capturing device 30 and an output device 40. Here, the terminal device 20, the image capturing device 30, and the output device 40 are connected to the information processing device 10 via a network 50. Alternatively, the terminal device 20, the image capturing device 30, and the output device 40 are connected to the information processing device 10 via dedicated cables or via wireless communication.

[0015] The information processing device 10 detects people from video (images) captured by the imaging device 30, tracks each of the detected people, classifies each person, and controls the output device 40 to output notifications to each person or some of the people depending on the classification results.

[0016] Based on the output from the output device 40 and / or upon receiving a notification from the information processing device 10, the terminal device 20 performs output processing so that the user of the terminal device 20 can easily identify / specify the person who is the target of the notification.

[0017] The image capturing device 30 is installed to capture an image of the monitoring area, and the captured image is input to the information processing device 10 .

[0018] The output device 40 is also installed within the monitoring area of ​​the image capturing device 30 or in the vicinity of the monitoring area, and outputs a notification to a target person under the control of the information processing device 10.

[0019] The information processing device 10 includes a tracking unit 11, a classification unit 12, an output control unit 13, and a storage unit 14. The tracking unit 11 has a function of tracking each of multiple people detected in video from the imaging device 30 within the video. The classification unit 12 has a function of classifying each of multiple people detected by the tracking unit 11. When classifying each person, the classification unit 12 performs the classification based on the results of facial authentication of the detected person. For this purpose, the classification unit 12 includes an authentication unit 15. When a person's face is confirmed while the tracking unit 11 is tracking the person, the authentication unit 15 performs face authentication processing using authentication information related to the person's face. The output control unit 13 controls output based on a notification pattern for notifications to the person from the output device 40, depending on the classification result by the classification unit 12. The storage unit 14 stores information including notification pattern information 14a and authentication information 14b. The notification pattern information 14a is information that associates the notification pattern to be output from the output device 40 with the person classification result. The authentication information 14b includes features for face authentication registered in advance for face authentication of a person detected by the authentication unit 15 of the classification unit 12, information about the person registered for face authentication, etc. Note that the authentication information 14b may be registered as an "Allowed List" which is a list of people who are permitted by authentication, or as a "Denied List" which is a list of people who are not permitted by authentication.

[0020] The terminal device 20 includes a display processing unit 21, a notification acquisition unit 22, a storage unit 23, an imaging device 24, a display device 25, and a speaker 26. The display processing unit 21 performs processing to display on the display device 25 an image obtained from the imaging device 24 and / or a notification or information received by the notification acquisition unit 22 from the information processing device 10 in a manner that makes it easy to identify the person receiving the notification. The display processing unit 21 also performs processing to output sound or audio from the speaker 26 in a manner that makes it easy to identify the person who is the target of the notification, based on the notification or information received by the notification acquisition unit 22 from the information processing device 10. The notification acquisition unit 22 receives information regarding a notification to a person who is the target of the notification from the information processing device 10. The notification acquisition unit 22 also receives from the information processing device 10 a display method or the like for displaying the notification in a manner that makes it easy to identify the person who is the target of the notification, depending on the notification pattern, and stores the information in the storage unit 23 as notification information 23a. The terminal device 20 may be a personal computer, or may be a pair of virtual eyeglasses or a head-mounted display that overlays a virtual image on a real image acquired by the image capturing device 24 and outputs the image to the display device 25. Alternatively, the terminal device 20 may be an earphone-type device. Depending on the output device 40 to which the information processing device 10 outputs, some of the notification acquisition unit 22, the storage unit 23, the image capturing device 24, the display device 25, and the speaker 26 may not be required.

[0021] The image capturing device 30 is installed to capture an area where people move, for example, a predetermined area near the entrance of a venue such as a movie theater or concert hall. The image capturing device 30 is installed so that it can capture the direction in which a person is approaching, and is installed at a height that allows the person's face to be recognized and overlooks the person from diagonally above, taking into account the person's height, so that the person's head and face can be captured. This is to enable efficient processing by the tracking unit 11 and authentication unit 15 of the information processing device 10. Details of the processing of the information processing device 10, including the tracking unit 11 and authentication unit 15, will be described in detail separately.

[0022] The information processing device 10 uses one or more of the output devices 40, such as a display device 40a, a projector 40b, a light 40c, a speaker 40d, a floor speaker 40e, a speaker-equipped ornament 40f, a floor LED 40g, and a floor display 40h.

[0023] FIG. 2 is a diagram showing an example of the configuration of the notification pattern information 14a. In FIG. 2, the notification pattern information 14a includes items such as classification items, classification results, and notification patterns. Here, the item "classification items" is data indicating the items that are conditions for classifying people and the contents of those items. The item "classification results" indicates the type of classification performed by the classification unit 12. The item "classification results" is the result of classification based on the item "classification items." The item "notification patterns" is data indicating the type of notification to be performed using the type of output device for the corresponding classification results. The notification pattern information 14a may also include an item "highlighting." The item "highlighting" is information regarding the display method, etc., used to display a person on the terminal device 20 so that the person can be easily identified based on the corresponding notification pattern.

[0024] Next, a description will be given of the operations of the information processing device 10 and the terminal device 20. FIG.

[0025] The tracking unit 11 of the information processing device 10 constantly analyzes the video input from the image capture device 30 and determines whether a new person has been detected (S10). Here, the detection of a new person is performed by detecting the head of a new person who is not currently being tracked in the video. This is because heads tend to be relatively unlikely to overlap in camera images taken from a position looking down on a person diagonally from above. Therefore, as described above, the image capture device 30 is installed at a height that looks down on the person diagonally from above, taking into account the person's height, so that the person's head can be captured. Head detection is performed by using learning results from machine learning that can detect heads. In order to track the person detected in step S10 within the video, the tracking unit 11 associates and manages an ID that uniquely identifies the person and feature quantities related to the detected head.

[0026] When a new person is detected (S10, Yes), the tracking unit 11 tracks the detected person (S11). The tracking unit 11 may track the newly detected person in the video in step S10 by tracking the head movement position of the detected person in the video and also by matching the detected person's clothing. In crowded areas, a part or all of a person to be tracked may be hidden by other people or obstructions. Therefore, when the person next appears in the video, the tracking accuracy may be improved by adopting a technology that matches clothing in addition to head matching. Note that the clothing of the tracked person may be identified using known techniques, such as learning results from machine learning that can identify a person associated with a detected head or learning results that can identify clothing from a person. For this purpose, the tracking unit 11 may also associate and manage the feature values ​​of the identified clothing with the ID of the target person. Furthermore, when the tracking unit 11 is unable to identify a person being tracked from the video and then identifies the person again, it determines whether the person matches the managed head and clothing features in addition to the person's position in the video. If the tracking unit 11 determines that a person who appeared in the video is a person whose tracking has been lost, it reassociates the ID it manages with the determined person and resumes tracking the person. Note that the tracking unit 11 may track a person not based on video, but by receiving location information from a smartphone carried by the target person or a signal from a transmitter worn by the target person. Furthermore, the tracking unit 11 may track a person by various methods, such as receiving a signal from a beacon installed in a hallway with a receiver worn by the target person. Furthermore, the tracking unit 11 may track a person by a combination of these methods.

[0027] The tracking unit 11 sequentially acquires video from the image capture device 30, tracks each person, and determines whether the face of the person being tracked is included in the video (S12). For this reason, as described above, the image capture device 30 should be installed in a position that allows the direction in which the person is moving to be captured from the front. The tracking unit 11 determines whether the face of the person being tracked is included in the video using known techniques, such as learning results from machine learning that can identify human faces. If the person's face is not included in the video (S12, No), the tracking unit 11 returns to the processing of step S11 to continue tracking the person and determining whether the person's face is included. If it is determined that the face of the person detected in step S10 is included in the video (S12, Yes), the tracking unit 11 provides an image of the identified face to the classification unit 12 along with an ID for managing the person.

[0028] The classification unit 12, which has received the image of the identified face, performs face authentication on the identified face using the authentication unit 15 (S13). The authentication unit 15 performs face authentication on the identified face using pre-registered authentication information 14b included in the storage unit 14. The authentication unit 15 also notifies the classification unit 12 of the authentication result, and the classification unit 12 associates the authentication result with the ID of the authenticated person.

[0029] The classification unit 12 classifies the target person based on the authentication result, etc. (S14). For example, in Fig. 2, in the item "authentication item," the value "α" is "authentication OK" and the value "β" is "authentication NG," and the corresponding items "classification item" are classification "A" and classification "B," respectively. Furthermore, if the tracking unit 11 is unable to recognize the person's face while tracking the person, that is, if face-based authentication processing is unable to be performed, this may also be added as a "classification item" so that it is classified differently from the authentication result.

[0030] In addition, at concert venues and the like, in addition to the authentication result, further classification items may be provided based on whether or not the person is carrying a large bag that violates regulations. In this case, the classification unit 12 obtains an image of the entire target person from the tracking unit 11, identifies the person's belongings from the image, and determines whether or not the person is carrying a large bag that violates regulations. The determination of whether or not the person is carrying a large bag is performed using known techniques such as image recognition technology and the use of learning results from machine learning. As another example, if the authentication information 14b is associated with information about the person's ticket, further classification items may be provided using the ticket information, such as S seat, A seat, B seat, etc.

[0031] When the classification unit 12 has classified the target person, the output control unit 13 uses the classification result and the current location of the target person to perform output control based on the notification pattern for the output device 40 and / or the terminal device 20 (S15). Note that the output control based on the notification pattern by the output control unit 13 will be described in detail separately.

[0032] The tracking unit 11 determines whether it has become impossible to track the target person in the video (S16). This is determined based on whether the target person has moved and disappeared from the video while being tracked. If it is determined that tracking is impossible (S16, Yes), the tracking unit 11 ends tracking of the target person (S17). On the other hand, if it is determined that tracking is not impossible (S16, No), the process proceeds to step S15 so that the tracking unit 11 continues tracking the target person and the output control unit 13 controls output based on the notification pattern for the person to be handled. Note that the tracking of the target person by the tracking unit 11 in step S15 is the same process as step S11.

[0033] 3 in parallel for all persons newly detected in step S10. In this manner, the information processing device 10 tracks and classifies all persons detected in the video from the image capturing device 30, and performs notification processing based on the classification results as needed.

[0034] Next, a description will be given of the operation of the terminal device 20. Fig. 4 is a flowchart showing the operation of the terminal device 20.

[0035] The display processing unit 21 of the terminal device 20 detects the notification pattern of the information processing device 10 (S20). The display processing unit 21 detects the notification pattern from an image captured by the imaging device 24 provided in the terminal device 20, or from information relating to the notification pattern transmitted from the information processing device 10.

[0036] The display processing unit 21 identifies the person who is the target of the notification pattern detected in step S21 (S21). The display processing unit 21 identifies the target person based on the image from the imaging device 24, the position of the person, etc.

[0037] The display processing unit 21 performs processing to highlight the person identified in step S21 on the display device 25 of the terminal device 20 (S22). The display processing unit 21 also controls the display device 25 to output the highlighted result superimposed on the image from the imaging device 24. The display processing unit 21 may determine the highlighting method based on notification information 23a stored in the storage unit 23. This notification information 23a is information that is notified in advance by the information processing device 10 to the notification acquisition unit 22 of the terminal device 20 and stored in the storage unit 23. The notification information 23a includes information associated with a highlighting method for a notification pattern. Note that, as shown in FIG. 2 , the notification information 23a is part of the notification pattern information 14a stored in the storage unit 14 of the information processing device 10.

[0038] The display processing unit 21 determines whether or not the notification pattern for the target person can no longer be detected during the process of continuously performing the process in step S21 (S23). If the notification pattern for the target person cannot be detected (Yes in S23), the highlighting of the target person is terminated (S24). If the notification pattern for the target person can be detected (No in S23), the display processing unit 21 proceeds to step S23 to detect a specific person from the image from the imaging device 24 and highlight the detected person.

[0039] The terminal device 20 performs the processes of steps S20 to S24 shown in Fig. 4 in parallel for all persons whose notification patterns have been detected. As a result, all persons whose notification patterns have been detected in the video from the image capture device 24 of the terminal device 20 are highlighted as needed. In this way, the information processing device 10 notifies multiple persons according to the classification results for each person. Furthermore, the terminal device 20 works in cooperation with the information processing device 10 to highlight persons who are the target of notification.

[0040] Depending on the notification pattern of the information processing device 10, the terminal device 20 may output a sound or voice from the speaker 26 instead of highlighting the person. Alternatively, the terminal device 20 may output a sound or voice from the speaker 26 in addition to highlighting the person. A specific example of the process shown in FIG. 4 will be described separately.

[0041] 3 by the output control unit 13 of the information processing device 10 and specific examples of processing by the terminal device 20 according to the notification pattern in the processing example will be described with reference to the drawings. Note that depending on the notification pattern by the output control unit 13, the terminal device 20 may not be used.

[0042] First Embodiment Fig. 5 is a diagram illustrating an example of a notification based on a notification pattern by the information processing device 10 in the first embodiment. As shown in Fig. 5, a display device 40a, a projector 40b, and a light 40c are used as the output device 40. The image capturing device 30 is installed on a path along which people travel to a venue or the like, so that it can capture images of people on their path from directly in front of them and diagonally above them.

[0043] The display device 40a is installed at the end position in the direction of movement of the movement path within the shooting range of the shooting device 30, or in a position near the outside of the shooting range of the shooting device 30, where the person can see the display content of the display device 40a when moving along the movement path.

[0044] The projector 40b is installed at a position where it can project within the shooting range of the shooting device 30 or in the vicinity thereof, and where, when a person moves along the moving route, the person and / or the guide can see the contents projected by the projector 40b onto the floor or wall surface.

[0045] The light 40c is installed in a position where it can irradiate a specific person from above within the shooting range of the image capture device 30 or in the vicinity thereof. The light 40c is, for example, a light using a beam laser, and is capable of irradiating light to cover the shooting range and its vicinity, and is also capable of projecting light like a spotlight onto multiple locations within the covered range. In addition, the light 40c is also capable of changing the color (wavelength) of the light when projecting it like a spotlight.

[0046] 5, three people A, B, and C are detected within the range of photography by the camera 30, and A and B are authenticated (OK), while C is not authenticated (NG). Also, in the example of Fig. 5, a guide is waiting near the edge of the range of photography by the camera 30 in the direction of travel of the people.

[0047] The output control unit 13 of the terminal device 20 issues a notification based on the notification pattern information 14a only to person C who has not been authenticated. The notification pattern here may be, for example, as follows: 1) Estimate the movement direction of person C from the tracking path of person C, and control the projector 40b to project a projection image of person C's guide path, etc. onto the floor or wall in front of person C. 2) Control the light 40c to project a spotlight of a predetermined color onto person C along the tracking path of person C. 3) Control is performed so that a predetermined guidance display, for example, "Please move along the guide path on the floor," etc. is output from the display device 40a together with the acquired facial image of person C.

[0048] In order to project a spotlight of a predetermined color onto person C, the information processing device 10 is provided with conversion data relating to the position for converting the range photographed by the photographing device 30 into the projection range of the light 40c. Using this data, the output control unit 13 controls the projection range of the spotlight by the light 40c. The output control unit 13 also controls the projection position of the projection image on the projector 40b in the same manner as it controls the illumination range of the light 40c.

[0049] When projecting a spotlight of a predetermined color onto person C, the light may be projected within a range around person C, for example, within a radius of about 1 meter from near the ceiling, so that person C can be distinguished from other people around them.

[0050] This allows person C to recognize that he or she is the target person from the content displayed on display device 40a, and enables the person who could not be authenticated by projector 40b to move along the guidance route, for example, to the location of a guide. Also, the guide can confirm the person to be guided by using a spotlight or the like from light 40c. In this case, the guide can perform face authentication on person C again, check the ticket, etc., or provide guidance for exit if person C cannot be authenticated.

[0051] In the first embodiment, the output control by the output control unit 13 can be modified as follows.

[0052] The light 40c changes the color of the light depending on whether authentication has been successful. For example, the light 40c is controlled to project blue light onto a person who has been authenticated and red light onto a person who has not been authenticated. If ticket information is registered in the authentication information 14b together with the authentication information, the display device 40a is controlled to display information for the person who has been authenticated, such as the seat number and the distance and time to the destination, when the person approaches the display device 40a. If ticket information is registered in the authentication information 14b together with the authentication information, the projector 40b is controlled to project, in the direction of movement of the person, a guide direction to the seat position as information for the person who has been authenticated. At this time, the projector 40b may also be controlled to project the distance and time to the destination as well. The output device 40 is controlled to output from one or more of the display device 40a, the projector 40b, and the light 40c.

[0053] In addition, if the output control unit 13 is unable to recognize the face contained in the image before the person reaches a specified range at the end of the shooting range of the shooting device 30 in the direction of travel of the person and therefore is unable to perform authentication processing by the authentication unit 15, the output control unit 13 may perform output control for such a person using the same notification pattern as for a person who cannot be authenticated.

[0054] The output control unit 13 may also control the light 40c so that the spotlight is projected onto the person who has failed authentication in a circular pattern surrounding the person, as shown in FIG.

[0055] As a result, each person or person who meets a predetermined condition can receive a notification when moving. Also, since the guide can distinguish between people who have been authenticated and people who have not been authenticated based on the notification pattern, it is possible to guide not only people who have not been authenticated, but also people who have been authenticated, for example, to their seats.

[0056] It should be noted that the modifications shown in the first embodiment can also be applied to the other embodiments described below.

[0057] Second Embodiment FIG. 7 is a diagram illustrating an example of a notification based on a notification pattern by the information processing device 10 in the second embodiment. The second embodiment differs from the first embodiment in that the light from the light 40c is light of a wavelength other than visible light. This is to prevent the person on whom the light is projected by the light 40c and people around him from noticing the light. For example, infrared light is used as light of a wavelength other than visible light. Alternatively, near-infrared light may be used if it is only necessary for people to be less likely to notice the light being projected. Another difference is that a terminal device 20 is used for the guide. Here, the terminal device 20 is a terminal device equipped with an imaging device 24 capable of detecting wavelengths other than visible light so that the notification pattern using light of that wavelength can be detected. Furthermore, considering the convenience of the guide when using the terminal device 20, the terminal device 20 may be a glasses-type terminal device 20. In this embodiment, the guide wears the glasses-type terminal device 20 to identify the person to be guided. Differences from the first embodiment are described below.

[0058] The output control unit 13 of the information processing device 10 controls the light 40c to project light of a wavelength other than visible light onto the person C along the tracking path of the person C whose authentication result was NG. Here, a case will be described in which infrared light is used as the light of a wavelength other than visible light.

[0059] The image capture device 24 of the terminal device 20 is an infrared detection camera capable of detecting infrared light emitted by the light 40c. The image capture device 24 is also capable of capturing visible light images. The display processing unit 21 of the terminal device 20 identifies a person illuminated by infrared light from the captured image based on known techniques such as image recognition or machine learning learning results. The display processing unit 21 then performs a process to highlight the identified person and controls the display device 25 mounted in the eyeglass lens portion of the eyeglass-type terminal device 20 to output the visible light image including the highlighting. Note that the highlighting may be, for example, a process to surround the identified person's face with a rectangle. The highlighting may be, instead of or in addition to the process of surrounding the person with a rectangle, the display of additional information superimposed on or near the identified person, such as superimposed text indicating that person C is the person to be guided.

[0060] As described above, a person who needs guidance from a guide, such as person C, cannot recognize that he or she or those around him or her are being guided, which is preferable from the perspective of protecting the privacy of person C. On the other hand, the guide can identify the person to be guided by using the terminal device 20. Furthermore, since the person to be guided can be identified by the terminal device 20 performing a detection process on the person on whom a spotlight of light other than visible light is projected, there is no need to share images or information in the information processing device 10 with the terminal device 20. As a result, even if the terminal device 20 is not connected online to the information processing device 10, the terminal device 20 can identify the person to be notified.

[0061] In addition, if the eyeglass lens portion of the eyeglass-type terminal device 20 is transparent and data can be projected onto the eyeglass lens portion, the imaging device 24 of the terminal device 20 does not need the function of capturing visible light, and the display processing unit 21 may be controlled to project a highlight onto the eyeglass lens portion.

[0062] In the present embodiment, the terminal device 20 is in the form of glasses, but is not limited to this. The terminal device 20 may be a normal personal computer equipped with a camera, and the guide may identify the person to be guided while checking the display screen of the personal computer. The terminal device 20 may also be a head-mounted display. In addition, the display processing unit 21 of the terminal device 20 may control the speaker 26 to play a sound, such as that the person to be guided has been detected, along with the highlighting.

[0063] Third Embodiment FIG. 8 is a diagram illustrating an example of a notification based on a notification pattern by the information processing device 10 in the third embodiment. The third embodiment differs from the first and second embodiments in that the light 40c illuminates the target person with light that changes over time. The "light that changes over time" may be a continuous flashlight, light whose intensity changes over time, or light whose wavelength changes over time. For example, light whose intensity changes over time may be ON / OFF modulation, such as illuminating light for only 1 millisecond at 1-second intervals, or light whose intensity changes with a 1-Hz sine wave. For example, light whose intensity is modulated or frequency modulated depending on the classification result may be illuminated. This allows the classification result of the person to be determined from the light illuminated on the person. Another difference from the first and second embodiments is that the eyeglass-type terminal device 20 performs image processing synchronized with the light that changes over time and displays the image on the display device 25 of the terminal device 20. The differences from the first and second embodiments are described below. In the following description, a case where light flashes will be taken as an example of light that changes over time.

[0064] The output control unit 13 of the information processing device 10 controls the light 40c to irradiate the person C, whose authentication result is NG, with light that flashes in short periods of time along the tracking path of the person C. Here, the light may be visible light.

[0065] The terminal device 20 may receive a signal from the information processing device 10 at any time to synchronize with the flashing of the light, or may be provided with a synchronization pattern from the information processing device 10 in advance to synchronize with the flashing of the light. The display processing unit 21 of the terminal device 20 then displays only frames in which the light is on, among frames captured by the image capturing device 24, on the display device 25. For example, as shown in FIG. 8 , if the light is on at times "10:00:00" and "10:00:02" but not at time "10:00:01," the display processing unit 21 controls the display device 25 to display only images corresponding to frames captured at times "10:00:00" and "10:00:02." This process allows the display device 25 of the terminal device 20 to constantly display people who are illuminated by the light, making it easier for the guide to identify the person to be guided. In addition, it is not necessary to use a signal or synchronization pattern for synchronization, and the display processing unit 21 of the terminal device 20 may detect and extract frames in which the light is on from each frame captured by the imaging device 24, and display only the frames in which the light is on on the display device 25.

[0066] In this embodiment, the output control unit 13 of the information processing device 10 may control the light 40c to change the light, for example, the frequency (color) or intensity of the light, or to change the blinking pattern of the light, depending on the classification result. In this case, the terminal device 20 identifies the classification result from the change in light according to the classification contained in the video captured by the image capture device 24, and identifies the person to whom the light is projected in the video. The terminal device 20 then controls the display device 25 to superimpose a highlighting according to the classification on the identified person on the video captured by the image capture device 24. Note that the highlighting here may be similar to the highlighting in the second embodiment. While the example of a flashing light has been described as the light that changes over time, other types of light that change over time can also be provided with a synchronized pattern with the time-varying light, similar to the flashing, to enable the terminal device 20 to identify the person to be notified and the content of the notification.

[0067] 9 is a diagram illustrating an example of a notification based on a notification pattern by the information processing device 10 in the fourth embodiment. The difference from the first embodiment is that the projection by the projector 40b projects a projection image onto a person in front of the target person. The differences from the first embodiment will be described below.

[0068] The output control unit 13 of the information processing device 10 estimates the movement direction of person C from the temporal change in the tracking position of person C whose authentication result was NG. The output control unit 13 also identifies people who are in the estimated movement direction of person C and have already been detected. For identified people who are in front of person C, the output control unit 13 identifies the body parts of those people using known technology such as image recognition. The output control unit 13 controls the projector 40b so that a message is projected behind the identified person, for example, on their back. Note that such projection by the projector 40b is not limited to person C whose authentication result was NG, and similar projection processing may also be performed on people whose authentication result was OK depending on the classification result.

[0069] This allows a person on the move to see the message addressed to them in front of them in the direction of their movement. Also, by adjusting their position, the guide can identify the person they are guiding.

[0070] 10 is a diagram illustrating an example of a notification based on a notification pattern by the information processing device 10 in the fifth embodiment. The fifth embodiment differs from the first embodiment in that the projection image by the projector 40b is an image of a character such as a dog, instead of an arrow or letters, projected onto the floor or wall in front of the target person. The differences from the first embodiment will be described below.

[0071] The output control unit 13 of the information processing device 10 estimates the movement direction of person C from the temporal change in the tracking position of person C whose authentication result was NG. Furthermore, the output control unit 13 identifies the floor or wall in front of person C from the estimated movement direction of person C. The output control unit 13 controls the projector 40b so that an image of a character is projected along the movement of person C onto the identified floor or wall to guide person C. Note that the person to be guided is not limited to person C, and may also be a person whose authentication was successful, such as person A or B.

[0072] This allows the moving person to move along the guided path. The guide can also identify the person to be guided by the image projected by the projector 40b.

[0073] Sixth Embodiment Fig. 11 is a diagram illustrating an example of a notification based on a notification pattern by the information processing device 10 in the sixth embodiment. The sixth embodiment differs from the first embodiment in that a speaker 40d is used as the output device 40. Furthermore, this speaker 40d is a highly directional speaker, and the direction in which sound is output can be controlled. The differences from the first embodiment will be described below.

[0074] The output control unit 13 of the information processing device 10 identifies the current position of person C whose authentication result is NG. The output control unit 13 also controls the speaker 40d to output an audible message near the head of person C at the estimated current position. Here, an example of the message is "Stop" as shown in FIG. 11 or "Please follow the arrow projected on the floor to the nearest guide."

[0075] As a result, the message from the speaker 40d cannot be heard by people other than the person to be guided, and the person to be guided can recognize that he or she has been selected as a guide target. The guide can also identify the person to be guided by output from an output device 40 other than the speaker 40d. Note that the message sent by the speaker 40d may be different depending on the classification result, even for people other than person C who has failed authentication.

[0076] Seventh Embodiment Fig. 12 is a diagram illustrating an example of a notification based on a notification pattern by the information processing device 10 in the seventh embodiment. The difference from the first and sixth embodiments is that a floor speaker 40e provided on the floor is used as the output device 40. Also, when a person to be guided comes to the installation position of the floor speaker 40e, a message is conveyed by outputting sound from the floor speaker 40e. Here, it is assumed that the floor speaker 40e is installed within the shooting range of the shooting device 30. The differences from the first and sixth embodiments will be described below.

[0077] The output control unit 13 of the information processing device 10 identifies the current position of person C, whose authentication result has been NG. The output control unit 13 also determines whether the current position of person C has reached the installation position of the floor speaker 40e from the tracking position of person C. When the current position of person C has reached the installation position of the floor speaker 40e, the output control unit 13 controls the floor speaker 40e so that a sound is played. Here, examples of the sound include short music, a chime sound, or a voice message such as "Follow the arrow projected on the floor surface to the nearest guide."

[0078] This allows the person who is the target of guidance to recognize that some kind of notification has been sent to them. Also, when a sound is played from the floor speaker 40e, the guide can identify the person who is at the installation position of the floor speaker 40e as the target of guidance.

[0079] In this embodiment, the guide may wear an earphone-type terminal device 20. In this case, the output control unit 13 of the information processing device 10 controls the earphone-type terminal device 20 to emit sound at the same time as sound is emitted from the floor speaker 40e. This makes it easier for the guide to identify the person at the installation position of the floor speaker 40e. Note that the message notification from the floor speaker 40e may be configured to have different message content depending on the classification result, even for people other than person C who has failed authentication.

[0080] Eighth Embodiment Fig. 13 is a diagram illustrating an example of a notification based on a notification pattern by the information processing device 10 in the eighth embodiment. The difference from the first, sixth, and seventh embodiments is that a figurine with a speaker 40f is used as the output device 40. Also, when a person to be guided reaches the vicinity of the installation position of the figurine with a speaker 40f, a message is output from the figurine with a speaker 40f. The differences from the first, sixth, and seventh embodiments will be described below.

[0081] The output control unit 13 of the information processing device 10 identifies the current location of person C, whose authentication result has been rejected. Furthermore, the output control unit 13 determines whether the current location of person C has reached the vicinity of the installation position of the speaker-equipped ornament 40f, based on the tracking position of person C. When the current location of person C has reached the vicinity of the installation position of the speaker-equipped ornament 40f, the output control unit 13 controls the speaker-equipped ornament 40f so that a message is played. Here, the message may be a voice message such as "Follow the arrow projected on the floor to the nearest guide."

[0082] This allows the person who is the target of guidance to recognize the message directed to them. Also, when the message is played from the ornament with speaker 40f, the guide can identify the person who is near the installation position of the ornament with speaker 40f as the person to be guided. Note that the message notification from the ornament with speaker 40f may be sent to people other than person C who was not authenticated, with different message content depending on the classification result.

[0083] Ninth Embodiment Fig. 14 is a diagram illustrating an example of a notification based on a notification pattern by the information processing device 10 in the ninth embodiment. The difference from the first embodiment is that a display device 40a capable of outputting a 3D image or a set of a screen and projector 40b capable of outputting a 3D image is used as the output device 40. Also, a message is output together with the 3D image using these devices. The differences from the first embodiment will be described below.

[0084] The output control unit 13 of the information processing device 10 identifies the current location of person C from the tracking location of person C whose authentication result was NG. The output control unit 13 also determines whether person C's current location has reached the front of the installation position of the display device 40a or the front of the screen for the projector 40b. When person C's current location has reached the front of the installation position of the display device 40a or the front of the screen for the projector 40b, the output control unit 13 controls the display device 40a or the projector 40b so that a message is displayed along with the projection of a 3D image. Here, the 3D image is a 3D image showing a guide, and the message is a voice message such as "Stop" or "Please stand still for a while" as shown in FIG. 14, or "Please follow the arrow projected on the floor." Alternatively, the output control unit 13 may control the display device 40a or the projector 40b so that a three-dimensional image such as a human character is always output, and control the display device 40a or the projector 40b so that when the current position of person C reaches the front of the installation position of the display device 40a or the front of the screen for the projector 40b, the three-dimensional image issues an audio message.

[0085] This allows the person who is the target of guidance to feel as if the 3D image is sending a message to him / her. Also, when the message is displayed, the guide can identify the person who is in front of the display device 40a or in front of the screen for the projector 40b as the target of guidance. Note that the message using the 3D image may be sent to people other than person C who failed authentication, with different message content depending on the classification result.

[0086] Tenth Embodiment Fig. 15 is a diagram illustrating an example of a notification based on a notification pattern by the information processing device 10 in the tenth embodiment. The difference from the first embodiment is that the output device 40 uses floor LEDs 40g installed on the floor in an area including all or part of the image capture range of the image capture device 30 or the vicinity thereof. Also, the floor LEDs 40g are used to perform output processing near a target person according to the person classification results. The differences from the first embodiment are described below.

[0087] The output control unit 13 of the information processing device 10 identifies the current positions of persons A, B, and C from the tracked positions. The output control unit 13 also controls the display of the floor LEDs 40g based on the current positions of persons A, B, and C. Here, as an example of the display control of the floor LEDs 40g, as shown in Fig. 15 , for persons A and B who have been authenticated, green is displayed on the left and right of persons A and B along the direction of travel, and for person C who has not been authenticated, red is displayed on the left and right of person C along the direction of travel.

[0088] This allows the person to be guided to recognize that there is some kind of notification for them from the difference in display color. Also, the guide can identify the person to be guided by the difference in display color that can distinguish the classification result of the floor LED 40g.

[0089] (Eleventh embodiment) Fig. 16 is a diagram illustrating an example of a notification based on a notification pattern by the information processing device 10 in the eleventh embodiment. The difference from the first and tenth embodiments is that a floor surface display 40h installed at a predetermined position within the shooting range of the shooting device 30 is used as the output device 40. Also, when a person to be notified reaches the installation position of the floor surface display 40h, output processing is performed from the floor surface display 40h using the floor surface display 40h. The differences from the first and tenth embodiments will be described below.

[0090] The output control unit 13 of the information processing device 10 identifies the current position of person C from the tracking position of person C whose authentication result was NG. The output control unit 13 also determines whether the current position of person C has reached the installation position of the floor surface speaker 40e. When the current position of person C has reached the installation position of the floor surface display 40h, the output control unit 13 controls the floor surface display 40h to display the current position. Here, an example of the display is output of a specific color such as red.

[0091] This allows the person who has become the guidance target to recognize that some kind of notification has been sent to him / her. Also, when the message is displayed on the floor surface display 40h, the guide can identify the person who is at the installation position of the floor surface display 40h as the guidance target. Note that the message notification using the floor surface display 40h may be sent to people other than person C who has failed authentication with different message content, for example, different color, depending on the classification result.

[0092] In this embodiment, the guide may wear an earphone-type terminal device 20. In this case, the output control unit 13 of the information processing device 10 controls the earphone-type terminal device 20 to play a silent voice so that the guide can recognize that the person is a target for guidance at the same time as the display on the floor display 40h. This makes it easier for the guide to identify the person at the installation position of the floor display 40h.

[0093] 12th Embodiment Fig. 17 is a diagram illustrating an example of a notification based on a notification pattern by the information processing device 10 in the 12th embodiment. The differences from the first to 11th embodiments are that: the classification unit 12 of the information processing device 10 classifies a person based on whether or not the person is within a predetermined range, without using the authentication result by the authentication unit 15; the output device 40 is within or near the predetermined range; and output control from the output device 40 is performed while the person is classified as being within the predetermined range. The differences from the first to 10th embodiments will be described below.

[0094] 17, the predetermined range is the area near the front of the display shelf. The image capturing device 30 is installed in a position where it can capture images of the display shelf and its front from an oblique angle. The output device 40 is a display device or light-emitting device 40i installed on or near the display shelf, and / or a speaker 40d installed on or near the display shelf.

[0095] The classification unit 12 of the information processing device 10 acquires the current location of the person detected by the tracking unit 11. The classification unit 12 classifies the detected person as a person to be notified while the person is within a predetermined range. The output control unit 13 controls output from the display device or light-emitting device 40i and / or output from the speaker 40d while a person classified by the classification unit 12 as a person to be notified is present. Here, examples of output from the display device or light-emitting device 40i include flashing of a specific color and, in the case of a display device, output of a character image representing a store clerk. Also, an example of output from the speaker 40d is an announcement such as, "Did you find a product that interests you?"

[0096] This allows people within a certain range to recognize that they are being monitored. In the case of application to display shelves, this also has the effect of preventing theft by people near the display shelves, as people within a certain range recognize that they are being monitored.

[0097] The information processing device 10 and the terminal device 20 operate in the above manner.

[0098] The processes of the information processing device 10 and the terminal device 20 in each of the above-described embodiments may be combined in any manner.

[0099] Furthermore, while the authentication performed by the authentication unit 15 of the information processing device 10 is described as facial recognition, this is not limited to this. For example, authentication for identifying a person may be performed using an ID card or ticket. The image capture device 30 is not limited to a single device, but multiple devices may be installed, and tracking, classification, and output control of a person may be performed based on the images from each device. In this case, if the image capture ranges of the image capture devices overlap, the information processing device 10 associates the same person captured simultaneously by multiple image capture devices based on the positional relationship between the image capture ranges of each image capture device and the position of the detected person within that image capture range, as well as the features of the detected person. This enables the information processing device 10 to perform more accurate tracking, authentication, classification, and output control processes.

[0100] In addition, in the above-described embodiments, particularly the first to eleventh embodiments, a person moving to a venue has been described as an example, but the present invention is not limited to this. For example, the present invention can be applied to entrance / exit management in an office or factory, entrance / exit management for transportation such as a train, in addition to event venues such as theme parks and movie theaters.

[0101] 18 is a diagram illustrating an example configuration of an information processing device 10 according to an embodiment of the present disclosure. The information processing device 10 includes a tracking unit 11, a classification unit 12, and an output control unit 13. The tracking unit 11 tracks each of the detected people. The classification unit 12 classifies each of the detected people. The output control unit 13 controls notifications using a notification pattern for the detected people based on the tracking positions by the tracking unit and the classification results by the classification unit.

[0102] 19 is a diagram illustrating an example configuration of a terminal device 20 according to an embodiment of the present disclosure. The terminal device 20 includes a display processing means 21, an imaging device 24, and a display device 25. The display processing means 21 detects a light notification pattern irradiated onto a person detected by the information processing device from an image acquired from the imaging device 24 based on a classification result of the person, and controls the display device 25 to display an image in which the person irradiated with the detected light notification pattern is highlighted based on the detected light notification pattern.

[0103] FIG. 20 is a block diagram showing an example of the hardware configuration of the information processing device 10 or the terminal device 20. The hardware configuration of the information processing device 10 or the terminal device 20 includes a CPU 61, a RAM (Random Access Memory) 62, a ROM (Read Only Memory) 63, a recording device 64, and the like. The ROM 63 and the recording device 64 store programs and information that realize the functions of the information processing device 10 or the terminal device 20. The recording device 64 serves as the storage unit 14 of the information processing device 10 or the storage unit 23 of the terminal device 20, storing information required for processing. The RAM 62 is used as a work area or the like for temporarily storing data and the like used by the CPU 61 or the like during operation. The information processing device 10 or the terminal device 20 may also include an input / output port 65 that serves as an interface with input / output devices such as a display device, a mouse, and a keyboard. The input / output port 65 may also function as a communication port for communication connection. The ROM 63 may be composed of an EEPROM (Electrically Erasable Programmable Read-Only Memory) or the like, and the recording device 64 may be composed of a hard disk, SSD, or the like, so that the computer programs for realizing the functions of the information processing device 10 and the terminal device 20 can be updated in the ROM 63 or the recording device 64.

[0104] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0105] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0106] (Supplementary Note 1) An information processing device comprising: a tracking means for tracking each of a plurality of detected persons; a classification means for classifying each of the detected plurality of persons; and an output control means for controlling notifications using a notification pattern for the detected plurality of persons based on the tracking position by the tracking means and the classification result by the classification means.

[0107] (Supplementary Note 2) The information processing device according to Supplementary Note 1, wherein the output control means controls notification using a notification pattern of light output with different frequencies based on the classification result.

[0108] (Supplementary Note 3) The information processing device according to Supplementary Note 1 or 2, wherein the output control means controls notification using a light notification pattern based on the output of light that changes over time based on the classification result.

[0109] (Supplementary Note 4) The information processing device according to Supplementary Note 2, wherein the output control means controls notification using a light notification pattern by outputting light of a wavelength other than visible light in response to a predetermined classification result.

[0110] (Supplementary Note 5) The information processing device according to any one of Supplementary Notes 2 to 4, wherein the output control means controls notification using the notification pattern of light by irradiating the tracking position of the target person with the light.

[0111] (Supplementary Note 6) The information processing device described in any one of Supplementary Notes 1 to 5, wherein the output control means estimates a moving direction based on a temporal change in the tracking position of the target person, and controls notification using a notification pattern by projecting a projection image forward in the moving direction relative to the tracking position of the target person.

[0112] (Supplementary Note 7) The information processing device according to any one of Supplementary Notes 1 to 6, wherein the output control means controls notification using a sound notification pattern by outputting a sound based on the classification result.

[0113] (Supplementary Note 8) The information processing device according to Supplementary Note 7, wherein the output control means controls to output the sound notification pattern from a forward directional speaker toward the tracking position of the target person.

[0114] (Supplementary Note 9) The information processing device according to Supplementary Note 7 or 8, wherein the output control means controls the terminal device to output the sound notification pattern when the tracking position of the target person reaches a predetermined position.

[0115] (Supplementary Note 10) An information processing device according to any one of Supplementary Notes 1 to 9, wherein the classification means performs a predetermined classification on the target person while the tracking position of the target person reaches a predetermined range, and the output control means, when the classification means performs the predetermined classification, controls output using a notification pattern in which output is performed from an output device within the predetermined range or in the vicinity of the predetermined range.

[0116] (Supplementary Note 11) The information processing device according to any one of Supplementary Notes 1 to 10, wherein the classification means includes an authentication means that authenticates the plurality of detected persons based on pre-registered authentication information, and the classification means classifies each of the plurality of detected persons based on an authentication result by the authentication means.

[0117] (Supplementary Note 12) The information processing device according to any one of Supplementary Notes 1 to 11, wherein the notification pattern includes a pattern in which no notification is performed.

[0118] (Supplementary Note 13) A terminal device comprising: an imaging device; a display device; and a display processing means that detects a light notification pattern irradiated onto a person detected by an information processing device from an image acquired from the imaging device based on a classification result of the person, and controls the display device to display an image in which the person irradiated with the detected light notification pattern is highlighted based on the detected light notification pattern.

[0119] (Supplementary Note 14) The terminal device according to Supplementary Note 13, wherein the photographing device is an infrared detection camera, and the display processing means detects the light notification pattern from the image captured by the infrared detection camera, and controls the display device to display an image in which a person illuminated by the detected light notification pattern is highlighted based on the detected light notification pattern.

[0120] (Supplementary Note 15) The terminal device according to Supplementary Note 13 or 14, wherein the display processing means detects the light notification pattern based on light that changes over time from the image captured by the imaging device, and controls the display device to display an image in which a person illuminated by the detected light notification pattern is highlighted based on the detected light notification pattern.

[0121] (Supplementary Note 16) The terminal device according to any one of Supplementary Notes 13 to 15, wherein the display processing means highlights a person illuminated with the detected light notification pattern by linking and displaying supplementary information based on the detected light notification pattern.

[0122] (Supplementary Note 17) The terminal device according to any one of Supplementary Notes 14 to 16, further comprising a notification acquisition means for acquiring a display method for the light notification pattern from the information processing device, wherein the display processing means highlights a person illuminated with the light notification pattern using the display method for the detected light notification pattern.

[0123] (Supplementary Note 18) An information processing system comprising: an information processing device according to any one of Supplementary Notes 1 to 5; and a terminal device according to any one of Supplementary Notes 13 to 16.

[0124] (Supplementary Note 21) An information processing method performed by a computer, which includes tracking each of a plurality of detected people, classifying each of the detected people, and controlling notifications using a notification pattern for the detected people based on the tracking position obtained by the tracking and the classification result obtained by the classification.

[0125] (Supplementary Note 22) The information processing method according to Supplementary Note 21, wherein notification control is performed using a notification pattern of light output with different frequencies based on the classification result.

[0126] (Supplementary Note 23) The information processing method according to Supplementary Note 21 or 22, wherein notification control is performed using a light notification pattern based on a light output that changes over time based on the classification result.

[0127] (Supplementary Note 24) The information processing method according to Supplementary Note 22, wherein notification control is performed using a light notification pattern by outputting light of a wavelength other than visible light for a predetermined classification result.

[0128] (Supplementary Note 25) The information processing method according to any one of Supplementary Notes 22 to 24, wherein notification control is performed using the notification pattern of light by irradiating the tracking position of the target person with the light.

[0129] (Supplementary Note 26) An information processing method according to any one of Supplementary Notes 21 to 25, wherein a moving direction of a target person is estimated based on the temporal progression of the tracking position of the target person, and notification control is performed using a notification pattern by projecting a projection image forward of the tracking position of the target person on the side of the moving direction.

[0130] (Supplementary Note 27) The information processing method according to any one of Supplementary Notes 21 to 26, further comprising: controlling notification using a sound notification pattern by outputting a sound based on the classification result.

[0131] (Supplementary Note 28) The information processing method according to Supplementary Note 27, further comprising controlling to output the sound notification pattern from a forward directional speaker with respect to the tracking position of the target person.

[0132] (Supplementary Note 29) The information processing method according to Supplementary Note 27 or 28, further comprising controlling the terminal device to output the sound notification pattern when the tracking position of the target person reaches a predetermined position.

[0133] (Supplementary Note 30) An information processing method described in any one of Supplementary Notes 21 to 29, wherein while the tracking position of the target person reaches a predetermined range, a predetermined classification is performed on the target person, and when the predetermined classification is performed by the classification means, output control is performed using a notification pattern in which output is performed from an output device within the predetermined range or in the vicinity of the predetermined range.

[0134] (Supplementary Note 31) The information processing method according to any one of Supplementary Notes 21 to 30, further comprising: authenticating the detected plurality of persons based on pre-registered authentication information; and classifying each of the detected plurality of persons based on the authentication result.

[0135] (Supplementary Note 32) The information processing method according to any one of Supplementary Notes 21 to 31, wherein the notification pattern includes a pattern in which no notification is performed.

[0136] (Supplementary Note 33) An information processing method performed by a computer: detecting a notification pattern of light irradiated onto a person detected by an information processing device from an image acquired from an imaging device based on a classification result of the person; and controlling a display device to display an image in which the person irradiated with the detected notification pattern of light is highlighted based on the detected notification pattern of light.

[0137] (Supplementary Note 34) The information processing method described in Supplementary Note 13, wherein the photographing device is an infrared detection camera, and the light notification pattern is detected from an image captured by the infrared detection camera, and the display device is controlled to display an image in which a person illuminated by the detected light notification pattern is highlighted based on the detected light notification pattern.

[0138] (Supplementary Note 35) The information processing method described in Supplementary Note 13 or 14, which detects a light notification pattern based on light that changes over time from an image captured by the imaging device, and controls the display device to display an image in which a person illuminated by the detected light notification pattern is highlighted based on the detected light notification pattern.

[0139] (Supplementary Note 36) The information processing method according to any one of Supplementary Notes 13 to 15, wherein a person illuminated by the detected light notification pattern is highlighted by linking and displaying supplementary information based on the detected light notification pattern.

[0140] (Supplementary Note 37) The information processing method described in any one of Supplementary Notes 14 to 16, further comprising: acquiring a display method for the light notification pattern from the information processing device; and highlighting a person illuminated by the acquired light notification pattern using the display method for the detected light notification pattern.

[0141] (Supplementary Note 38) A computer-implemented information processing method, comprising the method according to any one of Supplements 21 to 25 and the method according to any one of Supplements 33 to 36.

[0142] (Appendix 41) A computer-readable recording medium having recorded thereon a program that causes a computer to execute the following: tracking each of a plurality of detected persons; classifying each of the detected plurality of persons; and controlling notifications using notification patterns for the detected plurality of persons based on the tracking positions obtained by the tracking and the classification results obtained by the classification.

[0143] (Supplementary Note 42) The computer-readable recording medium according to Supplementary Note 41, wherein notification control is performed using a light notification pattern by outputting light of different frequencies based on the classification result.

[0144] (Supplementary Note 43) The computer-readable recording medium according to Supplementary Note 41 or 42, wherein notification control is performed using a light notification pattern with a light output that changes over time based on the classification result.

[0145] (Supplementary Note 44) The computer-readable recording medium according to Supplementary Note 42, wherein notification control is performed using a light notification pattern by outputting light of a wavelength other than visible light for a predetermined classification result.

[0146] (Supplementary Note 45) The computer-readable recording medium according to any one of Supplementary Notes 42 to 44, wherein notification control is performed using the notification pattern of light by irradiating the tracking position of the target person with the light.

[0147] (Appendix 46) A computer-readable recording medium according to any one of appendices 41 to 45, wherein a moving direction of a target person is estimated based on the temporal progression of the tracking position of the target person, and notification control is performed using a notification pattern by projecting a projection image forward of the tracking position of the target person on the side of the moving direction.

[0148] (Supplementary Note 47) The computer-readable recording medium according to any one of Supplementary Notes 41 to 46, wherein notification control is performed using a sound notification pattern by outputting a sound based on the classification result.

[0149] (Supplementary Note 48) The computer-readable recording medium according to Supplementary Note 47, further comprising controlling to output the sound notification pattern from a forward-directional speaker with respect to the tracking position of the target person.

[0150] (Supplementary Note 49) The computer-readable recording medium according to Supplementary Note 47 or 48, wherein when the tracking position of the target person reaches a predetermined position, the terminal device is controlled to output the sound notification pattern.

[0151] (Appendix 50) A computer-readable recording medium according to any one of Appendices 41 to 49, wherein while the tracking position of the target person reaches a predetermined range, a predetermined classification is performed on the target person, and when the predetermined classification is performed by the classification means, output control is performed using a notification pattern in which output is performed from an output device within the predetermined range or in the vicinity of the predetermined range.

[0152] (Supplementary Note 51) A computer-readable recording medium according to any one of Supplementary Notes 41 to 50, further comprising: authenticating the detected plurality of persons based on pre-registered authentication information; and classifying each of the detected plurality of persons based on the authentication results.

[0153] (Supplementary Note 52) The computer-readable storage medium according to any one of Supplementary Notes 41 to 51, wherein the notification pattern includes a pattern in which no notification is performed.

[0154] (Supplementary Note 53) A computer-readable recording medium having recorded thereon a program that causes a computer to execute the following steps: detect a light notification pattern irradiated onto a person detected by an information processing device from an image acquired from an imaging device based on a classification result of the person; and control a display device to display an image in which the person irradiated with the detected light notification pattern is highlighted based on the detected light notification pattern.

[0155] (Appendix 54) The computer-readable recording medium described in Appendix 53, wherein the photographing device is an infrared detection camera, and the light notification pattern is detected from the image captured by the infrared detection camera, and the display device is controlled to display an image in which a person illuminated by the detected light notification pattern is highlighted based on the detected light notification pattern.

[0156] (Appendix 55) A computer-readable recording medium as described in Appendix 53 or 54, which detects a light notification pattern based on light that changes over time from the image captured by the imaging device, and controls the display device to display an image in which a person illuminated by the detected light notification pattern is highlighted based on the detected light notification pattern.

[0157] (Supplementary Note 56) A computer-readable recording medium according to any one of Supplementary Notes 53 to 55, wherein a person illuminated by the detected light notification pattern is highlighted by linking and displaying additional information based on the detected light notification pattern.

[0158] (Appendix 57) A computer-readable recording medium described in any one of Appendices 54 to 56, which acquires a display method for the light notification pattern from the information processing device, and highlights a person illuminated by the acquired light notification pattern using the display method for the detected light notification pattern.

[0159] REFERENCE SIGNS LIST 10 Information processing device 11 Tracking unit 12 Classification unit 13 Output control unit 14 Memory unit 14a Notification pattern information 14b Authentication information 15 Authentication unit 20 Terminal device 21 Display processing unit 22 Notification acquisition unit 23 Memory unit 23a Notification information 24 Imaging device 25 Display device 26 Speaker 30 Imaging device 40 Output device 40a Display device 40b Projector 40c Light 40d Speaker 40e Floor speaker 40f Ornament 40h Floor display 40i Light-emitting device 50 Network

Claims

1. An information processing device comprising: a tracking means for tracking each of a plurality of detected persons; a classification means for classifying each of the detected persons; and an output control means for controlling notifications using a notification pattern for the detected persons based on the tracking position by the tracking means and the classification result by the classification means.

2. The information processing device according to claim 1, wherein the output control means controls notification using a light notification pattern by outputting light of different frequencies based on the classification results.

3. The information processing device according to claim 1, wherein the output control means controls notification using a light notification pattern based on the light output that changes over time based on the classification result.

4. The information processing device according to claim 2, wherein the output control means controls notification using a light notification pattern by outputting light of a wavelength other than visible light in response to a predetermined classification result.

5. An information processing device according to any one of claims 2 to 4, wherein the output control means controls notification using the light notification pattern by irradiating the light onto the tracking position of the target person.

6. The information processing device according to claim 1, wherein the output control means estimates the direction of movement of the target person based on the temporal progression of the tracking position of the target person, and controls notification using a notification pattern by projecting a projection image forward in the direction of movement relative to the tracking position of the target person.

7. The information processing device according to claim 1, wherein the output control means controls notification using a sound notification pattern by outputting a sound based on the classification result.

8. The information processing device according to claim 7, wherein the output control means controls the sound notification pattern to be output from a forward directional speaker toward the tracking position of the target person.

9. The information processing device according to claim 7 or 8, wherein the output control means controls the terminal device to output the sound notification pattern when the tracking position of the target person reaches a predetermined position.

10. The information processing device of claim 1, wherein the classification means performs a predetermined classification on the target person while the tracking position of the target person reaches a predetermined range, and the output control means, when the classification means performs the predetermined classification, controls output using a notification pattern in which output is performed from an output device within the predetermined range or in the vicinity of the predetermined range.

11. The information processing device according to claim 1, wherein the classification means includes authentication means for authenticating the plurality of detected persons based on pre-registered authentication information, and the classification means classifies each of the plurality of detected persons based on the authentication results by the authentication means.

12. The information processing device according to claim 1, wherein the notification pattern includes a pattern in which no notification is performed.

13. A terminal device comprising: an imaging device; a display device; and a display processing means for detecting a light notification pattern irradiated onto a person detected by an information processing device from an image acquired from the imaging device based on a classification result of the person, and controlling the display device to display an image in which the person irradiated with the detected light notification pattern is highlighted based on the detected light notification pattern.

14. The terminal device according to claim 13, wherein the photographing device is an infrared detection camera, and the display processing means detects the light notification pattern from the image captured by the infrared detection camera, and controls the display device to display an image in which a person illuminated by the detected light notification pattern is highlighted based on the detected light notification pattern.

15. The terminal device according to claim 13, wherein the display processing means detects the light notification pattern based on light that changes over time from the image captured by the imaging device, and controls the display device to display an image in which a person illuminated by the detected light notification pattern is highlighted based on the detected light notification pattern.

16. The terminal device according to claim 13, wherein the display processing means highlights a person illuminated by the detected light notification pattern by linking and displaying additional information based on the detected light notification pattern.

17. A terminal device according to any one of claims 14 to 16, further comprising a notification acquisition means for acquiring a display method for the light notification pattern from the information processing device, wherein the display processing means highlights a person illuminated by the light notification pattern using the display method for the detected light notification pattern.

18. An information processing system comprising an information processing device according to any one of claims 1 to 4 and a terminal device according to any one of claims 13 to 16.

19. An information processing method performed by a computer, which includes tracking each of a plurality of detected people, classifying each of the detected people, and controlling notifications using a notification pattern for the detected people based on the tracking position obtained by the tracking and the classification results obtained by the classification.

20. A computer-readable recording medium having recorded thereon a program that causes a computer to execute the following steps: track each of a plurality of detected people; classify each of the detected people; and control notifications using notification patterns for the detected people based on the tracking positions obtained by the tracking and the classification results obtained by the classification.

Images