Processing device, processing method, and program

Abstract

The present disclosure provides a processing device comprising: a registration unit that registers, in a storage unit, appearance information which at least indicates a person subject to monitoring and an accessory of the person subject to monitoring; and a determination unit that outputs the appearance information pertaining to the person subject to monitoring if the person subject to monitoring does not appear in a confirmation target image.

Description

Processing device, processing method, and program

[0001] The present disclosure relates to a processing device, a processing method, and a program.

[0002] A technology related to the present disclosure is disclosed in Patent Document 1. Patent Document 1 discloses a technology for taking roll calls on buses. This technology photographs passengers boarding the bus with a camera installed on the bus. This technology then calculates the number of passengers who have not returned to the bus based on the results of comparing the photographed image with pre-registered images. This technology then notifies the number of passengers who have not returned to the bus and the number of passengers who are still on board.

[0003] International Publication No. 2018 / 123843

[0004] When traveling in a group, it may be necessary to keep track of the number of passengers at various times. There is a need for technology that can efficiently keep track of the number of passengers. The information output by the technology described in Patent Document 1 makes it possible to know the "number of passengers who have not returned to the bus," but it does not make it possible to know "who has not returned to the bus." In this case, the supervisor must visually check who has not returned, which is inefficient.

[0005] In view of the above-mentioned problems, an example of an object of the present disclosure is to provide a processing device, a processing method, and a program that support efficient identification of personnel when acting in a group.

[0006] The first program of the present disclosure causes a computer to function as: a registration means for registering appearance information indicating at least a person to be monitored and accessories of the person to be monitored in a storage means; and a determination means for outputting the appearance information of the person to be monitored when the person to be monitored is not captured in an image to be confirmed.

[0007] The first processing device of the present disclosure also includes a registration means for registering appearance information indicating at least the person to be monitored and accessories of the person to be monitored in a storage means, and a determination means for outputting the appearance information of the person to be monitored when the person to be monitored is not included in the image to be confirmed.

[0008] Furthermore, the first processing method of the present disclosure includes one or more computers registering appearance information indicating at least the person being monitored and the accessories of the person being monitored in a storage means, and outputting the appearance information of the person being monitored if the person being monitored is not captured in the image to be confirmed.

[0009] According to one aspect of the present disclosure, a processing device, a processing method, and a program are realized that support efficient identification of personnel when acting as a group.

[0010] FIG. 1 is a diagram illustrating an example of a functional block diagram of a processing device according to the present disclosure. FIG. 2 is a flowchart illustrating an example of a processing flow of a processing device according to the present disclosure. FIG. 3 is a diagram illustrating an example of an implementation of a processing device according to the present disclosure. FIG. 4 is a diagram illustrating an example of a hardware configuration of a processing device according to the present disclosure. FIG. 5 is a diagram schematically illustrating an example of information processed by a processing device according to the present disclosure. FIG. 6 is a flowchart illustrating another example of a processing flow of a processing device according to the present disclosure. FIG. 7 is a diagram illustrating an example of information output by a processing device according to the present disclosure. FIG. 8 is a flowchart illustrating an example of a processing flow of a processing device according to the present disclosure. FIG. 9 is a diagram illustrating another example of information output by a processing device according to the present disclosure. FIG. 10 is a diagram illustrating another example of an implementation of a processing device according to the present disclosure.

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all drawings, like components are denoted by like reference numerals, and descriptions thereof will be omitted where appropriate.

[0012] <First embodiment> Fig. 1 is a functional block diagram showing an overview of a processing device 10. As shown in Fig. 1, the processing device 10 has a registration unit 11, a determination unit 12, and a storage unit 13. Note that the processing device 10 does not necessarily have to have the storage unit 13. In this case, an external device configured to be able to communicate with the processing device 10 has the storage unit 13.

[0013] The registration unit 11 registers appearance information indicating at least the person to be monitored and accessories of the person to be monitored in the storage unit 13. Accessories are objects that are attached to the person to be monitored, and include, for example, at least one of an object worn by the person to be monitored and an object possessed by the person to be monitored.

[0014] The determination unit 12 executes the process shown in the flowchart of FIG.

[0015] First, the determination unit 12 acquires a confirmation target image (S10). The confirmation target image is an image taken at a location during a personnel identification operation. The "person identification operation" is an operation to identify whether a person to be monitored is present at a location, such as a roll call.

[0016] Next, the determination unit 12 determines whether the person to be monitored is captured in the confirmation target image (S11). The determination unit 12 can make this determination by using a widely known image analysis technique.

[0017] If the monitoring target person is not captured in the confirmation image (No in S11), the determination unit 12 outputs the appearance information of the monitoring target person (S12). The determination unit 12 can acquire the appearance information of the monitoring target person from the storage unit 13.

[0018] When conducting personnel counting work, a supervisor of group activities can recognize, based on the output appearance information, that the person being monitored is not in the confirmation image, i.e., that the person being monitored is not at that location. Furthermore, based on the output appearance information, the supervisor can recognize the person being monitored who is not at that location and their accessories. Based on the information thus recognized, the supervisor can then search for the person being monitored who is not at that location.

[0019] On the other hand, if the person to be monitored is captured in the confirmation target image (Yes in S11), the determination unit 12 terminates the process without executing the process in S12. Note that, although not shown, the determination unit 12 may execute a process different from the process in S12 when the result in S11 is Yes. For example, if the result in S11 is Yes, the determination unit 12 may output information indicating that the person to be monitored is captured in the confirmation target image.

[0020] Based on the operation of the processing device 10 when the answer to S11 is Yes (such as not executing the processing of S12), the supervisor can recognize that the person being monitored is in the image being confirmed, i.e., that the person being monitored is present at that location.

[0021] According to the processing device 10 of the first embodiment, the supervisor only needs to photograph the location when conducting personnel counting work, and does not need to visually check whether the person to be monitored is present at the location.

[0022] Furthermore, the processing device 10 of the first embodiment can output appearance information of a monitored person when the monitored person is not captured in the confirmation target image. Based on the output appearance information, a supervisor can recognize that the monitored person is not captured in the confirmation target image, i.e., that the monitored person is not present at the location, during personnel identification work. Based on the output appearance information, the supervisor can also recognize monitored people and their accessories that are not present at the location. Based on the information thus recognized, the supervisor can then search for monitored people who are not present at the location.

[0023] According to the processing device 10 of the first embodiment, the efficiency of personnel identification work is improved.

[0024] Second Embodiment Overview A processing apparatus 10 according to a second embodiment is a specific embodiment of the configuration of the processing apparatus 10 according to the first embodiment. The configuration of the processing apparatus 10 will be described in detail below.

[0025] 3, the processing device 10 of the second embodiment is the server 1 of a server-client system. The processing device 10 cooperates with the mobile terminal 2 to execute the processes described below.

[0026] The mobile terminal 2 is a portable terminal carried by a supervisor of group activities. The mobile terminal 2 has a communication function for communicating with the server 1, a camera function, an input / output function, and a calculation function. The mobile terminal 2 is, for example, a smartphone, a tablet terminal, a laptop computer, a mobile phone, a smart watch, etc., but is not limited to these.

[0027] "Hardware Configuration" An example of the hardware configuration of the processing device 10 will be described. Each functional unit of the processing device 10 is realized by any combination of hardware and software. Those skilled in the art will understand that there are various variations in the realization method and device. Software includes programs that are pre-loaded in the device before shipping, and programs downloaded from recording media such as CDs (Compact Discs) or servers on the Internet.

[0028] FIG. 4 is a block diagram illustrating an example of the hardware configuration of a processing device 10. As shown in FIG. 4, the processing device 10 has a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. The processing device 10 does not necessarily have to have the peripheral circuit 4A. Note that the processing device 10 may be composed of multiple devices that are physically and / or logically separated. In this case, each of the multiple devices may have the above hardware configuration.

[0029] The bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input / output interface 3A to mutually transmit and receive data. The processor 1A is, for example, a processing unit such as a CPU or a graphics processing unit (GPU). The memory 2A is, for example, a random access memory (RAM) or a read-only memory (ROM). The input / output interface 3A includes interfaces for acquiring information from input devices, external devices, external servers, external sensors, cameras, etc., and interfaces for outputting information to output devices, external devices, external servers, etc. The input / output interface 3A also includes an interface for connecting to a communication network such as the Internet. Examples of input devices include a keyboard, mouse, microphone, physical buttons, touch panel, etc. Examples of output devices include a display, speaker, printer, mailer, etc. The processor 1A can issue commands to each module and perform calculations based on the results of those calculations.

[0030] "Functional Configuration" Next, the functional configuration of the processing device 10 will be described in detail. Fig. 1 shows an example of a functional block diagram of the processing device 10. As shown in the figure, the processing device 10 has a registration unit 11, a determination unit 12, and a storage unit 13. Note that the processing device 10 does not necessarily have to have the storage unit 13. In this case, an external device configured to be able to communicate with the processing device 10 has the storage unit 13.

[0031] These functional units realize the "pre-registration process," "registration process on the day of group activity," and "personnel identification work support process during group activity." Each process will be explained below.

[0032] - Pre-registration process - Pre-registration process is a process that is carried out before a group activity is carried out. In the pre-registration process, information on members of the group that will be performing the group activity is registered in the storage unit 13. The registered members become people to be monitored.

[0033] A "group" is a group of multiple people. Examples of groups include, but are not limited to, a family, a group of friends, a group of members who belong to the same organization (e.g., a company, school, local government, class, team, etc.), a group of guests who participated in the same tour, etc.

[0034] Examples of "group activities" include, but are not limited to, travel, sightseeing, training, shopping, expeditions, extracurricular activities, field trips, tours, etc.

[0035] "Member information" includes at least facial information of each member. The facial information includes at least one of a facial image and facial features extracted from the facial image. Member information may also include other information such as name and contact information. Contact information may include a telephone number, address, email address, etc.

[0036] The pre-registration process is realized by the registration unit 11. Each member of a group that is engaged in group activities operates a communication terminal to perform a process of registering their own facial information, contact information, etc. in the processing device 10. Note that a person other than each member may obtain facial information, contact information, etc. from each member and operate an arbitrary communication terminal to register the facial information, contact information, etc. of each member in the processing device 10.

[0037] Registration of such information can be achieved using any well-known technology. The registration may be performed online. For example, a registered user may perform the registration via a dedicated application installed on a communication terminal. A registered user is a user who registers member facial information, contact information, etc. in the processing device 10. The dedicated application is an application dedicated to the service provided by the processing device 10 (the same applies below). Alternatively, a registered user may access a predetermined web page via a web browser installed on the communication terminal and perform the registration on the web page. The communication terminal may be, but is not limited to, a smartphone, tablet terminal, PC, mobile phone, smart watch, etc.

[0038] When the registration unit 11 acquires member information transmitted from the communication terminal, it registers the information in the storage unit 13. Fig. 5 shows an example of member information registered in the storage unit 13. The monitored person information in Fig. 5 includes fields for identification information, name, contact information, face information, and appearance information. The identification information, name, contact information, and face information are registered in advance registration processing. The appearance information is registered in registration processing on the day of group activity, which will be described later.

[0039] The "Identification Information" column records information for identifying multiple members belonging to the same group from one another. The "Name" column records the name of each member. The "Contact Information" column records contact information for each member. The "Facial Information" column records information about the face of each member. The facial information column records, for example, the file name of each member's facial image and the facial features of each member. The "Appearance Information" column records appearance information for each member on the day of the group activity. Appearance information will be described later.

[0040] The processing device 10, which is the server 1 of the server-client system, may also be able to support personnel tracking for each of a plurality of independent organizations. In this case, the processing device 10 can store monitored person information as shown in FIG. 5, linked to each of the plurality of organizations. A registered user then specifies the organization to which they belong and registers the member information as described above. When the registration unit 11 acquires the member information transmitted from the communication terminal, it associates the acquired member information with the identification information of the specified organization and registers the acquired member information in the storage unit 13. The organization can be designated using any technology, such as by inputting information identifying the organization.

[0041] - Registration processing on the day of group activity - The registration processing on the day of group activity is processing that is performed after the above-mentioned advance registration processing and before the processing for supporting the work of identifying personnel during group activity, which will be described later. The registration processing on the day of group activity is performed on the day on which the group activity is to be performed. The registration processing on the day of group activity may be performed, for example, at the first gathering before the group activity begins on the day of the group activity. Note that if the above-mentioned advance registration processing has not been performed for some reason, the advance registration processing may be performed simultaneously with or after the registration processing on the day of group activity.

[0042] In the registration process for the day of group activity, appearance information of each member (person to be monitored) on the day of group activity is registered in the storage unit 13.

[0043] The "appearance information" is information that indicates at least the person to be monitored and their accessories. For example, the appearance information is information that indicates the person to be monitored and their accessories on the day of the group activity (i.e., the day of monitoring).

[0044] The appearance information is, for example, an appearance image showing the monitored person and their accessories. The appearance information may be, for example, an appearance image taken on the day of group activity showing the monitored person and their accessories. The appearance information may also be, for example, an appearance image showing the monitored person's entire body and their accessories. The appearance information may also be, for example, an appearance image taken on the day of group activity showing the monitored person's entire body and their accessories.

[0045] "Accessories" include at least one of items worn by the monitored person and items carried by the monitored person. Items worn by the monitored person include clothes, hats, shoes, watches, glasses, masks, gloves, scarves, accessories, etc. Items carried by the monitored person include bags, cameras, umbrellas, etc.

[0046] The information registered in the advance registration process and the information registered in the registration process on the day of the group activity have in common that they are both information about persons to be monitored, but the purposes for collecting each type of information are different.

[0047] The information registered in the pre-registration process is intended to collect facial information of the person being monitored. The facial information registered in the pre-registration process can be used in various situations, such as to verify the identity of the person being monitored when meeting them for the first time.

[0048] On the other hand, the information registered in the registration process on the day of group activity is information intended to collect information on the state (hairstyle, makeup, etc.) and accessories of the monitored person on the "day of group activity." The appearance of the monitored person can change due to hairstyle, makeup, accessories, etc. Information on the state (hairstyle, makeup, etc.) and accessories of the monitored person registered on the day of group activity improves the accuracy of authentication and search for the monitored person on the day of group activity.

[0049] The registration process on the day of the group activity is realized by the registration unit 11. The registration unit 11 executes the process shown in the flowchart of FIG.

[0050] First, the registration unit 11 acquires a registration target image (S20). The registration target image is an image to be registered as appearance information.

[0051] For example, the registration unit 11 can acquire a registration target image from the portable terminal 2 carried by a supervisor of the group activity. In this case, the supervisor operates the portable terminal 2 to photograph the person to be monitored and transmits the photographed image to the processing device 10 as a registration target image. An example of this process will be described below, but is not limited to this example. For example, the supervisor operates the portable terminal 2 to activate a dedicated application and then activates a mode for registration processing for the day of the group activity. In response to the activation of the registration processing, the camera is activated and becomes ready to capture images. In this mode, the supervisor photographs the person to be registered and transmits the photographed image to the processing device 10.

[0052] The captured image may be one or more still images, or may be a moving image composed of multiple frame images. Each still image and / or each frame image becomes an image to be registered. When a moving image is captured, the images may be transmitted to the processing device 10 in real time or in batch. For example, after the start of capturing a moving image, the captured images may be transmitted to the processing device 10 in real time while the capturing is being performed. Alternatively, after the start of capturing a moving image, and after the capturing is completed, the images captured during that time may be transmitted to the processing device 10.

[0053] The photographing may be carried out without the monitored person being aware that they are being photographed. For example, the supervisor photographs people at a gathering place when the group first gathers before the group activity begins on the day of the group activity. The monitored person may be included among the people at the gathering place. Alternatively, the photographing may be carried out with the monitored person being aware that they are being photographed. For example, when the group first gathers before the group activity begins on the day of the group activity, a headcount operation is carried out in which each monitored person verbally notifies the supervisor of their presence. At this time, the supervisor may photograph each monitored person.

[0054] Alternatively, the registration unit 11 may acquire images taken by a camera installed at a gathering place at the time of the first gathering before the start of the group activity on the day of the group activity. The images taken by the camera are input to the processing device 10 by any means. For example, the camera and the processing device 10 may be configured to be able to communicate with each other. The camera may then transmit the captured images to the processing device 10. Note that this means is merely an example, and the images taken by the camera may also be input to the processing device 10 by using other means that are manually implemented by a person such as a supervisor.

[0055] After S20, the registration unit 11 compares the face information of at least one person to be monitored who has been registered in advance in the storage unit 13 by a pre-registration process with the registration target image, and identifies the person to be monitored who appears in the registration target image (S21). Then, the registration unit 11 associates the registration target image with the person to be monitored identified in S21, and registers it in the storage unit 13 as appearance information (S22). This process realizes the registration of the appearance information of the person to be monitored in FIG. 5.

[0056] As a modified example, after photographing a certain person to be monitored, the supervisor may operate the portable terminal 2 to designate the person to be monitored, and register the photographed image as appearance information by linking it to the designated person to be monitored. The person to be monitored can be designated, for example, by selecting one person from at least one person to be monitored registered in the pre-registration process, or by inputting information identifying the person to be monitored (such as an identification number, name, or contact information), but is not limited to these. In this modified example, the registration unit 11 does not need to perform the matching process of S21.

[0057] -Processing to support work to identify people during group activity- The processing to support work to identify people during group activity is a process that is performed after the above-mentioned advance registration process and the registration process on the day of the group activity. The processing to support work to identify people during group activity is performed at any timing during the group activity. This timing can vary, but examples include when gathering after a break or free activity, or before starting to move.

[0058] The "personnel identification work support process" is a process for confirming whether a person to be monitored is present at a location during the personnel identification work. The "location" is the location where the personnel identification work is performed. For example, a meeting place or the like is the location where the personnel identification work is performed.

[0059] The process of supporting the task of identifying the number of people in a group is performed by the determination unit 12. The determination unit 12 executes the process shown in the flowchart of FIG.

[0060] First, the determination unit 12 acquires a confirmation target image (S30).

[0061] For example, the determination unit 12 can acquire, from the portable device 2, a confirmation target image taken with the portable device 2 carried by a supervisor of group activity. In this case, the supervisor operates the portable device 2 to take a picture of people in the location and transmits the captured image to the processing device 10 as a confirmation target image. An example of this process will be described below, but is not limited to this example. For example, the supervisor operates the portable device 2 to launch a dedicated application and then activates a mode for personnel identification work support processing. In response to the activation of the personnel identification work support processing, the camera is activated and becomes ready to capture images. In this mode, the supervisor takes pictures of people in the location and transmits the captured images to the processing device 10.

[0062] The captured image may be one or more still images, or may be a moving image composed of multiple frame images. Each still image and / or each frame image becomes the image to be confirmed. When a moving image is captured, the images may be transmitted to the processing device 10 in real time or in batches. For example, after the start of capturing a moving image, the captured images may be transmitted to the processing device 10 in real time while the image is being captured. Alternatively, after the start of capturing a moving image, and after the image capture is completed, the images captured during that time may be transmitted to the processing device 10.

[0063] The photographing may be performed without the person being aware that he or she is being photographed. For example, a supervisor photographs people in a location during a personnel counting operation. The person being monitored may be included among the people in the location. Alternatively, the photographing may be performed while the person being monitored is aware that he or she is being photographed. For example, during a personnel counting operation, multiple people being monitored gather in one place. At that time, the supervisor may photograph each person being monitored.

[0064] Alternatively, the determination unit 12 may acquire images captured by a camera installed at the location during personnel identification work, for example. The images captured by the camera are input to the processing device 10 by any means. For example, the camera and the processing device 10 may be configured to be able to communicate with each other. The camera may then transmit the captured images to the processing device 10. Note that this means is merely an example, and the images captured by the camera may also be input to the processing device 10 by using other means that are manually implemented by a person such as a supervisor.

[0065] After S30, the determination unit 12 determines whether there is a person under surveillance who is not captured in the confirmation target image (S31). If all of the multiple persons under surveillance are captured in any of the confirmation target images, the determination unit 12 determines that there is no person under surveillance who is not captured in the confirmation target image. On the other hand, if at least one of the multiple persons under surveillance is not captured in any of the confirmation target images, the determination unit 12 determines that there is a person under surveillance who is not captured in the confirmation target image.

[0066] The determination unit 12 may determine whether each person to be monitored is captured in the confirmation target image by comparing the face information of the person to be monitored registered in the storage unit 13 with the person captured in the confirmation target image. Alternatively, the determination unit 12 may determine whether each person to be monitored is captured in the confirmation target image by comparing the appearance information of the person to be monitored registered in the storage unit 13 with the person captured in the confirmation target image.

[0067] If there is a person to be monitored who is not in the confirmation target image (Yes in S31), the determination unit 12 outputs appearance information of the person to be monitored who is not in the confirmation target image (S32). The determination unit 12 acquires the appearance information of the person to be monitored from the storage unit 13. Then, the determination unit 12 transmits the acquired appearance information to the supervisor's mobile terminal 2, and can display it on the display of the mobile terminal 2.

[0068] FIG. 8 shows an example of information displayed on the display of the mobile terminal 2. In the example of FIG. 8, an appearance image is displayed, which is appearance information of a person to be monitored who is not in the confirmation target image. In addition, in the example of FIG. 8, the name of the person to be monitored is displayed. Based on this information, a supervisor can recognize a person to be monitored who is not in a certain location during personnel identification work. Based on this information, the supervisor can also recognize the condition and belongings of the person to be monitored who is not in a certain location on that day. Then, based on the information recognized in this way, the supervisor can search for the person to be monitored who is not in a certain location.

[0069] Although not shown in Fig. 8, the determination unit 12 may further display contact information for the monitored person who is not at the location. In this case, the supervisor can recognize the contact information for the monitored person who is not at the location based on the information and contact the monitored person by means of telephone or the like.

[0070] 8, the appearance information of one monitored person who is not at the location is displayed, but the appearance information of multiple monitored people who are not at the location may also be displayed. The appearance information of multiple monitored people who are not at the location can be displayed in various ways, such as a list display or by switching between displaying each person one by one.

[0071] Returning to FIG. 7 , if there are no monitored persons not appearing in the confirmation target image (No in S31), the determination unit 12 terminates the process without executing the process of S32. Although not shown, the determination unit 12 may execute a process different from the process of S32 when the result of S31 is No. For example, when the result of S31 is No, the determination unit 12 may output information indicating that all monitored persons appear in the confirmation target image. The determination unit 12 may transmit the information to the supervisor's mobile terminal 2 and display it on the display of the mobile terminal 2. Based on the process of the processing device 10 when the result of S31 is No and the operation of the mobile terminal 2 linked thereto (e.g., not executing the process of S32), the supervisor can recognize that all monitored persons appear in the confirmation target image. In other words, the supervisor can recognize that all monitored persons are present at the location during personnel identification work.

[0072] "Operational Effects" According to the processing apparatus 10 of the second embodiment, operational effects similar to those of the processing apparatus 10 of the first embodiment are realized.

[0073] Furthermore, the processing device 10 of the second embodiment can support personnel identification work based on images of the objects to be confirmed captured by the mobile terminal 2. The technology disclosed in Patent Document 1, which performs roll call based on images captured by a camera installed on a bus, is limited to use on buses. However, the processing device 10 of the second embodiment, which supports personnel identification work based on images of the objects to be confirmed captured by the mobile terminal 2, has fewer limitations on the locations where personnel identification work can be supported, and can be used in a variety of locations.

[0074] Furthermore, the processing device 10 of the second embodiment can register appearance information indicating the monitored person and their accessories on the day of the group activity. The processing device 10 can output the appearance information of the monitored person when the monitored person is not captured in the confirmation image (i.e., when the monitored person is not present at the location during the personnel identification operation). Based on the output appearance information, the supervisor can recognize the condition (hairstyle, makeup, etc.) and accessories of the monitored person who is not present at the location on that day. Based on the recognized information, the supervisor can then search for the monitored person who is not present at the location. The appearance of the monitored person may change due to hairstyle, makeup, accessories, etc. Information about the monitored person and their accessories registered on the day of the group activity improves the accuracy of searching for the monitored person on the day of the group activity.

[0075] <Third Embodiment> In a task of identifying people in a group, if there is a person to be monitored who is not captured in the confirmation target image, i.e., if there is a person to be monitored who is not at the location, the supervisor will search for that person. The processing device 10 of the third embodiment has a function to support the search for this person to be monitored. This will be described in detail below.

[0076] If there is a person to be monitored who is not captured in the confirmation target image, the determination unit 12 executes a process to assist in searching for that person to be monitored. Hereinafter, a "person to be monitored who is not captured in the confirmation target image" will be referred to as a "person to be searched for."

[0077] Specifically, the determination unit 12 determines whether the person to be searched for is included in the search processed image, and outputs the determination result.

[0078] A "search processing image" is, for example, an image taken with the portable terminal 2 carried by a supervisor of a group activity. In this case, the supervisor operates the portable terminal 2 to take a picture of the location being searched and transmits the captured image to the processing device 10 as a search processing image. An example of this process will be described below, but is not limited to this example. For example, the supervisor operates the portable terminal 2 to start a dedicated application and activates a search processing mode. In response to the activation of the search processing, the camera is activated and ready to capture images. In this mode, the supervisor takes pictures of people in the location being searched and transmits the captured images to the processing device 10.

[0079] The captured image may be one or more still images, or may be a moving image composed of multiple frame images. Each still image and / or each frame image becomes a search processing image. When a moving image is captured, the image may be transmitted to the processing device 10 in real time or in batch processing. For example, after the start of capturing a moving image, the captured images may be transmitted to the processing device 10 in real time while the capturing is being performed. Alternatively, after the start of capturing a moving image, and after the capturing is completed, the images captured during that time may be transmitted to the processing device 10.

[0080] Alternatively, the determination unit 12 may acquire images captured by a camera installed in the search location, for example. The images captured by the camera are input to the processing device 10 by any means. For example, the camera and the processing device 10 may be configured to be able to communicate with each other. The camera may then transmit the captured images to the processing device 10. Note that this means is merely an example, and the images captured by the camera may also be input to the processing device 10 by using other means that are manually implemented by a person such as a supervisor.

[0081] The "determination of whether or not the search target person is captured in the search processing image" is realized based on at least one of the facial information and appearance information of the surveillance target person registered in the storage unit 13. For example, the determination unit 12 may determine whether or not the search target person is captured in the search processing image by comparing the facial information of the surveillance target person registered in the storage unit 13 with the person captured in the search processing image. Alternatively, the determination unit 12 may determine whether or not the search target person is captured in the search processing image by comparing the appearance information of the surveillance target person registered in the storage unit 13 with the person captured in the search processing image.

[0082] The "output of the determination result" is realized via an output device such as a display, a speaker, or a vibrator. For example, when the determination unit 12 determines that the search target person is included in the search processing image, the determination unit 12 may display information indicating this on the display of the portable terminal 2. Alternatively, when the determination unit 12 determines that the search target person is included in the search processing image, the determination unit 12 may output a sound indicating this from the speaker of the portable terminal 2. Alternatively, when the determination unit 12 determines that the search target person is included in the search processing image, the determination unit 12 may cause a vibrator of the portable terminal 2 to vibrate.

[0083] The above-described search processing mode may be activated from, for example, a screen such as that shown in Fig. 9. The screen shown in Fig. 9 is a modified example of the "screen (Fig. 8) displayed on the supervisor's mobile terminal 2 when there is a monitored person who is not captured in the confirmation target image (Yes in S31)" described in the second embodiment. The screen shown in Fig. 9 has a "Search" button. For example, the search processing mode may be activated by pressing this "Search" button.

[0084] Here, the difference between the processing of the determination unit 12 in the support process for identifying people during group activity described in the second embodiment and the processing of the determination unit 12 in the search process will be described. In the processing of the support process for identifying people during group activity, all monitored persons are searched for within the image. Therefore, the information to be compared with the confirmation target image is information (face information and appearance information) of all monitored persons. On the other hand, in the processing of the determination unit 12 in the search process, the monitored persons are searched for within the image. Therefore, the information to be compared with the search process image is limited to the information of the monitored person. That is, in the search process, the determination unit 12 uses the information of the monitored person as reference information (the information to be compared with the search process image) to search for the monitored person within the search process image. In the search process, the determination unit 12 does not need to use the information of monitored persons other than the monitored person as reference information (the information to be compared with the search process image). Note that the number of monitored persons searched for at one time in the search process may be one or more. That is, the object to be searched for in the image during the search process may be one person under surveillance or multiple persons under surveillance, and the supervisor may designate at least one person under surveillance as a person under search by any means.

[0085] Other configurations of the processing apparatus 10 of the third embodiment are similar to those of the processing apparatus 10 of the first and second embodiments.

[0086] According to the processing apparatus 10 of the third embodiment, the same effects as those of the processing apparatus 10 of the first and second embodiments are achieved.

[0087] Furthermore, the processing device 10 of the third embodiment has a function to assist in the search for a person to be monitored who is not captured in the confirmation target image, i.e., a person to be monitored who is not present at the location during the personnel identification operation. Specifically, the processing device 10 searches for the person to be monitored within the captured image and outputs the search results. A supervisor using this function can, for example, visually search for the person to be monitored, while taking a picture of a location that the supervisor is not looking at with the mobile device 2 and have the processing device 10 execute a search at that location. With this processing device 10, a search for a person to be monitored who is not present at the location during the personnel identification operation can be efficiently performed. As a result, the processing device 10 of the third embodiment improves the efficiency of the personnel identification operation.

[0088] The processing device 10 of the fourth embodiment has a function of notifying a person to be monitored whose appearance information has not been registered at a predetermined timing, as will be described in detail below.

[0089] The registration unit 11 determines whether appearance information is registered for at least one person to be monitored who has been registered in advance in the advance registration process described in the second embodiment. This determination can be realized by checking whether appearance information is registered in association with each person to be monitored in the person to be monitored information as shown in FIG. 5, for example.

[0090] If there is a person to be monitored whose appearance information has not been registered at a predetermined timing, the registration unit 11 performs a notification process.

[0091] The "notification process" is performed via the supervisor's mobile terminal 2. For example, the registration unit 11 performs the notification process by means of push notification or the like via a dedicated application installed on the supervisor's mobile terminal 2. In the notification process, a notification is made that there is a monitored person whose appearance information is not registered. In addition, in the notification process, information indicating the monitored person whose appearance information is not registered may be notified. The information is information registered in the advance registration process, such as name, contact information, and face information.

[0092] The "predetermined timing" is a predetermined time or a timing when the user makes a predetermined input.

[0093] The "predetermined time" is a time set in advance by the supervisor. The supervisor can set, for example, the departure time on the day of the group activity, or a predetermined time before the departure time.

[0094] The "predetermined input" is an input that causes the processing device 10 to execute a process of determining whether there is a person to be monitored whose appearance information is not registered. For example, a supervisor operates the mobile terminal 2 to start a dedicated application and then performs the predetermined input.

[0095] Here, an example of the flow of processing executed by the registration unit 11 will be described with reference to the flowchart of FIG.

[0096] The registration unit 11 is in a waiting state for a predetermined timing (S40). Then, when the predetermined timing arrives (Yes in S40), the registration unit 11 determines whether there is a person to be monitored whose appearance information has not been registered at that timing (S41).

[0097] If there is a person to be monitored whose appearance information is not registered (Yes in S41), the registration unit 11 performs a notification process (S42). That is, the registration unit 11 notifies that there is a person to be monitored whose appearance information is not registered. The registration unit 11 may further notify information indicating the person to be monitored whose appearance information is not registered. The registration unit 11 can display, for example, information such as that shown in FIG. 11 on the display of the supervisor's mobile terminal 2.

[0098] Other configurations of the processing apparatus 10 of the fourth embodiment are similar to those of the processing apparatus 10 of the first to third embodiments.

[0099] According to the processing apparatus 10 of the fourth embodiment, the same effects as those of the processing apparatus 10 of the first to third embodiments are realized.

[0100] Furthermore, the processing device 10 of the fourth embodiment can notify a person to be monitored whose appearance information has not been registered at a predetermined timing. With this processing device 10, the supervisor can register the appearance information of the person to be monitored without forgetting.

[0101] The predetermined timing can be a timing designated by the supervisor. With this processing device 10, the supervisor can be notified at an appropriate time that there is a person to be monitored whose appearance information is not registered.

[0102] Furthermore, according to the processing device 10 of the fourth embodiment, the registration of appearance information can be used to grasp the number of people at the initial gathering before the start of the group activity on the day of the group activity. That is, the supervisor can recognize whether each monitored person is gathering at a location based on whether appearance information is registered. Then, based on the "notification of the presence of monitored persons whose appearance information is not registered" by the processing device 10, the supervisor can recognize that there are monitored persons who have not gathered at the gathering location. Furthermore, based on the "notification of information indicating monitored persons whose appearance information is not registered" by the processing device 10, the supervisor can recognize that there are monitored persons who have not gathered at the gathering location.

[0103] Fifth Embodiment Group activity may occur over multiple days. For example, a group may stay at the same accommodation facility and engage in group activity over multiple days. The appearance of a monitored person may change due to hairstyle, makeup, accessories, etc. The hairstyle, makeup, accessories, etc. of a monitored person may differ from day to day. Therefore, when group activity occurs over multiple days, the processing device 10 of the fifth embodiment registers appearance information of the monitored person for each day. This will be described in detail below.

[0104] When the days on which the group activity (days on which the monitored person is monitored) span multiple days, the registration unit 11 registers appearance information indicating the monitored person and accessories of the monitored person for each day. The registration of appearance information for each day is realized by processing similar to the registration processing for the day of the group activity described in the second embodiment. For example, processing similar to the registration processing for the day of the group activity described in the second embodiment is executed at the first gathering of each day.

[0105] The registration unit 11 may, for example, register appearance information by linking it to the registration date. This configuration makes it possible to recognize whether appearance information for each day has been registered. Additionally, the registration unit 11 may perform a reset process at a predetermined timing after the next day, in order to erase the appearance information for the previous day that has been registered at that time.

[0106] In the process of supporting work to identify people in group activities, if the person to be monitored is not captured in the image to be confirmed, the determination unit 12 outputs the appearance information of the person to be monitored that was registered on that day. Here, "that day" refers to the day on which it was determined that the person to be monitored was not captured in the image to be confirmed.

[0107] The processing device 10 of the fifth embodiment may also have the configuration of the processing device 10 of the fourth embodiment. In this case, if there is a monitored person whose appearance information has not been registered at a predetermined timing on the day to which the predetermined timing belongs, the registration unit 11 notifies that fact.

[0108] Other configurations of the processing apparatus 10 of the fifth embodiment are similar to those of the processing apparatus 10 of the first to fourth embodiments.

[0109] According to the processing apparatus 10 of the fifth embodiment, the same effects as those of the processing apparatus 10 of the first to fourth embodiments are achieved.

[0110] Furthermore, the processing device 10 of the fifth embodiment can register appearance information for each day when group activity spans multiple days. With this processing device 10, even when group activity spans multiple days, it is possible to output the appearance information for that day of a person to be monitored who is not at the location during the personnel identification operation. With this processing device 10, a search for a person to be monitored who is not at the location during the personnel identification operation can be efficiently performed each day. As a result, the processing device 10 of the fifth embodiment improves the efficiency of the personnel identification operation.

[0111] Furthermore, according to the processing device 10 of the fifth embodiment, which has the configuration of the processing device 10 of the fourth embodiment, the registration of appearance information for each day can be used to grasp the number of people at the first gathering before the start of group activities each day.

[0112] Here, a modified example of this embodiment will be described. For example, a group activity may occur in multiple time slots in a day, such as a morning session and an afternoon session. The appearance of the monitored person may change depending on the time slot. In this case, the processing device 10 may register appearance information of the monitored person for each of the multiple time slots in a day.

[0113] <Sixth embodiment> As explained in the first to fifth embodiments, if appearance information indicating the person to be monitored and the person's accessories on the day of group activity is registered in the processing device 10, the efficiency of personnel identification work is improved.

[0114] However, if inappropriate appearance information is registered, the efficiency of personnel identification work will not improve. Also, if inappropriate appearance information is registered, it may cause misunderstandings in the supervisor's recognition of the person being monitored, which may worsen the efficiency of personnel identification work.

[0115] Inappropriate appearance information is, for example, appearance information that does not show the monitored person and their accessories on the day of the group activity. Other inappropriate appearance information is, for example, appearance information that shows content different from the monitored person and their accessories on the day of the group activity. Specifically, an image that does not show the entire body may be inappropriate appearance information. Furthermore, an image taken on a day other than the day of the group activity may be inappropriate appearance information.

[0116] The processing device 10 of the sixth embodiment has a function to prevent the inconvenience of registering such inappropriate appearance information, as will be described in detail below.

[0117] When the registration unit 11 acquires a registration target image, it determines whether the registration target image satisfies the registration conditions for registering it as appearance information.

[0118] The registration conditions are, for example, any of the following conditions, or a combination of the following conditions connected by an AND: The image to be registered is an image taken on the same day; The image to be registered includes the entire body of the person to be monitored.

[0119] The registration unit 11 can determine whether the registration target image was taken on the same day, for example, based on the metadata of the registration target image. The metadata includes the date of taking the image. Furthermore, the registration unit 11 can determine whether the entire body of the person being monitored is captured in the registration target image, based on well-known image analysis technology.

[0120] If the registration conditions are met, the registration unit 11 registers the registration target image as appearance information.

[0121] On the other hand, if the registration conditions are not met, the registration unit 11 does not register the registration target image as appearance information. In this case, the registration unit 11 may notify the supervisor that the registration target image will not be registered as appearance information because the registration target image does not meet the registration conditions. For example, the registration unit 11 may notify the supervisor that "This registration target image cannot be registered as appearance information because it was not taken today" or "This registration target image cannot be registered as appearance information because it does not show the entire body of the person being monitored." The notification to the supervisor is realized via the mobile terminal 2 carried by the supervisor.

[0122] In this way, if the image to be registered was photographed on the same day, the registration unit 11 can register the image to be registered as appearance information. If the image to be registered was photographed on a day other than the same day, the registration unit 11 can perform notification processing without registering the image to be registered as appearance information.

[0123] Furthermore, if the registration target image shows the entire body of the person to be monitored, the registration unit 11 can register the registration target image as appearance information. If the registration target image does not show the entire body of the person to be monitored, the registration unit 11 can perform notification processing without registering the registration target image as appearance information.

[0124] Furthermore, if the image to be registered was taken on the same day and the entire body of the person to be monitored is captured in the image to be registered, the registration unit 11 can register the image to be registered as appearance information. If the image to be registered was taken on a day other than the same day and the image to be registered does not capture the entire body of the person to be monitored, the registration unit 11 can perform notification processing without registering the image to be registered as appearance information.

[0125] Other configurations of the processing apparatus 10 of the sixth embodiment are similar to those of the processing apparatus 10 of the first to fifth embodiments.

[0126] According to the processing apparatus 10 of the sixth embodiment, the same effects as those of the processing apparatuses 10 of the first to fifth embodiments are achieved.

[0127] Furthermore, the processing device 10 of the sixth embodiment can prevent the inconvenience of inappropriate appearance information being registered. According to the processing device 10 of the sixth embodiment, the efficiency of personnel identification work is improved.

[0128] <Modifications> The following describes modifications that can be applied to any of the processing apparatuses 10 according to the first to sixth embodiments. These modifications also achieve the same effects as the processing apparatuses 10 according to the first to sixth embodiments.

[0129] "Variation 1" In the first to sixth embodiments, the processing device 10 was the server 1 of a server-client system. In Variation 1, as shown in FIG. 12, a mobile terminal 2 carried by a supervisor functions as the processing device 10. The mobile terminal 2 may be a smartphone, tablet terminal, laptop computer, mobile phone, smart watch, or the like. The hardware configuration of such a mobile terminal 2 may be a widely known configuration.

[0130] In the advance registration process of the first modification, the monitored individual information as shown in Fig. 5 is registered in the server. Then, by the day of the group activity, the monitored individual information is input and stored in the supervisor's mobile terminal 2 by any means. For example, the supervisor can connect the mobile terminal 2 to the server via a wired and / or wireless connection and input (download, etc.) the monitored individual information into the mobile terminal 2.

[0131] As another example, in the pre-registration process of Modification 1, the monitored person information as shown in Fig. 5 may be registered in the supervisor's mobile terminal 2 from the beginning. That is, the monitored person information as shown in Fig. 5 does not need to be registered in the server. The supervisor operates his / her own mobile terminal 2 to input information about each monitored person into the mobile terminal 2 and store the information in the mobile terminal 2.

[0132] "Variation 2" In the first to sixth embodiments, facial information of the person to be monitored is registered in the pre-registration process. In Variation 2, other biometric information can be registered in the pre-registration process instead of or in addition to facial information. Examples of biometric information include, but are not limited to, iris, fingerprint, palm print, and voice print. In Variation 2, such biometric information is used to verify the identity of the person to be monitored.

[0133] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0134] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order. However, the order of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the drawings can be changed to the extent that the content is not affected. Furthermore, the above-described embodiments can be combined to the extent that the content is not contradictory.

[0135] Some or all of the above embodiments may be described as in the following supplementary notes, but are not limited to the following. 1. A program that causes a computer to function as: a registration means that registers appearance information indicating at least a person to be monitored and accessories of the person to be monitored in a storage means; and a determination means that outputs the appearance information of the person to be monitored if the person to be monitored is not captured in an image to be confirmed. 2. The program described in 1, wherein the determination means acquires the image to be confirmed captured with a mobile terminal. 3. The program described in 1 or 2, wherein the registration means registers the appearance information indicating the person to be monitored and accessories of the person to be monitored on a day on which the person to be monitored is monitored. 4. The program described in 3, wherein the registration means registers the appearance information indicating the person to be monitored and accessories of the person to be monitored for each day if the day on which the person to be monitored spans multiple days. 5. 6. The program according to any one of 1 to 4, wherein the registration means determines whether the appearance information of at least one monitored person who has been registered in advance is registered, and performs a notification process if there is a monitored person whose appearance information is not registered at a predetermined timing. 6. The program according to 5, wherein the predetermined timing is a predetermined time or a timing when the user makes a predetermined input. 7. The program according to 3 or 4, wherein the appearance information is an image, and the registration means acquires a registration target image that is to be registered as the appearance information, and registers the registration target image as the appearance information if the date the registration target image was taken is the current day, and performs a notification process if the date the registration target image was taken is not the current day. 8. The program according to any one of 1 to 7, wherein the accessories include at least one of an item worn by the monitored person and an item carried by the monitored person. 9. A processing device having: registration means for registering appearance information indicating at least the monitored person and accessories of the monitored person in a storage means; and determination means for outputting the appearance information of the monitored person if the monitored person is not shown in a confirmation target image. 10. 10. The processing device according to 9, wherein the determination means acquires the confirmation target image photographed by a mobile terminal.11. The processing device according to 9 or 10, wherein the registration means registers the appearance information indicating the person to be monitored and the accessories of the person to be monitored on a day when the person to be monitored is monitored. 12. The processing device according to 11, wherein the registration means registers the appearance information indicating the person to be monitored and the accessories of the person to be monitored for each day when the day when the person to be monitored is monitored spans multiple days. 13. The processing device according to any of 9 to 12, wherein the registration means determines whether the appearance information of at least one person to be monitored who has been registered in advance is registered, and performs notification processing if there is a person to be monitored whose appearance information is not registered at a predetermined timing. 14. The processing device according to 13, wherein the predetermined timing is a predetermined time, or a timing when a user makes a predetermined input. 15. 16. The processing device according to any one of 9 to 15, wherein the appearance information is an image, and the registration means acquires a registration target image that is an object to be registered as the appearance information, and if the date on which the registration target image was taken is the current day, registers the registration target image as the appearance information, and if the date on which the registration target image was taken is not the current day, performs notification processing. 17. A processing method in which one or more computers register appearance information indicating at least the person being monitored and the accessories of the person being monitored in a storage means, and if the person being monitored is not captured in a confirmation target image, outputs the appearance information of the person being monitored. 18. The processing method according to 17, wherein the one or more computers acquire the confirmation target image taken with a mobile terminal. 19. The processing method according to 17 or 18, wherein the one or more computers register the appearance information indicating the person being monitored and the accessories of the person being monitored on the day on which the person being monitored is monitored. 20. 19. The processing method according to claim 19, wherein the one or more computers, when monitoring the monitored person over multiple days, register the appearance information indicating the monitored person and the accessories of the monitored person for each day.21. The processing method according to any one of 17 to 20, wherein the one or more computers determine whether the appearance information of each of at least one of the monitored persons who has been registered in advance is registered, and perform a notification process if there is a monitored person whose appearance information is not registered at a predetermined timing. 22. The processing method according to 21, wherein the predetermined timing is a predetermined time or a timing when the user makes a predetermined input. 23. The processing method according to 19 or 20, wherein the appearance information is an image, and the one or more computers acquire a registration target image that is to be registered as the appearance information, and if the registration target image was taken on the same day, register the registration target image as the appearance information, and if the registration target image was taken on a date other than the same day, perform a notification process. 24. The processing method according to any one of 17 to 23, wherein the accessory includes at least one of an object worn by the monitored person and an object carried by the monitored person.

[0136] 10 Processing device 11 Registration unit 12 Determination unit 13 Storage unit 1A Processor 2A Memory 3A Input / output I / F 4A Peripheral circuit 5A Bus

Claims

1. Computer, a registration means for registering appearance information indicating at least a person to be monitored and an accessory of the person to be monitored in a storage means; a determination means for outputting the appearance information of the person to be monitored when the person to be monitored is not captured in the confirmation target image; A program that functions as a

2. The determination means The program according to claim 1 , wherein the confirmation target image is acquired by taking a photograph with a mobile terminal.

3. The registration means 3. The program according to claim 1, wherein the appearance information indicating the person to be monitored and the accessories of the person to be monitored on a day when the person to be monitored is monitored is registered.

4. The registration means The program according to claim 3, wherein when the days for monitoring the person to be monitored span multiple days, the appearance information indicating the person to be monitored and the accessories of the person to be monitored for each day is registered.

5. The registration means determining whether the appearance information of each of at least one of the monitoring target persons registered in advance is registered; 3. The program according to claim 1, wherein a notification process is performed when there is a person to be monitored whose appearance information is not registered at a predetermined timing.

6. The predetermined timing is 6. The program according to claim 5, wherein the timing is a predetermined time or when the user makes a predetermined input.

7. the appearance information is an image, The registration means acquiring a registration target image to be registered as the appearance information; If the image to be registered was taken on the same day, the image to be registered is registered as the appearance information; 4. The program according to claim 3, wherein a notification process is performed if the date of shooting of the image to be registered is not the same day.

8. 3. The program according to claim 1, wherein the attachment includes at least one of an object worn by the person to be monitored and an object carried by the person to be monitored.

9. a registration means for registering appearance information indicating at least a person to be monitored and an accessory of the person to be monitored in a storage means; a determination means for outputting the appearance information of the person to be monitored when the person to be monitored is not captured in the confirmation target image; A processing device having:

10. One or more computers registering appearance information indicating at least the person to be monitored and any accessories of the person to be monitored in a storage device; A processing method for outputting the appearance information of the person to be monitored when the person to be monitored is not captured in the confirmation target image.