Crew management device, crew management method, and crew management program

Abstract

The primary purpose is to provide a passenger management system that allows passengers who do not have their own user ID to board a vehicle, and manages those passengers using the user's user ID. [Solution] The crew management system 1 includes a crew information acquisition unit 41 that assigns a crew ID to each area 33 of a crew member 3 recognized from images 14 of cameras 12 and 13, a user information acquisition unit 42 that acquires the user ID of a user 5, and a crew information management unit 43 that associates the user ID with the crew ID. A companion information acquisition unit 44 is provided to acquire the number of companions 15 that have been pre-registered and associated with the user ID. The crew information management unit 43, when the number of accompanying persons 15 associated with the user ID is one or more, associates the user ID with a crew ID that was recognized before the user ID was associated with a crew ID, or after the user ID was associated with a crew ID, based on the number of accompanying persons 15.

Description

Technical Field

[0001] This invention relates to a passenger management device, a passenger management method, and a passenger management program.

Background Art

[0002] In a vehicle used by a plurality of users, for example, a driverless passenger vehicle, the user issues a user ID in advance and presents the user ID to the vehicle to board the vehicle (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When a user wants to board a vehicle with a companion, the companion may need to issue their own user ID and board the vehicle. On the other hand, when a companion without their own user ID boards the vehicle, the vehicle side cannot manage the companion using the companion's own user ID. Therefore, in a vehicle used by a plurality of users, a vehicle operation mechanism that allows a companion without their own user ID to board the vehicle and manages the boarded companion is desired.

[0005] An object of the present invention is to solve the problems related to companions.

Means for Solving the Problems

[0006] To address the above problems, the present invention provides a passenger management device for managing information on the passengers of a vehicle, comprising: a passenger information acquisition unit that assigns a passenger ID to each region in which a recognition target is recognized in an image taken inside or near the entrance to the vehicle; a user information acquisition unit that acquires the user ID of a terminal user who has applied to board the vehicle; a passenger information management unit that associates the user ID that has applied to board the vehicle with the passenger ID; and a companion information acquisition unit that acquires the number of companions accompanying the user who has been registered in advance in association with the user ID, wherein the passenger information management unit, when the number of companions associated with the user ID is one or more, associates the user ID with the passenger ID that was recognized before the user ID was associated with the passenger ID, or after the user ID was associated with the passenger ID, based on the number of companions. [Effects of the Invention]

[0007] With the above configuration, the present invention allows the passenger management device to allow a companion who does not have its own user ID to board the vehicle and manage the boarded companion using the user's user ID. [Brief explanation of the drawing]

[0008] [Figure 1] This is an overall configuration diagram of the crew management device according to this embodiment. [Figure 2A] This is a block diagram showing the configuration of the crew management system in Figure 1. [Figure 2B] This is a modified version of Figure 2A. [Figure 3] This diagram shows the reading status of the user ID by the terminal communication device, where (a) indicates a readable state and (b) indicates an unreadable state. [Figure 4] This diagram shows the state in which the user ID is acquired by the crew monitoring device. [Figure 5] (a) is a block diagram showing the internal configuration of the crew monitoring device, (b) is an example of the output from the monitoring unit, and (c) is a diagram showing the area detected in the image. [Figure 6]It is a block diagram showing the internal configuration of the storage unit of the occupant monitoring device. [Figure 7] It is a recording example of the occupant table. [Figure 8] It is a block diagram showing the internal configuration of the management unit. [Figure 9] It is a time chart showing the operation of the occupant management device. [Figure 10] It is a flowchart of the standby process of the user by the occupant monitoring device. [Figure 11] It is a flowchart of the notification process of the availability of acquiring the user ID. [Figure 12] It is a flowchart of the user ID acquisition process. [Figure 13] It is a flowchart of the temporary queue extraction process. [Figure 14] It is a flowchart showing the process of the specific part of the management unit. [Figure 15] It is a flowchart of the boarding determination logic. [Figure 16] (a) shows a state where no one is in the vehicle, and (b) shows a state where a new occupant is detected. [Figure 17] It is a flowchart of the movement determination logic. [Figure 18] It is a flowchart of the alighting determination logic. [Figure 19] It is a flowchart of the violation determination logic. [Figure 20] It is a time chart showing the operation of the occupant management device for violations.

Mode for Carrying Out the Invention

[0009] This embodiment will be described in detail with reference to FIGS. 1 to 20.

Examples

[0010] ? <Configuration>This example has the following configuration.

[0011] Figure 1 is an overall configuration diagram of the occupant management device 1 in this embodiment. The occupant management device 1 is a device that manages the occupants 3 of vehicle 2 by executing an occupant management method using an occupant management program, mainly using a computer equipped with a processor and memory, and constitutes the occupant management system of vehicle 2. In Figure 1, the upper side of vehicle 2, indicated by the arrow, is the front in the direction of travel.

[0012] Vehicle 2 is used, for example, for passenger services where multiple users 5 share a ride, and has multiple seats 4 inside. Vehicle 2 has a reception unit 8 at the entrance / exit 6 that acquires the user IDs of users 5. The user IDs acquired at the reception unit 8 are sent to the crew monitoring device 11, which monitors the crew 3, via a service control device 9 that controls various services performed by the crew management device 1.

[0013] The occupant monitoring device 11 is connected to a camera 12 installed inside the vehicle 2 and inputs images 14 of the interior of the vehicle taken by the camera 12. The occupant monitoring device 11 may also be connected to a camera 13 installed around the entrance / exit 6 and input images 14 of the interior and exterior of the vehicle around the entrance / exit 6 taken by the camera 13.

[0014] Vehicle 2 is, for example, a vehicle that can accommodate multiple users 5 who do not know each other. This embodiment mainly assumes an unmanned passenger vehicle. However, even if vehicle 2 is a manned passenger vehicle, the occupants 3 may be managed.

[0015] Crew 3 refers to those who are in, about to board, or have boarded vehicle 2. In this embodiment, Crew 3 refers to people detected by the crew monitoring device 11 from images 14 of cameras 12 and 13, and includes users 5 who use vehicle 2, and their companions 15. User 5 is a person who has their own user ID and intends to use or is using the passenger service provided by vehicle 2. When user 5 obtains a user ID in order to register in advance the use of vehicle 2, they register the number of companions 15. Hereafter, when companions 15 and user 5 are not distinguished, they will be referred to as Crew 3.

[0016] Companion 15 is a person who rides in vehicle 2 with user 5 and uses the passenger service without having a user ID. There may or may not be a limit on the number of companions 15. For example, in the case of a group of several people, one person who is issued a user ID becomes user 5, and the remaining members become companions 15 when they are registered as companions 15 when their user IDs are issued. In the case of a family with a parent and child, for example, the parent becomes user 5 when a user ID is issued, and the child becomes companion 15 upon registration.

[0017] In this way, allowing companions 15 eliminates the need for each member of a group or family to be issued a separate user ID, thus improving the convenience of passenger services. In this embodiment, the group of user 5 and companions 15 does not matter in terms of boarding or alighting, as long as they travel together and board or alight consecutively. For example, companion 15 may board or alight before or after user 5. User 5 may board or alight between companions 15 of the same group.

[0018] Seat 4 is an individual seat where one crew member 3 sits, and seats are installed in the vehicle in proportion to the passenger capacity. The configuration of seat 4 is not limited. For example, in Figure 1, crew member 3 consists of one group of two people: a passenger 5 seated in a middle row seat 4 and a companion 15 seated in a rear row seat 4, and another passenger 5 seated in a rear row seat 4. The entrance / exit 6 is an opening in the vehicle 2 through which people pass.

[0019] The user ID is associated with user 5 and is a permit that allows boarding vehicle 2. The user ID is mainly obtained by user 5 using their mobile device 16 to apply to the operating site's server via a public network connection, and is issued after permission is received from the operating site's server. The user ID and the number of accompanying persons 15 are stored in user 5's mobile device 16. The user ID and the number of accompanying persons 15 may also be sent from the operating site's server to user 5's mobile device 16 at any time by user 5's actions.

[0020] The user ID may be provided on a recording medium capable of displaying or recording the user ID, such as a paper ticket, magnetic ticket, magnetic card, or IC card, which is issued or registered at a counter or ticket machine. When a recording medium is used, the mobile terminal 16 shall be read as a recording medium. A paper ticket has a user ID printed on its surface. The printing is visual information printed or transferred onto the recording medium, consisting of letters, numbers, figures, symbols, codes (e.g., barcode, QR code®), or a combination of one or more of these. Magnetic tickets, magnetic cards, and IC cards store the user ID as data.

[0021] In this embodiment, the user ID and the number of accompanying persons 15 are described as being stored in user 5's mobile terminal 16. User 5's mobile terminal 16 may be a portable multi-functional communication terminal such as a smartphone or tablet.

[0022] The reception unit 8 is a terminal device that receives boarding requests from users 5 when boarding or alighting from vehicle 2, and reads the user ID and the number of accompanying persons 15 held by user 5. The reception unit 8 only needs to be able to read the user ID and the number of accompanying persons 15, and may be, for example, a terminal communication device 8a that performs short-range wireless communication with the user 5's mobile terminal 16. The reception unit 8 may also be an electromagnetic induction type user code reader 8b that reads the user ID code recorded on the mobile terminal 16 or the recording medium. The reception unit 8 may also be an optical reader that optically reads the display of the mobile terminal 16 or the surface of the recording medium.

[0023] The reception unit 8 may be contact-type or contactless. In this embodiment, a contact-type terminal communication device 8a is disclosed as an example of the reception unit 8. The reception unit 8 may operate independently, or it may be operated by a control signal from the service control device 9, the crew monitoring device 11, or a server at an external operating site. In this embodiment, the reception unit 8 operates in cooperation with the service control device 9 and the crew monitoring device 11.

[0024] The service control device 9 is connected between the reception unit 8 and the crew monitoring device 11, and transmits information between the reception unit 8 and the crew monitoring device 11. The service control device 9 may be configured as a computer equipped with a CPU and memory, for example. The service control device 9 may also be connected to the server at the operating site via a public network line. This allows the reception unit 8, the service control device 9, and the crew monitoring device 11 to transmit information to the server at the operating site and to be remotely operated from the server at the operating site as needed.

[0025] The crew monitoring device 11 is a main component of the crew management device 1. The crew monitoring device 11 may consist of a computer equipped with a CPU, GPU, and memory, for example. The crew monitoring device 11 may be mounted on the vehicle 2, or it may be installed outside the vehicle 2 (for example, on a server at the operating site) to remotely manage information on the crew 3 from the outside, or its functions may be divided and installed both in the vehicle 2 and externally. The CPU or GPU is sometimes referred to as a processor. In this embodiment, the crew monitoring device 11 is installed on the vehicle 2. Details of the crew monitoring device 11 will be described later.

[0026] Camera 12 photographs the interior of the vehicle and outputs images 14 taken inside the vehicle 2. Camera 12 may be installed one or more times inside the vehicle to photograph the condition of all seats 4. Camera 13 photographs the exterior of the vehicle 2 and the area around the entrance 6 and outputs images 14 taken near the outside entrance. Camera 13 is installed facing outwards from the entrance 6.

[0027] Image 14 from camera 12 is a video of the interior of the vehicle taken by camera 12, capturing the appearance of the occupant 3 inside the vehicle. Multiple images 14 from cameras 12 may be stitched together into a single image, or they may be switched between so that they follow each other. Image 14 from camera 13 is a video taken by camera 13 from outside the vehicle to the entrance / exit 6, mainly capturing the appearance of the user 5 and their companion 15 passing through the entrance / exit 6. Image 14 from camera 13 may be stitched together with image 14 from camera 12 into a single image, or they may be switched between so that they follow each other.

[0028] Figure 2A is a block diagram showing the configuration of the main components of the crew management device 1.

[0029] Vehicle 2 is equipped with a display 21 and a speaker 22. The display 21 and speaker 22 are installed in one or more locations inside the vehicle. The display 21 and speaker 22 may be installed for each individual occupant 3, for example, one at each seat 4.

[0030] Similar displays 21a and speakers 22a may be provided in the service provision device 23, as shown in the modified example in Figure 2B. The service provision device 23 is primarily used by the service control device 9 to provide various services in passenger services. The service provision device 23 is connected to the crew management device 1 by wire or wireless. The service provision device 23 may be installed at each seat 4. The service provision device 23 may be made into a portable terminal and temporarily lent to the accompanying person 15. In this case, the service provision device 23 may be lent to the accompanying person 15, for example, in advance or upon boarding, and collected upon disembarking.

[0031] The display 21b and speaker 22b can be provided in or around the reception area 8 for occupants 3 passing through the entrance / exit area 6. In this embodiment, as shown in Figure 3, the reception area 8 has the display 21b on its surface.

[0032] Displays 21, 21a, and 21b are either for displaying image 14 or for displaying visual information. Displays 21, 21a, and 21b may be, for example, liquid crystal panels, organic EL panels, or means of displaying visual information using light sources such as LEDs. Visual information may be, for example, at least one of characters, figures, marks, or colors. Visual information may be displayed in such a way that it appears to float when light is emitted from the light source. Speakers 22, 22a, and 22b can provide necessary guidance and warnings through voice, beeps, etc.

[0033] The crew monitoring device 11 comprises, as functional units, a monitoring unit 24, a management unit 25, and a storage unit 26.

[0034] The monitoring unit 24 is an image processing unit that monitors the crew member 3 by inputting images 14 from cameras 12 and 13 and performing image recognition. The monitoring unit 24 outputs the recognition results, which are the result of the image recognition, to the management unit 25. The recognition results from the monitoring unit 24 are stored in the storage unit 26 and used by the management unit 25.

[0035] The management unit 25 manages the information of the crew member 3. The management unit 25 manages the information of the crew member 3 using the recognition results of the monitoring unit 24 and the information from the reception unit 8 to the service control device 9. The management unit 25 may also be able to use information from the server of the operating site. The management unit 25 can send notifications such as guidance and warnings to the crew member 3 directly or via the service control device 9. In this case, the management unit 25 can send notifications as appropriate to the inside of the vehicle, the displays 21, 21b of the reception unit 8, the speakers 22, 22b, the mobile terminal 16 of the user 5, and the service provision device 23 of the accompanying person 15.

[0036] The memory unit 26 stores various data and information necessary for the crew management device 1 to manage the crew 3. The memory unit 26 is installed in the crew monitoring device 11 and is mainly used by the crew monitoring device 11, but it can also be used by the service control device 9, the reception unit 8, or the server at the operation site. The memory unit 26 may not be installed in the crew monitoring device 11, but at least a portion of the storage device provided in the service control device 9, the reception unit 8, or the server at the operation site may be used.

[0037] The mobile terminal 16 of user 5 in Figure 2A comprises a computing unit 28 that executes the installed application 27, a communication unit 29 that communicates with the reception unit 8 and the server of the operating site, a display 31, and a speaker 32.

[0038] App 27 is executed on the processing unit 28, communicates with the server of the operating site to issue a user ID, and registers the accompanying person 15 with the server. App 27 receives warnings from the crew monitoring device 11 directly or via the service control device 9, and displays the warning on the display 31 or generates a warning sound in the speaker 32. In this embodiment, App 27 is for an unmanned passenger vehicle. The processing unit 28 consists of a computer equipped with a CPU and memory.

[0039] The communication unit 29 accesses the server of the operating site via a public network such as the internet when user 5 issues a user ID and registers the number of accompanying persons 15. The communication unit 29 causes the reception unit 8 to read the user ID and the number of accompanying persons 15. The communication unit 29 may communicate with the reception unit 8 via short-range wireless communication, or it may be equipped with an antenna that receives the magnetic field generated by the electromagnetic induction type user code reader 8b.

[0040] The display 31 is equipped with a touch panel for receiving input from the user. The display 31 may also display the user ID and the number of accompanying persons 15, which can be optically read by the optical reader of the reception unit 8. The display 31 and speaker 32 may be used, like the service provision device 23, to provide various services (e.g., guidance, warnings) by the service control device 9.

[0041] The service provider 23 in Figure 2B may include a calculation unit 28a and a communication unit 29a, similar to the mobile terminal 16. The service provider 23 is a terminal that does not have the function of issuing a user ID, registering the number of accompanying persons 15, or causing the reception unit 8 to read the user ID and the number of accompanying persons 15. The service provider 23 can receive warnings from the management unit 25 directly or via the service control device 9 via wired or wireless connection, display them on the display 21a, and generate a warning sound from the speaker 22a.

[0042] As shown in Figure 3, user 5 applies to board vehicle 2 by presenting their mobile device 16 to the reception desk 8 upon boarding. Presenting the mobile device 16 to the terminal communication device 8a, which is an example of the reception desk 8, is done by touching or bringing the mobile device 16 close to the terminal communication device 8a. Hereafter, the act of the mobile device 16 touching or coming close to the terminal communication device 8a may be referred to as "touching".

[0043] As shown in Figure 4, the reception unit 8 reads terminal information, a pre-issued user ID, and the number of accompanying persons 15 through contact with the mobile terminal 16 that the user 5 has used to request to board the vehicle 2, and transmits this information to the service control device 9. The terminal information is unique to the mobile terminal 16 and is information that uniquely identifies the mobile terminal 16 that has been contacted.

[0044] The service control device 9 stores in its internal storage device 9a the terminal information corresponding to the mobile terminal 16 that the user 5 received from the reception unit 8 to request to board vehicle 2, along with the user ID issued in advance. The service control device 9 transmits the user ID that requested to board vehicle 2 and the number of accompanying persons 15 to the crew monitoring device 11.

[0045] In the crew monitoring device 11, as shown in Figures 5(a) to (c), the monitoring unit 24 performs image recognition processing on crew members 3 captured in the images 14 of cameras 12 and 13. For each recognized crew member 3, the monitoring unit 24 outputs a rectangular area 33 indicating the position of that crew member 3 within the image 14, along with an area number #, to the management unit 25. Note that the area number # is different from the crew member ID, which will be described later. If there are as many images 14 as there are cameras 12, the area 33 contains information that identifies which camera 12 corresponds to the image 14.

[0046] The monitoring unit 24 identifies various actions of crew member 3 captured in the images 14 of cameras 12 and 13 as events. In addition to the region number # and region 33, the monitoring unit 24 outputs the necessary event information to the management unit 25, as shown in the output example in Figure 5(b). The event information may include violation information such as the smoking status of each crew member 3.

[0047] The management unit 25 takes the output from the monitoring unit 24, adds the boarding detection time and the passenger ID as boarding time, creates a temporary queue for boarding detection information, and stores it in the storage unit 26 as "detection information." The boarding detection time is the reception time when the monitoring unit 24 receives the recognition result used for boarding detection, or the passenger monitoring time when the monitoring unit 24 detects passenger 3 and begins monitoring.

[0048] A temporary queue is, as the name suggests, a temporary queue. A queue is a data structure that stores data in a single line in the order it is received, and processes data using a first-in, first-out (FIFO) rule. In addition to boarding detection information, detection information also includes, for example, alighting detection information, which is created in much the same way as boarding detection information and stored in the temporary queue as "detection information".

[0049] As shown in Figure 6, the memory unit 26 includes at least a memory area 26a for temporarily storing the user ID described above, and a memory area 26b for temporarily storing the number of accompanying persons 15. The memory unit 26 also includes a memory area 26c for storing the temporary queue described above, and a memory area 26d for storing the passenger table. The memory unit 26 also includes a memory area 26e for holding a user ID acquisition flag (described later), a memory area 26f for holding a detection counter, and a memory area 26g for holding a passenger counter.

[0050] The User ID Acquisition Available Flag is primarily used for the waiting process of the reception unit 8, as described later. The Detection Counter counts the number of passengers 3 who pass through the entrance / exit unit 6. The Detection Counter is primarily used for acquiring User IDs when boarding and alighting. The Boarding Counter is used as the assignment value when passenger IDs are assigned to passengers 5 and their companions 15.

[0051] As shown in Figure 7, the passenger table described above includes a data area that stores the user ID, boarding time, passenger ID, and area 33. However, the passenger table may also include other data areas. The boarding time is the boarding request time, which is the time when the management unit 25 receives the user ID from the service control device 9, or the terminal contact time, which is the time when the mobile terminal 16 is brought into contact with the terminal communication device 8a, which is an example of the reception unit 8. The passenger table is recorded for each user ID, and each record contains data for the total number of passengers in the group, including user 5 and accompanying persons 15. In the passenger table of Figure 7, two records are recorded: one for user 5 with one accompanying person 15, corresponding to user ID 12345, and another for user 5 without an accompanying person 15, corresponding to user ID 54321, which is different from the above.

[0052] As shown in Figure 8, the management unit 25 of the crew monitoring device 11 comprises a table management unit 34 and an identification unit 35. The table management unit 34 creates and manages records in the crew table. The identification unit 35 monitors the actions of crew members 3 using the recognition results of image recognition by the monitoring unit 24 and performs identification processing to identify the crew member 3 who caused the event.

[0053] When creating a crew table, the table management unit 34 performs the reception unit 8's waiting process for user 5, and notifies the reception unit 8 or the display 21b around the reception unit 8 whether or not the user ID can be obtained. The table management unit 34 also notifies user 5's mobile terminal 16 whether or not the user ID can be obtained, and performs the user ID acquisition process for the management unit 25.

[0054] The identification unit 35 performs identification processing, including boarding detection processing, movement detection processing, disembarking detection processing, and violation detection processing. The creation of the passenger table and the identification processing can be performed in parallel, and the processing results can be used interchangeably.

[0055] As shown in Figure 8, the management unit 25 includes, as functional units, a crew information acquisition unit 41, a user information acquisition unit 42, a crew information management unit 43, and a companion information acquisition unit 44.

[0056] The occupant information acquisition unit 41 assigns an occupant ID to each region 33 in which a recognition target is recognized in the image 14 taken inside or near the entrance of the vehicle 2. The user information acquisition unit 42 acquires the user ID of user 5 from the terminal that applied to board the vehicle 2. The occupant information management unit 43 associates the user ID that applied to board the vehicle 2 with the occupant ID. The companion information acquisition unit 44 acquires the number of companions 15 accompanying user 5 that have been pre-registered and associated with the user ID. When the number of companions 15 associated with the user ID is 1 or more, the occupant information management unit 43 associates the user ID that applied to board the vehicle 2 with the occupant ID that was recognized before the user ID was associated with the occupant ID, or after the user ID was associated with the occupant ID, based on the number of companions 15.

[0057] As described above, the management unit 25 manages information on the occupants 3 currently in vehicle 2, the occupants 3 about to board, and the occupants 3 who were previously in vehicle 2. The images 14 from cameras 12 and 13 are images of the interior and exterior of the vehicle, including the area around the entrance / exit 6. The occupants 3 are the targets of image recognition. The area 33 is the rectangular area occupied by the targets of image recognition by the monitoring unit 24. The occupant information acquisition unit 41 assigns occupant IDs to the user 5 and accompanying persons 15 who are occupants 3.

[0058] The user information acquisition unit 42 acquires the user ID of user 5 who has applied to board vehicle 2. The crew information management unit 43 plays a central role in the management unit 25 by associating crew IDs with user IDs. When the user information acquisition unit 42 acquires the user ID, the companion information acquisition unit 44 acquires the number of companions 15 from user 5's mobile terminal 16 or recording medium via the reception unit 8 or service control device 9.

[0059] "When the number of accompanying persons 15 is 1 or more" means that there are registered accompanying persons 15. "Associating user IDs based on the number of accompanying persons 15" means that the same user ID is assigned to the total number of passenger IDs for the 5 users and 15 accompanying persons belonging to the same group. "Before the user ID is associated with the passenger ID, or after the user ID is associated with the passenger ID" means that the order of boarding does not matter.

[0060] As long as the number of passengers 3 recognized based on the images 14 taken inside or near the entrance of vehicle 2 is less than the number of accompanying persons 15 corresponding to the user ID plus 5 users, or greater than the number of accompanying persons 15 plus 5 users, the passenger information management unit 43 will not associate a different user ID with a passenger 3 recognized based on the images 14 taken inside or near the entrance of vehicle 2, even if the user information acquisition unit 42 acquires a different user ID.

[0061] "While the number of passengers 3 recognized based on the image 14 taken inside or near the entrance of vehicle 2 is less than the number of accompanying persons 15 corresponding to the user ID plus the number of users 5, or greater than the number of accompanying persons 15 plus the number of users 5" is when the number of passengers 3 detected by image recognition does not match the total number of people in the group of users 5 obtained from the reception unit 8.

[0062] A different user ID refers to the user ID of a different user 5, distinct from the user 5 whose user ID was obtained earlier. The statement "Do not associate a different user ID with the crew member 3 recognized from the images 14 of cameras 12 and 13" means that the crew member ID will not be associated with the other user 5 until the association has been completed for all members of the group of user 5 whose user ID was obtained earlier. This prevents the other user 5 from cutting in line to board.

[0063] If the number of passengers 3 recognized based on the image 14 taken inside or near the entrance of vehicle 2 is less than the number of accompanying persons 15 corresponding to the user ID plus the number of users 5, or greater than the number of accompanying persons 15 plus the number of users 5, and the user information acquisition unit 42 acquires a user ID different from the user ID, the processing unit 46 sends a notification to the user 5 with the different user ID. The processing unit 46 also sends a notification to the user 5 corresponding to the user ID that applied to board vehicle 2.

[0064] The management unit 25 processes notifications to terminals and sends notifications to the mobile terminals 16 of users 5 and others. Firstly, the notification is sent to the mobile terminal 16 of another user 5 with a different user ID. Secondly, the notification may also be sent to user 5 whose user ID was read earlier, for the purpose of raising awareness. Notifications to other users 5 and user 5's mobile terminal 16 may be made by the user information acquisition unit 42 or the crew information management unit 43 via the service control device 9 that stores terminal information.

[0065] The management unit 25 further includes a notification unit 45 that notifies the user 5 in a discerning manner whether the user information acquisition unit 42 is able to accept the user ID. The notification unit 45 notifies the user that the user ID cannot be accepted as long as the number of crew members 3 recognized from the images 14 of the cameras 12 and 13 is less than the number of accompanying persons 15 corresponding to the user ID plus the user 5, or greater than the number of accompanying persons 15 plus the user 5.

[0066] The notification unit 45 processes a notification regarding whether or not the user ID can be obtained and sends a notification to the reception unit 8 or a display 21b or speaker 22b around the reception unit 8. As a result, the display 21b displays a message indicating that the user ID cannot be received. The speaker 22b may also provide an audio message indicating that the reception cannot be received. The notification unit 45 may also notify the mobile terminal 16 of the aforementioned user 5 and another user 5.

[0067] In the above, the user waiting process is mainly performed by the user information acquisition unit 42, which works in conjunction with the service control device 9, controlling the reception unit 8. The notification process of whether or not the user ID can be acquired is performed by the notification unit 45 to the reception unit 8 or the display 21b around the reception unit 8. The notification process of whether or not the user ID can be acquired is performed by the management unit 25 or the notification unit 45 to the mobile terminal 16 of the user 5 who is applying to board the vehicle 2. The user ID acquisition process, which associates the user ID with the crew ID, is performed by the table management unit 34, mainly by the crew information management unit 43.

[0068] The management unit 25 further comprises a processing unit 46 and an identification unit 35 as functional units. The processing unit 46 sends a notification to the user 5 corresponding to the user ID. When the identification unit 35 acquires an event from a companion 15, it identifies the user ID corresponding to the companion 15 from whom the event was detected. The processing unit 46 sends a notification regarding the event to the terminal of the user 5 having the identified user ID.

[0069] The processing unit 46 does not need to notify for all events, but it will notify for certain events that may interfere with the operation of the vehicle 2 or cause inconvenience to other passengers 3 (for example, violations by passenger 3). From the standpoint of usability and privacy, it is preferable that notifications be made personally. Events by user 5 are notified directly to user 5's terminal (for example, mobile terminal 16). Events by companion 15 are notified to user 5's terminal so that user 5 can indirectly inform companion 15. The processing unit 46 transmits notifications via the service control device 9 which stores terminal information. In addition, events by companion 15 may be transmitted by the processing unit 46 directly or via the service control device 9 to the terminal functioning as a service provider 23 lent to companion 15, based on the passenger ID identified by the identification unit 35. The processing unit 46 may also perform a notification process to the terminal (for example, user 5's mobile terminal 16) regarding whether or not the user ID can be obtained.

[0070] An event is an action of the occupant 3 detected from the output of the monitoring unit 24, and includes boarding, moving, disembarking, and violations. A violation is, for example, various actions that violate the rules of operation of the passenger vehicle.

[0071] Violations include, for example, smoking, violent behavior in the vehicle, improper seating in seat 4 or any other seat, failure to wear a seat belt, and others. For example, an abnormality of occupant 3 due to illness, fainting, or other reasons is not a violation, but it is an event that can be detected from the output of the monitoring unit 24, just like a violation. For convenience, an abnormality of occupant 3 may be treated as part of a violation or as separate from a violation. An abnormality of occupant 3 can be interpreted as a passenger abnormality instead of a violation.

[0072] As shown in Figure 8, the management unit 25 is equipped with a boarding determination unit 51, a movement determination unit 52, a disembarking determination unit 53, and a violation determination unit 54 in the identification unit 35 so that boarding, movement, disembarking, and violations can be treated as events by the passenger 3. The boarding determination unit 51 performs the boarding determination process, the movement determination unit 52 performs the movement determination process, the disembarking determination unit 53 performs the disembarking determination process, and the violation determination unit 54 performs the violation determination process.

[0073] Furthermore, the table management unit 34 and the identification unit 35 may have overlapping functions without completely differentiating their functions. For example, the passenger information acquisition unit 41 may be configured using at least some of the functions of the boarding determination unit 51, movement determination unit 52, disembarking determination unit 53, and violation determination unit 54 of the identification unit 35. If the event to be notified is limited to violations, only the violation determination unit 54 may be made into the identification unit 35, and the boarding determination unit 51, movement determination unit 52, and disembarking determination unit 53 may be made into the passenger information acquisition unit 41. The notification unit 45 and the processing unit 46 may have their functions combined into one.

[0074] <Effect> The effect of this embodiment is as follows:

[0075] In this embodiment, the occupant management device 1 is a computer equipped with a processor and memory that executes the following occupant management method using an occupant management program. Specifically, the occupant management program causes the processor to execute an occupant information acquisition step of assigning an occupant ID to each region 33 in which a recognition target is recognized in an image 14 taken inside or near the entrance to the vehicle 2; a user information acquisition step of acquiring a user ID from a user 5 of a terminal that has applied to board the vehicle 2; an occupant information management step of associating the user ID that has applied to board the vehicle 2 with the occupant ID; and an accompanying person information acquisition step of acquiring the number of accompanying persons 15 who are accompanied by the user 5 and have been registered in advance in association with the user ID. In the occupant information management step, if the number of accompanying persons 15 associated with the user ID is 1 or more, the user ID is associated with an occupant ID that was recognized before the user ID was associated with the occupant ID, or after the user ID was associated with the occupant ID, based on the number of accompanying persons 15.

[0076] Specifically, the crew management device 1 operates as shown in the time chart in Figure 9. First, at time T1, user 5 obtains a user ID in advance using the application 27 on the mobile terminal 16 and registers the number of accompanying persons 15. The application 27 stores the user ID and the number of accompanying persons 15 from the operating site in the mobile terminal 16.

[0077] On the day of use, at time T2, user 5 touches their mobile terminal 16 to the reception unit 8 at the entrance / exit 6 of vehicle 2. At this time, the reception unit 8 is in standby mode under the control of the user information acquisition unit 42. During standby mode, the notification unit 45 informs user 5 whether it is possible to acquire a user ID, as shown in Figure 3(a), or whether it is not possible, as shown in Figure 3(b). The notification by the notification unit 45 is made using a light source such as a display 21b or LEDs in or around the reception unit 8.

[0078] This display allows user 5 to determine whether the contact is valid, that is, whether they can move from the "waiting process" to the "user ID acquisition process". At this time, the table management unit 34 or notification unit 45 of the management unit 25 may notify user 5 whether the contact is valid via the display 31 or speaker 32 of the mobile terminal 16. At the very least, it is preferable for the management unit 25 to notify user 5's mobile terminal 16 when the contact is invalid. The notification ensures that user 5 is reliably informed that boarding is permitted if the contact is valid, and that boarding is not permitted if the contact is invalid.

[0079] Meanwhile, at time T3, the crew management device 1 and the crew monitoring device 11 are activated when the vehicle 2 is powered on or after it is powered on, and begin monitoring the crew 3. At this time, the reception unit 8 is performing standby processing. Details of the standby processing will be described later.

[0080] When a user's mobile terminal 16 is brought into contact with or near a terminal communication device 8a, which is an example of a reception unit 8, the service control device 9 inputs the terminal information of the mobile terminal 16, the user ID, and the number of accompanying persons 15 obtained from the mobile terminal 16, and stores the terminal information and user ID in association. The service control device 9 outputs the user ID and the number of accompanying persons 15 to the user information acquisition unit 42.

[0081] At time T4, when the user ID and the number of accompanying persons 15 are input to the user information acquisition unit 42 of the crew monitoring device 11, the user information acquisition unit 42 stores the user ID and the boarding request time as the boarding time in the crew table. At time T5, the crew information acquisition unit 41 detects boarding from the output of the monitoring unit 24, which has processed the images 14 from cameras 12 and 13. At time T6, the crew information acquisition unit 41 stores the boarding detection information, which is the output of the monitoring unit 24, with the crew ID added to area 33, in the temporary queue of the storage unit 26. At time T7, the crew information management unit 43 retrieves the boarding ID and area 33 from the temporary queue, records them in the crew table, and deletes them from the temporary queue.

[0082] In the above, the crew monitoring device 11 primarily uses the user information acquisition unit 42 to cause the reception unit 8 to perform a waiting process for users 5, as shown in the flowchart in Figure 10. The "waiting process for users 5" is a process for accepting multiple users 5 in order. The reception unit 8 waits for users 5 to present their user IDs to the reception unit 8 through the waiting process.

[0083] In step S1, the process begins when power is turned on to the vehicle 2 and the occupant monitoring device 11. Upon power-on, the reception unit 8 awaits the user 5 to present their user ID, such as by touching their mobile terminal 16.

[0084] First, in step S2, the user information acquisition unit 42 turns on the user ID acquisition enabled flag. This allows the reception unit 8 to acquire the user ID. Acquiring the user ID corresponds to the reception and boarding permission process when boarding vehicle 2. The reception unit 8 uses the user ID acquisition enabled flag to prevent the simultaneous acquisition of multiple user IDs. The user ID acquisition enabled flag is initially set to ON.

[0085] In step S3, the notification unit 45 executes the "notification process for whether or not a user ID can be obtained" in parallel, which will be described later.

[0086] In step S4, the user information acquisition unit 42 determines whether the user's mobile terminal 16 has made contact with the reception unit 8. If there is no contact, i.e., step S4 is No, the process returns to step S4 and repeats the determination in step S4 until it is Yes. If there is contact, i.e., step S4 is Yes, the process proceeds to step S5.

[0087] In step S5, the user information acquisition unit 42 determines whether it is possible to acquire the user ID. As shown in Figure 3(b), if it is not possible to acquire it, i.e., if step S5 is No, the process proceeds to step S6.

[0088] In step S6, the management unit 25 sends a notification to the mobile terminal 16 that corresponds to a user ID different from the user ID currently being acquired.

[0089] In step S7, the management unit 25 sends a notification to the mobile terminal 16 corresponding to the user ID for which the user ID is currently being acquired. The notification in step S7 may be sent as needed. The notifications in steps S6 and S7 are sent by either the user information acquisition unit 42, the crew information management unit 43, or the notification unit 45. The process returns to step S4 and repeats the decision in step S4 until it becomes Yes. As shown in Figure 3(a), if acquisition is possible, i.e., if step S5 is Yes, the process proceeds to step S8.

[0090] In step S8, the management unit 25 turns OFF the user ID acquisition enabled flag. As a result, the reception unit 8 is unable to acquire the user ID.

[0091] In step S9, the user information acquisition unit 42 executes the user ID acquisition process in parallel. The user ID acquisition process, described later, is the process of having the crew monitoring device 11 acquire information such as the user ID presented by the user 5 to the reception unit 8 via the service control device 9. By executing this in parallel with the waiting process described above, the reception unit 8 becomes able to wait for another user 5 to present a different user ID. The process returns to step S4 and continues with the processing from step S4 onward.

[0092] The "notification process for obtaining a user ID" described above is carried out as shown in the flowchart in Figure 11.

[0093] The notification process for obtaining a user ID starts by branching off from the "user waiting process" in step S3 of Figure 10. First, in step S11, it is determined whether or not a user ID can be obtained. As shown in Figure 3(a), if the user ID acquisition flag is ON, i.e., if step S11 is Yes, step S12 notifies that the user ID acquisition process can proceed and returns to step S11. As shown in Figure 3(b), if the user ID acquisition flag is OFF, i.e., if step S11 is No, step S13 notifies that the user ID acquisition process cannot proceed and returns to step S11.

[0094] The notification is made by either the reception unit 8 or the display 21b or speaker 22b surrounding the reception unit 8. For example, the display 21b displays as shown in Figure 3(a) that the user ID acquisition process can proceed, and as shown in Figure 3(b) that the user ID acquisition process cannot proceed. However, the content displayed by the display 21b is not limited to these. In the case of Figure 3(a), the display 21b may use blue, green, or other reassuring or safe colors. In the case of Figure 3(b), the display 21b may use yellow, red, black, or other attention-grabbing or warning colors.

[0095] Step S9 in Figure 10, "Execute user ID acquisition process in parallel," is performed across the entire table management unit 34 of the management unit 25, as shown in the flowchart in Figure 12. The user ID acquisition process involves obtaining a user ID for each user 5 and, if there are accompanying persons 15, associating the obtained user ID with the accompanying person 15.

[0096] When processing begins, first, in step S21, the crew information acquisition unit 41 or the crew information management unit 43 sets the detection counter to 0. In step S22, the user information acquisition unit 42 obtains the user ID from the user's mobile terminal 16 via the reception unit 8 and the service control device 9.

[0097] The user information acquisition unit 42 of the crew monitoring device 11 records a record in the crew table that associates the user ID received from the service control device 9 with the boarding request time, and temporarily stores the number of accompanying persons 15 in the storage unit 26.

[0098] The user information acquisition unit 42 acquires the user ID when the mobile terminal 16 held by user 5 makes contact with the reception unit 8 and the user ID is available at the time of contact. The user information acquisition unit 42 does not acquire the user ID if the user ID is not available even when the mobile terminal 16 held by user 5 makes contact with the reception unit 8. However, if the monitoring unit 24 and the passenger information acquisition unit 41 detect passenger 3 before the user ID is acquired, that is, before user 5 boards the vehicle, the boarding detection information is already stored in the temporary queue of the storage unit 26. Therefore, past detection information in the temporary queue is retrospectively reflected in the passenger table. As a result, user 5 and their companion 15 can board or alight in any order within the group.

[0099] In step S23, the crew information management unit 43 determines whether the temporary queue is empty. If the temporary queue is empty, i.e., step S23 is No, the process proceeds to step S28. If the temporary queue is not empty, i.e., step S23 is Yes, in step S24, the crew information management unit 43 mainly performs the temporary queue retrieval process described later.

[0100] In step S25, the crew information management unit 43 determines whether the detection counter is 0 or the number of accompanying persons 15 plus 1, which is the number of users 5. A detection counter of 0 means that the boarding of crew member 3 has not been detected, and a detection counter of the number of accompanying persons 15 plus 1 means that the boarding of exactly the number of people in the group consisting of users 5 and accompanying persons 15 has been detected.

[0101] After recording the crew information in the crew table in step S22, the crew information management unit 43 enables the acquisition of other user IDs based on the image recognition results of the monitoring unit 24 and the number of accompanying persons 15 temporarily stored in the storage unit 26. The crew information management unit 43 enables the acquisition of other user IDs only when the detection counter is 0 or matches the total number of users 5 and accompanying persons 15. In other words, the crew information management unit 43 enables the acquisition of other user IDs only when no one has been detected, or when all members of the group of user 5 have been detected.

[0102] If step S25 is No, in step S26 the user information acquisition unit 42 or the crew information management unit 43 turns OFF the user ID acquisition possible flag and proceeds to step S28. As a result, the reception unit 8 is unable to acquire the user ID. If step S25 is Yes, in step S27 the user information acquisition unit 42 or the crew information management unit 43 turns ON the user ID acquisition possible flag and proceeds to step S28. As a result, the reception unit 8 is able to acquire the user ID.

[0103] In step S28, the user information acquisition unit 42 determines whether other user IDs have been acquired. If no other user IDs have been acquired, i.e., step S28 is No, the process returns to step S23 and repeats the process from step S23 onward. If other user IDs have been acquired, i.e., step S28 is Yes, in step S29, the companion information acquisition unit 44 or the crew information management unit 43 discards the information on the number of companions 15. When the acquisition of other user IDs begins, the crew information management unit 43 discards the number of companions 15 temporarily stored in the storage unit 26.

[0104] In step S30, if the detection counter is 0, the crew information management unit 43 deletes the record. That is, if the number of crew IDs in the crew table records becomes 0 due to disembarking, the crew information management unit 43 deletes that record from the crew table. The process ends.

[0105] The "temporary queue retrieval process" in step S24 of Figure 12 is performed as shown in the flowchart in Figure 13. When the process starts, first in step S31, the crew information management unit 43 retrieves the detection information recorded by the crew information acquisition unit 41 from the temporary queue of the storage unit 26.

[0106] In step S32, if boarding detection information is retrieved from the temporary queue as detection information, the crew information management unit 43 identifies the record in the crew table that has the boarding request time closest to the boarding detection time of user 5. For example, the crew information management unit 43 identifies the record of user ID obtained in step S22 of Figure 12. The crew information management unit 43 adds the crew ID and area 33 to the identified record and increments the detection counter by 1.

[0107] In step S33, if disembarkation detection information is retrieved from the temporary queue as detection information, the passenger information management unit 43 deletes the passenger ID and the area 33 corresponding to the passenger ID from the record of the acquired user ID in the passenger table and decrements the detection counter by 1. The temporary queue retrieval process is then completed.

[0108] The "event identification processing" by the identification unit 35 of the management unit 25 is performed as follows. The identification processing involves boarding determination, movement determination, disembarking determination, and violation determination using the recognition results of image recognition by the monitoring unit 24. The identification unit 35, together with the passenger information acquisition unit 41, performs the following image recognition processing as shown in the flowchart of Figure 14.

[0109] When processing begins upon power-on, the boarding determination unit 51 waits for the monitoring unit 24 to detect that a passenger 3 has boarded. In step S41, the boarding determination unit 51 executes the boarding determination logic. In step S42, the boarding determination unit 51 determines whether boarding has been detected. If boarding is not detected, i.e., step S42 is No, the process proceeds to step S44. If boarding is detected, i.e., step S42 is Yes, in step S43 the boarding determination unit 51 stores the boarding detection information in a temporary queue and proceeds to step S44.

[0110] When boarding is detected by the boarding detection logic described later, the boarding detection unit 51 creates a passenger ID for each boarding detection. The passenger ID becomes the value of the boarding counter. The initial value of the boarding counter is 0, and it increases by 1 each time boarding is detected. Unlike the detection counter, the boarding counter does not decrease when passengers disembark and is shared among multiple users 5. The boarding detection unit 51 associates each passenger ID, the region 33 in each detection, and the boarding detection time in each detection and stores them in a temporary queue as boarding detection information.

[0111] In step S44, the movement determination unit 52 executes the movement determination logic. In step S45, the movement determination unit 52 determines whether movement has been detected. If no movement is detected, i.e., step S45 is No, the unit proceeds to step S47. If movement is detected, i.e., step S45 is Yes, the movement determination unit 52 updates area 33 in step S46 and proceeds to step S47.

[0112] Based on the movement determination logic described later, the movement determination unit 52 detects the movement of the occupant 3 within the vehicle 2 and identifies the occupant ID of the occupant 3 that has moved. Based on the detected movement, the movement determination unit 52 updates the area 33 corresponding to the identified occupant ID.

[0113] In step S47, the disembarkation determination unit 53 executes the disembarkation determination logic. In step S48, the disembarkation determination unit 53 determines whether or not a disembarkation has been detected. If no disembarkation is detected, i.e., step S48 is No, the process proceeds to step S50. If disembarkation is detected, i.e., step S48 is Yes, in step S49 the disembarkation determination unit 53 decrements the detection counter by 1 and deletes the disembarkation detection information. Alternatively, the disembarkation determination unit 53 stores the disembarkation detection information in a temporary queue. The process then proceeds to step S50.

[0114] The detection counter is decremented by 1 to delete the disembarkation detection information if a passenger disembarks due to boarding at the wrong time when starting the passenger service. Disembarkation detection information is stored in a temporary queue when the passenger service has ended.

[0115] Based on the disembarkation determination logic described later, the disembarkation determination unit 53 detects that passenger 3 has disembarked and identifies the passenger ID. At this time, if the user ID acquisition process is not underway by the user information acquisition unit 42 and the passenger information management unit 43, the disembarkation determination unit 53 deletes the passenger ID and area 33 from the passenger table. If, as a result of the deletion, the number of passenger IDs in the records of the passenger table becomes 0, the passenger information management unit 43 deletes that record. If the user ID acquisition process is underway, the passenger information management unit 43 stores the passenger ID as disembarkation detection information in a temporary queue.

[0116] In step S50, the violation determination unit 54 executes the violation determination logic. In step S51, the violation determination unit 54 determines whether a violation has been detected. If step S51 is No, the process proceeds to step S41. If step S51 is Yes, in step S52 the processing unit 46 and the service control device 9 notify of the violation and the process proceeds to step S41.

[0117] The violation determination unit 54 detects a violation by occupant 3 within the vehicle 2 using the violation determination logic described later, and identifies the occupant ID of the occupant 3 who committed the violation. Based on the detected violation, the processing unit 46 notifies the mobile terminal 16 of user 5, which has a user ID corresponding to the identified occupant ID, of the violation via the service control device 9. Alternatively, the processing unit 46 notifies the service provision device 23 lent to the identified occupant ID of the violation, either directly or via the service control device 9. Alternatively, the processing unit 46 may notify the display 21 or speaker 22 installed in the seat 4 of the identified occupant ID, either directly or via the service control device 9, of the violation.

[0118] More specifically, the "boarding determination logic" performed by the boarding determination unit 51 is carried out as shown in the flowchart in Figure 15. When the processing in step S41 in Figure 14 begins, first, in step S61, the boarding determination unit 51 receives information for one passenger 3 output by the monitoring unit 24. The monitoring unit 24 processes the images 14 from cameras 12 and 13 and outputs information for one passenger 3. The boarding determination logic is performed for each passenger 3 output by the monitoring unit 24.

[0119] In step S62, the passenger determination unit 51 searches for a passenger ID in the memory unit 26 that has an area 33 that overlaps with the area 33 of the entered passenger 3.

[0120] In step S63, the boarding determination unit 51 determines whether the search result is "no match". If there is a match, i.e., step S63 is "No", the boarding determination unit 51 terminates the process. If there is no match, i.e., step S63 is "Yes", the boarding determination unit 51 determines in step S64 that a passenger has boarded and terminates the process.

[0121] For example, as shown in Figure 16(a), when the vehicle is empty, and as shown in Figure 16(b), when a passenger 3 not stored in the memory unit 26 is newly detected, the passenger determination unit 51 determines that a passenger has boarded, issues a passenger ID, and stores the area 33 in the memory unit 26.

[0122] The "movement determination logic" performed by the movement determination unit 52 is carried out as shown in the flowchart in Figure 17. When the processing in step S44 in Figure 14 begins, in step S71, the movement determination unit 52 receives information for one crew member 3 output by the monitoring unit 24. The monitoring unit 24 processes the images 14 from the cameras 12 and 13 and outputs information for one crew member 3. The movement determination logic is performed for each crew member 3 output by the monitoring unit 24. If multiple cameras 12 and 13 are used, the monitoring unit 24 may perform tracking processing between the cameras 12 and 13 using existing technology.

[0123] In step S72, the movement determination unit 52 searches for a crew member ID in the memory unit 26 that has an area 33 that overlaps with the area 33 of the input crew member 3.

[0124] In step S73, the movement determination unit 52 determines whether there is a match in the search results. If there is a match, i.e., step S73 is No, the movement determination unit 52 terminates the process. If there is no match, i.e., step S73 is Yes, the movement determination unit 52 determines in step S74 that there has been a movement, identifies the crew member ID with the largest overlapping area 33 among the matched areas 33 as the moved crew member 3, and terminates the process.

[0125] The "disembarkation determination logic" performed by the disembarkation determination unit 53 is carried out as shown in the flowchart in Figure 18. When the processing in step S47 in Figure 14 begins, first, in step S81, the disembarkation determination unit 53 receives information for one passenger 3 held by the memory unit 26. The monitoring unit 24 processes the images 14 from the cameras 12 and 13 and outputs information for one passenger 3. The disembarkation determination logic is performed for each passenger 3 output by the monitoring unit 24.

[0126] In step S82, the disembarkation determination unit 53 searches for an area 33 in the output of the monitoring unit 24 that overlaps with an area 33 in the memory unit 26 that corresponds to the occupant ID.

[0127] In step S83, the disembarkation determination unit 53 determines whether the search result indicates that there is no corresponding area 33. If there is a corresponding area, i.e., step S83 is No, the disembarkation determination unit 53 terminates the process. If there is no corresponding area, i.e., step S83 is Yes, in step S84, the disembarkation determination unit 53 determines that the passenger has disembarked. The passenger 3 in the input storage unit 26 is identified as the passenger 3 who disembarked, and the process terminates.

[0128] The "violation determination logic" performed by the violation determination unit 54 is carried out as shown in the flowchart in Figure 19. When the processing in step S50 in Figure 14 begins, first, in step S91, the violation determination unit 54 receives information for one crew member 3 output by the monitoring unit 24. The monitoring unit 24 processes the images 14 from cameras 12 and 13 and outputs information for one crew member 3. The violation determination logic is performed for each crew member 3 output by the monitoring unit 24.

[0129] In step S92, the violation determination unit 54 determines whether the information for one crew member 3 contains information about a violation. If step S92 is No, the violation determination unit 54 terminates the process. If step S92 is Yes, the violation determination unit 54 proceeds to step S93 and searches for the crew member ID in the memory unit 26 that has an area 33 that overlaps with the area 33 of the entered crew member 3.

[0130] In step S94, the violation determination unit 54 determines whether the search result is correct or incorrect. If there is no match, i.e., step S94 is No, the violation determination unit 54 terminates the process. If there is a match, i.e., step S94 is Yes, the violation determination unit 54 identifies the crew member ID with the largest overlapping area 33 among the matched areas 33 in step S95 as the crew member 3 who committed the violation, and terminates the process.

[0131] Based on the results of the violation detection logic, the management unit 25 of the crew monitoring device 11 operates as shown in the time chart in Figure 20. At time T11, the crew monitoring device 11 identifies the ID of the offending crew member and sends the user ID and violation information to the service control device 9.

[0132] The identification unit 35 identifies the crew ID of the crew member 3 who committed the violation based on the image recognition results from the monitoring unit 24. The processing unit 46 transmits the user ID corresponding to the identified crew member ID and the violation information to the service control device 9.

[0133] The service control device 9 identifies the offending user ID at time T12 and sends the offending information to the mobile terminal 16.

[0134] The service control unit 9 identifies the mobile device 16 owned by user 5 based on the terminal information corresponding to the received user ID. The service control unit 9 notifies the application 27 on the mobile device 16 owned by user 5 of the violation information.

[0135] The mobile terminal 16 issues a "warning of violation" at time T13. The application 27 on the mobile terminal 16 uses at least one of the mobile terminal 16's display 31 and speaker 32 to prompt the user ID to stop the violation by the user ID or their companion 15. Alternatively, the service control device 9 may prompt the companion 15, who possesses the service providing device 23, to stop the violation.

[0136] If the violation is not corrected despite warnings from the mobile device 16, the service control unit 9 will issue a warning about the violation using at least one of the display 21 or speaker 22 inside the vehicle 2, urging the driver to stop the violation.

[0137] <Effects> The effects of this embodiment are as follows:

[0138] The occupant management device 1, occupant management method, and occupant management program for managing information on the occupants 3 of the vehicle 2, as shown in Figure 8, include: an occupant information acquisition unit 41 that assigns an occupant ID to each region 33 in which a recognition target is recognized in an image 14 taken inside or near the entrance outside the vehicle; a user information acquisition unit 42 that acquires the user ID of a terminal user 5 that has applied to board the vehicle 2; an occupant information management unit 43 that associates the user ID that has applied to board the vehicle 2 with the occupant ID; and an accompanying person information acquisition unit 44 that acquires the number of accompanying persons 15 accompanying the user 5 that have been registered in advance in association with the user ID. When the number of accompanying persons 15 associated with the user ID is 1 or more, the occupant information management unit 43 associates the user ID with an occupant ID that was recognized before the user ID was associated with the occupant ID, or after the user ID was associated with the occupant ID, based on the number of accompanying persons 15.

[0139] Therefore, before and after user 5 applies to board and boards the vehicle, accompanying persons 15 who are traveling with user 5 can board without applying to board themselves. The crew information management unit 43 can identify the recognized crew member 3 as a member of the same group as user 5. The number of accompanying persons 15 that have been registered in advance is associated with the same user ID as a member of the same group. Therefore, for example, an unmanned passenger vehicle can carry accompanying persons 15 who do not have their own user ID, and the accompanying persons 15 can be managed using user 5's user ID.

[0140] The crew management device 1, crew management method, and crew management program according to this embodiment include a processing unit 46 that sends a notification to a user 5 corresponding to a user ID, and an identification unit 35 that, upon acquiring an event by a companion 15, identifies the user ID corresponding to the companion 15 or the user 5 themselves where the event was detected. The processing unit 46 sends a notification regarding the event to a terminal corresponding to the identified user ID, for example, a mobile terminal 16.

[0141] Therefore, if any of the accompanying passengers 15 violate the rules of vehicle 2, the terminal of user 5, which corresponds to the user ID of accompanying passenger 15, can be notified of the rule violation. Thus, for example, an unmanned passenger vehicle can manage accompanying passengers 15 who do not have a user ID of the passenger who boarded vehicle 2, using their user IDs. The identification unit 35 uses the output of the monitoring unit 24 to track events by the occupant 3 and continuously associates user 5 and accompanying passengers 15 by user ID even after boarding. As a result, the identification unit 35 identifies user 5 and accompanying passengers 15 who have violated the rules of vehicle 2, and in the case of accompanying passenger 15, identifies user 5, which corresponds to the user ID of accompanying passenger 15. The processing unit 46 notifies user 5's mobile terminal 16, which corresponds to the identified user ID, to stop the violation. Thus, for example, an unmanned passenger vehicle can manage violations by accompanying passengers 15 who do not have a user ID of the passenger who boarded vehicle 2, using their user IDs.

[0142] In this embodiment, the occupant information management unit 43 does not associate a different user ID with an occupant 3 recognized based on an image 14 taken inside or near the entrance of the vehicle 2, as long as the number of occupants 3 recognized based on an image 14 taken inside or near the entrance of the vehicle 2 is less than the number of accompanying persons 15 corresponding to the user ID plus the number of users 5, or greater than the number of accompanying persons 15 plus the number of users 5. Therefore, even if a different user 5 corresponding to a different user ID tries to interrupt after user 5 has finished boarding or alighting, but before all accompanying persons 15 have boarded or alighted, the management unit 25 does not make the association. As a result, the management unit 25 can prohibit interruptions and correctly associate occupant IDs with user IDs.

[0143] In the passenger management device 1, passenger management method, and passenger management program according to this embodiment, the processing unit 46, while the number of passengers 3 recognized based on images 14 taken inside or near the entrance outside the vehicle is less than the number of accompanying persons 15 corresponding to the user ID plus the number of users 5, or greater than the number of accompanying persons 15 plus the number of users 5, sends a notification to another user 5 corresponding to the different user ID or to user 5 of the user ID when the user information acquisition unit 42 acquires a user ID different from the user ID. As a result, the management unit 25 can inform another user 5 or user 5 themselves of the situation that has cut in line to board or alight from the group of user 5 and alert them to the situation.

[0144] The crew management device 1, crew management method, and crew management program according to this embodiment further include a notification unit 45 that notifies the user 5 in a discriminable manner whether the user information acquisition unit 42 can acquire a user ID. The notification unit 45 notifies whether it is possible to acquire a user ID as long as the number of crew members 3 recognized based on the image 14 taken inside or near the entrance outside the vehicle is less than the number of accompanying persons 15 corresponding to the user ID plus the user 5, or greater than the number of accompanying persons 15 plus the user 5. Therefore, the notification prevents another user 5 possessing a mobile terminal 16 with a different user ID from attempting to intrude into a group of users 5 corresponding to a user ID already recognized by the user information acquisition unit 42. [Explanation of symbols]

[0145] 1: Crew management device, 2: Vehicle, 3: Crew, 5: User, 12: Camera, 13: Camera, 14: Image, 15: Companion, 25: Management department, 33: Area, 35: Identification department, 41: Crew information acquisition department, 42: User information acquisition department, 43: Crew information management department, 44: Companion information acquisition department

Claims

1. A crew management device that manages information about the occupants of a vehicle, An occupant information acquisition unit that assigns an occupant ID to each area in which a recognition target is recognized in an image taken inside or near the entrance of the vehicle, A user information acquisition unit that acquires the user ID of the terminal user who applied to board the aforementioned vehicle, A crew information management unit associates the aforementioned crew ID with the aforementioned user ID that applied to board the vehicle, The system includes a companion information acquisition unit that acquires the number of companions accompanying the user, which are registered in advance in association with the user ID. The crew information management unit is characterized in that, when the number of accompanying persons associated with the user ID is one or more, it associates the user ID with a crew ID that was recognized before the user ID was associated with the crew ID, or after the user ID was associated with the crew ID, based on the number of accompanying persons.

2. A crew management device according to claim 1, A processing unit that sends a notification to the user corresponding to the user ID, The system includes an identification unit that, upon acquiring an event by the aforementioned accompanying person, identifies the user ID corresponding to the accompanying person or the user themselves who detected the event, The crew management device is characterized in that the processing unit transmits a notification regarding the event to a terminal corresponding to the identified user ID.

3. A crew management device according to claim 1, The occupant information management unit shall, as long as the number of occupants recognized based on images taken inside the vehicle or near the exterior entrance is less than the number of accompanying persons corresponding to the user ID plus the number of users, or greater than the number of accompanying persons plus the number of users, The occupant management device is characterized in that, even if the user information acquisition unit acquires a user ID different from the user ID, it does not associate the different user ID with the occupant recognized based on an image taken inside the vehicle or near the outside entrance.

4. A crew management device according to claim 2, The processing unit, while the number of passengers recognized based on images taken inside the vehicle or near the exterior entrance is less than the number of accompanying persons corresponding to the user ID plus the number of users, or greater than the number of accompanying persons plus the number of users, A crew management device characterized in that, when the user information acquisition unit acquires a user ID different from the user ID, it sends a notification to another user corresponding to the different user ID or to the user with the user ID.

5. A crew management device according to claim 1, The system further includes a notification unit that notifies the user in a discriminable manner whether the user information acquisition unit is able to acquire the user ID. The notification unit continues as long as the number of passengers recognized based on images taken inside the vehicle or near the exterior entrance is less than the number of accompanying persons corresponding to the user ID plus the number of users, or greater than the number of accompanying persons plus the number of users. A crew management device characterized by notifying whether it is possible to obtain the aforementioned user ID.

6. A crew management method for managing information about the occupants of a vehicle, The process includes an occupant information acquisition step in which an occupant ID is assigned to each area in which a recognition target is recognized in an image taken inside or near the entrance of the vehicle, A user information acquisition step, which involves obtaining a user ID from the user of the terminal that applied to board the aforementioned vehicle, A crew information management step that associates the aforementioned crew ID with the aforementioned user ID, The step includes obtaining the number of accompanying persons who are registered in advance in association with the user ID, and obtaining the number of accompanying persons who are accompanying the user, A crew management method characterized in that, in the crew information management step, when the number of accompanying persons associated with the user ID is one or more, the user ID is associated with a crew ID that was recognized before the user ID was associated with the crew ID, or after the user ID was associated with the crew ID, based on the number of accompanying persons.

7. A crew management program to be executed on a computer having a processor and memory, The crew management program is provided to the processor, A passenger information acquisition step involves assigning a passenger ID to each area where a recognition target is identified in an image taken inside or near the vehicle's entrance, A user information acquisition step, which involves obtaining a user ID from the user of the terminal that applied to board the aforementioned vehicle, A crew information management step that associates the aforementioned crew ID with the aforementioned user ID that applied to board the aforementioned vehicle, The process involves executing a companion information acquisition step, which involves acquiring the number of companions accompanying the user that have been pre-registered in association with the user ID, A crew management program characterized in that, in the crew information management step, when the number of accompanying persons associated with the user ID is one or more, the program associates the user ID with a crew ID that was recognized before the user ID was associated with the crew ID, or after the user ID was associated with the crew ID, based on the number of accompanying persons.

Images