Passenger safety situation detection device and passenger safety situation detection method

Abstract

To provide a passenger safety situation detecting device and a passenger safety situation detecting method which can detect safety situations of passengers in a vehicle such as a bus having an area where the passengers are standing, without visual confirmation by a driver or a watcher.SOLUTION: The passenger safety situation detecting device, which is provided in a vehicle having an area where passengers can stand, comprises: a detecting part, installed in the vicinity of a ceiling of the vehicle, which detects passenger information; a person detecting part that detects heads of passengers using the passenger information to discriminate whether the passengers are in the vehicle or not on the basis of the detected heads, and when the passengers are in the vehicle, detects the number of the passengers and respective positions of the heads of the passengers; a posture detecting part that detects. for each of the detected passengers, a posture of the passenger, by analyzing an area in a predetermined range around the head, using the passenger information; a determining part that determines, for each of the detected passengers, whether the passenger is in a safe situation or not, on the basis of the posture of the passenger; and an output part that outputs the determined results.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a passenger safety situation detection device and a passenger safety situation detection method. [Background technology]

[0002] For example, a wide range of age groups, from children to the elderly, use route buses. In recent years, there have been many accidents of passengers falling on route buses. For this reason, route buses make announcements such as "Please hold on to the straps and handrails" on board to prevent passengers from falling, and the driver or a guide or other supervisor checks that passengers are seated before the bus departs.

[0003] In recent years, demonstration experiments of self-driving buses have been conducted in various places. In recent years, there have been cases where routes with high demand have been reduced or abolished due to a shortage of drivers caused by depopulation and labor shortages. For this reason, self-driving buses are about to be used to operate without a driver in the vehicle. In such self-driving buses without a driver, the driver or supervisor cannot check whether passengers are seated or hanging on to the straps before departure. For this reason, a system has been proposed for self-driving buses without a driver, which recognizes passengers at high risk of falling based on information read from a transportation IC card when the passengers board the bus and guides the passengers to take a seat (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-3936 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology described in Patent Document 1 guides only passengers who are at high risk of falling to take their seats, but all passengers who are not seated are at risk of falling. The technology described in Patent Document 1 may not be able to guide all standing passengers in the vehicle to take their seats, and may not be able to prevent passengers from falling.

[0006] The present invention has been made in consideration of the above problems, and has an object to provide a passenger safety condition detection device and a passenger safety condition detection method that can detect the safety condition of passengers in a vehicle that has an area where passengers stand, such as a bus, without the driver or supervisor having to visually check. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, a passenger safety situation detection device according to one embodiment of the present invention comprises a detection unit installed near the ceiling of a vehicle having an area where passengers can stand, and detects passenger information regarding the passengers; a human detection unit that detects the head of the passenger using the passenger information and determines whether the passenger is present in the vehicle based on the detected head; a posture detection unit that, if a passenger is present in the vehicle, detects the posture of the passenger for each detected passenger by analyzing an area within a predetermined range around the detected head using the passenger information; a determination unit that determines whether the passenger is in a safe situation for each detected passenger based on the detected posture of the passenger; and an output unit that notifies the determination result.

[0008] In addition, in a passenger safety situation detection device according to one aspect of the present invention, the detection unit, the human detection unit, the judgment unit, and the output unit may repeat processing until it is determined that all passengers present in the vehicle are in a safe situation.

[0009] In addition, the passenger safety situation detection device according to one embodiment of the present invention may further include a grip detection unit, wherein the posture detection unit detects whether the passenger is standing or seated, and for each detected passenger, when the passenger is standing, the grip detection unit detects a gripping state of whether the passenger is gripping a grip part inside the vehicle by analyzing an area within a predetermined range around the detected head based on the detected posture of the passenger, and the determination unit may determine whether the passenger is in a safe situation by also using the detected gripping state of the passenger, for each detected passenger.

[0010] In addition, in a passenger safety situation detection device according to one aspect of the present invention, the output unit may be multiple, and when the judgment unit judges that the passenger is not in a safe situation, it may cause the output unit located closest to the passenger judged to be in an unsafe situation to notify information indicating that the passenger is in an unsafe situation.

[0011] In addition, in a passenger safety situation detection device according to one embodiment of the present invention, there may be multiple detection units, and the posture detection unit may associate the same passenger in the first passenger information detected by the first detection unit and the second passenger information detected by the second detection unit, and detect the posture of the passenger based on the first detection information and the second detection information.

[0012] In addition, in a passenger safety status detection device according to one embodiment of the present invention, the person detection unit may, when passengers are present, detect the number of passengers and the position of the head of each detected passenger, and the posture detection unit may determine, for each detected passenger, whether the passenger is seated or standing based on a change in the detected head position.

[0013] In addition, in a passenger safety status detection device according to one embodiment of the present invention, the detection unit may be a photographing device or a distance sensor, and if the detection unit is a photographing device, the passenger information is an image and the detection unit is installed in an area where the passenger cannot be standing while the vehicle is traveling and in a position where the passenger can be overlooked, and if the detection unit is a distance sensor, the passenger information is distance information from the detection unit and the detection unit is installed in a position where the passenger can be seen through.

[0014] In order to achieve the above-mentioned object, a passenger safety situation detection method according to one embodiment of the present invention includes, in a vehicle having an area where passengers can stand, a detection unit installed near the ceiling of the vehicle detects passenger information regarding the passengers, a human detection unit detects the heads of the passengers using the passenger information and determines whether the passengers are present in the vehicle based on the detected heads, and if the passengers are present, detects the number of passengers and the position of the head for each detected passenger, a posture detection unit detects the posture of the passenger for each detected passenger by analyzing an area within a predetermined range around the detected head using the passenger information, a judgment unit judges whether the passenger is in a safe situation based on the detected posture of the passenger for each detected passenger, and an output unit reports the judgment result. Effect of the Invention

[0015] According to the present invention, the safety status of passengers in a vehicle having an area where passengers stand, such as a bus, can be detected without the driver or supervisor having to visually check. [Brief description of the drawings]

[0016] [Figure 1] 1 is a block diagram showing an example of the configuration of a passenger safety situation detection device according to an embodiment; [Diagram 2] 4A to 4C are diagrams illustrating examples of positions of chairs, a detection unit, handrails, and hanging straps in a vehicle according to an embodiment. [Diagram 3] 5A to 5C are diagrams illustrating an example of the arrangement and number of detection units according to the embodiment. [Figure 4]5A to 5C are diagrams illustrating an example of an image captured by a first detection unit according to the embodiment. [Diagram 5] 4 is a flowchart of a processing procedure performed by a passenger safety situation detection device according to the embodiment. [Figure 6] 2 is a diagram for explaining an outline of a process performed by a passenger safety situation detection device according to an embodiment. FIG. [Figure 7] 1A and 1B are diagrams showing example images of a head and a hand extracted from a captured image; [Figure 8] FIG. 11 is a diagram illustrating an example of a tracking process according to the embodiment. [Figure 9] FIG. 2 is a diagram showing an example of seat positions (coordinates) according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the drawings used in the following description, the scale of each component is appropriately changed so that each component is of a recognizable size.

[0018] Fig. 1 is a block diagram showing an example of the configuration of a passenger safety situation detection device 1 according to this embodiment. As shown in Fig. 1, the passenger safety situation detection device 1 includes a detection unit 11, an analysis unit 12, a storage unit 13, and an output unit 14. The detection unit 11 includes a first detection unit 111, a second detection unit 112, and a third detection unit 113. The analysis unit 12 includes an acquisition unit 121, a person determination analysis unit 122 (person detection unit), a person tracking unit 123, a posture analysis unit 124 (posture detection unit), a grip analysis unit 125 (posture detection unit, grip detection unit), and a determination unit 126. In the configuration example of Figure 1, the detection unit 11 is an example equipped with three imaging devices (first detection unit 111 to third detection unit 113), but the number of imaging devices is not limited to three, and may be three or less or four or more.

[0019] The passenger safety situation detection device 1 detects the safety situation of passengers in a vehicle (such as a route bus, shuttle bus, school bus, shuttle bus, chartered bus, bus-type truck, train, ship, etc.) having an area where passengers are standing. The vehicle may be either an automatic or manual driving vehicle.

[0020] The detection unit 11 is an imaging device, and is installed in a position where it can capture images of passengers including their heads (passenger information related to passengers) and images of the inside of the vehicle, for example, on the ceiling or near the ceiling at the front, center, and rear of the vehicle. The detection unit 11 captures images of the inside of the vehicle and outputs the captured images to the analysis unit 12. The images may be still images, continuous still images, or videos. The images captured by the detection unit 11 include images of a specified area inside the vehicle. The installation position of the detection unit 11 will be described later.

[0021] The analysis unit 12 detects the safety status of passengers inside the vehicle based on the captured images.

[0022] The acquisition unit 121 acquires the captured image and outputs the acquired image to the human determination analysis unit 122.

[0023] The person determination analysis unit 122 extracts a head image from the image output by the acquisition unit 121. The head image is, for example, an image from the neck up. In addition, the person determination analysis unit 122 may calculate the probability of detection by a known method at the time of detection, and store it by linking it to the head image. In addition, the person determination analysis unit 122 may extract the head image by a pattern matching method. If the person determination analysis unit 122 can extract a head image, it determines that a person (passenger) is present in the vehicle, and if the head image cannot be extracted, it determines that a person (passenger) is not present in the vehicle. In addition, if a person is present in the vehicle, the person determination analysis unit 122 detects the number of people present, and further detects the head positions of the passengers present. In addition, the person determination analysis unit 122 may extract the judgment of whether or not a person is present using a trained model with a teacher. In the case of machine learning the model, an image taken inside the vehicle is input to the model, and the image range of the person's head is trained as teacher data. The head position is, for example, a coordinate with a predetermined position inside the vehicle as the origin, and also includes a height coordinate. The person determination analysis unit 122 outputs the determination result and the detection result to the posture analysis unit 124.

[0024] The human tracking unit 123 tracks the position of the head image detected by the human determination analysis unit 122 using a known image processing method.

[0025] The posture analysis unit 124 acquires the image output by the acquisition unit 121 and the judgment result and detection result output by the person judgment analysis unit 122. When a person is present in the vehicle, the posture analysis unit 124 extracts an image of an area within a predetermined range around the area where the person judgment analysis unit 122 detects the head from the acquired image based on the acquired image and the detection result. The posture analysis unit 124 analyzes the posture of each person detected by the person judgment analysis unit 122 using the extracted image and, for example, a trained model with a teacher. When the model is machine-learned, an image taken inside the vehicle is input to the model, and the model is trained as teacher data on whether the person is standing, seated, or standing, and the movement of the passenger tracked at each time after boarding. The posture is, for example, a standing state or a seated state. In addition, in the learning, images of the standing state and the seated state are used for each of a plurality of vehicle types. In this way, the posture analysis is limited to the area around the area where the person judgment analysis unit 122 detects the head. The posture analysis unit 124 may detect a change in posture for each person present in the vehicle based on the results detected by the person tracking unit 123. Alternatively, the posture analysis unit 124 may detect a change in the position of the head based on the position of the head image detected by the person determination analysis unit 122 and the tracking results of the person tracking unit 123, and determine that the person is seated when the height becomes lower than a predetermined value. The posture analysis unit 124 outputs information indicating the posture analyzed for each person detected by the person determination analysis unit 122 to the grip analysis unit 125 as a detection result.

[0026] The grip analysis unit 125 acquires the image output by the acquisition unit 121 and the detection result output by the posture analysis unit 124. The grip analysis unit 125 estimates the position of the hands (including arms) from the position of the head image by referring to the physique information stored in the storage unit 13 for each standing person, and extracts the image of the hands from the image. The grip analysis unit 125 may extract the image of the hands from the image by using, for example, a trained model with a teacher. The grip analysis unit 125 may calculate the likelihood of detection by a known method at the time of detection, and store it in association with the image of the hands. If the grip analysis unit 125 cannot extract the image of the hands, it cannot confirm that the passenger is gripping the grip (handrail or strap), and therefore determines that the passenger is not gripping the grip. When the grip analysis unit 125 can extract the hand image, it analyzes the grip state of each person standing, whether or not the grip is being gripped, using the extracted hand image and, for example, a learned model with a teacher. The grip analysis unit 125 may detect a change in whether or not each person present in the vehicle is gripping, based on the result detected by the person tracking unit 123. When detecting the hand, for example, if the hands of two passengers are crossed, the image of the hand of one passenger is interrupted by the hand of the other passenger. For this reason, when detecting the hand image, the grip analysis unit 125 determines that the hand within a predetermined range from the head and continuing from the head is the hand of the passenger to be detected.

[0027] The determination unit 126 determines whether or not the person in the vehicle is in a safe situation based on the determination result determined and the detection result detected by the person determination analysis unit 122, the detection result detected by the posture analysis unit 124, and the detection result detected by the grip analysis unit 125. The determination unit 126 outputs the determination result as the analysis result to the output unit 14. Note that, for example, in a vehicle that does not have a grip unit, the determination unit 126 may determine whether or not the person in the vehicle is in a safe situation based on the determination result determined and the detection result detected by the person determination analysis unit 122, and the detection result detected by the posture analysis unit 124, without using the result detected by the grip analysis unit 125.

[0028] The memory unit 13 stores physique information (e.g., distance information between the head and hands) related to a person's physique used when detecting hands. The memory unit 13 stores the positions of seats, handrails, and straps. The memory unit 13 stores, for example, images to be compared in pattern matching. The memory unit 13 stores programs, constants, thresholds, etc. used by the analysis unit 12 for analysis.

[0029] The output unit 14 outputs the analysis result output by the analysis unit 12. The output unit 14 is, for example, an image display device, a speaker, a buzzer, a lamp, etc. The output result is notified to passengers in the vehicle. For example, if the notification result indicates that the situation is not safe, a passenger who is standing and not holding the gripping part will grasp the gripping part upon confirming the notification result. The output unit 14 may be provided at multiple locations in the vehicle. In this case, the determination unit 126 may instruct the output unit 14 installed at a position closest to the head position of the passenger who has been determined to be in an unsafe situation to output information indicating that the situation is not safe.

[0030] <Examples of the positions of seats, detectors, handrails, and straps inside a vehicle> Next, an example of the positions of the seats, the detection unit, the handrails, and the straps in the vehicle will be described. Fig. 2 is a diagram showing an example of the positions of the seats, the detection unit, the handrails, and the straps in the vehicle according to this embodiment. As described above, the vehicle 500 is, for example, a bus. In Fig. 2, the longitudinal direction of the vehicle is the x-axis direction, and the lateral direction is the y-axis direction.

[0031] 2, seats 501 are installed on the left, right, and rear inside vehicle 500. Handrails 511 are installed on the left and right sides of the aisle of vehicle 500. Hanging straps 521 are installed on the ceiling of vehicle 500. Also, first detection unit 111 to third detection unit 113 are installed in the longitudinal direction (x-axis direction) of the ceiling of the vehicle.

[0032] The arrangement and number of the detectors 11 in FIG. 2 are merely an example, and are not limited thereto. 3 is a diagram showing an example of the arrangement and number of detection units 11 according to this embodiment. As in the area indicated by reference symbol g101, there may be one detection unit 11. In this case, the detection unit 11 is attached, for example, to the ceiling or near the ceiling in the center of the vehicle. As in the area indicated by reference symbol g102, there may be four detection units 11. In this case, the detection units 11 are attached, for example, to the left and right, front and rear, near the ceiling of the vehicle.

[0033] The mounting position of the detection unit 11 is a position where the head can be photographed. The detection unit 11 may be mounted at a position and in a number according to the width of the vehicle interior, the position of the seat, the position of the handrail, the position of the hanging strap, the width of the aisle, the boarding entrance, the disembarking entrance, the performance of the detection unit 11 (angle of view, resolution, etc.), and the like. The mounting position of the detection unit 11 is preferably a position where passengers do not overlap in the photographed image, and is preferably a position where the detection unit 11 is not blocked by passengers. For this reason, the mounting position of the detection unit 11 is more preferably a position where no passengers are standing, such as the area indicated by reference symbol g102. Alternatively, in the case of an express bus, for example, where passengers are required to sit, there are no passengers standing during the journey, and therefore the mounting position of the detection unit 11 may be a single location near the ceiling in the center of the vehicle, such as the area indicated by reference symbol g101. As described above, the installation position of the detection unit 11 in this embodiment is in an area where passengers cannot stand (seats, areas where passengers cannot enter, etc.) in a vehicle that has an area where passengers can stand, and is a position where the passengers can be overlooked.

[0034] <Example of an image captured by the detection unit 11> Next, an example of an image captured by the detection unit 11 will be described. Fig. 4 is a diagram showing an example of an image captured by the first detection unit 111 according to this embodiment. In the example of Fig. 4, seven passengers (HU1 to 7) are riding in the captured area. In the example of Fig. 4, passengers HU1, 3 to 5, and 7 are seated, and passengers HU2 and 6 are standing. Also, in the example of Fig. 4, passenger HU2 is holding onto a handrail, but passenger HU7 is not holding onto a handrail or a strap. In this embodiment, the safety status of passengers is determined by detecting the number of passengers in the vehicle, the postures of the passengers, and the gripping state of standing passengers based on the captured images.

[0035] <Processing Overview> Next, a process performed by the passenger safety situation detection device 1 will be described. FIG. 5 is a flowchart of a processing procedure performed by the passenger safety situation detection device 1 according to this embodiment.

[0036] (Step S11) The detection unit 11 captures an image of the inside of the vehicle. (Step S12) The acquisition unit 121 acquires the captured image.

[0037] (Step S13) The human determination analysis unit 122 extracts a head image from the acquired image.

[0038] (Step S14) The human determination analysis unit 122 determines whether or not there are people (passengers) in the vehicle based on whether or not a head image has been extracted. If the human determination analysis unit 122 has determined that there are people (passengers) in the vehicle when a head image has been extracted (Step S14; YES), it detects the number of passengers from the number of head images that have been extracted, and detects the head position of each passenger based on the head images that have been extracted. After processing, the human determination analysis unit 122 proceeds to processing of step S15. If the human determination analysis unit 122 has determined that there are no people (passengers) in the vehicle when a head image has not been extracted (Step S14; NO), it proceeds to processing of step S19.

[0039] (Step S15) The posture analysis unit 124 extracts an image of the area surrounding the extracted head image from the acquired image.

[0040] (Step S16) The posture analysis unit 124 determines whether each passenger corresponding to the detected head image is in a seated position or a standing position based on the extracted image. If the posture analysis unit 124 determines that the passenger corresponding to the detected head image is in a standing position (Step S16; standing), the process proceeds to step S17. If the posture analysis unit 124 determines that all passengers corresponding to the detected head images are in a seated position (Step S16; seated), the process proceeds to step S19.

[0041] (Step S17) The grip analysis unit 125 extracts hand images from the captured images for each passenger corresponding to the image of the standing passenger, and analyzes whether the passenger is gripping a grip based on the extracted hand images. Note that the grip analysis unit 125 also determines that the passenger is not gripping a grip if it cannot confirm an image of the hand.

[0042] (Step S18) The gripping analysis unit 125 judges whether or not the standing passengers are gripping the gripping parts. If the gripping analysis unit 125 judges that all the standing passengers are gripping the gripping parts (Step S18; YES), the process proceeds to Step S19. If the gripping analysis unit 125 judges that the standing passengers are not gripping the gripping parts (Step S18; NO), the process proceeds to Step S20.

[0043] (Step S19) The determination unit 126 determines that the passengers are in a safe situation when one of the following is true: no passengers are on board (Step S14; NO), all passengers are seated (Step S16; seated), and all standing passengers are gripping the gripping parts (Step S18; YES). The output unit 14 then outputs information indicating that the passengers are in a safe situation.

[0044] (Step S20) If some or all of the passengers are standing (Step S16; Standing) and even one of the standing passengers is not gripping the gripping part (Step S18; NO), the determination unit 126 determines that the passengers are not in a safe situation. Then, the output unit 14 outputs information indicating that the passengers are not in a safe situation.

[0045] The passenger safety status detection device 1 repeats the above process from steps S11 to S20 until the situation is one of the following: no passengers are on board, all passengers are seated, and all standing passengers are holding the gripping parts, and all passengers are in a safe situation.

[0046] 5 when the vehicle stops at a bus stop and the boarding door is opened, and repeats the process until the vehicle departs. The passenger safety situation detection device 1 may also perform the process in FIG. 5 while the vehicle is in operation.

[0047] <Processing Overview> Next, an outline of the process performed by the passenger safety situation detection device 1 will be described. FIG. 6 is a diagram for explaining an outline of the process performed by the passenger safety situation detection device 1 according to this embodiment. The first detection unit 111 captures an image of the inside of the vehicle. The acquisition unit 121 acquires an image g201 captured by the first detection unit 111. Three passengers are captured in the image g201.

[0048] Next, the human determination analysis unit 122 extracts head images g202 to g204 of passengers from the image g201.

[0049] Next, the posture analysis unit 124 extracts an image g205 of the area surrounding the head image g202. The posture analysis unit 124 extracts an image g206 of the area surrounding the head image g203. The posture analysis unit 124 extracts an image g207 of the area surrounding the head image g204.

[0050] Next, the posture analysis unit 124 determines that the passenger is standing based on the extracted image g205. Next, the grip analysis unit 125 extracts a hand image g208 from the extracted image g205 or the captured image g201, and determines that the passenger is not gripping the grip based on the extracted image. Next, the determination unit 126 determines that the standing passenger is not gripping the grip, and therefore is not in a safe situation (g209).

[0051] The posture analysis unit 124 determines that the passenger is seated based on the extracted image g206. Next, the determination unit 126 determines that the situation is safe because the passenger is seated (g210).

[0052] The posture analysis unit 124 determines that the passenger is standing based on the extracted image g207. Next, the grip analysis unit 125 determines that the hand could not be confirmed because the hand image could not be extracted from the extracted image g207 or the captured image g201. Next, the determination unit 126 determines that the situation is not safe (g212) because the hand could not be confirmed (g211) and the standing passenger is not gripping the grip.

[0053] 6, an example of determining based on an image captured by one detection unit 11 has been described, but the present invention is not limited to this. For example, even if a hand image cannot be extracted from an image g207 extracted from an image captured by the first detection unit 111, the hand image may be extracted from an image extracted from an image captured by the second detection unit 112. In such a case, the grip analysis unit 125 determines the grip state based on images captured by multiple detection units. Similarly, when the passenger's posture cannot be determined from an image captured by one detection unit, the posture analysis unit 124 determines the posture state based on images captured by multiple detection units. Moreover, the passenger safety situation detection device 1 repeats such processing until all passengers are in a safe situation, as described above.

[0054] Here, examples of head images and hand images extracted from a captured image will be described. 7 is a diagram showing example head images and hand images extracted from a captured image. The captured first image g300 shows four passengers, and head images g301 to g304 and a hand image g311 are detected. In the example of the first image g300, the passengers corresponding to the head images g301 and g302 are determined to be seated, and the passengers corresponding to the head images g303 and g304 are determined to be standing. Then, the grip analysis unit 125 determines that the situation is not safe for the passenger corresponding to the head image g303 because an image of the hands could not be detected. The grip analysis unit 125 determines that the situation is safe for the passenger corresponding to the head image g304 because an image of the hands can be detected and the passenger is gripping the back of the chair.

[0055] The captured second image g350 is an example in which there are three passengers, head images g351 to g353 are detected, and hand images g361 and g362 are detected. It should be noted that the head image does not have to include the entire face, as in the head image g353, and may include the entire face, as in the head images g301 to g304, g351, and g352. In the example of the second image g350, the passengers corresponding to the head images g351, g352, and g353 are determined to be standing. Then, the grip analysis unit 125 determines that the passenger corresponding to the head image g351 is in a safe situation because an image of the hands can be detected and the passenger is gripping the grip (handrail). The grip analysis unit 125 determines that the passenger corresponding to the head image g352 is in an unsafe situation because an image of the hands can be detected but the passenger is not gripping the grip. The grip analysis unit 125 determines that the passenger corresponding to the head image g353 is in an unsafe situation because an image of the hands cannot be detected.

[0056] <Tracking process> In the above example, the passenger is stationary, but in reality, the passenger who boards the vehicle through the boarding gate moves around the vehicle, sits down, grips the gripping part, etc. Also, the passenger may change from a standing state to a seated state.

[0057] For this reason, the passenger safety situation detection device 1 may perform a process of tracking the head position of the passenger who has boarded the vehicle. FIG. 8 is a diagram showing an example of tracking processing according to this embodiment. For ease of explanation, FIG. 8 shows an example in which there is one passenger in a vehicle. The person tracking unit 123 assigns identification information to each passenger detected in an image captured by one image capture unit. For example, when three head images are detected from an image captured by the first detection unit 111, the identification information IDs are H1(1), H2(1), and H1(1). When two head images are detected from an image captured by the second detection unit 112, the identification information IDs are H1(2) and H2(2).

[0058] At time t1, the person determination analysis unit 122 extracts a head image of person HU5 from the captured image (HU5(t1)). Next, the person tracking unit 123 tracks the position (x, y, z) of the head image corresponding to the detected person HU5 at times t2 and t3, and outputs the tracked position information to the posture analysis unit 124 and the grip analysis unit 125. Note that the longitudinal direction of the vehicle is the x-axis direction, the lateral direction of the vehicle is the y-axis direction, and the height direction of the vehicle is the z-axis direction.

[0059] At times t1 and t2, the determination unit 126 determines that the passenger is standing and not holding anything, and therefore the situation is not safe. At time t3, the determination unit 126 determines that the passenger is seated because the value of the head position in the z-axis direction is lower than that at time t1 or t2. This causes the determination unit 126 to determine that the passenger's situation has changed to a safe one.

[0060] The effect of performing the tracking process in this manner will be further explained. For example, if the range Z1 is the shooting range of the first detection unit 111, the range Z2 is the shooting range of the second detection unit 112, and the range Z3 is the shooting range of the third detection unit 113, at time t2, the passenger HU5 exists across the ranges Z1 and Z3. In order to determine the posture and gripping state of the passenger HU5, it is necessary to detect that the HU5 is the same person, and to determine the posture and gripping state by integrating the image captured by the first detection unit 111 and the image captured by the second detection unit 112. Alternatively, in order to determine the posture and gripping state of the customer HU5, it is necessary to detect that the HU5 is the same person, determine the posture and gripping state for the image captured by the first detection unit 111, and determine the posture and gripping state for the image captured by the second detection unit 112, and integrate the determination results. In this embodiment, tracking information detected by the person tracking unit 123 is used for such integration.

[0061] In addition, tracking may be performed, for example, from when a passenger boards at a first stop until he or she gets off at a second stop. This allows the boarding section for each passenger to be obtained, which can be used, for example, to set the boarding fare for each boarding section.

[0062] <Seat location> In the above example, the posture analysis unit 124 determines the posture of the passenger based on the captured image, but the present invention is not limited to this example. The posture analysis unit 124 may determine the posture of the passenger using the head position (x, y) and the seat position (x, y). 9 is a diagram showing an example of the position (coordinates) of a seat according to this embodiment. The position of the reference coordinates g400(0,0) is, for example, near the boarding entrance. For a single seat, the coordinates (g401 to g407) are set to a specific position (or range) of the seat. In the case of a seat that can accommodate several people, the coordinates g408 and g409 are set in a strip, for example.

[0063] The posture analysis unit 124 may compare the head coordinates extracted by the person determination analysis unit 122 with the position of the seat to determine that the passenger is in the seat, i.e., seated.

[0064] As described above, in this embodiment, by extracting a head image from a captured image, it is determined whether or not a passenger is present in a vehicle, and the number of passengers and the position of the passengers are detected. In addition, in this embodiment, an image of a region surrounding an image from which a head image is extracted is extracted to determine the posture of each passenger (whether the passenger is standing or seated). Furthermore, in this embodiment, for a standing passenger, an image of a hand is extracted, and it is determined whether or not the passenger is gripping a grip part based on the extracted image. Then, in this embodiment, the safety status of the passenger is determined based on the presence or absence of the passenger and the posture, or based on the presence or absence of the passenger, the posture, and the gripping state. That is, in this embodiment, first, a head image of the passenger is extracted from a captured image, and it is determined that the passenger is a passenger based on the head image, and then the safety status of the passenger is determined by determining the posture of each passenger (including the gripping state).

[0065] As a result, according to this embodiment, the safety status of passengers in a vehicle such as a bus or train can be automatically detected without the driver or supervisor having to visually check.

[0066] Here, a case will be described in which the entire body of a passenger is detected and determined from a captured image. In this case, the entire body is rarely captured in an image captured inside a vehicle. In contrast, it has been confirmed through experiments that by capturing an image from above, such as from the ceiling of the vehicle, as in this embodiment, the captured image often includes the head of the passenger. Therefore, in this embodiment, it is possible to determine whether or not a passenger is present in the vehicle based on the head image, and to detect the number of passengers and their positions.

[0067] In addition, depending on the positional relationship between the position of the detection unit 11 and the passenger, the entire body of the passenger may not appear in the captured image, for example, as in image g207 in Fig. 6. In contrast, in this embodiment, by having multiple detection units, images captured from different positions can be integrated, or evaluations of each image can be integrated, allowing the posture and gripping state to be determined more accurately than when images are captured using a single detection unit.

[0068] <First Modification> In the above embodiment, the detection unit performs determination and detection for each captured image. However, the present invention is not limited to this. The person determination analysis unit 122 may perform determination and detection on the image captured by the first detection unit 111, perform determination and detection on the image captured by the second detection unit 112, perform determination and detection on the image captured by the third detection unit 113, and integrate the determination results and the detection results. Alternatively, the person determination analysis unit 122 may integrate the images captured by the first detection unit 111, the second detection unit 112, and the third detection unit 113, and perform determination and detection on the integrated image. Note that, when only a part of the head image is captured at the time of integration, the person determination analysis unit 122 may compare the likelihood values ​​associated with the head images captured by each of the two imaging units, and detect the head based on the more likely one.

[0069] The posture analysis unit 124 may perform judgment and detection on the image captured by the first detection unit 111, perform detection on the image captured by the second detection unit 112, and perform detection on the image captured by the third detection unit 113, and integrate the detection results. Alternatively, the posture analysis unit 124 may integrate the images captured by the first detection unit 111, the second detection unit 112, and the third detection unit 113, and perform detection on the integrated image. Note that, when integrating, if only a part of the body is captured in the image captured by one imaging unit, the posture analysis unit 124 may compare the likelihood values ​​associated with the head images captured by each of the two imaging units, and detect the posture based on the more likely one.

[0070] The grip analysis unit 125 may perform judgment and detection on the image captured by the first detection unit 111, judgment and detection on the image captured by the second detection unit 112, and judgment and detection on the image captured by the third detection unit 113, and integrate the detection results. Alternatively, the grip analysis unit 125 may integrate the images captured by the first detection unit 111, the second detection unit 112, and the third detection unit 113, and perform detection on the integrated image. Note that, when only a part of the image of the hand is captured during integration, the grip analysis unit 125 compares the likelihood values ​​associated with the images of the hand, and determines whether or not the hand is being gripped based on the more likely one. For example, if an image of a passenger's left hand can be extracted from an image captured by the first detection unit 111, and an image of the passenger's right hand can be extracted from an image captured by the second detection unit 112, the grip analysis unit 125 may integrate the information by determining whether or not the left or right hand is gripping the grip based on the information captured by these two imaging units.

[0071] <Second Modification> In the above example, the detection unit 11 is an image capture device, and an example has been described in which a head image is extracted from the captured image, and an image of the area around the head is further extracted from the captured image to determine and detect, but the present invention is not limited to this. The detection unit 11 may be, for example, a non-contact distance detection device using light. In this case, the detection unit 11 may detect the position (passenger information) of the head, shoulder, hand, etc., and the position of the chair, etc., by measuring the distance from the detection unit 11. In this case, the detection unit 11 is attached to a position (for example, on the aisle and on the seat) through which the passenger's head, etc. can be transmitted. Then, the person determination analysis unit 122 may detect the head based on the distance information detected by the detection unit 11. The person tracking unit 123 may be configured so that the person determination analysis unit 122 tracks the position of the head based on the distance detected by the detection unit 11. The posture analysis unit 124 may detect the area around the area where the head is detected by the person determination analysis unit 122 based on the distance detected by the detection unit 11, and determine the posture based on the detection result. For example, the posture analysis unit 124 may determine whether the passenger is in a seated posture or a standing posture based on the distance to the head. The grip analysis unit 125 may determine the grip state based on the distance detected by the detection unit 11 and on the information extracted by the posture analysis unit 124. The determination unit 126 may determine the safety status of the passenger based on the determination results and detection results of the person determination analysis unit 122, the person tracking unit 123, the posture analysis unit 124, and the grip analysis unit 125.

[0072] As described above, in the second modified example, a non-contact type detection device is used to detect the heads of passengers, etc. In this case, as in the case where an image is used, the safety status of passengers in a vehicle such as a bus or train can be automatically detected without the driver or supervisor having to visually check.

[0073] <Third Modification> In the above example, a route bus or the like in which passengers may be standing while traveling has been described. In the third modified example, an example of a vehicle such as an express bus in which passengers are required to be seated while traveling will be described. In this case, the analysis unit 12 does not need to include the grip analysis unit 125.

[0074] In such a case, the judgment unit 126 may judge whether or not the people inside the vehicle are in a safe situation based on the judgment result and detection result detected by the person judgment analysis unit 122 and the detection result detected by the posture analysis unit 124.

[0075] According to the third modified example, the safety condition of a passenger can be determined based on the passenger's posture without detecting the gripping state.

[0076] In addition, a program for realizing all or part of the functions of the analysis unit 12 in the present invention may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to perform all or part of the processing performed by the analysis unit 12. Note that the term "computer system" as used herein includes hardware such as an OS and peripheral devices. The term "computer system" also includes a WWW system equipped with a homepage providing environment (or display environment). The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system. The term "computer-readable recording medium" also refers to storage devices that hold a program for a certain period of time, such as volatile memory (RAM) inside a computer system that becomes a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line.

[0077] The above program may also be transmitted from a computer system in which the program is stored in a storage device or the like to another computer system via a transmission medium, or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The above program may also be for realizing part of the above-mentioned functions. Furthermore, it may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0078] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0079] Reference Signs List 1: Passenger safety status detection device, 11: Detection unit, 12: Analysis unit, 13: Memory unit, 14: Output unit, 111: First detection unit, 112: Second detection unit, 113: Third detection unit, 121: Acquisition unit, 122: Person determination analysis unit, 123: Person tracking unit, 124: Posture analysis unit, 125: Grasping analysis unit, 126: Determination unit

Claims

1. a detection unit that is installed in a vicinity of a ceiling of a vehicle having an area where passengers can stand, and detects passenger information relating to the passengers; a human detection unit that detects a head of the passenger by using the passenger information and determines whether the passenger is present in the vehicle based on the detected head; a posture detection unit that detects a posture of each of the detected passengers by analyzing a region within a predetermined range around the detected head of the passenger using the passenger information when the passenger is present in the vehicle; a determination unit that determines, for each of the detected passengers, whether or not the passengers are in a safe situation based on the detected posture of the passengers; an output unit that notifies the result of the determination; A passenger safety situation detection device comprising:

2. The detection unit, the human detection unit, the determination unit, and the output unit repeating the process until all of the passengers present in the vehicle are determined to be in a safe condition; 2. The passenger safety condition detection device according to claim 1.

3. A grip detection unit is further provided, The posture detection unit detects whether the passenger is standing or sitting, the grip detection unit detects, for each of the detected passengers, a grip state as to whether or not the passenger is gripping a grip part in the vehicle by analyzing an area within a predetermined range around the detected head based on the detected posture of the passenger when the passenger is standing; The determination unit determines, for each of the detected passengers, whether or not the passengers are in a safe situation by also using the detected gripping state of the passengers.

3. A passenger safety situation detection device according to claim 1 or 2.

4. The output unit is a plurality of units, When the determination unit determines that the passenger is not in a safe situation, the determination unit causes the output unit located closest to the passenger determined to be in an unsafe situation to notify information indicating that the passenger is not in a safe situation. A passenger safety situation detection device according to any one of claims 1 to 3.

5. The detection unit is a plurality of detection units, the posture detection unit associates the same passenger in the first passenger information detected by the first detection unit and the second passenger information detected by the second detection unit, and detects the posture of the passenger based on the first detection information and the second detection information. A passenger safety situation detection device according to any one of claims 1 to 4.

6. the person detection unit detects, when the passengers are present, the number of the passengers and the position of the head of each of the detected passengers; the posture detection unit determines, for each of the detected passengers, whether the passenger is seated or standing based on the detected change in head position. A passenger safety situation detection device according to any one of claims 1 to 5.

7. the detection unit is an imaging device or a distance sensor, When the detection unit is a photographing device, the passenger information is an image, and the detection unit is installed in an area where the passenger standing while the vehicle is moving cannot be present and at a position where the passenger can be overlooked, When the detection unit is a distance sensor, the passenger information is distance information from the detection unit, and the detection unit is installed at a position where the passenger can be seen through. A passenger safety situation detection device according to any one of claims 1 to 6.

8. In a vehicle having an area where passengers can stand, a detection unit installed near a ceiling of the vehicle detects passenger information related to the passengers; a person detection unit detects heads of the passengers using the passenger information, determines whether or not the passengers are present in the vehicle based on the detected heads, and, if the passengers are present, detects the number of the passengers and the position of the head for each of the detected passengers; a posture detection unit detects a posture of each of the detected passengers by analyzing an area within a predetermined range around the detected head using the passenger information; a determination unit that determines, for each of the detected passengers, whether or not the passengers are in a safe situation based on the detected posture of the passengers; The output unit notifies the determined result. Passenger safety situation detection method.

Images