In-vehicle monitoring system, safety status monitoring and control program

Abstract

The system monitors the availability of seats inside the vehicle and, based on that availability, develops appropriate announcements for passengers. [Solution] In the passenger bus 12, the content of the announcement to encourage passenger safety is selected appropriately according to the availability of seats 16. Without the driver having to check the interior of the bus regularly, flexible announcements can be made, such as encouraging passengers to hold on to handrails, etc., when the bus is full, and encouraging them to sit down when there are empty seats. In the seating determination based on the skeletal model of passenger 14, a threshold for difficulty is set for the determination, and if the difficulty level is above a predetermined level, measures are taken to prevent errors in selecting the announcement content. If the announcement content is selected based on a misdetermined determination, an announcement encouraging passenger 14 to sit in an occupied seat will be selected. Therefore, by reducing the number of seats 16 with a difficulty level above the threshold in advance, it is possible to avoid announcements that passenger 14 may find uncomfortable.

Description

Technical Field

[0001] The present invention relates to an in-vehicle monitoring device and a safety situation monitoring control program.

Background Art

[0002] In a vehicle, such as a bus, where seated passengers and standing passengers are mixed, the crew has an obligation to monitor the interior of the vehicle so that a situation where passengers lose their balance does not occur. In the case of a bus, the driver, who is a crew member, has a greater burden while driving and monitoring the interior of the vehicle compared to when there is a dedicated monitor. For this reason, automating the process of monitoring the interior situation of the vehicle has been proposed.

[0003] Patent Document 1 describes acquiring a determination result of individually determining the risk of falling for passengers boarding a vehicle, tracking high-risk individuals determined to have a high risk of falling inside the passenger compartment, individually grasping the state of the high-risk individuals inside the passenger compartment, and restricting the running of the vehicle based on the state of each high-risk individual.

[0004] Patent Document 2 describes imaging an image of a monitoring target space by an imaging unit to generate a skeletal model representing a person included in the image, distinguishing and determining, as the state of the person corresponding to the skeletal model, a state where the person is standing and a state where the person is sitting based on the generated skeletal model, and executing a fall prevention process according to the determination result.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, if, for example, a system is used to monitor passengers' posture using cameras or other means to determine their safety, and the accuracy of the determination is low, it could result in inappropriate announcements being made to passengers.

[0007] For example, displaying a message such as "Please sit in any available seat while the train is in motion" when there are no empty seats could cause discomfort to passengers.

[0008] The present invention aims to provide an in-vehicle monitoring device and a safety situation monitoring and control program that can monitor the vacancy status of seats inside a vehicle and construct appropriate announcements to passengers based on that vacancy status, taking the above facts into consideration. [Means for solving the problem]

[0009] The in-vehicle monitoring device according to the first invention includes a confirmation unit that, in a vehicle equipped with seats where passengers sit, determines the seating status of each passenger for each seat identified by images captured by a camera capable of capturing the range of movement of the passengers, and excludes seats whose difficulty level when determined is above a predetermined threshold from the subject of seating status determination or determines that a passenger is seated, thereby confirming the seating status; and a provision unit that provides warning information to the passengers for safe operation based on the seating status confirmed by the confirmation unit.

[0010] According to the first invention, for each seat identified by images captured by a camera capable of capturing the passenger's movement range, the seating status of the passenger is determined, and for example, the confirmation unit performs a process to exclude seats above a threshold from the seating status determination (process 1) or to determine that a seat is occupied (process 3), thereby confirming the seating status, and based on the confirmed seating status, provides the passenger with cautionary information for safe operation.

[0011] This allows for monitoring the availability of seats inside the vehicle and, based on that availability, creating appropriate announcements for passengers.

[0012] The in-vehicle monitoring device according to the second invention includes a confirmation unit that, in a vehicle equipped with seats where passengers sit, determines the seating status of each passenger for each seat identified by images captured by a camera capable of capturing the range of movement of the passengers, and corrects a standard value for determining that the vehicle is full if the difficulty of determination is above a predetermined threshold, and confirms the seating status; and a provision unit that provides warning information to the passengers for safe operation based on the seating status confirmed by the confirmation unit.

[0013] According to the second invention, instead of process 1 or process 3, the verification unit may perform a process (process 2) to correct the criteria value for determining that the venue is full if the difficulty level is above a predetermined threshold.

[0014] This allows for monitoring the availability of seats inside the vehicle and, based on that availability, creating appropriate announcements for passengers.

[0015] In the first or second invention, the confirmation unit is characterized in that it confirms the seating status of the passengers according to the difference between the number of seats included in a predetermined range of image area captured by the camera and the number of passengers included in the image area.

[0016] The first or second invention is characterized by further comprising an adjustment unit that adjusts the content of the warning information when the seating status of the passenger confirmed by the confirmation unit is in a specific condition.

[0017] In the first or second invention, if the confirmation unit confirms the existence of a vacant seat, the adjustment unit adjusts the information to include a warning message that the vacant seat exists.

[0018] In the first or second invention, if the confirmation unit confirms the existence of an empty seat, the adjustment unit adjusts the information to include a warning message that identifies the location of the empty seat.

[0019] In the first invention or the second invention, when there is luggage on the seat, the confirmation unit regards the seat where the luggage is present as the seat on which the passenger is seated.

[0020] In the first invention or the second invention, when the confirmation unit confirms a safety confirmation state including all passengers being seated, the providing unit stops providing the alert information.

[0021] In the first invention or the second invention, the grasping of the safety situation and the detection of empty seats are performed by the same camera.

[0022] In the first invention or the second invention, it further has a generating unit that generates a skeletal model of a passenger specified from an image captured by the camera, and the confirmation unit confirms the seating situation for each seat from the skeletal model generated by the generating unit.

[0023] The in-vehicle monitoring device according to the third invention is provided in a vehicle having a seat on which a passenger sits and a support member for the passenger to hold and stand. A generating unit that generates a skeletal model of a passenger specified from an image captured by capturing the moving range of the passenger, a seat position acquired from the captured image by the camera, and the skeletal model of the passenger generated by the generating unit. A confirmation unit that confirms the seating situation of the passenger for each seat, a determination unit that determines the safety situation of a passenger who is not seated based on a predetermined skeletal point of the skeletal model generated by the generating unit, a providing unit that provides alert information to the passenger for safe operation, and an adjustment unit that adjusts the content of the alert information when the seating situation confirmed by the confirmation unit and the safety situation determined by the determination unit are in a specific situation.

[0024] According to the third invention, in a vehicle having a seat on which a passenger sits and a support member for the passenger to hold and stand, a skeletal model of a passenger specified from an image captured by capturing the moving range of the passenger is generated.

[0025] Based on the seat positions obtained from the captured images by the camera and the generated passenger skeleton models, check the seating status of passengers for each seat, determine the safety status of passengers who are not seated based on predetermined skeleton points of the generated skeleton models, and provide warning information to the passengers for safe operation.

[0026] Here, when the confirmed seating status and the determined safety status are in a specific situation, adjust the content of the warning information.

[0027] Thereby, the vacant seat status in the vehicle can be monitored, and an appropriate announcement to the passengers can be constructed based on the vacant seat status.

[0028] In the third invention, the determination unit sets as the first condition that a predetermined skeleton point of the skeleton model generated by the generation unit exists within a predetermined specific area that specifies the positions of the seat and the support member, and sets the fulfillment of further conditions of the skeleton points of the passenger as the second condition. When the first condition and the second condition are fulfilled, it is determined as safe.

[0029] The safety status monitoring control program according to the fourth invention causes a computer to operate as the above-described in-vehicle monitoring device.

Effect of the Invention

[0030] As described above, in the present invention, the vacant seat status in the vehicle can be monitored, and an appropriate announcement to the passengers can be constructed based on the vacant seat status.

Brief Description of the Drawings

[0031] [Figure 1] It is a side view of a bus equipped with the in-vehicle monitoring device according to the first embodiment and the second embodiment. [Figure 2] It is a plan view of the bus according to the first embodiment, where (A) shows the right front shooting area, (B) shows the left front shooting area, (C) shows the right rear shooting area, and (D) shows the left rear shooting area. [Figure 3]This is a schematic diagram of an in-vehicle monitoring device according to the first embodiment, installed in a passenger bus. [Figure 4] These are skeletal model diagrams generated in the first and second embodiments. [Figure 5] This is a flowchart showing a safety status monitoring and control routine according to the first embodiment. [Figure 6] This is a control flowchart showing the announcement control interrupt routine according to the first embodiment. [Figure 7] This is a front view of an image captured by a surveillance camera. [Figure 8] This is a control flowchart illustrating the details of the determination processing subroutine executed in step 116 of Figure 5, relating to a second embodiment. [Figure 9] This is an example of monitoring in the in-vehicle monitoring device according to the second embodiment, where (A) is an image captured by a monitoring camera showing a state in which three passengers in different riding positions are present, (B) is a magnified view of the seated passenger in Figure 9(A), (C) is a magnified view of the standing passenger (without holding on) in Figure 9(A), (D) is a magnified view of the standing passenger (with holding on) in Figure 9(A), and (E) is a modified example, a magnified view of the passenger holding on to the handrail. [Modes for carrying out the invention]

[0032] (First Embodiment) Figure 1 is a side view of a passenger bus 12 equipped with the in-vehicle monitoring device 10 (see Figure 3) according to the first embodiment. Note that the passenger bus 12 is just one example; the in-vehicle monitoring device 10 can be applied to any vehicle carrying passengers 14. However, a passenger bus is used as an example of a vehicle where passengers 14 seated in seats 16 and passengers 14 standing are mixed.

[0033] The passenger bus 12 has a driver's seat 18 at the front of the vehicle, and seats 16 for passengers 14 are arranged behind the driver's seat 18. In this configuration, the passenger bus 12 is driven by a driver seated in the driver's seat 18, but it may also be a so-called autonomous bus in which the driver does not directly drive or does not require a driver. An autonomous bus performs various autonomous driving controls on its own, but it is preferable to build a system in which an operator remotely monitors the system.

[0034] The passenger bus 12 is equipped with an entrance / exit 20 for passengers, and passengers 14 board and alight through this entrance / exit 20. In Figure 1, only the entrance / exit 20 on the front side of the vehicle is shown, but there are also passenger buses that have entrance / exits in the middle or rear of the vehicle, and the number of entrances / exits is not limited.

[0035] The floor surface 22 on which the seats 16 are located is stepped due to the vehicle's structure, and the steps rise towards the rear of the vehicle, making it less safe for passengers 14 to move around compared to a flat surface. Even if it were a flat surface (a low-floor bus), the driver 24 seated in the driver's seat 18 has an obligation to monitor the interior of the vehicle using rearview mirrors (not shown) etc., while driving, to prevent situations where passengers 14 might lose their balance. With a stepped floor surface 22, even more attention must be paid to the interior of the vehicle.

[0036] In other words, from the perspective of safety monitoring, passengers 14 seated in seat 16 are evaluated as "safe," while passengers 14 not seated are evaluated as "unsafe."

[0037] As shown in Figure 2, surveillance cameras 30, which constitute the in-vehicle monitoring system 10, are installed in each of the four corners (front right, front left, rear right, and rear left) of the passenger bus 12. The surveillance cameras 30 are designed to capture images of the area in which passengers 14 can move within the vehicle without creating blind spots. Existing safety confirmation cameras installed to understand the safety situation inside the vehicle may be used as the surveillance cameras 30, or new dedicated cameras for determining whether seats are full or empty may be installed. Using existing safety confirmation cameras leads to a reduction in the number of devices, while using dedicated cameras can improve the accuracy of the determination, so both may be used in combination depending on the appropriate application.

[0038] Figure 2(A) shows the field of view of the front right surveillance camera 30A, indicated by the dashed line A. It can be seen that the field of view of surveillance camera 30A is the seat 16 located on the front right side of the passenger bus 12.

[0039] Figure 2(B) shows the field of view of the left front surveillance camera 30B, indicated by the dashed line B. It can be seen that the field of view of surveillance camera 30B is the seat 16 located on the left front of the passenger bus 12.

[0040] Figure 2(C) shows the field of view of the right rear surveillance camera 30C, indicated by the dashed line C. It can be seen that the field of view of the surveillance camera 30C is the seat 16 located at the right rear of the passenger bus 12.

[0041] Figure 2(D) shows the area of ​​view of the left rear surveillance camera 30D, indicated by the dashed line D. It can be seen that the area of ​​view of surveillance camera 30D is the seat 16 located at the left rear of the passenger bus 12.

[0042] In this first embodiment, the shooting areas of surveillance camera 30C and surveillance camera 30D partially overlap. Since the seat 16 located in this overlapping shooting area can be determined by the two surveillance cameras 30C and 30D, it is possible to improve the reliability of the seating status determination result. In this embodiment, the shooting areas of surveillance camera 30C and surveillance camera 30D partially overlap, but overlapping areas may also be provided for two or more other surveillance cameras 30.

[0043] (In-vehicle monitoring device 10) Figure 3 is a schematic diagram of the in-vehicle monitoring device 10 according to the first embodiment, which is installed in a passenger bus 12 (see Figure 1).

[0044] The in-vehicle monitoring device 10 includes n monitoring cameras 30 (where n is a positive integer, and in the first embodiment, n=4) and a safety status monitoring control unit 32.

[0045] When installing multiple surveillance cameras 30, the cameras 30 do not need to be the same model or have the same specifications. Instead, the most suitable camera 30 should be used for each specific location, with the optimal specifications (magnification, zoom range, wide-angle range, aperture, shutter speed, pan / tilt angle, etc.).

[0046] Furthermore, the surveillance camera 30 only needs to be able to capture the outlines of the passengers 14, and is not limited to images based on density, but may also be images based on temperature differences using infrared light, etc.

[0047] The surveillance camera 30 is connected to the image acquisition unit 34 of the safety situation monitoring control unit 32.

[0048] The image acquisition unit 34 is connected to the passenger identification unit 36 ​​and sends the acquired image information to the passenger identification unit 36.

[0049] The passenger identification unit 36 ​​uses image recognition technology to identify the location of the passengers 14 and sends it to the target determination unit 40. The target determination unit 40 is connected to the skeleton model generation unit 44. The skeleton model generation unit 44 is connected to the judgment instruction unit 46, and when a judgment instruction trigger is input to the judgment instruction unit 46, the judgment instruction unit 46 instructs the skeleton model generation unit 44 to generate a skeleton model for each passenger 14.

[0050] Possible triggers for the judgment instruction include, for example, periodic events (trigger 1), when the acceleration of the passenger bus 12 changes by more than a predetermined threshold (trigger 2), and when the driving history information of the passenger bus 12 (starting, stopping, accelerating, decelerating, turning) is detected (trigger 3).

[0051] The skeletal model is generated using, for example, a posture estimation AI. This AI is a technology that visualizes a person's posture by detecting feature points such as their face and joints through their clothing. As shown in Figure 4, the posture of the person (passenger 14) is represented by connecting skeletal points No. 1 to No. 16. While conventional methods required markers on the body surface or the use of inertial sensors, the posture estimation AI allows for the detection of skeletal information simply by capturing images with the surveillance camera 30.

[0052] The skeletal model generation unit 44 is connected to the person counting unit 48 within the shooting area. Using each skeletal model, the unit counts the number of passengers seated in the seats 16 within the shooting area by determining, for example, whether the nose position (No. 0 in Figure 4) and both shoulders (No. 5 and No. 6 in Figure 4) of each passenger 14 are within the range of the seats 16 in the area shown in Figure 2.

[0053] The person counting unit 48 within the shooting area is connected to the seat number adjustment unit 50 and the full / empty seat determination unit 52.

[0054] The seating adjustment unit 50 adjusts the number of seats if there are seats 16 with a difficulty level higher than a predetermined level. The adjustment of the number of seats is performed by executing the following process 1. The difficulty level can be expressed, for example, by the ratio of the area that obstructs the screen of seat 16 in the shooting area captured by the camera 30 (blind spot ratio). If the blind spot ratio is above a predetermined threshold (for example, 30%), it is determined that the difficulty level is high.

[0055] (Process 1) Seat 16, which is judged to be difficult, is excluded from the evaluation. For example, if there are 5 seats in the shooting area and 1 seat 16 is judged to be difficult, seat 16 itself is recognized as 4 seats.

[0056] Furthermore, since the presence or absence of blind spots in the shooting area of ​​camera 30, or the area of ​​the blind spot, differs depending on the installation location, the threshold value may be changed for each camera 30. Also, the threshold value may be changed depending on the time of day, such as during off-peak hours and peak hours. Furthermore, the difficulty of determination is not limited to the blind spot ratio of seat 16, but if the number of standing seats 16 exceeds a certain number, the shooting range of camera 30 will be obstructed by the standing seats 16, so the difficulty of determination may be uniformly set to "high". In addition, if seat 16 is hidden by seat 16 or luggage, it may be determined that it is not an empty seat.

[0057] The seat number adjustment unit 50 reads the number of seats present within the determined shooting area from the seat number storage unit 54 (for example, 5 seats in shooting area A in Figure 2(A)), subtracts the number of people received from the shooting area person count unit 48, and sends the result to the full / empty seat determination unit 52.

[0058] The occupancy / vacancy determination unit 52 compares the count value received from the person count unit 48 within the shooting area with the adjusted number of seats received from the seat number adjustment unit 50, and determines whether each corresponding shooting area (see shooting areas A to D in Figures 2(A) to (D)) is full or vacant.

[0059] In the above description, process 1 is executed as the adjustment of the number of seats in the seat adjustment unit 50, but process 2 or process 3 below may also be executed.

[0060] (Process 2) If there are 16 seats that are judged to be difficult to use, the occupancy rate (threshold for determining full occupancy), which is the standard value for determining full occupancy, is corrected (in other words, the number of seats used to determine whether the room is full or not is set to a negative value). For example, if there are 5 seats in the shooting area, and under normal circumstances a seating rate of 5 / 5 is used to determine full occupancy, but one seat is judged to be difficult to use, then the room will be determined to be full occupancy when the seating rate reaches 4 / 5. In this case, the standard seating rate of 4 / 5 may be recognized as a fixed threshold (100%), or the threshold itself may be made variable, and in the above case (4 / 5), the threshold may be set to 80%.

[0061] (Process 3) For seats 16 that are judged to be difficult to occupy, the judgment is made by assuming that a passenger 14 is seated there. For example, if there are a total of 5 seats 16, and 4 of the seats 16 that are not difficult to occupy are judged to be occupied, and 1 of the seats that are difficult to occupy is judged to be occupied, the occupancy rate is 5 / 5 and the seats are judged to be full.

[0062] Process 1 is an algorithm that reduces the number of seats that can be occupied within the shooting area, Process 2 is an algorithm that corrects the number of seats that can be identified, and Process 3 is an algorithm that determines that seat 16, which is difficult to identify, is occupied by passenger 14. The result will be the same regardless of whether Process 1 to Process 3 is used.

[0063] The occupancy / vacancy determination unit 52 is connected to the announcement content selection unit 56 and sends the determination result (occupancy or vacancies) to the announcement content selection unit 56.

[0064] The announcement content selection unit 56 has multiple types of announcement content pre-registered for specific situations (pre-defined situations including full capacity and empty seats). Note that the specific situations include not only full capacity and empty seats, but also situations that take into account the distance between standing passengers 14 and empty seats, and situations where passengers 14 identified as elderly or children are standing, etc.

[0065] For example, if the train is full (no seats available), an announcement such as "Please hold on tightly to the handrails or straps while the train is in motion" will be selected.

[0066] On the other hand, if there are empty seats, the announcement will be something like, "Please sit in any available seat while the train is in motion, or hold on tightly to the handrails or straps."

[0067] Furthermore, by associating each shooting area (see shooting areas A to D in Figures 2(A) to (D)), if the front of the passenger bus 12 (shooting area A in Figure 2(A) and shooting area B in Figure 2(B)) is determined to be full, and the rear (shooting area C in Figure 2(C) or shooting area D in Figure 2(D)) is determined to have empty seats, the announcement content selected will be, "There are empty seats in the rear, please feel free to sit down."

[0068] Furthermore, if it is possible to determine not only the seating status of passengers 14 but also the posture of standing passengers 14 (whether or not they are holding onto a strap 26, etc.), the announcement content will be selected to include a message such as, "Please hold onto a strap or handrail while the train is in motion." Note that determining passengers 14 other than their seating status requires specific skeletal structure determination processing, which will be explained in detail in the second embodiment.

[0069] The announcement content selection unit 56 is connected to the announcement execution unit 58, and the selected announcement content is sent to the announcement execution unit 58. The announcement execution unit 58 controls an output device (e.g., a speaker or monitor) to output the selected announcement content as audio or display.

[0070] The operation of the first embodiment will be described below with reference to the flowcharts in Figures 5 and 6.

[0071] Figure 5 shows the safety status monitoring control routine in the safety status monitoring control unit 32 according to the first embodiment.

[0072] Step 100 determines whether a trigger for starting the judgment has been input. Triggers include periodic events (trigger 1), such as immediately before departure from a bus stop (after the doors close), changes in acceleration detected by sensors (trigger 2), and travel history information.

[0073] If no trigger inputs such as Trigger 1 to Trigger 3 are received, and the result in step 100 is negative, the process proceeds to step 132 to determine whether or not the operation of the passenger bus 12 has ended. If the result in step 132 is negative, the process returns to step 100; if the result is positive, this routine terminates.

[0074] If triggers such as Trigger 1 to Trigger 3 are input and a positive result is obtained in step 100, the system proceeds to step 102 to identify the passenger 14 on board, and then proceeds to step 112.

[0075] In step 112, the following loop processing steps 114 and 116 is started in a predetermined order, corresponding to the number of passengers (14) currently on board.

[0076] (Steps 114 and 116 of the loop processing) In step 114, a skeletal model of the selected passenger 14 is generated (skeletal points are identified), and then the process proceeds to step 116 to count the number of seated passengers for each imaging area A to D. Hereafter, when referring to the number of people, unless otherwise specified, it refers to the number of seated passengers, excluding infants and toddlers sitting on laps.

[0077] In the next step, 118, the loop ends when the judgment process for all 14 passengers on board is complete, and the program proceeds to step 120.

[0078] Step 120 determines whether there are any judgments that were counted with a judgment difficulty level (in this case, the blind spot ratio of seat 16) above a threshold.

[0079] If the result in step 120 is positive, it is determined that there is an ambiguous judgment. At this point, it is unclear whether the judgment is correct or incorrect.

[0080] If the judgment is incorrect, and one of the errors is a situation where passenger 14 is seated in seat 16 but is not actually seated, then selecting an announcement based on this incorrect judgment will result in an announcement prompting passenger 14 to move to an empty seat.

[0081] To avoid such an undesirable choice, in the first embodiment, if a positive result is obtained in step 120, the system proceeds to step 124 to adjust the number of seats in the corresponding shooting areas A to D (-1), and then proceeds to step 126. In other words, the seats 16 are excluded from the selection of seats for announcement content. If a negative result is obtained in step 120, it is determined that there is no ambiguity, and the system proceeds to step 126.

[0082] Step 126 determines whether the number of people (seated) in each shooting area A to D is equal to or greater than the adjusted number of seats.

[0083] If the result in step 126 is positive, each imaging area A to D will be determined to be full, and the process will proceed to step 128. If the result in step 130 is negative, each imaging area A to D will be determined to be vacant, and the process will proceed to step 132.

[0084] Step 132 determines whether or not the operation of the passenger bus 12 has ended. If the result in step 132 is negative, the process returns to step 100; if the result is positive, this routine ends.

[0085] Figure 6 is a flowchart showing the announcement control routine that is interrupted in response to the process shown in Figure 5.

[0086] Step 140 determines whether or not it is appropriate to make an announcement. Examples include before and after passengers 14 board or alight, before traveling on a curve with a radius of curvature greater than a specified value, during acceleration or deceleration greater than a specified value, and other times when standing passengers 14 may become unstable. The announcement may be made regularly or irregularly during operating hours, or at the discretion of the driver. Alternatively, multiple timings for making the announcement may be combined.

[0087] If the result in step 140 is negative, it is determined that it is not time to make an announcement, and this routine ends. If the result in step 140 is positive, it is determined that it is time to make an announcement, and the process proceeds to step 142.

[0088] In step 142, the "full" or "available" determination result for each shooting area A to D is obtained, and then the process proceeds to step 144, where the announcement content is selected based on the determination result. An example of the announcement content based on the determination result is shown below.

[0089] (If the train is full (no seats available)) "Please hold on tightly to the handrails or straps while the train is in motion."

[0090] (If there are empty seats) "Please sit in any available seat while the train is in motion, or hold on tightly to the handrails or straps."

[0091] Here, you can also select the announcement content by relating the judgments for each shooting area A to D.

[0092] (If the front is full and the back is empty) "There are empty seats in the back, please feel free to sit down."

[0093] In the next step, 146, the announcement is made based on the selected announcement content. While the announcement is typically made via speaker output, a monitor may be installed on the wall of the passenger bus 12 (such as in a section of the advertising area) to display the announcement content. The monitor may also display the location of available seats.

[0094] As described above, in the first embodiment, the passenger bus 12 is configured to appropriately select the content of announcements promoting the safety of passengers 14 according to the availability of seats 16. For example, without the driver having to check the interior of the bus regularly, flexible announcements can be made, such as encouraging passengers to hold on to handrails, etc., when the bus is full, and encouraging them to sit down when there are empty seats.

[0095] Furthermore, in the seating determination based on the skeletal model of passenger 14, a threshold is set for the blind spot ratio of seat 16, which represents the difficulty of determination. If the blind spot ratio is above the threshold, it is considered difficult, and measures are taken to prevent errors in selecting the announcement content. In other words, if the announcement content is selected based on a misdetermination (a situation where passenger 14 is seated in seat 16 but is not seated), an announcement urging passenger 14 to move to an empty seat will be selected. Therefore, by reducing the number of seats 16 that are considered difficult (blind spot ratio above the threshold) in advance (-1), the announcement content will not be selected. This makes it possible to avoid announcements that passenger 14 may find uncomfortable.

[0096] (Second Embodiment) A second embodiment of the present invention will be described below.

[0097] In the second embodiment, the same reference numerals are used for components identical to those in the first embodiment, and their descriptions are omitted.

[0098] The second embodiment is characterized by a configuration aimed at improving the accuracy of determining the seating status of passengers 14 based on a skeletal model, which was also applied in the first embodiment.

[0099] As shown in Figure 7, the passenger bus 12 is equipped with support members such as hand straps 26, support bars 26A that support the hand straps 26, and handrails 28 to prevent standing passengers 14 from losing their balance. By holding onto the support members (by holding on), standing passengers 14 can maintain their posture against the swaying of the bus during travel, compared to passengers 14 who are not holding on. The determination of whether or not a passenger is holding onto a support member will be described later. Other support members include assist grips provided at the upper corners of the seat backs.

[0100] In other words, from the perspective of safety monitoring, passengers 14 seated in seats 16 and passengers 14 holding on to a seat are evaluated as "safe," while passengers 14 not holding on to a seat are evaluated as "unsafe."

[0101] The skeletal model generation unit 44 is equipped with a judgment processing function and uses each skeletal model to determine the safety (safe / unsafe) of each passenger 14. Details of the judgment processing will be described later.

[0102] The judgment processing function outputs the judgment result to a predetermined device. The predetermined device is a monitor installed in the driver's seat 18, which displays the positions of passengers 14 in a safe state and passengers 14 in an unsafe state as a simplified image (for example, a color-coded circular image). This allows the driver 24 to check the display on the monitor with less eye movement than when checking with mirrors, etc. (while maintaining forward visibility for driving), and to understand the status of the passengers 14.

[0103] (Decision process) Figure 7 shows an image captured by the surveillance camera 30, with predetermined regions E0 to E10 (see the solid line frame in Figure 7) set in advance. The setting of regions E0 to E10 may be performed automatically using image recognition. In Figure 7, region E10 is set for the handrail 28, but the handrail image itself may be designated as region E10. Furthermore, the number of regions is not limited to E0 to E10, and varies depending on the vehicle to which the in-vehicle monitoring device 10 is applied.

[0104] The reason why the hand strap area includes a support bar 26A for the hand strap 26 is that, for example, if a passenger 14 who is taller than average boards the train, it may be difficult for them to grasp the hand strap 26 which is positioned at a height appropriate for their average height, and they may instead grasp the support bar 26A located above it.

[0105] Areas E0 to E10 represent the areas where the passenger's face and upper body are located when the passenger 14 is seated on seat 16, the hand strap 26, the support bar 26A that supports the hand strap 26, and the area where the passenger 14's hands are located when holding onto the handrail 28. In the following, the area including the hand strap 26 and the support bar 26A will be collectively referred to as the hand strap area.

[0106] "Provisional judgment conditions (Condition 1)" In the first embodiment, a provisional determination condition is made to determine whether specific skeletal points (No. 1 to No. 16) of the skeletal model are located in a specific region (E0 to E10 shown in Figure 7) (see Determination 1 and Determination 2 below).

[0107] (Determination 1) Determine whether the person is seated by checking whether all of the skeletal points of the nose and both shoulders (No. 1 to No. 6 shown in Figure 4) are included in the seat area (E0 to E7 shown in Figure 7). In the first embodiment, since three points (nose and both shoulders) are used compared to the case where only one skeletal point is used to determine the seat area, misdeterminations can be suppressed.

[0108] (Determination 2) Determine whether the skeletal points of the wrist (No. 9 or No. 10 shown in Figure 4) are included in the strap area (E8 or E9 shown in Figure 7) or the handrail area (E10 shown in Figure 7), and determine whether the standing passenger 14 is holding onto the strap 26, support bar 26A, or handrail 28.

[0109] "Confirmed judgment condition (second condition)" In this case, under the provisional judgment conditions, situations may arise where it is uncertain whether the person is seated or holding onto the strap 26.

[0110] Therefore, in the second embodiment, as a determination condition to confirm the provisional determination which is executed when the provisional determination condition is met, the determination of the provisional determination condition is verified based on the angles and relative positional relationships of specific skeletal points (No. 1 to No. 16) of the skeletal model (see determination 3 and determination 4 below).

[0111] (Decision 3) If the conditions in Decision 1 above are met, the system further determines whether the angle of a predetermined skeletal point in the lower body, for example, the angle of the knee or the hip (No. 11 to No. 14), is below a threshold, and confirms that the person is seated. In the first embodiment, the system checks whether the angle of the knee or hip is below a threshold, that is, whether the knee or hip is bent. Note that the angle determination may be performed on both the knee and the hip. This prevents the system from mistakenly determining that a passenger 14 who is deviating from a proper seated position is seated safely.

[0112] Furthermore, in terms of judgment accuracy, rather than seeking absolute accuracy, it may be acceptable to shift the judgment criteria towards judging a situation as unsafe in ambiguous circumstances, from the perspective of drawing attention to passengers 14 in unsafe conditions.

[0113] (Decision 4) If the conditions in Decision 2 are met, further, it is determined whether the hand is higher than the shoulder based on the relative positional relationship of two specific skeletal points on the upper body, for example, whether the wrist skeletal points (No. 9 and No. 10) are higher than the shoulder skeletal points (No. 5 and No. 6), and it is confirmed that the person is standing and holding onto the strap 26. According to Decision 4, compared to Decision 2, which simply determines whether the person is holding onto the strap 26 based on the wrist skeletal points alone, it is possible to suppress misjudgments, for example, caused by the wrist being captured in the image of another strap area due to the angle of view of the surveillance camera 30.

[0114] In the first embodiment, the height determination between the wrist skeletal points (No. 9 and No. 10) and the shoulder skeletal points (No. 5 and No. 6) was performed using two-dimensional coordinates. However, the height position may also be determined by three-dimensional analysis (depth analysis) of the image captured by the surveillance camera 30.

[0115] In the above, the judgment criteria are set mainly based on the gripping state of the strap 26, but the judgment specifications are the same for the support bar 26A and the handrail rod 28, and the judgment criteria values ​​can be adjusted as needed.

[0116] In other words, the relative positions of the skeletal points used for grip detection may need to be different for the handrail 28 (which extends in the vertical direction) and the strap 26. For example, an example of grip detection for the handrail 28 is shown below.

[0117] (Example of gripping determination for handrail 28) One possible cause of false detection is simply that the hand of passenger 14, who is standing near the handrail 28, is within area E10.

[0118] Therefore, as a preliminary determination, we will determine whether or not the skeletal points of the wrist fall within the region E10 of the handrail 28.

[0119] Next, as a final determination, it is judged whether the wrist skeletal point is between the shoulder skeletal point and the waist skeletal point in the height direction. For example, in the state shown in Figure 9(E), the wrist skeletal point of passenger 14 is between the shoulder skeletal point and the waist skeletal point in the height direction, so it is judged as safe. The natural gripping position of the handrail 28 is slightly below shoulder height, and it is unlikely that a person would deliberately grip the pole at a position higher than their shoulder when gripping the handrail 28. When the hands are lowered, the wrist is normally lower than the waist.

[0120] The operation of the second embodiment will be explained below with reference to the flowchart in Figure 8.

[0121] Figure 8 shows a determination processing subroutine for determining the state of passenger 14, which is a prerequisite for performing the seating count processing to determine the number of people seated for each shooting area A to D, as performed in step 116 of Figure 5.

[0122] More specifically, it is determined whether the seated passengers 14, the standing passengers 14, and the standing passengers 14 are holding onto support members (such as straps 26, support bars 26A, and handrails 28).

[0123] In step 150, as the first provisional determination condition, it is determined whether the skeletal points of the nose and both shoulders (No. 1 to No. 6 shown in Figure 4) are all included in the seat area (E0 to E7 shown in Figure 7), and whether the person is seated or not.

[0124] If a positive result is obtained in step 150, there is a high probability that passenger 14 is seated, but it is possible that they are simply bending over with their face within the seating area (E0~E7 shown in Figure 7). Therefore, if a positive result is obtained in step 150, the process moves to step 152, where, as the third determination condition for confirmation, it is determined whether the angle of either the knees or hips (NO.11~NO.14) is below a threshold. If a positive result is obtained, the process moves to step 154 ​​to confirm that the passenger is seated. With this confirmation of seating, passenger 14 is determined to be "safe," and the process moves to step 166, where the determination result is output and the process returns.

[0125] On the other hand, if a negative determination is made in step 150, or in step 152, it is determined that passenger 14 is not seated, and the process proceeds to step 156, where a provisional determination is made that the passenger is standing (unsafe), and the process proceeds to step 158.

[0126] Furthermore, taking into consideration situations such as a seated passenger 14 temporarily standing up and sitting down again (hereinafter referred to as a "temporary unsafe state"), it is possible to determine that an unsafe state is not an unsafe state immediately, but rather when the temporary unsafe state continues for a predetermined time (a predetermined number of imaging frames).

[0127] In step 158, as the second determination of the provisional determination condition, it is determined whether the skeletal point of the wrist (No. 9 or No. 10 shown in Figure 4) is included in the handrail area (E8 or E9 shown in Figure 7), and it is determined whether the standing passenger 14 is holding onto a handrail.

[0128] If a positive determination is made in step 158, it is determined that passenger 14 is standing and there is a high probability that they are holding onto the strap 26. However, due to the position and angle of the surveillance camera 30, it may appear as if the wrist is inside the area even though it is not actually there, and the system may determine that the arm is inside the strap area (E8 or E9 shown in Figure 7). Therefore, if a positive determination is made in step 158, the system proceeds to step 160, where, as the fourth determination for confirmation, it is determined whether the position of the wrist skeletal points (No. 9 and No. 10 shown in Figure 4) is higher than the shoulder skeletal points (No. 5 and No. 6 shown in Figure 4). If this determination is positive, the system proceeds to step 162, where it is confirmed that passenger 14 is standing and holding onto the strap 26. With this confirmation that passenger 14 is standing and holding onto the strap 26, the system determines that passenger 14 is "safe," and proceeds to step 166, where the determination result is output and the system returns.

[0129] In step 160 above, the angle determination setting is specific to the strap 26, but in parallel, the determination may also be performed by setting numerical values ​​for the support bar 26A and the handrail 28.

[0130] On the other hand, if a negative determination is made in step 158 or step 160, it is determined that passenger 14 is standing and not holding onto the strap 26, and the process proceeds to step 164, where a determination of non-holding of the strap (unsafe) is made, and the process proceeds to step 166, where the determination result is output and the system returns.

[0131] By reflecting this assessment and selecting the content of the announcement accordingly, it becomes possible to deliver even more detailed warning announcements.

[0132] (Examples) The following describes an example of determining the state of passenger 14 in the second embodiment.

[0133] Figure 9(A) is an image captured by a surveillance camera 30 showing the in-vehicle image shown in Figure 7, with three passengers 14 in different configurations.

[0134] Figure 9(B) shows passenger 14 seated in seat 16. Skeletal points No. 1 to No. 6 are included in a specific region E0 (Judgment 1 is met), and the angle of the knee (No. 11) is below the threshold (Judgment 3). Therefore, seating is confirmed, and the judgment is "stable".

[0135] Figure 9(C) shows passenger 14 in a standing position (Judgment 1 and Judgment 2 are not met), resulting in a judgment of "unstable".

[0136] Figure 9(D) shows passenger 14 in a standing position (Judgment 1 is not met, Judgment 2 is met), and the position of the wrist skeletal point (No. 10) is higher than the shoulder skeletal point (No. 6), so it is confirmed that the passenger is holding onto the strap 26 and is judged as "stable".

[0137] As described above, in the second embodiment, in addition to a preliminary determination (determination of whether specific skeletal points (No.1 to No.16) of the skeletal model exist in a specific region (E0 to E10)), a final determination (determination based on the angle and relative positional relationship of specific skeletal points (No.1 to No.16) of the skeletal model) is performed to verify the determination of the preliminary determination conditions. This makes it possible to accurately determine whether the passenger 14 is seated, standing, or holding onto the handrail 26 while standing.

[0138] Furthermore, in the second embodiment, appropriate announcements can also be made to standing passengers 14. For example, if the seats 16 are full and there are standing passengers 14, but some of these passengers 14 are not holding onto any support members, the system can select the most appropriate announcement for standing passengers, such as, "Standing passengers, please hold onto a handrail or similar if possible."

[0139] In the first and second embodiments, the skeletal points used for determining seating were the face and both shoulders. However, the determination is not limited to the face and both shoulders; the positions of other skeletal points or the relative positions of other skeletal points may also be used.

[0140] Furthermore, in the first and second embodiments, safety / unsafety determinations were made for all passengers 14. However, passengers 14 may be classified into categories such as children (e.g., 12 years old and under), adults (13 to 59 years old), and the elderly (60 years old and over), and safety / unsafety determinations may be limited to children and the elderly. Adults have physical strength and can maintain a balanced standing posture without holding on, so to reduce the processing burden, the subjects of determination may be selected. In this case, classification can be performed by facial recognition based on images captured by the surveillance camera 30.

[0141] Furthermore, in the second embodiment (particularly the determination processing subroutine in Figure 8), a determination was made as to whether or not the standing passenger 14 was holding onto the strap 26, but this could be replaced with holding onto other support members such as handrails.

[0142] Furthermore, in judgment 4, for example, it was determined whether the position of the wrist skeletal points (No. 9 and No. 10) was higher than the shoulder skeletal points (No. 5 and No. 6). However, depending on the height of the passenger 14, it is possible that the passenger is holding onto the strap 26 even if the position of the wrist skeletal points (No. 9 and No. 10) is lower than the shoulder skeletal points (No. 5 and No. 6). Therefore, a correction may be made depending on the height of the passenger 14 being judged. That is, with the correction, for example, it may be determined that it is safe even if the height of the wrist is the same as or a certain amount lower than the height of the shoulder.

[0143] Furthermore, in addition to the passenger's height, the positional relationship between the wrist skeletal points (No. 9 and No. 10) and the shoulder skeletal points (No. 5 and No. 6) may shift depending on the position and angle of the surveillance camera 30. For this reason, when using multiple surveillance cameras 30, it may be necessary to change the correction value (threshold) for each surveillance camera.

[0144] Furthermore, there may be luggage placed on seat 16. In such cases, it is conceivable that an announcement could be made to identify the luggage and prompt passenger 14 to place the luggage on their lap. However, in the first and second embodiments, priority was given to preventing any confusion among passengers 14, etc., and seat 16 with luggage on it was treated as occupied, and the system was adapted accordingly. Moreover, a seated passenger 14 may be holding a child, but if the child is also detected using image recognition technology, it is possible to confirm that the seat is occupied (even if there are two faces within the area).

[0145] In this embodiment, image recognition technology was used to identify the passengers on board. However, since passengers frequently get on and off the bus 10, bounding technology may be used to identify the position of each passenger 14, assign an identification symbol to each passenger 14, and track the movement of each passenger 14.

[0146] In other words, this tracking allows us to add passengers 14 who have boarded and remove passengers 14 who have disembarked, thereby determining which passengers 14 will follow the movement.

[0147] [Note] (Note 1)

[0148] In a vehicle equipped with seats for passengers, a confirmation unit determines the seating status of each passenger for each seat identified by images captured by a camera capable of capturing the passenger's movement range, and excludes seats where the difficulty of determination exceeds a predetermined threshold from the seating status determination or determines that a passenger is seated, thereby confirming the seating status. Based on the seating status confirmed by the confirmation unit, a provision unit provides passengers with cautionary information for safe operation. An in-vehicle monitoring device having the following features.

[0149] (Note 2) In a vehicle equipped with seats for passengers, a confirmation unit determines the seating status of each passenger for each seat identified by images captured by a camera capable of capturing the passenger's movement range, and corrects the criteria value for determining full occupancy if the difficulty of determination exceeds a predetermined threshold, thereby confirming the seating status. Based on the seating status confirmed by the confirmation unit, a provision unit provides passengers with cautionary information for safe operation. An in-vehicle monitoring device having the following features.

[0150] (Note 3) The in-vehicle monitoring device according to claim 2, wherein the reference value is the denominator of a fraction representing the seating rate at which the vehicle is determined to be full, or a threshold value for the seating rate at which the vehicle is determined to be full.

[0151] (Note 4) The aforementioned verification unit, An in-vehicle monitoring device according to any one of Appendix 1 to Appendix 3, which compares the number of seats included in a predetermined range of image area captured by the camera with the number of passengers included in the image area to confirm the seating status of the passengers.

[0152] (Note 5) An in-vehicle monitoring device according to any one of the appendices 1 to 4, further comprising an adjustment unit that adjusts the content of the warning information when the seating status of the passenger confirmed by the confirmation unit is in a specific condition.

[0153] (Note 6) If the confirmation unit confirms the existence of an empty seat, the adjustment unit adjusts the in-vehicle monitoring device as described in Appendix 5 to include warning information that the empty seat exists.

[0154] (Note 7) If the confirmation unit confirms the existence of an empty seat, the adjustment unit adjusts the in-vehicle monitoring device as described in Appendix 5 or Appendix 6 to include warning information that identifies the location of the empty seat.

[0155] (Note 8) The in-vehicle monitoring device described in any one of Appendix 1 to Appendix 7, wherein the confirmation unit deems that a seat with luggage is occupied by a passenger if luggage is present in that seat.

[0156] (Note 9) The aforementioned verification unit is an in-vehicle monitoring device as described in any one of the appendices 1 to 8, which divides the seat area into sections and makes judgments using multiple cameras.

[0157] (Note 10) The in-vehicle monitoring device according to any one of Appendix 1 to Appendix 9, wherein the confirmation unit changes the threshold value of the difficulty level between shooting areas.

[0158] (Note 11) The in-vehicle monitoring device described in any one of Appendix 1 to Appendix 10, wherein when the confirmation unit confirms that all passengers are seated and the safety status is confirmed, the provision unit discontinues providing the warning information.

[0159] (Note 12) An in-vehicle monitoring device described in any one of the appendices 1 to 11, which uses the same camera to assess safety conditions and detect empty seats.

[0160] (Note 13) An in-vehicle monitoring device according to any one of Appendix 1 to Appendix 12, further comprising a generation unit that generates a skeletal model of a passenger identified from an image captured by the aforementioned camera, and a confirmation unit that confirms the seating status of each seat from the skeletal model generated by the generation unit.

[0161] (Note 14) In a vehicle equipped with seats for passengers to sit on and support members for passengers to hold onto to stand, a generation unit generates a skeletal model of the passenger identified from images taken with a camera capable of capturing the passenger's range of movement, A confirmation unit that checks the seating status of each passenger for each seat based on the seat position obtained from the image captured by the aforementioned camera and the passenger's skeletal model generated by the generation unit, A determination unit determines the safety status of a passenger who is not seated based on predetermined skeletal points of the skeletal model generated by the generation unit, A service department that provides passengers with cautionary information for safe operation, If the seating status confirmed by the confirmation unit and the safety status determined by the determination unit are in a specific state, the adjustment unit adjusts the content of the warning information, An in-vehicle monitoring device having the following features.

[0162] (Note 15) The determination unit, An in-vehicle monitoring device as described in Appendix 14, wherein the first condition is that predetermined skeletal points of the skeletal model generated by the generation unit are located within a predetermined specific area that identifies the positions of the seat and the support member, and the second condition is that further conditions for the skeletal points of the passenger are met, and safety is determined when the first and second conditions are met.

[0163] (Note 16) Computers, A safety status monitoring and control program that operates as an in-vehicle monitoring device as described in any one of the appendices 1 to 15. [Explanation of Symbols]

[0164] 10 In-vehicle monitoring device, 12 Passenger bus, 14 Passengers, 16 Seats, 18 Driver's seat, 20 Entrance / exit, 22 Floor surface, 24 Driver, 26 Handrail, 28 Handrail, 30 (30A, 30B, 30C, 30D) Surveillance camera (camera), 32 Safety situation monitoring control unit, 34 Image acquisition unit, 36 Passenger identification unit, 40 Target determination unit, 44 Skeleton model generation unit, 46 Judgment support unit, 48 Person count unit within shooting area, 50 Seat number adjustment unit, 52 Full / empty seat determination unit, 54 Seat number storage unit

Claims

1. In a vehicle equipped with seats for passengers, a confirmation unit determines the seating status of each passenger for each seat identified by images captured by a camera capable of capturing the passenger's movement range, and excludes seats where the difficulty of determination exceeds a predetermined threshold from the seating status determination or determines that a passenger is seated, thereby confirming the seating status. Based on the seating status confirmed by the confirmation unit, a provision unit provides passengers with cautionary information for safe operation. An in-vehicle monitoring device having the following features.

2. In a vehicle equipped with seats for passengers, a confirmation unit determines the seating status of each passenger for each seat identified by images captured by a camera capable of capturing the passenger's movement range, and corrects the criteria value for determining full occupancy if the difficulty of determination exceeds a predetermined threshold, thereby confirming the seating status. Based on the seating status confirmed by the confirmation unit, a provision unit provides passengers with cautionary information for safe operation. An in-vehicle monitoring device having the following features.

3. The in-vehicle monitoring device according to claim 2, wherein the reference value is the denominator of a fraction representing the seating rate at which a vehicle is determined to be full, or a threshold value for the seating rate at which a vehicle is determined to be full.

4. The aforementioned verification unit, The in-vehicle monitoring device according to claim 1, which compares the number of seats included in a predetermined range of image area captured by the camera with the number of passengers included in the image area to confirm the seating status of the passengers.

5. The in-vehicle monitoring device according to claim 1, further comprising an adjustment unit that adjusts the content of the warning information when the seating status of the passenger confirmed by the confirmation unit is in a specific state.

6. If the confirmation unit confirms the existence of an empty seat, the adjustment unit adjusts the information to include a warning message indicating the existence of the empty seat, as described in claim 5.

7. If the confirmation unit confirms the existence of an empty seat, the adjustment unit adjusts the information to include warning information that identifies the location of the empty seat, as described in claim 5.

8. The in-vehicle monitoring device according to claim 1, wherein the confirmation unit deems that a seat with luggage is occupied by a passenger if luggage is present in that seat.

9. The in-vehicle monitoring device according to claim 1, wherein the confirmation unit divides the seat area into multiple cameras and makes a determination.

10. The in-vehicle monitoring device according to claim 1, wherein the confirmation unit changes the threshold value of the difficulty level between shooting areas.

11. The in-vehicle monitoring device according to claim 1, wherein when the confirmation unit confirms that all passengers are seated and that safety conditions have been met, the provision unit ceases providing the warning information.

12. The in-vehicle monitoring device according to claim 1, which uses the same camera to assess safety conditions and detect empty seats.

13. The system further includes a generation unit that generates a skeletal model of a passenger identified from an image captured by the aforementioned camera. The in-vehicle monitoring device according to claim 1, wherein the confirmation unit confirms the seating status of each seat from the skeletal model generated by the generation unit.

14. In a vehicle equipped with seats for passengers to sit on and support members for passengers to hold onto to stand, a generation unit generates a skeletal model of the passenger identified from images captured by a camera capable of capturing the passenger's range of movement, A confirmation unit that checks the seating status of each passenger for each seat based on the seat position obtained from the image captured by the aforementioned camera and the passenger's skeletal model generated by the generation unit, A determination unit determines the safety status of a passenger who is not seated based on predetermined skeletal points of the skeletal model generated by the generation unit, A service department that provides passengers with cautionary information for safe operation, If the seating status confirmed by the confirmation unit and the safety status determined by the determination unit are in a specific state, the adjustment unit adjusts the content of the warning information, An in-vehicle monitoring device having the following features.

15. The determination unit, The in-vehicle monitoring device according to claim 14, wherein a first condition is that predetermined skeletal points of the skeletal model generated by the generation unit are located within a predetermined specific area that identifies the positions of the seat and the support member, and a second condition is that further conditions for the skeletal points of the passenger are met, and safety is determined when the first and second conditions are met.

16. Computers, A safety status monitoring and control program that operates as an in-vehicle monitoring device according to any one of claims 1 to 15.

Images