method

Abstract

To improve technology to prevent vehicle occupants from falling over. [Solution] The information processing device 10 acquires information regarding the posture of the vehicle occupants and information regarding the use of the occupants' support equipment. Next, the information processing device 10 determines the degree of attention to be given to the occupants based on the information regarding their posture and the information regarding the use of the support equipment. Then, the information processing device 10 gives a notification to the occupants according to the degree of attention to be given.

Description

Technical Field

[0001] This disclosure relates to a method executed by an information processing apparatus.

Background Art

[0002] Conventionally, technologies for preventing the fall of vehicle occupants (passengers) are known. For example, Patent Document 1 discloses an invention for controlling the acceleration of a vehicle based on the relationship between the driving state when assuming that the vehicle will drive on a road in the future and the risk of the occupant falling in that driving state.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Improvements are desired in the technology of games using vehicles.

[0005] In view of such circumstances, an object of the present disclosure is to improve the technology for preventing the fall of vehicle occupants.

Means for Solving the Problems

[0006] A method according to an embodiment of the present disclosure is a method executed by an information processing apparatus, including: acquiring information regarding the posture of a vehicle occupant and information regarding the presence or absence of use of the support equipment for the occupant; determining the degree of alertness for the occupant based on the information regarding the posture and the information regarding the presence or absence of use of the support equipment; and sending a notification corresponding to the degree of alertness to the occupant.

Effects of the Invention

[0007] According to one embodiment of the present disclosure, the technology for preventing vehicle occupants from falling over is improved. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing the schematic configuration of the information processing device related to this disclosure. [Figure 2] This is a flowchart showing the operation of the information processing device related to this disclosure. [Modes for carrying out the invention]

[0009] The embodiments of this disclosure will be described below.

[0010] (Summary of the embodiment) Referring to Figure 1, the configuration of the information processing device 10 according to this embodiment will be described. The information processing device 10 is, for example, a computer such as an in-vehicle device mounted in a vehicle or a server device that communicates with a vehicle, or a general-purpose or dedicated electronic device. The information processing device 10 may also be composed of multiple devices such as an in-vehicle device and a server device.

[0011] First, an overview of this embodiment will be described, and details will be described later. The information processing device 10 acquires information regarding the posture of the vehicle occupants and information regarding the use of the occupants' support equipment. Next, the information processing device 10 determines the degree of attention to be given to the occupants based on the information regarding their posture and the information regarding the use of the support equipment. Then, the information processing device 10 gives a notification to the occupants according to the degree of attention to be given.

[0012] The risk of a vehicle occupant falling over can vary not only depending on whether the occupant is seated or not, but also on their other postures and / or conditions. In contrast, according to this embodiment, the degree of alerting the occupant is determined based on information regarding the occupant's posture and information regarding the use of support equipment for the occupant. For example, even if the occupant is standing, different alerts can be given depending on whether or not they are using a handrail. Also, for example, even if the occupant is not using a handrail, if the occupant is in a posture that is determined to have a high risk of falling, a higher alert can be given to the occupant than in other postures. Therefore, according to this embodiment, the technology for preventing vehicle occupants from falling over is improved in that it is possible to give the occupant an alert that takes into account various conditions of the occupant.

[0013] Next, the configuration of the information processing device 10 will be described in detail.

[0014] (Configuration of information processing device) The information processing device 10 comprises a communication unit 11, an output unit 12, an imaging unit 13, a storage unit 14, a control unit 15, and a sensor unit 16.

[0015] The communication unit 11 includes one or more communication interfaces connected to a network. These communication interfaces are compatible with, but are not limited to, mobile communication standards such as 4G (4th Generation) or 5G (5th Generation). The communication unit 11 receives information used in the operation of the information processing device 10 and transmits information obtained through the operation of the information processing device 10. The communication unit 11 enables the information processing device 10 to communicate via the network with, for example, a server that controls a part of the vehicle or provides information about other vehicles.

[0016] The output unit 12 includes one or more output devices for outputting information. These output devices are, for example, displays, monitors, or signage that output information as images, or speakers that output information as sound, but are not limited to these. Alternatively, the output unit 12 may include an interface for connecting external output devices.

[0017] The imaging unit 13 includes one or more cameras that are installed in the vehicle and capable of photographing the surroundings or interior of the vehicle.

[0018] The storage unit 14 includes one or more memories. The memories are, for example, semiconductor memories, magnetic memories, or optical memories, but are not limited to these. Each memory included in the storage unit 14 may function as, for example, a main memory, an auxiliary memory, or a cache memory. The storage unit 14 stores any information used in the operation of the information processing device 10. In addition, for example, the storage unit 14 may store system programs, application programs, and embedded software. For example, the information stored in the storage unit 14 may be updatable with information obtained from a network via, for example, the communication unit 11.

[0019] The control unit 15 includes at least one processor, at least one programmable circuit such as an FPGA (field-programmable gate array), at least one dedicated circuit such as an ASIC (application-specific integrated circuit), or any combination thereof. The processor is a general-purpose processor such as a CPU (central processing unit) or GPU (graphics processing unit), or a dedicated processor specialized for a specific process. The control unit 15 controls each part of the information processing device 10 and executes processes related to the operation of the information processing device 10.

[0020] The sensor unit 16 includes any sensors such as, for example, a steering angle sensor, an acceleration sensor, a vehicle speed sensor, a position sensor, and a gyro sensor. The sensor unit 16 may include, for example, a GPS (Global Positioning System) receiver or the like.

[0021] (Operation flow of the information processing device) Referring to FIG. 2, the operation of the information processing device 10 according to the present embodiment will be described.

[0022] S100: The control unit 15 of the information processing device 10 acquires information regarding the posture of the vehicle occupants and information regarding the use of the occupant support equipment.

[0023] Specifically, the control unit 15 detects the occupants from the video inside the vehicle captured by the imaging unit 13. Next, the control unit 15 determines the posture and the use of the support equipment for each of the occupants detected from the video. Note that AI may be used to acquire information regarding the posture of the vehicle occupants and information regarding the use of the support equipment. Specifically, the control unit 15 may input the video inside the vehicle into a learning model and acquire the estimated posture and the use of the support equipment output as information regarding the posture and information regarding the use of the support equipment, respectively. Note that the learning model includes, but is not limited to, skeleton estimation, and may include any learning model. Note that different learning models may be used for acquiring information regarding the posture of the occupants and information regarding the use of the support equipment. Further, the sensor unit 16 may be used to acquire information regarding the posture of the vehicle occupants and information regarding the use of the support equipment. For example, sensors may be provided for each of the support equipment to acquire information regarding the use of the support equipment.

[0024] Information regarding "posture" includes seated posture and one or more other postures different from seated posture, such as stable posture and unstable posture. Here, "stable posture" refers to a state in which the body's balance is not easily disrupted while standing, such as an upright posture. "Unstable posture" refers to a state in which the body's balance is disrupted while standing, such as being half-crouched, or the body being significantly tilted to the left or right or forward or backward, or swaying.

[0025] "Support equipment" refers to any equipment that vehicle occupants can hold onto, such as handrails and straps.

[0026] S101: The control unit 15 determines the degree to which attention should be given to the occupant based on information regarding the occupant's posture and information regarding the use of the occupant's support equipment.

[0027] The "degree of caution" indicates the likelihood of the occupants tipping over. For example, the degree of caution may be expressed as a numerical value using integers in any range, such as 0 to 3, in descending order of caution, or as an alphabetical rank, but any other format is acceptable. In the following explanation, we will use a format where the degree of caution is 0, 1, 2, and 3 in descending order of caution.

[0028] For example, if the information regarding the occupant's posture indicates a seated position, the control unit 15 may determine the level of alert as 0. If the information regarding posture is not a seated position, and there is information regarding the use of support equipment, the control unit 15 may determine the level of alert as 1. If the information regarding posture indicates a stable position, and there is no information regarding the use of support equipment, the control unit 15 may determine the level of alert as 2. If the information regarding posture indicates an unstable position, and there is no information regarding the use of support equipment, the control unit 15 may determine the level of alert as 3.

[0029] Furthermore, if, for example, the information regarding the occupant's posture is not that of a seated position, the control unit 15 may acquire information regarding the occupant's movements and determine the degree of alerting based on that information regarding movements. For example, if the information regarding the occupant's posture is not that of a seated position, the control unit 15 may input the in-vehicle video into a learning model and acquire the estimated movements output as information regarding movements. Note that information regarding movements may be acquired simultaneously with information regarding posture and information regarding the use of support equipment.

[0030] Information regarding "actions" refers to any action that an occupant may perform, such as being stationary, moving, or standing on one leg.

[0031] For example, if information regarding the occupant's movements indicates that they are not stationary, the control unit 15 may determine the level of alert to be 3. This is because when an occupant is in motion, their balance is disrupted, and there is a high probability of them falling.

[0032] S102: The control unit 15 provides a notification to the crew member according to the degree of attention required.

[0033] Specifically, the control unit 15 uses the output unit 12 to make an announcement to the occupant according to the degree of attention required. The more important the degree of attention required, the more the announcement may be repeated and the louder it may be. The content of the announcement may also be more attention-seeking to the occupant as the degree of attention required increases. Alternatively, the control unit 15 may use the output unit 12 to display a message or image according to the degree of attention required on a display, monitor, or signage. Alternatively, the notification may be sent to the terminal device used by the occupant. Furthermore, if there are multiple occupants, the control unit 15 may send a notification to each of the occupants according to the determined degree of attention required. Alternatively, the control unit 15 may send a notification to all of the occupants according to the highest degree of attention required among the occupants, or it may send it only to the occupant with the highest degree of attention required. Furthermore, if the level of alert is 0, the control unit 15 does not need to issue a notification.

[0034] While this disclosure has been described based on the drawings and embodiments, it should be noted that those skilled in the art may make various modifications and alterations based on this disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of this disclosure. For example, the functions, etc., included in each component or step can be rearranged in a logically consistent manner, and multiple components or steps can be combined into one or divided into two.

[0035] For example, in the embodiment described above, it is also possible to distribute the configuration and operation of the information processing device 10 to multiple computers that can communicate with each other. For example, if the information processing device 10 is composed of an in-vehicle device and a server, the in-vehicle device may perform some operations, and the server may perform operations that the in-vehicle device does not perform. Also, for example, the information processing device 10 according to the embodiment described above may be used to provide MaaS (Mobility as a Service), which is a service that utilizes mobility.

[0036] Furthermore, for example, in the embodiment described above, the control unit 15 may again acquire information regarding the occupant's posture and information regarding the use of the occupant's support equipment after a first predetermined time has elapsed since the notification in S102. If the posture is not a seated posture, or if the support equipment is not being used, the control unit 15 may make the notification again. Note that if the level of attention is 1 or less, the control unit 15 does not need to perform this modification.

[0037] Furthermore, for example, in the embodiment described above, if the level of alert is 3, the control unit 15 may again acquire information regarding the occupant's posture, information regarding the use of occupant support equipment, and information regarding the occupant's movements after a second predetermined time has elapsed since the notification in S102. Then, if the control unit 15 performs S101 again and the level of alert is still 3, it may notify the vehicle operator, such as a remote operator, of this fact. This is because the risk of tipping over remains very high, and notification to the occupant alone may not be effective.

[0038] Furthermore, for example, in the embodiment described above, if the information regarding posture is an unstable posture and there is no information regarding the use of support equipment, the control unit 15 may determine the degree of alerting the occupant based on whether or not a third predetermined time has elapsed since that state. Specifically, the control unit 15 may set the degree of alert to 3 if a third predetermined time has elapsed since that state, and to 2 otherwise.

[0039] "First predetermined time," "second predetermined time," and "third predetermined time" are any time that can be expressed in terms of seconds, minutes, etc.

[0040] Furthermore, for example, in the embodiment described above, the control unit 15 may acquire information on the vehicle's travel route and information on other vehicles, and determine the degree of the first risk of the vehicle shaking based on the information on the travel route and information on other vehicles. The control unit 15 may then determine the degree of warning based on the degree of the first risk.

[0041] The "Level of Risk 1" indicates the likelihood of the vehicle shaking. The Level of Risk 1 may be determined based on the Level of Risk 2, which is determined based on information about the driving route, and the Level of Risk 3, which is determined based on information about other vehicles. The Levels of Risk 1, Risk 2, and Risk 3 may use a format similar to, for example, the Level of Warning, but may use any format without limitation. Furthermore, in the following explanation, the Levels of Risk 1, Risk 2, and Risk 3 will be expressed in a format where the risk levels are 0, 1, 2, and 3 in descending order of risk.

[0042] Specifically, the control unit 15 acquires information about the vehicle's travel route stored in the memory unit 14 and determines, for example, whether there are any swaying locations on the travel route within a fourth predetermined time after the execution of S101 described above. If there are swaying locations, the control unit 15 may determine the degree of the second risk to be 1, and if there are no such locations, it may determine the degree of the second risk to be 0. Alternatively, the control unit 15 may acquire information about the vehicle's travel route using the imaging unit 13. Next, the control unit 15 acquires information about other vehicles from, for example, a server that manages the operation of vehicles, and determines the percentage of swaying occupants relative to the total occupants excluding those seated in other vehicles that have previously passed the location in question. If the percentage of swaying occupants is less than a predetermined percentage, the control unit 15 may determine the degree of the third risk to be 0, if it is between a predetermined percentage and 100%, it may determine the degree of the third risk to be 1, and if it is 100%, it may determine the degree of the third risk to be 2. Note that the predetermined percentage may be changed. The control unit 15 then calculates the degree of the first risk by adding the degree of the third risk to the degree of the second risk. Note that the degree of the first risk may be determined not only by addition but also by any calculation method.

[0043] If the level of alert is determined to be 1 in S101 and the level of the first risk is less than 3, the control unit 15 may change the level of alert to 0. If the level of alert is determined to be 2 in S101 and the level of the first risk is less than 2, the control unit 15 may change the level of alert to 1. If the level of alert is determined to be 3 in S101 and the level of the first risk is less than 1, the control unit 15 may change the level of alert to 2. Also, if the level of alert is determined to be 3 based on the actions of the crew, the control unit 15 may keep the level of alert at 3 regardless of the value of the first risk.

[0044] The "fourth predetermined time" is any time that can be expressed in terms of seconds, minutes, etc. Furthermore, the first predetermined time, the second predetermined time, and the third predetermined time may be changed in accordance with the fourth predetermined time. For example, the first predetermined time, the second predetermined time, or the third predetermined time may be any time that is smaller than the fourth predetermined time.

[0045] Furthermore, it is also possible to implement an embodiment in which a general-purpose computer functions as the information processing device 10 according to the above embodiment. Specifically, a program describing the processing content that realizes each function of the information processing device 10 according to the above embodiment is stored in the memory of the general-purpose computer, and the processor reads and executes the program. Therefore, this disclosure can also be implemented as a program that can be executed by a processor, or as a non-temporary computer-readable medium that stores the program. [Explanation of Symbols]

[0046] 10 Information processing unit, 11 Communication unit, 12 Output unit, 13 Imaging unit, 14 Storage unit, 15 Control unit, 16 Sensor unit

Claims

1. A method executed by an information processing device, To obtain information regarding the posture of the vehicle occupants and information regarding the use of the occupant support equipment, Based on the information regarding the posture and the information regarding the use of the support equipment, the degree to which the occupant should be alerted is determined. The crew members shall be notified according to the degree of the aforementioned warning, Methods that include...

2. The method according to claim 1, To obtain information on the vehicle's travel route and information on other vehicles, Based on the information of the aforementioned travel route and the information of the other vehicles, the degree of the first risk of the vehicle shaking is determined, It further includes, The information processing device is a method for determining the degree of warning based on the degree of the first risk.

3. The method according to claim 1, The aforementioned posture is one of the following: a seated posture and one or more other postures different from the seated posture. The method, if the posture is not the seated posture, obtains information regarding the occupant's movements. It further includes, The information processing device further determines the degree of the warning based on the information regarding the operation.

4. The method according to claim 1, A method for obtaining information regarding the posture of the occupant includes inputting in-vehicle video footage into a learning model and obtaining the output estimated posture as information regarding the posture.

5. The method according to claim 1, The aforementioned posture is one of the following: a seated posture and one or more other postures different from the seated posture. After the aforementioned notification has been made and a first predetermined time has elapsed, information regarding the posture of the occupant and information regarding whether or not the occupant is using the support equipment shall be acquired again. If the aforementioned posture is not a seated posture, or if the aforementioned support equipment is not in use, the aforementioned notification shall be given again. Methods that further include the above.

6. A method for providing MaaS (Mobility as a Service) using the information processing device described in claim 1.

Images