Information processing method

Abstract

The present invention addresses the problem of improving a technique for preventing a vehicle occupant from falling down. An information processing method is performed by an information processing apparatus. The information processing apparatus acquires information about a posture of an occupant of a vehicle and information about whether or not the occupant uses a support device. Next, the information processing device determines the degree of attention alert to the occupant on the basis of the information relating to the posture and the information relating to whether or not to use the support device. The information processing device gives a notification to the occupant in accordance with the degree of attention alert.

Description

Information processing methods Technical Field

[0001] This invention relates to an information processing method performed by an information processing device. Background Technology

[0002] Previously, technologies for preventing vehicle occupants (passengers) from falling were known. For example, Patent Document 1 discloses an invention that controls the acceleration of a vehicle based on the relationship between the vehicle's driving state assuming it is traveling on a road it will travel on in the future and the risk of the occupants falling under that driving state.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2023-101066 Summary of the Invention

[0004] The goal is to improve the technology used in games that involve vehicles.

[0005] The object of the present invention, which was made in view of this situation, is to improve the technology for preventing vehicle occupants from falling.

[0006] One embodiment of the present invention relates to a method executed by an information processing device, the method comprising the following steps: acquiring information related to the posture of a vehicle occupant and information related to whether the occupant is using a support device; determining the degree of attention reminder to the occupant based on the information related to the posture and the information related to whether the support device is used; and notifying the occupant of the degree of attention reminder.

[0007] Invention Effects

[0008] According to one embodiment of the present invention, the technology for preventing vehicle occupants from falling can be improved. Attached Figure Description

[0009] Figure 1 is a block diagram showing the schematic structure of the information processing device involved in the present invention.

[0010] Figure 2 is a flowchart illustrating the operation of the information processing device involved in this invention. Detailed Implementation

[0011] The embodiments of the present invention will be described below.

[0012] (Summary of the implementation method)

[0013] Referring to FIG1, the structure of the information processing apparatus 10 according to this embodiment will be described. The information processing apparatus 10 is, for example, a computer or a general or special electronic device, such as an in-vehicle device mounted on a vehicle or a server device that communicates with the vehicle. Furthermore, the information processing apparatus 10 may be composed of multiple devices, such as an in-vehicle device and a server device.

[0014] First, a general overview of this embodiment will be given; detailed descriptions will follow. The information processing device 10 acquires information related to the posture of the vehicle occupants and information related to whether the occupants are using support devices. Next, based on the posture-related information and the information regarding the use of support devices, the information processing device determines the level of attention reminder required for the occupant. Furthermore, the information processing device sends a notification to the occupant corresponding to the level of attention reminder.

[0015] The risk of vehicle occupants falling varies not only depending on whether the occupant is in a sitting position, but also depending on the occupant's posture and / or state other than sitting. In contrast, according to this embodiment, the level of attentional warning to the occupant is determined based on information related to the occupant's posture and whether the occupant is using support equipment. Therefore, for example, even when the occupant is standing, different levels of attentional warning can be given depending on whether a handrail is used. Furthermore, for example, even when the occupant is not using a handrail, if the occupant is in a posture deemed to have a high risk of falling, a higher level of attentional warning can be given compared to other postures. Therefore, according to this embodiment, from the viewpoint of being able to provide attentional warnings to the occupant considering various occupant states, the technology for preventing vehicle occupant falls is improved.

[0016] Next, the structure of the information processing device 10 will be described in detail.

[0017] (Composition of information processing device)

[0018] The information processing device 10 includes a communication unit 11, an output unit 12, a camera unit 13, a storage unit 14, a control unit 15, and a sensor unit 16.

[0019] The communication unit 11 includes one or more communication interfaces connected to a network. These communication interfaces may correspond to, for example, fourth-generation (4G) or fifth-generation (5G) mobile communication standards, but are not limited to these. The communication unit 11 receives information for the operation of the information processing device 10 and transmits information obtained through the operation of the information processing device 10. The communication unit 11 enables the information processing device 10 to communicate via a network, for example, with a server that controls a part of the vehicle or provides information about other vehicles.

[0020] The output unit 12 includes one or more output devices for outputting information. These output devices may be, for example, a display, monitor, or electronic signage that outputs image information, or a speaker that outputs sound information, but are not limited thereto. Alternatively, the output unit 12 may also include an interface for connecting an external output device.

[0021] The camera unit 13 includes one or more cameras capable of capturing images of the surroundings or interior of the vehicle, which are mounted on the vehicle.

[0022] Storage unit 14 includes one or more memories. These memories may be, for example, semiconductor memories, magnetic memories, or optical memories, but are not limited to these. Each memory included in storage unit 14 may function as a main storage device, an auxiliary storage device, or a cache memory. Storage unit 14 stores any information used for the operation of information processing device 10. Furthermore, storage unit 14 may store, for example, system programs, application programs, and embedded software. For example, information stored in storage unit 14 can be updated via information obtained from a network through communication unit 11.

[0023] The control unit 15 includes at least one processor, at least one programmable circuit such as a field-programmable gate array (FPGA), at least one application-specific integrated circuit (ASIC), or any combination thereof. The processor is a general-purpose processor such as a central processing unit (CPU) or a graphics processing unit (GPU), or a dedicated processor specifically designed for particular processing. The control unit 15 controls the various parts of the information processing device 10 while performing processing related to the operation of the information processing device 10.

[0024] The sensor unit 16 may include, for example, any sensor such as a steering angle sensor, an acceleration sensor, a vehicle speed sensor, a position sensor, and a gyroscope sensor. The sensor unit 16 may also include, for example, a Global Positioning System (GPS) receiver.

[0025] (The operation flow of the information processing device)

[0026] Referring to FIG2, the operation of the information processing device 10 according to this embodiment will be described.

[0027] S100: The control unit 15 of the information processing device 10 acquires information related to the posture of the vehicle occupants and information related to whether the occupants are using support equipment.

[0028] Specifically, the control unit 15 detects occupants based on images of the vehicle interior captured by the camera unit 13. Then, the control unit 15 determines the posture and the presence or absence of support equipment for each occupant detected from the images. Furthermore, AI can be used to acquire information related to the posture of the vehicle occupants and information related to the use of support equipment. Specifically, the control unit 15 can input the images of the vehicle interior into a learning model, and acquire the output estimated posture and the presence or absence of support equipment as posture-related information and support equipment-related information, respectively. The learning model includes skeletal estimation, but may include any learning model not limited to this. Different learning models can be used to acquire information related to the occupant's posture and information related to the use of support equipment. Furthermore, the sensor unit 16 can be used to acquire information related to the posture of the vehicle occupants and information related to the use of support equipment. For example, sensors can be installed on the support equipment to acquire information related to the use of support equipment.

[0029] Information related to "posture" includes sitting postures and one or more other postures that differ from sitting postures, such as stable postures and unstable postures. Furthermore, a "stable posture" here refers, for example, an upright standing posture, where the body's balance is not easily disrupted, rather than a sitting posture. An "unstable posture" refers, for example, a standing posture where the body's balance is disrupted, such as bending over, tilting significantly to the left or right, or swaying forward or backward, rather than a sitting posture.

[0030] "Supporting equipment" refers to any equipment, such as handrails or grab bars, that vehicle occupants can hold onto.

[0031] S101: The control unit 15 determines the degree of attention reminder to the occupant based on information related to the occupant's posture and information related to whether the occupant is using support equipment.

[0032] "Level of alert" indicates the degree of likelihood of an occupant falling. For example, the level of alert can be a numerical value of any range of integers from 0 to 3 in descending order, or it can be a letter-based ranking, but it can also be any form not limited to these. Furthermore, in the following description, the level of alert is set in descending order to 0, 1, 2, 3.

[0033] For example, if the information related to the occupant's posture is a sitting posture, the control unit 15 can determine the level of attention reminder as 0. If the information related to posture is not a sitting posture but information related to whether or not a support device is used exists, the control unit 15 can determine the level of attention reminder as 1. If the information related to posture is a stable posture and information related to whether or not a support device is used does not exist, the control unit 15 can determine the level of attention reminder as 2. If the information related to posture is an unstable posture and information related to whether or not a support device is used does not exist, the control unit 15 can determine the level of attention reminder as 3.

[0034] Furthermore, for example, if the information related to the occupant's posture is not a sitting posture, the control unit 15 can acquire information related to the occupant's actions and further determine the level of attention alert based on the information related to those actions. For example, if the information related to the occupant's posture is not a sitting posture, the control unit 15 can input images of the vehicle interior into the learning model and acquire the output estimated actions as action-related information. Additionally, information related to posture and information related to whether support devices are used can be acquired simultaneously, along with action-related information.

[0035] Information related to "movement" refers to any movement that an occupant can perform, such as when stationary, moving, or standing on one leg.

[0036] For example, if the information related to the occupant's actions is not static, the control unit 15 can determine the level of attention alert to be 3. This is because, in this case, the occupant's balance would be disrupted when they perform actions, thus increasing the likelihood of a fall.

[0037] S102: The control unit 15 provides the occupant with a notification corresponding to the level of attention required.

[0038] Specifically, the control unit 15 uses the output unit 12 to broadcast to the occupant a message corresponding to the level of attention required. The higher the level of attention required, the more times the broadcast can be repeated and the louder the volume can be. Furthermore, the content of the broadcast can be designed to emphasize the occupant's attention as the level of attention increases. Alternatively, the control unit 15 can use the output unit 12 to display a message or image corresponding to the level of attention required on a display, monitor, or electronic signage. Alternatively, the notification can be sent to the occupant's terminal device. Furthermore, when there are multiple occupants, the control unit 15 can send a notification to each of the multiple occupants corresponding to the level of attention required for each of the multiple occupants. Alternatively, the control unit 15 can send a notification to each of the multiple occupants corresponding to the highest level of attention required for each of the multiple occupants, or it can send a notification only to the occupant with the highest level of attention required. Furthermore, when the level of attention required is 0, the control unit 15 may not send a notification.

[0039] Although the present invention has been described with reference to the accompanying drawings and embodiments, those skilled in the art will recognize that various modifications and alterations can be made to the present invention. Therefore, it should be noted that such modifications and alterations are included within the scope of the present invention. For example, the functions included in each component or step can be reconfigured in a logically consistent manner, and multiple components or steps can be combined into one or divided.

[0040] For example, in the above embodiments, it is also possible to implement an embodiment in which the structure and operation of the information processing device 10 are distributed among multiple computers capable of communicating with each other. For example, in the case where the information processing device 10 is composed of an in-vehicle device and a server, the in-vehicle device can perform some operations, and the server can perform operations that the in-vehicle device does not perform. Furthermore, for example, the information processing device 10 according to the above embodiments can be used to provide a Mobility as a Service (MaaS) service that utilizes mobility.

[0041] Furthermore, for example, in the above embodiment, the control unit 15 can, after issuing a notification in S102 and after a first predetermined time, re-acquire information related to the occupant's posture and whether the occupant is using the support device. Moreover, if the posture is not a sitting posture or the support device is not being used, the control unit 15 can issue the notification again. Additionally, if the level of attention reminder is 1 or less, the control unit 15 may not perform this variation.

[0042] Furthermore, for example, in the above embodiment, when the level of attention reminder is 3, the control unit 15 can, after notifying in S102 and after a second predetermined time, reacquire information related to the occupant's posture, whether the occupant is using support equipment, and the occupant's actions. Moreover, if the result of re-implementing S101 is that the level of attention reminder is 3, the control unit 15 can notify the vehicle operator, such as a remote operator, of this information. This is because, in this situation, the risk of falling remains very high, and notifying only the occupant may be ineffective.

[0043] Furthermore, for example, in the above embodiment, if the posture-related information is an unstable posture and there is no information related to whether a support device is used, the control unit 15 can further determine the degree of attention reminder to the occupant based on whether a third predetermined time has elapsed since that state. Specifically, if a third predetermined time has elapsed since that state, the control unit 15 can set the degree of attention reminder to 3; otherwise, it can set the degree of attention reminder to 2.

[0044] "First specified time", "Second specified time" and "Third specified time" are, for example, any time that can be expressed in seconds, minutes, etc.

[0045] Furthermore, for example, in the above embodiment, the control unit 15 can acquire information about the vehicle's driving route and information about other vehicles, and determine the degree of the first risk of vehicle swaying based on the driving route information and information about other vehicles. Moreover, the control unit 15 can further determine the degree of attention reminder based on the degree of the first risk.

[0046] "Level of Risk 1" indicates the degree of likelihood of vehicle swaying. The level of risk 1 can be determined based on the level of risk 2 determined by information about the driving route and the level of risk 3 determined by information about other vehicles. The levels of risk 1, risk 2, and risk 3 can be expressed in the same form as the level of caution warning, but can also be expressed in any form not limited to this. Furthermore, in the following description, the levels of risk 1, risk 2, and risk 3 are expressed in the form of 0, 1, 2, and 3, with the risks ranked from lowest to highest.

[0047] Specifically, the control unit 15 acquires information about the vehicle's travel route stored in the storage unit 14. For example, after the execution of S101 described above, it determines whether a swaying part exists on the travel route within a fourth predetermined time period. If a swaying part exists, the control unit 15 can determine the degree of the second risk as 1; otherwise, it can determine the degree of the second risk as 0. Alternatively, the control unit 15 can acquire information about the vehicle's travel route using the camera unit 13. Next, the control unit 15 acquires information about other vehicles, for example, from a server that manages vehicle operations, and determines the riding status and the proportion of swaying occupants among the total occupants in other vehicles besides the swaying part. If the proportion of swaying occupants is less than a predetermined proportion, the control unit 15 can determine the degree of the third risk as 0; if it is above the predetermined proportion but less than 100%, it can determine the degree of the third risk as 1; and if it is 100%, it can determine the degree of the third risk as 2. Furthermore, the predetermined proportion can be changed. Moreover, the control unit 15 adds the degree of the third risk to the degree of the second risk to calculate the degree of the first risk. Furthermore, the degree of risk can be determined not only by addition, but also by any calculation method.

[0048] If the level of attention alert is determined to be 1 in S101 and the level of the first risk is less than 3, the control unit 15 can change the level of attention alert to 0. If the level of attention alert is determined to be 2 in S101 and the level of the first risk is less than 2, the control unit 15 can change the level of attention alert to 1. If the level of attention alert is determined to be 3 in S101 and the level of the first risk is less than 1, the control unit 15 can change the level of attention alert to 2. Furthermore, if the level of attention alert is determined to be 3 based on the occupant's actions, the control unit 15 can maintain the level of attention alert at 3 even if the first risk is any value.

[0049] "The fourth specified time" can be any time that can be expressed in seconds, minutes, etc. Furthermore, the first, second, and third specified times can be changed in correspondence with the fourth specified time. For example, the first, second, or third specified time can be any time shorter than the fourth specified time.

[0050] Furthermore, for example, it is also possible to implement a general-purpose computer as the information processing apparatus 10 according to the above embodiments. Specifically, a program containing processing content for implementing the various functions of the information processing apparatus 10 according to the above embodiments is stored in the memory of a general-purpose computer, and the program is read and executed by a processor. Therefore, the present invention can also be implemented as a program executable by a processor or as a non-transitory computer-readable medium storing the program.

[0051] Symbol Explanation

[0052] 10-Information processing device, 11-Communication unit, 12-Output unit, 13-Camera unit, 14-Storage unit, 15-Control unit, 16-Sensor unit.

Claims

1. A method performed by an information processing device, characterized in that, The method includes the following steps: acquiring information related to the posture of a vehicle occupant and information related to whether the occupant is using a support device; determining the degree of attention reminder to the occupant based on the information related to the posture and the information related to whether the support device is used; and notifying the occupant of the degree of attention reminder.

2. The method according to claim 1, characterized in that, It also includes the following steps: obtaining information about the vehicle's driving route and information about other vehicles; and determining the degree of a first risk of vehicle swaying based on the information about the driving route and the information about other vehicles, wherein the information processing device further determines the degree of the attention reminder based on the degree of the first risk.

3. The method according to claim 1, characterized in that, The posture is any one of a sitting posture and one or more other postures different from the sitting posture. The method further includes the step of acquiring information related to the occupant's actions when the posture is not the sitting posture. The information processing device further determines the degree of attention reminder based on the information related to the actions.

4. The method according to claim 1, characterized in that, Obtaining information related to the occupant's posture includes the following steps: inputting images inside the vehicle into a learning model, and obtaining the output estimated posture as information related to the posture.

5. The method according to claim 1, characterized in that, The posture is any one of a sitting posture and one or more other postures different from the sitting posture. The method further includes the following steps: after a first predetermined time has elapsed after the notification, obtaining information related to the posture of the occupant and information related to whether the occupant is using the support device; and if the posture is not a sitting posture or the support device is not used, issuing the notification again.

6. A method for providing mobile-as-a-service using the information processing apparatus of claim 1.

Images