Information processing systems and programs

Abstract

Compared to simply detecting and notifying of postural abnormalities in a person's posture from an image of that person being monitored, this makes it easier to understand the nature of the detected postural abnormality. [Solution] The information processing system of the present disclosure includes a processor, which continuously captures images including a person to be monitored, estimates the skeleton of the person in the captured images to detect whether or not there is a postural abnormality in the person, and when a postural abnormality is detected, the system inputs the image in which the postural abnormality of the person was detected to a large-scale language model that converts the contents of the input image into text information and outputs it, thereby obtaining text information regarding the state of the person in the input image, and notifying a pre-set notification destination that a postural abnormality has occurred in the person to be monitored, along with the acquired text information and the image in which the postural abnormality was detected.

Description

Technical Field

[0001] The present disclosure relates to an information processing system and a program.

Background Art

[0002] Patent Document 1 discloses an action detection system that notifies a predetermined notification destination of the detection of an action when any of a first notification condition set for each action class specified based on the action of a person included in an image and a second notification condition such as the duration of the action, the degree of congestion around the person, and the time zone in which the image was taken is satisfied.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, due to the development of AI (Artificial Intelligence) technology, AI technology has been utilized in various fields. For example, a system has been proposed that captures a person to be monitored with a camera and applies AI technology to the captured image to detect the person's fall. In the fields of medical care and welfare, the aim is to quickly notice a fall and raise an alert, leading to early rescue activities. Also, in places where dangerous work is performed, such as factories, it may be used to stop the machine when a dangerous action such as a fall is detected.

[0005] However, even if, when a person to be monitored is captured and it is detected that the captured person has fallen, a notification to that effect is sent to a preset notification destination, the person who receives this notification must check the captured image, and there are cases where it is not easy to grasp the state of the person to be monitored.

[0006] The purpose of this disclosure is to provide an information processing system and program that makes it easier to understand the state of detected postural abnormalities compared to simply detecting and notifying of postural abnormalities of a person being monitored from images of that person. [Means for solving the problem]

[0007] An information processing system in a first aspect of this disclosure comprises a processor which continuously captures images including a person being monitored. The system estimates the skeleton of a person in the captured image and detects whether or not there is a postural abnormality in that person. When a postural abnormality is detected, a large-scale language model that converts the content of an input image into text information and outputs it is input to the image in which the person's postural abnormality was detected, thereby obtaining text information about the person's state in the input image. The system notifies a pre-configured recipient that a postural abnormality has occurred in the person being monitored, along with the acquired text information and the image in which the postural abnormality was detected.

[0008] In the second aspect of this disclosure, the information processing system is such that, in the information processing system of the first aspect, the posture abnormality detected by skeletal estimation is a fall of the monitored person.

[0009] In the third aspect of the information processing system of the present disclosure, in the information processing system of the second aspect, the processor inputs an image in which the person being monitored has fallen, and images before and after the fall, into the large-scale language model to obtain text information containing information about the state of the person being monitored before and after the fall.

[0010] The information processing system of the fourth aspect of this disclosure, in the information processing system of the first aspect, if a postural abnormality of the person is detected by skeletal estimation, the processor determines whether or not the detected postural abnormality is a fall of the monitored person. If it is determined that the detected postural anomaly is not a fall of the monitored person, the image in which the postural anomaly was detected is input to the large-scale language model to obtain text information about the person's state in the input image. The system notifies a pre-configured recipient that a postural abnormality has occurred in the person being monitored, along with the acquired text information and the image in which the postural abnormality was detected.

[0011] The information processing system of the fifth aspect of this disclosure, in the information processing system of the fourth aspect, when the processor determines that the detected posture abnormality is a fall of the monitored person, inputs the image in which the fall of the person was detected, and images before and after the fall, into the large-scale language model to obtain text information containing information about the state of the monitored person before and after the fall. The system notifies a pre-configured recipient that a fall has occurred in the person being monitored, along with the acquired text information and the image showing the fall.

[0012] The sixth aspect of the present disclosure includes the step of continuously capturing images that include a person being monitored, The steps include: estimating the skeleton of a person in a captured image to detect whether or not there is a postural abnormality in the person; The process involves inputting an image in which a posture abnormality of a person has been detected into a large-scale language model that converts the content of an input image into text information and outputs it, thereby obtaining text information about the state of the person in the input image. This program causes a computer to perform the following steps: notify a pre-configured recipient that a postural abnormality has occurred in a person being monitored, along with the acquired text information and an image in which the postural abnormality was detected. [Effects of the Invention]

[0013] According to the information processing system of the first aspect of this disclosure, it becomes easier to understand the state of the detected posture abnormality compared to simply detecting and notifying of a posture abnormality of a person from an image of the person being monitored.

[0014] According to the information processing system of the second aspect of the present disclosure, it is possible to make it easier to grasp the detected fall state as compared with the case of only detecting and notifying the fall of a person to be monitored from an image of the person.

[0015] According to the information processing system of the third aspect of the present disclosure, a person who has received a notification that a person to be monitored has fallen can grasp the urgency of the notification based on text information.

[0016] According to the information processing system of the fourth aspect of the present disclosure, it is possible to detect an abnormal posture other than the fall of a person to be monitored and notify a preset notification destination.

[0017] According to the information processing system of the fifth aspect of the present disclosure, when a person to be monitored has fallen, it is possible to notify more detailed content based on text information than when an abnormal posture other than the fall has occurred.

[0018] According to the program of the sixth aspect of the present disclosure, it is possible to make it easier to grasp the state of the detected abnormal posture as compared with the case of only detecting and notifying the abnormal posture of a person to be monitored from an image of the person.

Brief Description of Drawings

[0019] [Figure 1] It is a diagram showing the system configuration of the information processing system according to an embodiment of the present disclosure. [Figure 2] It is a block diagram showing the hardware configuration of the management server 10 in an embodiment of the present disclosure. [Figure 3] It is a block diagram showing the functional configuration of the management server 10 in an embodiment of the present disclosure. [Figure 4] It is a diagram for explaining a processing example in LLM16. [Figure 5] It is a flowchart for explaining the operation of the information processing system according to an embodiment of the present disclosure. [Figure 6]It is a diagram showing an example of notification when the fallen state of the person 21 to be monitored continues. [Figure 7] It is a diagram showing an example of notification when the person 21 to be monitored stands up after falling. [Figure 8] It is a flowchart for explaining the operation of an information processing system when detecting postures other than a fall and notifying a terminal device 40. [Figure 9] It is a diagram showing a first example of notification when detecting a posture abnormality other than a fall. [Figure 10] It is a diagram showing a second example of notification when detecting a posture abnormality other than a fall.

Mode for Carrying Out the Invention

[0020] Next, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0021] FIG. 1 is a diagram showing the system configuration of an information processing system according to an embodiment of the present disclosure.

[0022] One embodiment of this information processing system is, for example, a system for a remote user to monitor for postural abnormalities such as falls in a person 21 who is a resident of a nursing care facility. As shown in Figure 1, the information processing system of this embodiment consists of a management server 10, a camera 20, and a terminal device 40 that are interconnected by a network such as the Internet 30. The camera 20 is installed in the nursing care facility and continuously photographs the actions of the person 21 who is a resident of the facility. Images captured by the camera 20 are transmitted to the management server 10 via the Internet 30. The management server 10 performs object detection processing on the images transmitted from the camera 20 using AI technology. If the presence of a person is detected in the image by the object detection processing, the management server 10 performs skeletal estimation processing on the area where the presence of the person was detected, obtains skeletal coordinate information, and detects whether or not there is a postural abnormality in that person. If a postural abnormality such as a fall is detected in the person 21 who is a resident of the facility, the management server 10 notifies the monitor's terminal device 40 that a postural abnormality has occurred in the person 21 who is a resident of the facility.

[0023] In this embodiment, a configuration is described in which the management server 10 performs object detection processing in images and skeletal estimation processing of detected persons, but this disclosure is not limited to such a configuration. The functions of the management server 10 can also be provided on an on-premises server or an edge server. Furthermore, the functions of the management server 10 can be provided on the camera 20 to configure it as an endpoint camera.

[0024] Next, Figure 2 shows the hardware configuration of the management server 10 in the information processing system of this embodiment. As shown in Figure 2, the management server 10 includes a CPU 11, memory 12, storage devices such as a hard disk drive 13, a communication interface (IF) 14 for sending and receiving data to and from external devices via the Internet 30, a user interface (UI) device 15 including a touch panel or liquid crystal display and a keyboard, and a large language model (LLM (Large Language Models)) 16. These components are connected to each other via a control bus 17. Details of the LLM 16 will be described later.

[0025] The CPU 11 is a processor that controls the operation of the management server 10 by executing predetermined processes based on a control program stored in the memory 12 or storage device 13. In this embodiment, the CPU 11 is described as reading and executing a control program stored in the memory 12 or storage device 13, but it is not limited to this. This control program may be provided in the form of a computer-readable recording medium. For example, this program may be provided in the form of a CD (Compact Disc)-ROM and DVD (Digital Versatile Disc)-ROM recorded on an optical disc, or in the form of a USB (Universal Serial Bus) memory and memory card recorded on a semiconductor memory. Furthermore, this control program may be acquired from an external device via a communication line connected to the communication interface 14. In addition, this control program may be provided as a standalone application software, or it may be incorporated into the software of each device as a function of the management server 10.

[0026] Figure 3 is a block diagram showing the functional configuration of the management server 10 realized by the execution of the control program described above.

[0027] As shown in Figure 3, the management server 10 of this embodiment includes an operation input unit 31, a display unit 32, a data transmission / reception unit 33, a control unit 34, an LLM 16, and a data storage unit 35.

[0028] The data transmission / reception unit 33 transmits and receives data with external devices such as the terminal device 40 and the camera 20. The display unit 32 is controlled by the control unit 34 and displays various information to the user. The operation input unit 31 receives various operation information performed by the user.

[0029] The control unit 34 receives image data captured from the camera 20 via the data transmission / reception unit 33, and if it detects an abnormal posture of the person 21 being monitored using the received image data, it performs control to notify the terminal device 40 of this fact. The data storage unit 35 stores various data, such as image data, received by the data transmission / reception unit 33.

[0030] Next, an example of processing in LLM16 will be explained with reference to Figure 4.

[0031] LLM16 has the function of converting the content of an input image into text information and outputting it. For example, as shown in Figure 4, if an image of people playing baseball is input to LLM16, the text information "People are playing baseball." will be output. Also, for example, if an image of a cat sitting on a car is input to LLM16, the text information "A cat is on a car." will be output. In this way, by inputting an image to LLM16, text information that describes the content of that image can be obtained.

[0032] In this embodiment of the information processing system, the person 21 to be monitored is photographed by the camera 20, and AI technology is applied to the captured image to detect any postural abnormalities such as the person falling, and the terminal device 40 is notified that a postural abnormality has occurred in the person 21 to be monitored.

[0033] However, even if a notification is sent to a pre-configured recipient such as a terminal device 40 that a postural abnormality has occurred in the monitored person 21, the person who receives this notification must actually look at the captured image to confirm it, and may not be able to easily understand the condition of the monitored person 21.

[0034] Therefore, in this embodiment, the information processing system not only notifies the monitored person 21 that a posture abnormality has occurred, but also notifies them along with text information acquired by the LLM 16, making it easier to understand the state of the detected posture abnormality.

[0035] In this embodiment, the camera 20 continuously captures images including the person 21 being monitored and transmits them to the management server 10.

[0036] Then, in the management server 10, the control unit 34 performs skeletal estimation of a person in the captured image and detects whether or not the person has a postural abnormality such as falling. When a postural abnormality is detected in the monitored person 21, the control unit 34 inputs the image in which the postural abnormality of the monitored person 21 was detected into the LLM 16, thereby obtaining text information about the state of the person 21 in the input image. The control unit 34 then notifies the terminal device 40, which is a pre-configured notification destination, that a postural abnormality has occurred in the monitored person 21, along with the text information obtained from the LLM 16 and the image in which the postural abnormality was detected.

[0037] In this embodiment, we will describe the case where the postural abnormality detected by skeletal estimation is a fall of the monitored person 21. However, postural abnormalities detected by skeletal estimation are not limited to falls, and other states that are different from the normal state, such as a state where a person is slumped over a table and not moving, or a state where a person is leaning against a wall and not moving, are also included in postural abnormalities.

[0038] Furthermore, if the control unit 34 detects that the monitored person 21 has fallen, it inputs the image in which the person 21 has fallen, as well as images before and after the fall, into the LLM 16 to obtain text information containing information about the monitored person 21's state before and after the fall. The control unit 34 may then notify the terminal device 40 of the obtained text information containing information about the monitored person 21's state before and after the fall, along with the fact that a postural abnormality has occurred in the monitored person 21, the image in which the postural abnormality was detected, and the images before and after the fall.

[0039] The reason why information about the person being monitored 21 not only when they fall, but also information before and after the fall, is that if the person being monitored 21 gets up immediately after falling, the situation is not considered urgent. However, if they remain in the fallen position, the situation is likely to be urgent and require a rapid response.

[0040] Next, the operation of the information processing system of this embodiment will be described in detail with reference to the drawings.

[0041] Figure 5 is a flowchart illustrating the operation of the information processing system in this embodiment. The following description will explain the case where a fall of the monitored person 21 is detected and the detection is notified to the terminal device 40.

[0042] First, in step S101, the camera 20 acquires an image of the person 21 being monitored. Then, in step S102, the image acquired by the camera 20 is sent to the management server 10, where the control unit 34 performs object detection processing on the transmitted image.

[0043] Then, in step S103, the control unit 34 performs skeletal estimation processing on the person detected in the image and obtains skeletal coordinate information. Furthermore, in step S104, the control unit 34 uses the obtained skeletal coordinate information to detect if the person in the image has fallen.

[0044] Next, in step S105, it is determined whether or not a fall of a person in the image has been detected. If a fall of a person in the image is not detected (no in step S105), the process returns to step S101 and steps S101 to S104 are repeated. If a fall of a person in the image is detected (yes in step S105), in step S106, the control unit 34 inputs images of the fall and images before and after the fall to the LLM16. Then, in step S107, the control unit 34 obtains the text information output from the LLM16.

[0045] Finally, in step S108, the control unit 34 notifies the terminal device 40, which is the designated notification destination, that the monitored person 21 has fallen, along with the text information obtained from the LLM 16 and images taken at the time of the fall and before and after the fall.

[0046] Furthermore, if images captured by camera 20 were continuously input into LLM16 and converted into text information, the processing load would be enormous. Therefore, in this embodiment, as described above, by inputting the image of the person 21 being monitored into LLM16 and converting it into text information only when a fall is detected, the processing load is reduced compared to the case where images are continuously input into LLM16 and converted into text information.

[0047] Next, Figures 6 and 7 show examples of notifications sent to the terminal device 40 as a result of the processing described above. Figure 6 is an example of a notification when the monitored person 21 remains in a fallen state. Figure 7 is an example of a notification when the monitored person 21 stands up after falling.

[0048] Referring to Figure 6, it is shown that person 21, who was walking normally before falling, falls and remains in a fallen state without getting up afterward. In such a case, by inputting images of the fall and before and after the fall into the LLM16, the LLM16 outputs text information such as, "A person was walking but fell. They are still lying on the ground." Therefore, the control unit 34 notifies the terminal device 40 that the monitored person 21 has fallen, along with the images of the fall and before and after the fall, and the text information, "A person was walking but fell. They are still lying on the ground." Upon receiving such a notification, the user can easily determine that the monitored person 21 has not gotten up after falling and that the situation is urgent.

[0049] Furthermore, as shown in Figure 7, a person 21 who was walking normally before falling falls and then immediately stands up afterward. In such cases, by inputting images of the fall and the images before and after the fall into the LLM 16, the LLM 16 outputs text information such as, "A person was walking but fell. They are now standing up." Therefore, the control unit 34 notifies the terminal device 40 that the monitored person 21 has fallen, along with the images of the fall and the images before and after the fall, and the text information, "A person was walking but fell. They are now standing up." Upon receiving such a notification, the user can easily determine that the monitored person 21 fell but then stood up, and therefore the urgency is low.

[0050] [Differentiation] The above explanation uses the example of detecting that the monitored person 21 has fallen and notifying a pre-configured recipient, but notifications may also be sent to the pre-configured recipient for other postural abnormalities.

[0051] For example, if the control unit 34 detects a postural abnormality in the monitored person 21 through skeletal estimation, it determines whether the detected postural abnormality is a fall by the monitored person 21. If it determines that the detected postural abnormality is not a fall by the monitored person 21, the control unit 34 inputs the image in which the postural abnormality was detected into the LLM 16 and obtains text information about the person's state in the input image. The control unit 34 then notifies a pre-configured notification destination that a postural abnormality has occurred in the monitored person 21, along with the obtained text information and the image in which the postural abnormality was detected.

[0052] In this modified version, if the detected postural abnormality is a fall of the monitored person 21, the control unit 34 inputs the image in which the fall of the person 21 was detected, as well as images before and after the fall, into the LLM 16 to obtain text information containing information about the state of the monitored person 21 before and after the fall. The control unit 34 then notifies a pre-set notification destination that a fall has occurred to the monitored person 21, along with the acquired text information and the image in which the fall was detected.

[0053] Next, Figure 8 shows the flowchart of the information processing system's operation when detecting posture abnormalities other than falls and notifying the terminal device 40. In the flowchart of Figure 8, the same reference numerals are used for processes that are the same as those in the flowchart of Figure 5, and their explanations are omitted.

[0054] The flowchart in Figure 8 is the same as the flowchart in Figure 5, but with step S104 replaced by step S104A, and the processing of steps S201 to S204 added.

[0055] In this modified case, the control unit 34 uses the acquired skeletal coordinate information to detect posture abnormalities, including falls, of the person in the image in step S104A.

[0056] Then, in step S201, it is determined whether or not any postural abnormality has been detected in the person in the image. If no postural abnormality has been detected in the person in the image (no in step S201), the process returns to step S101 and steps S101 to S104A are repeated. If any postural abnormality has been detected in the person in the image (yes in step S201), the control unit 34 determines in step S105 whether or not a fall has been detected in the person in the image.

[0057] If the control unit 34 determines that a fall has been detected in the image (yes in step S105), it executes the same processing as described above in steps S106 to S108. If the control unit 34 determines that a fall has not been detected in the image (no in step S105), it inputs the image at the time of posture abnormality detection to the LLM 16 in step S202. Then, in step S203, the control unit 34 obtains the text information output from the LLM 16. Finally, in step S204, the control unit 34 notifies the terminal device 40, which is the designated notification destination, that a posture abnormality has occurred in the monitored person 21, along with the text information obtained from the LLM 16 and the image at the time of posture abnormality detection.

[0058] Figures 9 and 10 show examples of notifications when a postural abnormality other than a fall is detected by the process described above.

[0059] Figure 9 shows an example of a notification when a postural abnormality is detected in the monitored person 21, who is slumped over on a desk. Referring to Figure 9, by inputting the image at the time the postural abnormality was detected into the LLM16, the LLM16 outputs text information such as "A person is slumped over on a desk." Therefore, the control unit 34 notifies the terminal device 40 that a postural abnormality has occurred in the monitored person 21, along with the image at the time the postural abnormality was detected and the text information "A person is slumped over on a desk." A user who receives such a notification can easily determine what kind of postural abnormality occurred in the monitored person 21 simply by reading the text information.

[0060] Figure 10 also shows an example of a notification when a postural abnormality is detected in which the monitored person 21 is lying on a chair. Referring to Figure 10, when the image at the time of postural abnormality detection is input to the LLM 16, the LLM 16 outputs text information such as "A person is lying on a chair." Therefore, the control unit 34 notifies the terminal device 40 that a postural abnormality has occurred in the monitored person 21, along with the image at the time of postural abnormality detection and the text information "A person is lying on a chair." A user who receives such a notification can easily determine what kind of postural abnormality has occurred in the monitored person 21 simply by reading the text information.

[0061] In this embodiment, each process is executed on any computer. Furthermore, any computer may execute these processes using a processor as hardware, a program as software, or a combination thereof. In that case, the processor is configured to work in cooperation with the program to execute the various processes in this embodiment, and can function as a unit or means in this embodiment. Also, the execution order of the processes by the processor is not limited to the order described and may be changed as appropriate. Any computer may be a general-purpose computer, a computer designed for a specific purpose, a workstation, or any other system capable of executing each process.

[0062] A processor may consist of one or more hardware components, and the type of hardware is not limited. For example, a processor may consist of a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a programmable logic device such as an FPGA (Field Programmable Gate Array), a dedicated circuit for executing a specific process such as an ASIC (Application Specific Integrated Circuit), a GPU (Graphic Processing Unit), or an NPU (Neural Processing Unit). Furthermore, the type of hardware may be a combination of different types of hardware. When multiple hardware components are configured to execute one or more processes of a given processor, these components may reside in physically separate devices or in the same device. Also, in any embodiment, the order of each process performed by the processor is not limited to the order described above and may be changed as appropriate. Hardware is composed of electrical circuits (circuitry) that combine circuit elements such as semiconductor elements.

[0063] Furthermore, the program may be firmware or software such as microcode. Alternatively, the program may be, for example, a group of program modules, each function of which may be implemented by a processor configured to perform its respective function. The program may be program code or multiple code segments stored on one or more non-temporary computer-readable media (e.g., storage media or other storage). The program may be divided and stored on multiple non-temporary computer-readable media located on physically separate devices. The program code or code segments may represent any combination of procedures, functions, subprograms, routines, subroutines, modules, software packages, classes, or instructions, data structures, or program statements. The program code or code segments may be connected to other code segments or hardware circuits by sending and receiving information, data, arguments, parameters, or memory contents. The program of this application may also be provided as a program product.

[0064] Furthermore, the processor operations in each of the above embodiments may not be performed by a single processor, but may also be performed by multiple processors located in physically separate locations working together. Also, the order of the processor operations is not limited to the order described in each of the above embodiments, and may be changed as appropriate.

[0065] In this embodiment, "system" includes both systems composed of multiple devices and systems composed of a single device.

[0066] [Note] (((1))) Equipped with a processor, The aforementioned processor, Continuously take images including the person being monitored, The system estimates the skeleton of a person in the captured image and detects whether or not there is a postural abnormality in that person. When a postural abnormality is detected, a large-scale language model that converts the content of an input image into text information and outputs it is input to the image in which the person's postural abnormality was detected, thereby obtaining text information about the person's state in the input image. The system notifies a pre-configured recipient that a postural abnormality has occurred in the person being monitored, along with the acquired text information and the image in which the postural abnormality was detected. Information processing system. (((2))) Postural abnormalities detected by skeletal estimation indicate that the monitored person has fallen. The information processing system described in (((1))). (((3))) The processor inputs an image in which the person being monitored has fallen, and images before and after the fall, into the large-scale language model to obtain text information containing information about the state of the person being monitored before and after the fall. The information processing system described in (((2))). (((4))) The aforementioned processor, If a postural abnormality is detected in the person through skeletal estimation, it is determined whether the detected postural abnormality is a fall of the person being monitored. If it is determined that the detected postural anomaly is not a fall of the monitored person, the image in which the postural anomaly was detected is input to the large-scale language model to obtain text information about the person's state in the input image. The system notifies a pre-configured recipient that a postural abnormality has occurred in the person being monitored, along with the acquired text information and the image in which the postural abnormality was detected. The information processing system described in (((1))). (((5))) The aforementioned processor, If the detected postural abnormality is determined to be a fall of the monitored person, the image in which the fall was detected, as well as images before and after the fall, are input into the large-scale language model to obtain text information containing information about the monitored person's state before and after the fall. The system notifies a pre-configured recipient that a fall has occurred in the person being monitored, along with the acquired text information and the image in which the fall was detected. The information processing system described in (((4))). (((6))) The steps include continuously capturing images that include the person being monitored, The steps include: estimating the skeleton of a person in a captured image to detect whether or not there is a postural abnormality in the person; The process involves inputting an image in which a posture abnormality of a person has been detected into a large-scale language model that converts the content of an input image into text information and outputs it, thereby obtaining text information about the state of the person in the input image. The steps include notifying a pre-configured recipient that a postural abnormality has occurred in the person being monitored, along with the acquired text information and the image in which the postural abnormality was detected, A program that causes a computer to execute something.

[0067] According to the information processing system (((1))), it becomes easier to understand the state of detected postural abnormalities compared to simply detecting and notifying of postural abnormalities of a person from images taken of the person being monitored. According to the information processing system (((2))), it becomes easier to understand the detected fall compared to simply detecting and notifying a person of a fall from an image taken of the person being monitored. According to the information processing system (((3))), a person who receives a notification that a person under surveillance has fallen will be able to understand the urgency of the notification through text information. According to the information processing system (((4))), it is possible to detect postural abnormalities other than falls of the monitored person and notify pre-set recipients. According to the information processing system (((5))), if the monitored person falls, it is possible to notify them of more detailed information in text format than in the case of any other postural abnormality. According to the program in (((6))), it becomes easier to understand the state of the detected postural abnormality compared to simply detecting and notifying of postural abnormalities in a person from an image taken of the person being monitored. [Explanation of Symbols]

[0068] 10 Management Server 11 CPU 12 memory 13 Storage device 14. Communication Interface 15. User Interface Device 16. Large-Scale Language Models (LLMs) 17 Control bus 20 cameras 21 People 30 Internet 31 Operation Input Section 32 Display section 33 Data transmission and reception unit 34 Control Unit 35 Image data storage unit 40 Terminal devices

Claims

1. Equipped with a processor, The aforementioned processor, Continuously take images including the person being monitored, The system estimates the skeleton of a person in the captured image and detects whether or not there is a postural abnormality in that person. When a postural abnormality is detected, a large-scale language model that converts the content of an input image into text information and outputs it is input to the image in which the person's postural abnormality was detected, thereby obtaining text information about the person's state in the input image. The system notifies a pre-configured recipient that a postural abnormality has occurred in the person being monitored, along with the acquired text information and the image in which the postural abnormality was detected. Information processing system.

2. Postural abnormalities detected by skeletal estimation indicate that the monitored person has fallen. The information processing system according to claim 1.

3. The processor inputs an image in which the person being monitored has fallen, and images before and after the fall, into the large-scale language model to obtain text information containing information about the state of the person being monitored before and after the fall. The information processing system according to claim 2.

4. The aforementioned processor, If a postural abnormality is detected in the person through skeletal estimation, it is determined whether the detected postural abnormality is a fall of the person being monitored. If it is determined that the detected postural anomaly is not a fall of the monitored person, the image in which the postural anomaly was detected is input to the large-scale language model to obtain text information about the person's state in the input image. The system notifies a pre-configured recipient that a postural abnormality has occurred in the person being monitored, along with the acquired text information and the image in which the postural abnormality was detected. The information processing system according to claim 1.

5. The aforementioned processor, If the detected postural abnormality is determined to be a fall of the monitored person, the image in which the person fell, as well as images before and after the fall, are input into the large-scale language model to obtain text information containing information about the monitored person's state before and after the fall. The system notifies a pre-configured recipient that a fall has occurred in the person being monitored, along with the acquired text information and the image in which the fall was detected. The information processing system according to claim 4.

6. The steps include continuously capturing images that include the person being monitored, The steps include: estimating the skeleton of a person in a captured image to detect whether or not there is a postural abnormality in the person; The process involves inputting an image in which a posture abnormality of a person has been detected into a large-scale language model that converts the content of an input image into text information and outputs it, thereby obtaining text information about the state of the person in the input image. The steps include notifying a pre-configured recipient that a postural abnormality has occurred in the person being monitored, along with the acquired text information and the image in which the postural abnormality was detected, A program that causes a computer to execute something.

Images