Monitoring and prompting method, device and system

Abstract

The invention discloses a monitoring and prompting method, device and system. Acquiring a video and a risk level of the target object (S210); identifying the video to obtain feature information representing an activity situation of the target object (S220), the feature information at least comprising: a body posture and position information of the target object; wherein the body posture is obtained by analyzing a plurality of posture key points capable of reflecting the outline of the target object, and the position information comprises the position relation of the target object relative to the sickbed; if the body posture is a first preset posture and the position information meets a first preset position condition, determining that the target object has an off-bed tendency, and outputting off-bed prompt information corresponding to the risk level (S230); the first preset position condition comprises that the target object is located in the sickbed but moves towards the edge of the sickbed, and / or part of the body area of the target object is located outside the edge of the sickbed, so that the risk that the target object falls off the sickbed due to non-standard getting-out of the sickbed is reduced, and nursing management on the target object is improved.

Description

Monitoring and prompting method, device and system

[0001] manual Technical Field

[0002] The present application generally relates to the field of medical device technology, and more specifically to a monitoring and prompting method, device and system. Background Art

[0003] A fall from bed is an uncontrolled, unintentional fall from a patient's bed. It is a primary nursing adverse event to prevent in clinical practice. Falls from bed in the hospital can not only cause bodily harm and worsen a patient's condition, but also undermine their trust in medical safety, adversely impact subsequent treatment, and lead to medical disputes. Therefore, they should be avoided whenever possible.

[0004] Comprehensive measures to prevent falls from bed include: comprehensive assessment of patients' fall risk, bedside warning cards for high-risk patients, placement of bed rails for high-risk patients, education on the process of getting out of bed for patients, strengthening nursing management, and maintaining close attention on high-risk patients.

[0005] At present, in clinical practice, medical staff mainly rely on paying close attention to patients at high risk of falling out of bed to strictly prevent the incidents. However, since medical staff cannot always be at the bedside, it may happen that they cannot remind patients in time when they get out of bed improperly.

[0006] Summary of the Invention

[0007] The present application is made to solve at least one of the above problems.

[0008] Specifically, the first aspect of the present application provides a monitoring and prompting method, characterized in that the method includes:

[0009] Obtain the target object’s video and risk level;

[0010] Identify the video to obtain characteristic information reflecting the activity of the target object, wherein the characteristic information includes at least: body posture and position information of the target object;

[0011] The body posture is obtained based on an analysis of a plurality of posture key points that can reflect the outline of the target object, and the position information includes a positional relationship of the target object relative to the hospital bed;

[0012] If the body posture is a first preset posture and the position information meets the first preset position condition, determining that the target subject has a tendency to leave the bed, and outputting bed leaving prompt information corresponding to the risk level;

[0013] The first preset position condition includes: the target object is located in the bed but moves toward the edge of the bed, and / or a part of the body area of ​​the target object is located outside the edge of the bed.

[0014] A second aspect of the present application provides a monitoring and prompting method, the method comprising:

[0015] Get the target object's video;

[0016] Identify the video to obtain characteristic information reflecting the activity of the target object, wherein the characteristic information includes at least: body posture and position information of the target object;

[0017] The body posture is obtained based on an analysis of a plurality of posture key points that can reflect the outline of the target object, and the position information includes: the area of ​​the target object outside the edge of the bed or the center of mass position of the target object;

[0018] If the body posture is a second preset posture and the position information meets the second preset position condition, it is determined that the target object has fallen out of bed, and a fall-out prompt message is output;

[0019] The second preset position condition includes: the area of ​​the target object outside the edge of the bed is greater than a second preset threshold, or the center of mass of the target object moves from inside the bed to outside the bed.

[0020] A third aspect of the present application provides a monitoring and prompting method, the method comprising:

[0021] Obtain the target object’s video and risk level;

[0022] Identify the video to obtain characteristic information reflecting the activity of the target object, wherein the characteristic information at least includes: body posture or position information of the target object;

[0023] If the body posture is a first preset posture and the position information meets the first preset position condition, determining that the target subject has a tendency to leave the bed, and outputting bed leaving prompt information corresponding to the risk level;

[0024] The first preset position condition includes: the target object is located in the bed but moves toward the edge of the bed, and / or a part of the body area of ​​the target object is located outside the edge of the bed.

[0025] A fourth aspect of the present application provides a monitoring and prompting method, the method comprising:

[0026] Obtain the target object’s video and risk level;

[0027] Identify the video to obtain feature information reflecting the activity of the target object;

[0028] Determine whether the characteristic information meets at least one prompt condition corresponding to the risk level; if so, provide a prompt in a prompt manner corresponding to the prompt condition met by the characteristic information.

[0029] A fifth aspect of the present application provides a device for monitoring and prompting, the device comprising:

[0030] A communication interface, configured to communicate with a video acquisition device to obtain a video of a target object acquired by the video acquisition device;

[0031] a memory for storing executable program instructions;

[0032] at least one processor, configured to execute the program instructions stored in the memory, so that the processor performs the aforementioned method;

[0033] A display is used to display various visual information.

[0034] A sixth aspect of the present application provides a system for monitoring and prompting, the system comprising:

[0035] Video capture equipment, used to capture video of the target object;

[0036] a video processing unit, communicatively connected to the video acquisition device, configured to acquire the video of the target object captured by the video acquisition device and execute the aforementioned method;

[0037] The interactive device is communicatively connected to the video processing unit and is used to receive various data information output by the video processing unit and present the various data information.

[0038] According to the monitoring and prompting methods of the first, third and fourth aspects of the present application, the video is identified to obtain characteristic information reflecting the activity of the target object, and the characteristic information includes at least: the body posture and position information of the target object. If the body posture is a first preset posture and the position information meets the first preset position condition, it is determined that the target object has a tendency to leave the bed, and bed leaving prompt information corresponding to the risk level is output to promptly prompt the target object with a tendency to leave the bed, so that the target object can get out of bed in accordance with the bed leaving prompt information or remind the target object not to get out of bed, thereby reducing the risk of falling out of bed due to the target object's improper getting out of bed and improving the nursing management of the target object.

[0039] According to the monitoring and prompting method of the second aspect of the present application, the accuracy of judging whether the target object has fallen out of bed by combining body posture with position information is higher, and when it is determined that the target object has fallen out of bed, a fall-out prompt message is output so that medical personnel can be informed of the fall-out in a timely manner, so that they can provide help or treatment to the target object who has fallen out of bed more promptly.

[0040] The monitoring and prompting device and system of the present application have the same advantages as the aforementioned method because they can execute the aforementioned method. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0042] FIG1 shows a schematic block diagram of a device for monitoring and prompting in one embodiment of the present application;

[0043] FIG2 shows a schematic flow chart of a monitoring and prompting method in one embodiment of the present application;

[0044] FIG3 shows a schematic diagram of posture key points in one embodiment of the present application;

[0045] FIG4 is a schematic diagram showing a falling-out-of-bed prompt message in one embodiment of the present application;

[0046] FIG5 is a schematic diagram showing detailed information and video information of a falling-out-from-bed event in one embodiment of the present application;

[0047] FIG6 is a schematic diagram showing detailed information of a fall-out event and a fall-out prompt in one embodiment of the present application;

[0048] FIG7 shows a schematic flow chart of a monitoring and prompting method in another embodiment of the present application;

[0049] FIG8 shows a schematic flow chart of a monitoring and prompting method in yet another embodiment of the present application;

[0050] FIG9 shows a schematic flow chart of a monitoring and prompting method in yet another embodiment of the present application;

[0051] FIG10 shows a schematic block diagram of a system for monitoring and prompting in one embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present application more apparent, the following is a detailed description of example embodiments of the present application with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application described in this application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of this application.

[0053] In the following description, a large number of specific details are provided to provide a more thorough understanding of the present application. However, it will be apparent to those skilled in the art that the present application can be implemented without one or more of these details. In other examples, some technical features well known in the art are not described in order to avoid confusion with the present application.

[0054] It should be understood that the present application can be implemented in different forms and should not be interpreted as being limited to the embodiments set forth herein. On the contrary, providing these embodiments will make the disclosure thorough and complete and will fully convey the scope of the present application to those skilled in the art.

[0055] The purpose of the terms used herein is only to describe specific embodiments and is not intended to limit the present application. When used herein, the singular forms "a", "an", and " / the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "comprising" and / or "including", when used in this specification, identify the presence of features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or groups. When used herein, the term "and / or" includes any and all combinations of the relevant listed items.

[0056] In order to fully understand the present application, a detailed structure will be provided in the following description to illustrate the technical solution proposed by the present application. The optional embodiments of the present application are described in detail below. However, in addition to these detailed descriptions, the present application may also have other implementation methods.

[0057] Specifically, the present application will be described in detail below with reference to the accompanying drawings. Unless there is any conflict, the features of the following embodiments and implementations may be combined with each other.

[0058] Furthermore, the present application also provides a device for monitoring and prompting. The device 100 for monitoring and prompting in the embodiment of the present application includes, but is not limited to, medical equipment, central monitoring equipment, workstation display equipment, any suitable PC, etc. Medical equipment includes, but is not limited to, monitors, ventilators, infusion pumps, etc., wherein monitors include, but are not limited to, bedside monitoring devices or wearable monitoring devices, etc. Furthermore, the monitoring and prompting device of the present application can also be a video processing device, which can communicate with other devices such as monitors or central monitoring devices, and can also communicate with video acquisition devices, wherein the video processing device and the video acquisition device can also be the same device.

[0059] As an example, as shown in FIG1 , a device 100 for monitoring and prompting includes one or more processors 101, a display 102, a memory 103, and a communication interface 104. These components are interconnected via a bus system and / or other connection mechanisms (not shown). It should be noted that the components and structure of the device 100 for monitoring and prompting shown in FIG1 are merely exemplary and non-limiting. The device 100 for monitoring and prompting may also have other components and structures as needed.

[0060] The memory 103 is used to store various data and executable programs, such as system programs for monitoring and prompting equipment, various application programs or algorithms for implementing various specific functions, and medical parameter information collected by medical equipment, video information collected by video acquisition equipment, etc.

[0061] The memory 103 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. When the device for monitoring and prompting obtains data information sent by various devices such as monitors, ventilators, ultrasound equipment, video acquisition equipment, etc., if there is data that needs to be stored locally, it can be stored in the memory 103.

[0062] The processor 101 may be a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the device for monitoring and prompting to perform desired functions. For example, the processor may include one or more embedded processors, processor cores, microprocessors, logic circuits, hardware finite state machines (FSMs), digital signal processors (DSPs), graphics processing units (GPUs), or combinations thereof.

[0063] In one example, the monitoring and prompting device 100 further includes a communication interface 104 for communicating between various components within the device, and between various components of the monitoring and prompting device and other devices outside the system (e.g., a video capture device, a medical device server, medical devices, etc.), to exchange information between the various devices. The information exchange method between the various devices can be found in the description of the medical system above.

[0064] The communication interface 104 is an interface that can be any currently known communication protocol, such as a wired interface or a wireless interface, wherein the communication interface 104 may include one or more serial ports, USB interfaces, Ethernet ports, WiFi, wired networks, DVI interfaces, device integrated interconnect modules or other suitable various ports, interfaces, or connections. The device 100 for monitoring and prompting can also access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G or a combination thereof. In an exemplary embodiment, the communication interface 104 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication interface 104 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0065] In one example, the monitoring and prompting device 100 further includes an input device (not shown). The input device can be a device used by a user to input commands, and can include one or more of a keyboard, a trackball, a mouse, a microphone, a touch screen, or other input devices consisting of control buttons. For example, a user can input commands for viewing various data information through the input device.

[0066] The device 100 for monitoring and prompting of the embodiment of the present application also includes an output device, which can output various information (such as images or sounds) to the outside (such as a user), and can include one or more of a display 102, a speaker, etc.

[0067] In one example, the device 100 for monitoring and prompting also includes one or more displays 102. The display 102 may be a touch screen, LCD screen, etc. built into the device 100 for monitoring and prompting, or it may be an independent display device such as an LCD display, a television, etc. that is independent of the device 100 for monitoring and prompting, or it may be a display screen on an electronic device such as a mobile phone or a tablet computer, etc.

[0068] The device for monitoring and prompting 100 further includes a user interface, through which a user of the device for monitoring and prompting can control the operation of the device for monitoring and prompting. The user interface can be provided by the display 102, which can include a touch screen for allowing the user to input operating instructions to the device for monitoring and prompting via the display 102, and / or include one or more control panels, through which the user can control the operation of the device for monitoring and prompting.

[0069] In one example, the processor 101 is configured to execute a program stored in the memory 103 , so that the processor 101 performs the monitoring and prompting method described below.

[0070] Below, we will combine the accompanying drawings 2 to

[0071] As an example, as shown in FIG2 , the present application provides a monitoring and prompting method 200 , which includes the following steps:

[0072] In step S210, the video and risk level of the target object are obtained;

[0073] In step S220, the video is recognized to obtain feature information reflecting the activity of the target object, the feature information including at least: body posture and position information of the target object, wherein the body posture is obtained by analyzing a plurality of posture key points that can reflect the outline of the target object, and the position information includes the position relationship of the target object relative to the bed;

[0074] In step S230, if the body posture is the first preset posture and the position information meets the first preset position condition, it is determined that the target object has a tendency to leave the bed, and bed leaving prompt information corresponding to the risk level is output, wherein the first preset position condition includes: the target object is located in the bed but moving toward the edge of the bed, and / or part of the target object's body area is already outside the edge of the bed.

[0075] According to the monitoring and prompting method and system of the present application, the video is identified to obtain characteristic information reflecting the activity of the target object. The characteristic information includes at least: the body posture and position information of the target object. If the body posture is a first preset posture and the position information meets the first preset position condition, it is determined that the target object has a tendency to leave the bed, and bed leaving prompt information corresponding to the risk level is output to promptly prompt the target object with a tendency to leave the bed, so that the target object can get out of bed in accordance with the bed leaving prompt information or remind the target object not to get out of bed, thereby reducing the risk of falling out of bed due to the target object's improper getting out of bed and improving the nursing management of the target object.

[0076] Specifically, in step S210, the video of the target object can be captured by a video capture device, which may also include audio information, etc. The video capture device can be, for example, any device with an image capture function, such as a camera. For example, the video capture device can be an RGB camera or a depth camera. When there are multiple target objects, each target object can correspond to one or more video capture devices. The video captured by the video capture device can be obtained. The video can be a real-time captured video. Subsequently, by performing recognition processing on the real-time captured video, information such as the posture of the target object on the bed can be obtained in real time, and timely bed leaving information prompts can be provided to reduce the risk of falling out of bed.

[0077] In some embodiments, the risk level is a rating of the target subject's risk of falling from bed. For example, the risk level for some target subjects may be allowed to move around in bed or can get out of bed, while the risk level for some target subjects may be strictly bedridden. Generally, the risk level for strictly bedridden subjects is higher than that for allowed to move around in bed or can get out of bed. The risk level may be determined by medical personnel based on the target subject's overall condition.

[0078] The risk level of the target object can be obtained by any suitable means, for example, the risk level of the target object can be obtained from an electronic medical record system, for example, the electronic medical record system stores information about the risk level of the target object, which can be retrieved when needed. For example, the corresponding risk level information can be searched and retrieved in the electronic medical record system using the target object's identification information, such as bed number, name, ID card, etc. For another example, the risk level of the target object can be obtained by receiving input information associated with the risk level of the target object, for example, inputting the risk level of the target object through a display interface of an interactive device. The input information can be input by a user through a remote terminal device (such as a mobile phone, tablet, notebook, wearable device) or input by an interactive device such as a monitoring device or a central station. Optionally, the interactive device can communicate with a video processing device to transmit the input information to the video processing device, which obtains the risk level and performs subsequent processing and analysis based on the video. The video processing device can be a processing device specifically for video processing, such as a video server, or it can also be a video acquisition device. The results of the analysis and processing can be subsequently transmitted to the interactive device for presentation.

[0079] In some embodiments, in step S220, the body posture is obtained based on an analysis of multiple posture key points that can reflect the outline of the target object (or the skeleton of the target object), and the position information includes the positional relationship of the target object relative to the bed. The body posture and position information can be used to determine whether the target object has a tendency to leave the bed.

[0080] Exemplarily, the video is identified to obtain characteristic information reflecting the activity of the target object, including: processing multiple frames of video images in the video to obtain change sequence data of the target object's posture key points; based on the change sequence data of the posture key points, the body posture of the target object is identified, and the body posture includes any of the following: lying posture, sitting posture, posture in the process of changing from lying to sitting, standing posture, walking posture, falling out of bed posture or falling posture, turning over, etc., or other types of body postures. It is worth mentioning that in this application, the target object can be a patient, or it can also be a person who needs care and monitoring, such as the elderly or children.

[0081] The change sequence data of the target object's posture key points can be obtained by any suitable method. For example, multiple frames of video images in a video can be processed first to identify the position data of the target object's posture key points in each frame of video image; the position data of the posture key points in each frame of video image can be recorded in chronological order to obtain the change sequence data of the target object's posture key points. By obtaining the change sequence data of the posture key points, the target object's body posture can be identified based on the change sequence data of the posture key points. Optionally, the change sequence data of the posture key points can be output each time a preset number of video frames are obtained, or can be randomly obtained based on the position data of the posture key points corresponding to the multiple frames of video images recorded in chronological order.

[0082] In some embodiments, the multiple pose key points include multiple joint points used to represent the joints of the target object. For example, they may be 18 or 25 joint points extracted by processing multiple frames of video images in a video based on the Open-pose algorithm. These joint points may be joint points of the arms, head, torso, legs, and other parts of the human body.

[0083] In addition to the Open-pose algorithm, any other suitable algorithm capable of identifying the joint points of the target object may also be used.

[0084] Further, the body posture of the target object can be identified according to the change sequence data of the posture key points. For example, the change sequence data of the posture key points are input into a pre-trained recognition model for processing to identify the body posture of the target object. Alternatively, the pre-trained recognition model can be an ST-GCN model or other applicable models. The ST-GCN model is a spatiotemporal graph convolutional neural network model for solving the behavior recognition analysis problem based on human skeleton key points (i.e., human skeleton joint points). The training of this model can adopt any applicable training method well known to those skilled in the art.

[0085] Furthermore, in some embodiments, the video is identified to obtain characteristic information reflecting the activity of the target object, and the process also includes: processing the video to identify a first positioning target frame corresponding to the target object and a second positioning target frame corresponding to the hospital bed, wherein the position of the target object can be located by the first positioning target frame, and the position of the hospital bed can be located by the second positioning target frame. The edge contour of the hospital bed can also be determined by processing the video, and the pixel position of the closed figure outlining the edge of the hospital bed can be output, or the edge contour of the target object can be obtained. In some embodiments, the edge contour of the target object can be the same as or different from the first positioning target frame. For example, the first positioning target frame can be a rectangular frame or other suitable shape frame, which can surround the target object. The edge contour of the hospital bed can be the same as or different from the second positioning target frame. For example, the second positioning target frame can be a rectangular frame or other suitable shape frame, which can surround the entire hospital bed.

[0086] The video can be identified and processed using a target detection algorithm to determine and identify the target object and the hospital bed in the video image, and then the target object and the hospital bed are located using the first positioning target frame and the second positioning target frame. For example, the target detection algorithm can be any one of the following: VJ, HOG, DPM Detector, RCNN, SPPNet, Fast RCNN, Faster RCNN, FPN, Cascade RCNN, Yolo v1, v2, v3, v4, v5, X, SSD, RetinaNet, CornerNet, CenterNet, FCOS, DETR, etc.

[0087] As an example, the method of the present application further includes calculating the area of ​​the first positioning target frame outside the second positioning target frame as the area of ​​the target object outside the edge of the bed. By obtaining the area of ​​the target object outside the edge of the bed, the positional relationship between the target object and the bed can be determined to assist in determining whether the target object has a tendency to leave the bed or fall out of bed.

[0088] Optionally, the area of ​​the region where the target object is located outside the edge of the bed may also be obtained by calculating the area of ​​the region where the target object is located outside the edge of the bed.

[0089] After the above steps, step S230 can be performed to determine whether the target subject has an intention to leave the bed. For example, if the body posture is a first preset posture and the position information meets the first preset position condition, then the target subject is determined to have an intention to leave the bed. The body posture combined with the position information can more accurately determine whether the target subject has an intention to leave the bed. The intention to leave the bed refers to the target subject's intention to prepare or be about to leave the bed. The first preset position condition includes: the target subject is located within the bed but moving toward the edge of the bed, and / or a portion of the target subject's body area is located outside the edge of the bed. The first preset posture includes the posture during the transition from a recumbent position to a sitting position. Since a target subject originally in a recumbent position changes to a sitting position, there are two possibilities: either simply sitting in the bed or preparing to leave the bed. Therefore, accurate determination of the intention to leave the bed based solely on posture is not possible. However, combining the position information can provide an accurate determination of the intention to leave the bed. For example, if the target subject is located within the bed but moving toward the edge of the bed, this indicates that the target subject is preparing to leave the bed, and thus, an intention to leave the bed can be determined. Alternatively, if a portion of the target subject's body area is located outside the edge of the bed, this indicates that the target subject is preparing to leave the bed, and thus, an intention to leave the bed can be determined.

[0090] It is worth mentioning that the positional relationship between the target object and the bed can be reflected in other suitable ways, in addition to the above-mentioned target object being located in the bed but moving toward the edge of the bed, and / or part of the target object's body area being located outside the edge of the bed, such as by the position of the target object's center of mass or center of gravity relative to the bed.

[0091] In some embodiments, part of the target object's body area is located outside the edge of the bed means that the area of ​​the target object outside the edge of the bed is greater than a first preset threshold. This can assist in determining that the target object may have a tendency to leave the bed, wherein the first preset threshold can be reasonably set according to actual needs and is not specifically limited here.

[0092] Furthermore, when it is determined that the target subject has a tendency to leave the bed, bed leaving prompt information corresponding to the risk level is output. By prompting the target subject with a tendency to leave the bed in a timely manner, the target subject can get out of bed in accordance with the bed leaving prompt information or be reminded not to get out of bed, thereby reducing the risk of falling out of bed due to the target subject's improper getting out of bed and improving the nursing management of the target subject.

[0093] Among them, different risk levels can correspond to different bed-leaving prompt information. For example, the higher the risk level, the more prominent the presentation of the bed-leaving prompt information. For example, when the risk level is the first risk level, the bed-leaving prompt information is: playing the rule steps that the target object should follow when getting out of bed, wherein the target object of the first risk level usually performs the activity of getting out of bed; when the risk level is the second risk level, the bed-leaving prompt information is: outputting an alarm message, wherein the risk level of the first risk level is lower than the risk level of the second risk level, that is, the risk level of falling out of bed at the first risk level is lower than the risk level of falling out of bed at the second risk level, that is, the target object at the second risk level is more likely to fall out of bed than the target object at the first risk level. By outputting an alarm message when the target object with a higher risk level has a tendency to get out of bed, the target object is warned or prompted not to get out of bed, thereby reducing the risk of falling out of bed.

[0094] The rule steps can be played to the target object through voice broadcast or other means to remind the target object to get out of bed in a standardized manner. For example, the rule steps include: first lying down and maintaining a lying posture for at least a first preset time, then sitting up and maintaining a sitting posture for at least a second preset time, and then standing up and maintaining a standing posture for at least a third preset time. Among them, the first preset time, the second preset time, and the third preset time can be reasonably set according to the actual situation or risk level of the target object. For example, the first preset time, the second preset time, and the third preset time can all be 30 seconds. That is, reminding patients to follow the "three steps to get out of bed": lying down for 30 seconds, sitting up for 30 seconds, and standing for 30 seconds can effectively prevent falling out of bed.

[0095] The rule steps are only used as examples, and other suitable steps or rules for regulating patients getting out of bed may also be applicable to this application.

[0096] In some embodiments, the alarm information can be output in a variety of ways, such as by alerting the target patient's corresponding medical personnel. For example, the alarm information can be output in a preset presentation mode, including at least one of the following: a text description of the second risk level, a symbol for the second risk level, an audible alarm, or a visual alarm. The alarm information can be presented on an interactive device, such as a monitoring device, a central station, or a remote terminal device (e.g., a mobile phone, tablet, laptop, or wearable device). In some embodiments, a voice alarm can be played through a speaker, or an audible or visual alarm can be used to remind the target patient not to get out of bed.

[0097] Optionally, the first risk level includes being allowed to move around in bed, or being able to get out of bed to move around; the second risk level includes absolute bed rest, or may include more risk level divisions, which are not specifically limited here.

[0098] Once a fall occurs, medical staff must promptly understand the circumstances surrounding the fall, particularly the patient's posture at the time of the fall, to identify the point of impact and quickly assess any potential injuries. However, inpatients are often physically weak and may experience confusion after a fall, making it difficult for them to clearly describe the fall to medical staff.

[0099] In view of the above problems, the method of the present application can also judge whether the target object has fallen out of bed and output a prompt. For example, the position information also includes the area of ​​the target object outside the edge of the bed or the center of mass position of the target object. The method of the present application also includes: if the body posture is a second preset posture and the position information meets the second preset position condition, then it is determined that the target object has fallen out of bed and a fall-out prompt message is output, wherein the second preset position condition includes: the area of ​​the target object outside the edge of the bed is greater than a second preset threshold, or the center of mass position of the target object moves from inside the bed to outside the bed. The accuracy of judging whether the target object has fallen out of bed by combining body posture with position information is higher, and when it is determined that the target object has fallen out of bed, a fall-out prompt message is output, so that medical staff can promptly know that the target object has fallen out of bed, so that they can provide more timely help or treatment to the target object who has fallen out of bed.

[0100] Optionally, the second preset posture includes a falling-out-of-bed posture or a falling-down posture or other postures indicating that the target object may be in a falling-out-of-bed event.

[0101] If the target object's area outside the bed edge is greater than a second preset threshold, and most of the target object's body is already outside the bed edge, then the posture can be used to determine that a fall from bed has occurred. Alternatively, if the target object's center of mass moves from inside the bed to outside the bed and is now outside the bed, then the target object is also outside the bed, and then the posture can be used to determine that a fall from bed has occurred. The second preset threshold is greater than the aforementioned first preset threshold, and these thresholds can be different for different target objects, and can be reasonably set based on the target object's actual height and size.

[0102] In some embodiments, a fall out of bed can also be determined by the following conditions. For example, when the target object is in the second preset posture, the height of the target object in the direction perpendicular to the surface of the bed is not higher than a first threshold height, for example, not higher than half of the target object's body height, or other suitable set values. This indicates that the patient may have fallen to the ground in a sitting, lying or other posture. For another example, when the target object is in the second preset posture, the height of the target object in the direction perpendicular to the surface of the bed is lower than the length in the direction parallel to the surface of the bed, and / or most of the target object is located under the bed. These conditions combined with the target object's body posture can be used to determine whether the target object has fallen out of bed with higher accuracy.

[0103] In other embodiments, the positional relationship between the target object and the ground may be identified through video, and combined with the body posture to determine whether the target object has fallen out of bed.

[0104] In order to determine whether a person has fallen out of bed, the center of mass position of the target object can be determined by the following method. For example, multiple frames of video images in a video are processed to identify the target object in the multiple frames of video images. For example, the target object is identified based on the aforementioned target detection algorithm, and the target object is located by a first positioning target frame. The area enclosed by the first positioning target frame can be used as a region of interest (ROI) for motion monitoring of the target object; a motion heat map corresponding to the target object in the multiple frames of video images is calculated, for example, a motion heat map within the ROI is calculated; and the center of mass position of the target object in the multiple frames of video images is calculated based on the position information of multiple pixel points in the motion heat map corresponding to the multiple frames of video images.

[0105] The motion heat map can be calculated by any suitable method known to those skilled in the art, such as by the difference method between adjacent frames. In some embodiments, the motion heat map corresponding to the target object in the multi-frame video image is calculated, including: for the video image of the target frame in the multi-frame video image, performing the following steps: subtracting the pixel values ​​of the corresponding pixel points in the video image of the target frame from the pixel values ​​of the corresponding pixel points in the video image of the previous frame to obtain the absolute values ​​of the frame differences corresponding to the multiple pixel points in the video image of the target frame, or calculating with the video image of the next frame, or accumulating the absolute values ​​of the frame differences corresponding to the frame difference images before the video image of the target frame to the absolute values ​​of the frame differences corresponding to the multiple pixel points in the video image of the target frame; obtaining the frame difference image corresponding to the video image of the target frame based on the absolute values ​​of the frame differences corresponding to the multiple pixel points, wherein the frame difference image represents Motion amplitude distribution within the ROI; filtering the frame difference image (such as median filtering or other suitable filtering methods) to obtain a filtered frame difference image. This filtering step can remove the influence of fluctuations between pixels; setting the absolute value of the frame difference in the filtered frame difference image that is less than the preset frame difference threshold to 0 to obtain a motion heat map corresponding to the video image of the target frame. The preset frame difference threshold can be reasonably set according to actual conditions. This step can ignore areas with small motion amplitudes, that is, areas with small motion amplitudes are approximated as static. The motion heat map can reflect the distribution of moving areas and static areas, and the pixel values ​​in the moving areas, such as the absolute value of the frame difference, reflect the motion amplitude distribution within the ROI.

[0106] In one embodiment, before subtracting the pixel values ​​of corresponding pixel points in the video image of the target frame and the video image of its previous frame to obtain the absolute values ​​of the frame differences corresponding to multiple pixel points in the video image of the target frame, the video image of the target frame and the video image of its previous frame can also be converted into grayscale images, and subsequent calculations can be performed using the grayscale images.

[0107] In some embodiments, the center of mass position of the target object in the multi-frame video image is calculated based on the position information of multiple pixels in the motion heat map corresponding to the multi-frame video image, including: multiplying the horizontal axis coordinate value and the vertical axis value of the multiple pixel points in the motion heat map by their respective corresponding weights and then summing them to obtain the horizontal axis coordinate value and the vertical axis coordinate value of the center of mass position of the target object, and the weights corresponding to the multiple pixel points in the motion heat map are the ratio between the absolute value of the frame difference corresponding to the multiple pixel points and the sum of the absolute value of the frame difference of all pixel points. This method can obtain the center of mass position of the target object, and then judge the approximate position of the target object by the center of mass position, and then combine the center of mass position to assist in judging whether the target object has fallen out of bed.

[0108] Furthermore, it is determined that the target object has fallen out of bed and a fall-out prompt message is output. The accuracy of judging whether the target object has fallen out of bed by combining body posture with position information is higher. When it is determined that the target object has fallen out of bed, the fall-out prompt message is output so that medical personnel can promptly learn that the target object has fallen out of bed and can provide assistance or treatment to the target object that has fallen out of bed more promptly. The fall-out prompt message includes at least one of the following prompt messages: a schematic diagram of key points of the posture when the target object contacts the ground after falling out of bed, a schematic diagram of key points of the posture when falling out of bed, a video screenshot of the fall-out, a video within a preset time before and after the fall-out, or other suitable prompt messages. After the patient falls out of bed, the body posture of the patient when falling out of bed is automatically extracted as the fall-out prompt message and displayed on the medical equipment so that the medical personnel can find the point of impact when falling out of bed through the fall-out prompt message, assist the medical personnel in making a quick judgment on the injury caused by falling out of bed, and judge the possible body damage as soon as possible.

[0109] Exemplarily, after falling out of bed and during falling out of bed can be different moments in the process of falling out of bed. After falling out of bed can be when the target object has fallen out of bed and contacted the ground, and during falling out of bed can be when the target object falls from the bed or is in the process of falling.

[0110] The posture key point diagram may be formed by connecting multiple posture key points through lines according to the arrangement of the human skeleton, as shown in FIG3 , for example.

[0111] In some embodiments, the display device of the interactive device can provide a display observation interface, and the fall prompt information can be displayed through the display device of the interactive device. For example, the interactive device can be a monitoring device, and the monitoring device has a display device for displaying the information of the corresponding target object. Alternatively, the interactive device can also be a central station, and the display device of the central station can be used to display the information of multiple target objects. For example, the display observation interface of the display device of the central station includes at least one display area, and each display area corresponds to displaying at least part of the medical parameter information of a target object. For example, the display observation interface shown in Figure 4 provides 4 display areas, and the 4 display areas simultaneously display at least part of the medical parameter information of 4 target objects, such as vital signs information.

[0112] Furthermore, the user can also operate one of the multiple display areas, for example, by using a mouse or a touch screen to open a detail interface of the corresponding selected target object. The detail interface can be used to display real-time information and review information of the selected target object. Specifically, when a selection instruction for one of the multiple display areas is obtained (for example, based on clicking the corresponding display area through a mouse or a touch screen), the display device is controlled to display the detail interface of the selected target object corresponding to the selected display area. The detail interface is provided with a review button (not shown). Optionally, the detail interface is also provided with a real-time information viewing button (not shown). Optionally, the detail interface can be located on one side of the display observation interface and does not cover the multiple display areas displayed by the display observation interface, so that while viewing the selected target object, partial medical parameter information of other target objects, such as vital signs information, can also be viewed.

[0113] When a viewing instruction for the real-time information viewing button is obtained, the control display device displays the real-time interface of the selected target object on the details interface. The real-time interface is used to display real-time information of the selected target object, such as real-time medical parameter information such as vital signs parameter information and real-time video information.

[0114] In addition to displaying a real-time information viewing button and a review button, the details interface may also include corresponding buttons for patient management. These buttons allow users to switch between the real-time, review, and patient management interfaces. It should be noted that real-time and review are relative terms. For example, the details interface may include multiple review buttons, each used to display review information from different perspectives.

[0115] In some examples, as shown in FIG4 , outputting the bed-falling prompt information includes: presenting the bed-falling prompt information in the form of a text description and / or an icon, for example, displaying the bed-falling prompt information in the form of a text description or an icon on the display of the interactive device, such as “The patient may fall out of bed” in FIG4 .

[0116] In some embodiments, the bed-falling reminder information can also be displayed in a preset display mode, for example, the event information is a text description of the target event. The preset display mode can be different from the display mode of the displayed medical parameter information, such as a different background color (a red background can be used for a patient who may fall out of bed as shown in FIG4 ), a different font, or a flashing display, or can be accompanied by an output sound and / or light prompt. In this way, medical staff can be reminded in a timely manner that the target patient is in a bed-falling event so that they can provide timely assistance or treatment.

[0117] When the real-time interface of the selected target object is displayed on the details interface, the processor is further used to: when an operation instruction for a fall-out prompt information described in text or the like (e.g., a patient may fall out of bed as shown in FIG4 ) is obtained, relevant prompt information of the posture of the selected target object when the fall-out event occurs is obtained, and relevant prompt information of the posture of the selected target object when the fall-out event occurs is controlled by the display device, such as a posture key point schematic diagram as shown in FIG4 , for example, a posture key point schematic diagram when the patient contacts the ground after falling out of bed, a posture key point schematic diagram when falling out of bed, or a video screenshot when falling out of bed, a video within a preset time before and after falling out of bed, or other suitable prompt information to enable medical personnel to know the posture of the target object when falling out of bed. Wherein, the relevant prompt information of the posture of the selected target object when the fall-out event occurs can be displayed in an information display window provided by the display device 102, and the information display window can be displayed in the form of a floating window on the current interface of the details interface, or can also be displayed in the real-time interface of the target object, or displayed near the display area corresponding to the target object, etc.

[0118] In certain embodiments, the method of the present application also includes: in response to an operation instruction for a fall-out prompt message (such as a text prompt message as shown in Figure 4, or an icon prompt message, etc.), synchronously displaying a video within a preset time before and after the fall-out and a schematic diagram of key points of posture when falling out, or, a video within a preset time before and after the fall-out, for example, the preset time can be a time of any length, such as 10s, 20s, etc., wherein the schematic diagram of key points of posture when falling out can be displayed in superposition with the video, or, both can be displayed in different display windows, or can also be displayed in different areas of the same display window, or, can also be that the schematic diagram of key points of posture is synchronously displayed in a partial area of ​​the video. By displaying these information, so that medical staff can find the point of application when falling out through the fall-out prompt message, assisting medical staff in making a quick judgment on the injury of falling out, and judging possible body damage as soon as possible.

[0119] In one example, each display area is further configured to display at least a portion of the medical parameter information and target event corresponding to a target object. The target event is the identification of video information for each target object. For example, as shown in FIG4 , when a target object corresponding to any display area is identified as being in a target event, the display area corresponding to the target object in the target event displays at least a portion of the medical parameter information and event information of the target event. For example, in FIG4 , if the patient in bed 02 is in a target event indicating a patient may fall from bed, a text description of the patient's potential fall from bed is displayed in the display area corresponding to bed 02. The event information of the target event can be displayed in a sub-area of ​​the corresponding display area without overlapping with the displayed medical parameter information. Alternatively, in some embodiments, the event information of the target event can be displayed in the form of a pop-up window. For example, the event information includes, but is not limited to, at least one of a text description of the target event and the time when the target event occurred. In some embodiments, the event information can also include video information or image information of the target event occurring, and / or video information within a preset time period before and after the target event occurs. Optionally, the pop-up window can partially cover the corresponding display area, or the pop-up window can cover a portion of the display observation interface.

[0120] In some embodiments, as shown in FIG5 , the review information also includes a list of alarm events for the selected target object. The alarm event list is used to display the alarm event information of the selected target object within a preset time. The alarm event information includes the alarm event type. Optionally, the alarm event type includes at least one of the following: target event, vital sign abnormality alarm, and device abnormality alarm. The target event is obtained by identifying the video information of each target object captured by the video capture device. The target event includes at least one of the following information: facial expression, body part movement, wherein the body part movement includes at least one of the following: the target object may fall out of bed, the target object is turning over, the target object's hands are moving irregularly, or other movements that may put the target object in danger. Through this alarm event list, medical staff can conveniently review the various times that the target object is in within the preset time.

[0121] In one example, the alarm event information also includes at least one of the following information: medical parameter information when the alarm event occurs, medical parameter information before and within a preset time before and after the alarm event, video information within a preset time before and after the alarm event, image information when the alarm event occurs, or other suitable information.

[0122] In some embodiments, the alarm event list is arranged in chronological order according to the time when the alarm events occurred, so that the user can easily find the event corresponding to the time he wants to query in a timely manner.

[0123] In one example, multiple alarm events can be marked in a differentiated manner, such as with differentiated colors, or different alarm types correspond to identifications with different display methods, and different alarm levels correspond to identifications with different display methods. For example, the alarm levels can include: critical, serious, technical and other levels according to the alarm urgency from high to low, and different alarm levels correspond to identifications with different display methods, such as identifications with different colors.

[0124] In one embodiment, as shown in FIG5 , the method of the present application further includes: in response to a user's detailed viewing instruction for the first alarm event in the alarm event list, controlling the display to display detailed information of the first alarm event; in response to a video review instruction input by the user (e.g., in response to a video review instruction input by the user via the video review button 214 ), controlling the display to display a video of the target object corresponding to the first alarm event or within a preset time before and after the first alarm event on the alarm event details interface. For example, as shown in FIG5 , when the first alarm event is a fall from bed event, a video screenshot of the fall from bed, a video within a preset time before and after the fall from bed, or other suitable prompt information is displayed to inform the medical staff of the target object's posture during the fall from bed event. Alternatively, as shown in FIG6 , when the first alarm event is a fall from bed event, controlling the display to display fall from bed prompt information corresponding to the fall from bed event on the alarm event details interface, such as a schematic diagram of the key points of the posture when the fall from bed contacts the ground, a schematic diagram of the key points of the posture when the fall from bed occurs. The time when the first alarm event occurs is a moment within the preset time. The video review button 214 can be set at any suitable position, such as the top, left, or other suitable position of the alarm event details interface. In this application, the various display interfaces and display windows can be reasonably arranged according to actual needs and are not specifically limited here.

[0125] The detailed information of the first alarm event includes but is not limited to the alarm occurrence time, alarm settings, vital sign parameter data information at the time of the alarm, waveform graph, trend graph, etc.

[0126] Optionally, the preset time includes the time when the first alarm event occurs, a first time period before the time when the first alarm event occurs, and a second time period after the time when the first alarm event occurs. The total time of the preset time can be reasonably set as needed, and the time of the first time period and the second time period can also be arbitrarily adjusted. Optionally, the preset time can include 10 seconds, 20 seconds, 30 seconds, 40 seconds, etc.

[0127] In summary, by identifying the video, characteristic information reflecting the activity of the target object is obtained, and the characteristic information includes at least: the body posture and position information of the target object. If the body posture is the first preset posture and the position information meets the first preset position condition, it is determined that the target object has a tendency to leave the bed, and the bed leaving prompt information corresponding to the risk level is output to promptly prompt the target object with the tendency to leave the bed, so that the target object can get out of bed in accordance with the bed leaving prompt information or remind the target object not to get out of bed, thereby reducing the risk of falling out of bed due to the target object's irregular getting out of bed, improving the nursing management of the target object, and at the same time, after detecting that the patient has fallen out of bed, the patient's body posture when falling out of bed will be automatically extracted and displayed on the medical equipment to assist medical staff in making a quick judgment on the injury caused by falling out of bed.

[0128] Furthermore, referring to FIG7 , the present application also provides a monitoring and prompting method 700 , which includes the following steps:

[0129] Step S710, obtaining a video of the target object;

[0130] Step S720: Recognize the video to obtain feature information reflecting the target object's activity, where the feature information includes at least the target object's body posture and position information;

[0131] The body posture is obtained by analyzing multiple posture key points that can reflect the outline of the target object. The position information includes: the area of ​​the target object outside the edge of the bed or the center of mass position of the target object;

[0132] Step S730: If the body posture is the second preset posture and the position information meets the second preset position condition, it is determined that the target object has fallen out of bed, and a fall-out prompt message is output;

[0133] The second preset position condition includes: the area of ​​the target object outside the edge of the bed is greater than a second preset threshold, or the center of mass position of the target object moves from inside the bed to outside the bed.

[0134] The accuracy of judging whether the target object has fallen out of bed by combining body posture with position information is higher. When it is determined that the target object has fallen out of bed, a fall prompt message is output so that medical staff can be informed of the fall in time and can provide help or treatment to the target object in a more timely manner.

[0135] For details of the relevant steps in this embodiment, please refer to the relevant descriptions in the previous embodiments and will not be repeated here.

[0136] In some embodiments, the second preset posture includes a falling posture or a falling posture, or other suitable postures that can represent a possible falling posture.

[0137] In some embodiments, when in the second preset posture, the height of the target object in the direction perpendicular to the bed surface is not higher than a first threshold height, and / or, when in the second preset posture, the height of the target object in the direction perpendicular to the bed surface is lower than its length in the direction parallel to the bed surface.

[0138] In some embodiments, the video is recognized to obtain characteristic information reflecting the activity of the target object, including: processing multiple frames of video images in the video to obtain change sequence data of the target object's posture key points; based on the change sequence data of the posture key points, the body posture of the target object is recognized, and the body posture includes any one of the following information: lying posture, sitting posture, posture in the process of changing from lying position to sitting position, standing posture, walking posture, falling out of bed posture or falling posture.

[0139] In some embodiments, multiple frames of video images in a video are processed to obtain change sequence data of posture key points of a target object, including: processing multiple frames of video images in a video to identify position data of posture key points of the target object corresponding to each frame of video image; recording position data of posture key points of each frame of video image in chronological order to obtain change sequence data of posture key points of the target object.

[0140] In some embodiments, identifying the body posture of the target object based on the change sequence data of the posture key points includes: inputting the change sequence data of the posture key points into a pre-trained recognition model for processing to identify the body posture of the target object.

[0141] In some embodiments, the plurality of pose keypoints includes a plurality of joint points for representing joints of the target object.

[0142] In some embodiments, the video is identified to obtain characteristic information reflecting the activity of the target object, and the method also includes: processing the video to identify a first positioning target frame corresponding to the target object and a second positioning target frame corresponding to the bed; and calculating the area of ​​the first positioning target frame outside the second positioning target frame as the area of ​​the target object outside the edge of the bed.

[0143] In some embodiments, identifying a video to obtain feature information reflecting the activity of a target object also includes: processing multiple frames of video images in the video to identify the target object in the multiple frames of video images; calculating a motion heat map corresponding to the target object in the multiple frames of video images; and calculating the center of mass position of the target object in the multiple frames of video images based on the position information of multiple pixel points in the motion heat map corresponding to the multiple frames of video images.

[0144] In some embodiments, calculating a motion heat map corresponding to a target object in a multi-frame video image includes: for a video image of a target frame in the multi-frame video image, performing the following steps: subtracting the pixel values ​​of the corresponding pixel points in the video image of the target frame from the pixel values ​​in the video image of its previous frame to obtain the absolute values ​​of the frame differences corresponding to multiple pixel points in the video image of the target frame; obtaining a frame difference image corresponding to the video image of the target frame based on the absolute values ​​of the frame differences corresponding to the multiple pixel points; filtering the frame difference image to obtain a filtered frame difference image; setting the absolute values ​​of the frame differences in the filtered frame difference image that are less than a preset frame difference threshold to 0 to obtain a motion heat map corresponding to the video image of the target frame.

[0145] In some embodiments, the center of mass position of the target object in the multi-frame video image is calculated based on the position information of multiple pixel points in the motion heat map corresponding to the multi-frame video image, including: multiplying the horizontal axis coordinate values ​​and vertical axis values ​​of the multiple pixel points in the motion heat map by their respective corresponding weights and then summing them to obtain the horizontal axis coordinate value and vertical axis coordinate value of the center of mass position of the target object, and the weights corresponding to the multiple pixel points in the motion heat map are the ratio between the absolute values ​​of the frame differences corresponding to the multiple pixel points and the sum of the absolute values ​​of the frame differences of all pixel points.

[0146] In some embodiments, the falling-out-of-bed prompt information includes at least one of the following prompt information: a schematic diagram of the key points of the posture when contacting the ground after falling out of bed, a schematic diagram of the key points of the posture when falling out of bed, a video screenshot of the falling out of bed, and a video within a preset time before and after the falling out of bed. By displaying these body postures representing the target object falling out of bed and displaying them on medical equipment, medical staff are assisted in making a quick judgment on the injury caused by falling out of bed.

[0147] In some embodiments, outputting the fall-out-from-bed prompt information includes: presenting the fall-out-from-bed prompt information in the form of text description and / or icon.

[0148] In some embodiments, the method further includes: in response to an operation instruction for the fall-out-of-bed prompt information, synchronously displaying a video within a preset time before and after the fall-out and a schematic diagram of key points of the posture during the fall-out, or a video within a preset time before and after the fall-out.

[0149] Furthermore, referring to FIG8 , the present application also provides a monitoring and prompting method 800 , which includes the following steps:

[0150] Step S810, obtaining the target object's video and risk level;

[0151] Step S820: Recognize the video to obtain characteristic information reflecting the target object's activity, the characteristic information at least including: the target object's body posture or position information;

[0152] Step S830: If the body posture is the first preset posture and the position information meets the first preset position condition, it is determined that the target subject has a tendency to leave the bed, and bed leaving prompt information corresponding to the risk level is output;

[0153] The first preset position condition includes: the target object is located in the bed but moves toward the edge of the bed, and / or a part of the body area of ​​the target object is located outside the edge of the bed.

[0154] When determining that the target subject has a tendency to leave the bed, the method of the present application outputs bed leaving prompt information corresponding to the risk level, so as to promptly prompt the target subject with the tendency to leave the bed, so that the target subject can get out of bed in a standardized manner according to the bed leaving prompt information or remind the target subject not to get out of bed, thereby reducing the risk of falling out of bed due to the target subject's improper getting out of bed and improving the nursing management of the target subject.

[0155] Some details of the relevant steps in the embodiments of the present application can also be referred to the relevant description above and will not be described one by one here.

[0156] In some embodiments, the first preset posture includes a posture in the process of changing from a lying position to a sitting position.

[0157] In some embodiments, the partial body area of ​​the target object is located outside the edge of the hospital bed means that the area of ​​the target object outside the edge of the hospital bed is greater than a first preset threshold.

[0158] For example, when the risk level is level 1, the bed exit reminder message plays the following steps that the target subject should follow when getting out of bed. Patient education on getting out of bed is a key clinical measure to prevent falls. For example, reminding patients to follow the "three steps to getting out of bed": lying down for 30 seconds, sitting up for 30 seconds, and standing for 30 seconds can effectively prevent falls. These reminders can address the problem of poor compliance among many high-risk patients, who often ignore the correct steps of the "three steps" when getting out of bed, suddenly change their position, and thus fall out of bed.

[0159] Exemplarily, when the risk level is the second risk level, the bed leaving prompt information is: outputting an alarm message, wherein the risk level of the first risk level is lower than the risk level of the second risk level.

[0160] Exemplarily, obtaining the risk level of the target object includes: obtaining the risk level of the target object from an electronic medical record system; or

[0161] The risk level of the target object is obtained by receiving input information associated with the risk level of the target object.

[0162] Furthermore, referring to FIG9 , the present application also provides a monitoring and prompting method 900 , which includes the following steps:

[0163] Step S910, obtaining the video and risk level of the target object;

[0164] Step S920: Recognize the video to obtain feature information reflecting the target object's activity;

[0165] Step S930: determine whether the characteristic information meets at least one prompt condition corresponding to the risk level. If so, provide a prompt in a prompt manner corresponding to the prompt condition met by the characteristic information.

[0166] The method of the present application can promptly prompt a target subject who has a tendency to leave the bed, so that the target subject can get out of bed in a standardized manner according to the bed leaving prompt information or remind the target subject not to get out of bed, thereby reducing the risk of falling out of bed due to the target subject's improper getting out of bed and improving the nursing management of the target subject.

[0167] Optionally, the characteristic information includes at least: the target object's body posture and the target object's position information relative to the bed, or other information that can be used to determine whether the target object has left the bed.

[0168] In some embodiments, at least one prompt condition includes: the body posture is a first preset posture and / or the position information meets the first preset position condition, and the first preset position condition includes: the target object is located in the bed but moves toward the edge of the bed, and / or, part of the target object's body area is already outside the edge of the bed. It is worth mentioning that there may also be different prompt conditions for target objects of different risk levels. For example, their first preset posture and / or first preset position conditions may be different. Usually, the prompt conditions corresponding to target objects with high risk levels are stricter than those corresponding to target objects with low risk levels. For example, it may trigger a prompt only when the target object has the first preset posture, or, part of the target object's body area is already outside the edge of the bed means that the area of ​​the target object outside the edge of the bed is greater than the first preset threshold, then the first preset threshold corresponding to the target object with high risk level may be smaller than that of the target object with low risk level.

[0169] In some embodiments, the first preset posture includes a posture in the process of changing from a lying position to a sitting position.

[0170] In some embodiments, the partial body area of ​​the target object is located outside the edge of the hospital bed means that the area of ​​the target object outside the edge of the hospital bed is greater than a first preset threshold.

[0171] In some embodiments, when the risk level is the first risk level, the prompting method is: playing the rule steps that the target object should follow when getting out of bed;

[0172] When the risk level is the second risk level, the prompting method is: outputting an alarm message, wherein the risk level of the first risk level is lower than the risk level of the second risk level.

[0173] Next, referring to FIG. 10 , a system 1000 for monitoring and prompting of the present application is described. The system 1000 includes:

[0174] Video capture device 1010, used to capture video of the target object;

[0175] The video processing device 1020 is in communication with the video capture device 1010 and is used to obtain the video of the target object captured by the video capture device and execute the monitoring and prompting method of the aforementioned embodiment;

[0176] The interactive device 1030 is in communication with the video processing device 1020 and is configured to receive various data information output by the video processing device 1020 and present the various data information.

[0177] Since the monitoring and prompting system 1000 of the present application can execute the aforementioned monitoring and prompting method, relevant details of the monitoring and prompting method can be referred to the previous description.

[0178] The video capture device 1010 is used to capture video and / or image information of the target object, and may also include audio information, etc. When there are multiple target objects, each target object may correspond to one or more video capture devices 1010. The video capture device can be, for example, any device with image capture function, such as a camera.

[0179] The video capture device can communicate with other devices in the medical system through wireless communication or wired communication to transmit the captured video information.

[0180] The interactive device 1030 can be a device for displaying or presenting information, such as a monitor or a central monitoring device, wherein the monitor includes a bedside monitor or a wearable monitor. The interactive device can also include a remote terminal device, such as a mobile phone, tablet, laptop, or wearable device.

[0181] For example, when the medical device is a monitor, the monitor can be used to collect vital sign information of the target object. Optionally, the vital sign information includes at least one of the following information: vital sign parameter information of the target object and alarm event information.

[0182] Vital sign parameter information includes, for example, data information of various vital sign parameters related to hemodynamics, and data information of some other basic parameters (i.e., basic physiological parameters), such as HR (heart rate), PR (pulse rate), SpO2 (blood oxygen saturation), body temperature, RESP (respiratory rate), BP (blood pressure), ECG (electrocardiogram), etc., and also includes trend graphs, waveform graphs or trend tables of various vital sign parameters.

[0183] The video processing device 1020 can be implemented by at least one server. The video processing server and the video acquisition device 1010 are directly or indirectly connected in communication. The video processing device 1020 can be used to obtain the real-time video captured by the video acquisition device 1010 and forward it. It can also further store the obtained video information to facilitate subsequent user review and viewing.

[0184] The video processing device 1020 can also be used to execute the monitoring and prompting method 900 of the aforementioned embodiment. For details regarding the monitoring and prompting method, please refer to the above description. Alternatively, the video processing device 1020 can perform the relevant data processing, while the interactive device 1030 can present or display the relevant information.

[0185] The video processing device 1020 can identify and process the video information through any suitable intelligent algorithm to obtain the processing results related to the video of the target object. For example, the video can be identified to obtain characteristic information reflecting the activity of the target object. The characteristic information at least includes: the body posture and position information of the target object. The body posture includes any one of the following: lying posture, sitting posture, posture in the process of changing from lying position to sitting position, standing posture, walking posture, falling out of bed posture or falling posture, etc.

[0186] Since the system of the present application can execute the aforementioned method, it has the same advantages as the aforementioned method.

[0187] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely illustrative and are not intended to limit the scope of the present application. Various changes and modifications may be made therein by those skilled in the art without departing from the scope and spirit of the present application. All such changes and modifications are intended to be included within the scope of the present application as required by the appended claims.

[0188] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0189] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is merely a logical function division. In actual implementation, other division methods may be used. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not performed.

[0190] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0191] Similarly, it should be understood that in order to streamline the present application and aid in understanding one or more of the various inventive aspects, in the description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, this approach of the present application should not be interpreted as reflecting the intention that the application claimed for protection requires more features than those explicitly recited in each claim. More precisely, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with fewer features than all the features of a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present application.

[0192] Those skilled in the art will understand that, except where mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus disclosed herein may be combined in any combination. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that provides the same, equivalent, or similar purpose.

[0193] Furthermore, those skilled in the art will appreciate that although some embodiments herein include certain features included in other embodiments but not other features, combinations of features from different embodiments are intended to be within the scope of this application and to form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.

[0194] The various component embodiments of the present application can be implemented in hardware, or in a software module running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functions of some modules according to the embodiments of the present application. The application can also be implemented as a part or all of a device program (e.g., a computer program and a computer program product) for performing the method described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0195] It should be noted that the above embodiments illustrate rather than limit the present application, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbols placed between brackets should not be construed as limiting the claims. The present application may be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

Claims

1. A monitoring and prompting method, characterized in that: The method comprises: Obtain the target object’s video and risk level; Identify the video to obtain characteristic information reflecting the activity of the target object, wherein the characteristic information includes at least: body posture and position information of the target object; The body posture is obtained based on an analysis of a plurality of posture key points that can reflect the outline of the target object, and the position information includes a positional relationship of the target object relative to the hospital bed; If the body posture is a first preset posture and the position information meets the first preset position condition, determining that the target subject has a tendency to leave the bed, and outputting bed leaving prompt information corresponding to the risk level; The first preset position condition includes: the target object is located in the bed but moves toward the edge of the bed, and / or a part of the body area of ​​the target object is located outside the edge of the bed.

2. The method according to claim 1, wherein The first preset posture includes a posture in the process of changing from a lying position to a sitting position.

3. The method according to claim 1, wherein The partial body area of ​​the target object is located outside the edge of the hospital bed means that the area of ​​the target object outside the edge of the hospital bed is greater than a first preset threshold.

4. The method according to claim 1, wherein When the risk level is the first risk level, the bed leaving prompt information is: playing the rules and steps that the target subject should follow when getting out of bed; When the risk level is the second risk level, the bed leaving prompt information is: outputting an alarm message, wherein the risk level of the first risk level is lower than the risk level of the second risk level.

5. The method according to claim 4, wherein The regular steps include: first lying down and maintaining the lying down posture for at least a first preset time, then sitting up and maintaining the sitting up posture for at least a second preset time, and then standing up and maintaining the standing posture for at least a third preset time.

6. The method according to claim 4, wherein The outputting of the alarm information includes: outputting the alarm information in a preset presentation mode, wherein the preset presentation mode includes at least one of the following: a text description of the second risk level, a sign of the second risk level, a sound alarm, and a light alarm.

7. The method according to claim 4, wherein The first risk level includes allowing bed activities or getting out of bed activities; the second risk level includes absolute bed rest.

8. The method according to claim 1, wherein Obtaining the risk level of the target object includes: Obtain the risk level of the target subject from the electronic medical record system; or The risk level of the target object is obtained by receiving input information associated with the risk level of the target object.

9. The method according to claim 1, wherein The identifying of the video to obtain characteristic information reflecting the activity of the target object includes: Processing multiple frames of video images in the video to obtain change sequence data of posture key points of the target object; According to the change sequence data of the posture key points, the body posture of the target object is identified, and the body posture includes any one of the following: lying posture, sitting posture, posture in the process of changing from lying position to sitting position, standing posture, walking posture, falling out of bed posture or falling posture.

10. The method according to claim 9, wherein Processing multiple frames of video images in the video to obtain change sequence data of posture key points of the target object includes: Processing multiple frames of video images in the video to identify position data of posture key points of the target object in each frame of video image; The position data of the posture key points in each frame of video image are recorded in chronological order to obtain the change sequence data of the posture key points of the target object.

11. The method according to claim 9, wherein Identifying the body posture of the target object according to the change sequence data of the posture key points includes: inputting the change sequence data of the posture key points into a pre-trained recognition model for processing to identify the body posture of the target object.

12. The method according to any one of claims 1 to 11, characterized in that The plurality of posture key points include a plurality of joint points for representing joints of the target object.

13. The method according to claim 1 or 3, wherein The identifying the video to obtain characteristic information reflecting the activity of the target object further includes: Processing the video to identify a first positioning target frame corresponding to the target object and a second positioning target frame corresponding to the hospital bed; The area of ​​the first positioning target frame outside the second positioning target frame is calculated as the area of ​​the target object outside the edge of the bed.

14. The method according to claim 1, wherein The location information also includes the area of ​​the target object outside the edge of the bed or the center of mass position of the target object; the method further includes: If the body posture is a second preset posture and the position information meets the second preset position condition, it is determined that the target object has fallen out of bed, and a fall-out prompt message is output; The second preset position condition includes: the area of ​​the target object outside the edge of the bed is greater than a second preset threshold, or the center of mass of the target object moves from inside the bed to outside the bed.

15. The method according to claim 14, wherein The second preset posture includes a falling-out-of-bed posture or a falling-down posture.

16. The method according to claim 14, wherein When in the second preset posture, the height of the target object in the direction perpendicular to the surface of the bed is not higher than a first threshold height, and / or, when in the second preset posture, the height of the target object in the direction perpendicular to the surface of the bed is lower than the length in the direction parallel to the surface of the bed.

17. The method according to claim 14, wherein The identifying the video to obtain characteristic information reflecting the activity of the target object further includes: Processing multiple frames of video images in the video to identify the target object in the multiple frames of video images; Calculating a motion heat map corresponding to the target object in the multiple frames of video images; The center of mass position of the target object in the multiple frames of video images is calculated according to the position information of multiple pixel points in the motion heat map corresponding to the multiple frames of video images.

18. The method according to claim 17, wherein The calculating of the motion heat map corresponding to the target object in the multiple frames of video images includes: executing the following steps for the video image of the target frame in the multiple frames of video images: Subtracting pixel values ​​of corresponding pixels in the video image of the target frame from those in the video image of the previous frame to obtain absolute values ​​of frame differences corresponding to a plurality of pixels in the video image of the target frame; Obtaining a frame difference image corresponding to the video image of the target frame based on the frame difference absolute values ​​corresponding to the multiple pixel points; filtering the frame difference image to obtain a filtered frame difference image; The absolute values ​​of the frame differences in the filtered frame difference image that are less than a preset frame difference threshold are set to 0 to obtain the motion heat map corresponding to the video image of the target frame.

19. The method according to claim 18, wherein Calculating the center of mass position of the target object in the multiple frames of video images according to the position information of multiple pixel points in the motion heat map corresponding to the multiple frames of video images includes: The horizontal axis coordinate values ​​and vertical axis coordinate values ​​of the multiple pixel points of the motion heat map are multiplied by their respective corresponding weights and then summed up to obtain the horizontal axis coordinate value and vertical axis coordinate value of the center of mass position of the target object. The weights corresponding to the multiple pixel points in the motion heat map are the ratios between the absolute values ​​of the frame differences corresponding to the multiple pixel points and the sum of the absolute values ​​of the frame differences of all pixel points.

20. The method of claim 14, wherein: The falling-out-of-bed prompt information includes at least one of the following prompt information: Schematic diagram of the key points of the posture when contacting the ground after falling out of bed, schematic diagram of the key points of the posture when falling out of bed, video screenshots of the falling out of bed, and videos within a preset time before and after the falling out of bed.

21. The method of claim 14, wherein: The outputting of the falling-out-of-bed prompt information includes: The falling-out-of-bed prompt information is presented in the form of text description and / or icon.

22. The method according to claim 14 or 21, wherein: The method further comprises: In response to the operation instruction for the falling-out-of-bed prompt information, the video within the preset time before and after the falling-out-bed and the schematic diagram of the key points of the posture when falling-out-of-bed are synchronously displayed, or the video within the preset time before and after the falling-out-bed is displayed.

23. A monitoring and prompting method, characterized in that: The method comprises: Get the target object's video; Identify the video to obtain characteristic information reflecting the activity of the target object, wherein the characteristic information includes at least: body posture and position information of the target object; The body posture is obtained based on an analysis of a plurality of posture key points that can reflect the outline of the target object, and the position information includes: the area of ​​the target object outside the edge of the bed or the center of mass position of the target object; If the body posture is a second preset posture and the position information meets the second preset position condition, it is determined that the target object has fallen out of bed, and a fall-out prompt message is output; The second preset position condition includes: the area of ​​the target object outside the edge of the bed is greater than a second preset threshold, or the center of mass of the target object moves from inside the bed to outside the bed.

24. The method according to claim 23, wherein The second preset posture includes a falling-out-of-bed posture or a falling-down posture.

25. The method of claim 23, wherein: When in the second preset posture, the height of the target object in the direction perpendicular to the surface of the bed is not higher than a first threshold height, and / or, when in the second preset posture, the height of the target object in the direction perpendicular to the surface of the bed is lower than the length in the direction parallel to the surface of the bed.

26. The method of claim 23, wherein: The identifying of the video to obtain characteristic information reflecting the activity of the target object includes: Processing multiple frames of video images in the video to obtain change sequence data of posture key points of the target object; According to the change sequence data of the posture key points, the body posture of the target object is identified, and the body posture includes any one of the following information: lying posture, sitting posture, posture in the process of changing from lying position to sitting position, standing posture, walking posture, falling out of bed posture or falling posture.

27. The method according to claim 26, wherein Processing multiple frames of video images in the video to obtain change sequence data of posture key points of the target object includes: Processing multiple frames of video images in the video to identify position data of posture key points of the target object corresponding to each frame of video image; The position data of the posture key points of each frame of video image are recorded in chronological order to obtain the change sequence data of the posture key points of the target object.

28. The method of claim 26, wherein: Identifying the body posture of the target object according to the change sequence data of the posture key points includes: inputting the change sequence data of the posture key points into a pre-trained recognition model for processing to identify the body posture of the target object.

29. The method of claim 23, wherein: The plurality of posture key points include a plurality of joint points for representing joints of the target object.

30. The method of claim 23, wherein: The identifying the video to obtain characteristic information reflecting the activity of the target object further includes: Processing the video to identify a first positioning target frame corresponding to the target object and a second positioning target frame corresponding to the hospital bed; The area of ​​the first positioning target frame outside the second positioning target frame is calculated as the area of ​​the target object outside the edge of the bed.

31. The method of claim 23, wherein: The identifying the video to obtain characteristic information reflecting the activity of the target object further includes: Processing multiple frames of video images in the video to identify the target object in the multiple frames of video images; Calculating a motion heat map corresponding to the target object in the multiple frames of video images; The center of mass position of the target object in the multiple frames of video images is calculated according to the position information of multiple pixel points in the motion heat map corresponding to the multiple frames of video images.

32. The method of claim 31, wherein The calculating of the motion heat map corresponding to the target object in the multiple frames of video images includes: executing the following steps for the video image of the target frame in the multiple frames of video images: Subtracting pixel values ​​of corresponding pixels in the video image of the target frame from those in the video image of the previous frame to obtain absolute values ​​of frame differences corresponding to a plurality of pixels in the video image of the target frame; Obtaining a frame difference image corresponding to the video image of the target frame based on the frame difference absolute values ​​corresponding to the multiple pixel points; filtering the frame difference image to obtain a filtered frame difference image; The absolute values ​​of the frame differences in the filtered frame difference image that are less than a preset frame difference threshold are set to 0 to obtain the motion heat map corresponding to the video image of the target frame.

33. The method of claim 32, wherein: Calculating the center of mass position of the target object in the multiple frames of video images according to the position information of multiple pixel points in the motion heat map corresponding to the multiple frames of video images includes: The horizontal axis coordinate values ​​and vertical axis coordinate values ​​of the multiple pixel points of the motion heat map are multiplied by their respective corresponding weights and then summed up to obtain the horizontal axis coordinate value and vertical axis coordinate value of the center of mass position of the target object. The weights corresponding to the multiple pixel points in the motion heat map are the ratios between the absolute values ​​of the frame differences corresponding to the multiple pixel points and the sum of the absolute values ​​of the frame differences of all pixel points.

34. The method of claim 23, wherein: The falling-out-of-bed prompt information includes at least one of the following prompt information: Schematic diagram of the key points of the posture when contacting the ground after falling out of bed, schematic diagram of the key points of the posture when falling out of bed, video screenshots of the falling out of bed, and videos within a preset time before and after the falling out of bed.

35. The method of claim 23, wherein: The outputting of the falling-out-of-bed prompt information includes: The falling-out-of-bed prompt information is presented in the form of text description and / or icon.

36. The method according to claim 23 or 35, wherein The method further comprises: In response to the operation instruction for the falling-out-of-bed prompt information, the video within the preset time before and after the falling-out-bed and the schematic diagram of the key points of the posture when falling-out-of-bed are synchronously displayed, or the video within the preset time before and after the falling-out-bed is displayed.

37. A monitoring and prompting method, characterized in that: The method comprises: Obtain the target object’s video and risk level; Identify the video to obtain characteristic information reflecting the activity of the target object, wherein the characteristic information at least includes: body posture or position information of the target object; If the body posture is a first preset posture and the position information meets the first preset position condition, determining that the target subject has a tendency to leave the bed, and outputting bed leaving prompt information corresponding to the risk level; The first preset position condition includes: the target object is located in the bed but moves toward the edge of the bed, and / or a part of the body area of ​​the target object is located outside the edge of the bed.

38. The method of claim 37, wherein: The first preset posture includes a posture in the process of changing from a lying position to a sitting position.

39. The method of claim 37, wherein: The partial body area of ​​the target object is located outside the edge of the hospital bed means that the area of ​​the target object outside the edge of the hospital bed is greater than a first preset threshold.

40. The method of claim 37, wherein When the risk level is the first risk level, the bed leaving prompt information is: playing the rules and steps that the target subject should follow when getting out of bed; When the risk level is the second risk level, the bed leaving prompt information is: outputting an alarm message, wherein the risk level of the first risk level is lower than the risk level of the second risk level.

41. The method of claim 37, wherein: Obtaining the risk level of the target object includes: Obtain the risk level of the target subject from the electronic medical record system; or The risk level of the target object is obtained by receiving input information associated with the risk level of the target object.

42. A monitoring and prompting method, characterized in that: The method comprises: Obtain the target object’s video and risk level; Identify the video to obtain feature information reflecting the activity of the target object; Determine whether the characteristic information meets at least one prompt condition corresponding to the risk level; if so, provide a prompt in a prompt manner corresponding to the prompt condition met by the characteristic information.

43. The method of claim 42, wherein: The characteristic information includes at least: the body posture of the target object and the position information of the target object relative to the hospital bed.

44. The method of claim 43, wherein: The at least one prompt condition includes: the body posture is a first preset posture and / or the position information meets a first preset position condition, The first preset position condition includes: the target object is located in the bed but moves toward the edge of the bed, and / or a part of the body area of ​​the target object is located outside the edge of the bed.

45. The method of claim 44, wherein: The first preset posture includes a posture in the process of changing from a lying position to a sitting position.

46. ​​The method of claim 44, wherein: The partial body area of ​​the target object is located outside the edge of the hospital bed means that the area of ​​the target object outside the edge of the hospital bed is greater than a first preset threshold.

47. The method of claim 42, wherein: When the risk level is the first risk level, the prompting method is: playing the rule steps that the target object should follow when getting out of bed; When the risk level is the second risk level, the prompting manner is: outputting an alarm message, wherein the risk level of the first risk level is lower than the risk level of the second risk level.

48. A device for monitoring and prompting, characterized in that: The device comprises: A communication interface, configured to communicate with a video acquisition device to obtain a video of a target object acquired by the video acquisition device; a memory for storing executable program instructions; at least one processor, configured to execute the program instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 47; A display is used to display various visual information.

49. A system for monitoring and prompting, characterized in that: The system comprises: Video capture equipment, used to capture video of the target object; a video processing unit, communicatively connected to the video capture device, configured to obtain a video of a target object captured by the video capture device and execute the method according to any one of claims 1 to 47; The interactive device is communicatively connected to the video processing unit and is used to receive various data information output by the video processing unit and present the various data information.

50. The system of claim 49, wherein: The interactive device includes a monitor or a central monitoring device.

Images