Monitoring method, device, system, equipment, storage medium and program product

Abstract

The embodiment of the invention provides a monitoring method, device and system, equipment, a storage medium and a program product, and relates to the field of artificial intelligence. The method comprises the following steps: in a process of tracking an object appearing in a target area based on a monitoring image of the target area, acquiring skeleton information of a first object based on a first monitoring image with the first object; and obtaining a position vector of the head of the first object relative to the hip based on the head key point information and the hip key point information in the skeleton information. And determining whether the first object has a suspected fall event based on the position vector. And if the fall event is suspected to occur, determining whether an alarm condition is satisfied based on a second monitoring image located behind the first monitoring image. And if yes, outputting alarm information that the first object falls down. According to the method, the tumble event monitoring and response speed is increased, it is ensured that help is provided in the shortest time, and the risk of personnel injury is reduced to the maximum extent.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a monitoring method, apparatus, system, equipment, storage medium and program product. Background Art

[0002] With the rapid development of science and technology, self-service outlets have been widely used in various fields. Self-service outlets can provide users with 24-hour uninterrupted convenient services, effectively improving the user's work efficiency, reducing labor costs, and enhancing the user's overall experience.

[0003] However, self-service outlets are usually unmanned, and users often fall down in self-service outlets for various reasons. If this situation is not handled in time, the person who falls may not be rescued in time. Traditional security monitoring systems rely on manual monitoring, which is inefficient and difficult to detect abnormal situations in self-service outlets in the first place. Summary of the invention

[0004] The present application provides a monitoring method, device, system, equipment, storage medium and program product to solve the technical problem that traditional security monitoring systems rely heavily on manual monitoring, which is inefficient and difficult to detect abnormal situations in self-service outlets in the first place.

[0005] In a first aspect, the present application provides a monitoring method, the method comprising:

[0006] In a process of tracking an object appearing in a target area based on a monitoring image of the target area, acquiring skeleton information of the first object based on a first monitoring image in which the first object exists;

[0007] Based on the head key point information and the hip key point information in the skeleton information, obtaining a position vector of the head relative to the hip of the first object;

[0008] Based on the position vector, determining whether a suspected fall event occurs to the first object;

[0009] If a fall event is suspected to have occurred, determining whether an alarm condition is met based on a second monitoring image located after the first monitoring image;

[0010] If the conditions are met, an alarm message indicating that the first object has fallen is output.

[0011] In a second aspect, the present application provides a monitoring device, the monitoring device comprising:

[0012] an acquisition module, configured to acquire skeleton information of the first object based on a first surveillance image in which the first object exists, during a process of tracking an object appearing in the target area based on the surveillance image of the target area; and to acquire a position vector of the head relative to the hip of the first object based on the head key point information and the hip key point information in the skeleton information;

[0013] a processing module, configured to determine whether a suspected fall event occurs to the first object based on the position vector; and if a suspected fall event occurs, determine whether an alarm condition is met based on a second monitoring image located after the first monitoring image;

[0014] The output module is used to output the alarm information that the first object has fallen down if the alarm condition is met.

[0015] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0016] The memory stores computer-executable instructions;

[0017] The processor executes the computer-executable instructions stored in the memory to implement the method as described in any one of the first aspects.

[0018] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method as described in any one of the first aspects.

[0019] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the method described in any one of the first aspects when executed by a processor.

[0020] In a sixth aspect, the present application provides a monitoring system, which is used to execute the method as described in any one of the first aspects.

[0021] The monitoring method, device, system, equipment, storage medium and program product provided by the present application can identify and track the target object in the monitoring image, obtain the skeletal information of the target object, obtain the head key point information and the hip key point information in the skeletal information of the target object by calculation, and judge whether the target object is suspected of having a fall event based on the relative position of the head key point information and the hip key point information. After detecting a suspected fall event, it is possible to further determine whether the first object meets the alarm conditions. Therefore, it is possible to monitor and respond to the fall event in real time. Ensure that help is provided in the shortest time and minimize the risk of injured persons. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0023] Figure 1 A flow chart of a monitoring method provided in an embodiment of the present application;

[0024] Figure 2 A schematic diagram of a method for taking measures after determining that an alarm condition is met provided in an embodiment of the present application;

[0025] Figure 3 A schematic diagram of the calculation principle of a SIoU loss function;

[0026] Figure 4 A flowchart of a method for implementing a fall monitoring method provided in an embodiment of the present application;

[0027] Figure 5 A schematic diagram of the structure of a monitoring device provided in an embodiment of the present application;

[0028] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0029] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0030] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0031] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0032] In addition, this application involves conducting big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.), and using artificial intelligence technology to make automated decisions, and making technical solutions that have a significant impact on personal rights and interests based on the results of automated decisions. The application provides users with corresponding operation entrances for them to choose to agree or reject the results of automated decisions; if the user chooses to reject, the expert decision-making process will be entered.

[0033] It should be noted that the monitoring methods, devices, systems, equipment, storage media and program products provided in this application can be used in the field of artificial intelligence, and can also be used in any field other than artificial intelligence. The application fields of the monitoring methods, devices, systems, equipment, storage media and program products in this application are not limited.

[0034] With the rapid development of science and technology, self-service outlets are widely used in various fields. Self-service outlets are a kind of unattended service facilities, common forms include: bank self-service outlets, unmanned stores, etc. Through the integration of automated equipment and user-friendly interface, it provides users with 24-hour uninterrupted convenient services. Self-service outlets can effectively improve users' work efficiency, reduce labor costs, and enhance users' overall experience.

[0035] However, self-service outlets are usually unmanned, and users often fall down in self-service outlets for various reasons. If this situation is not handled in a timely manner, the person who falls may not be rescued in time, and it may also bring legal and reputation risks to the operators of self-service outlets.

[0036] Traditional security monitoring systems mostly rely on manual monitoring, which is inefficient and makes it difficult to detect abnormal situations in self-service outlets in the first place.

[0037] In view of this, the present application provides a monitoring method that can automatically analyze whether a user has fallen based on monitoring images, and can automatically output an alarm message after identifying that a user has fallen, so that the person who fell can receive timely help and minimize the risk of injury.

[0038] The method of the embodiment of the present application can be applied to any monitoring system, and can perform fall detection and alarm based on the images captured by the front-end image acquisition sensor.

[0039] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0040] Figure 1 A flow chart of a monitoring method provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the method includes:

[0041] S101. In a process of tracking an object appearing in a target area based on a monitoring image of the target area, based on a first monitoring image in which the first object exists, obtaining skeleton information of the first object.

[0042] The target area refers to the area covered by the monitoring view. The object refers to an object that can move in the target area, such as a living being or an intelligent robot. Taking the aforementioned self-service outlet as an example, the target area may be part or all of the self-service outlet.

[0043] The monitoring system can track objects that appear in the target area based on the image of the target area captured by the image acquisition sensor (such as a camera, etc.). The tracking may include, for example, detecting the object that appears through a target detection frame, and tracking the object using the target detection frame to obtain its motion trajectory in the target area. For the content of tracking based on the detection frame, please refer to the prior art.

[0044] In this embodiment, the monitoring system can obtain the skeleton information of the first object based on the first monitoring image in which the first object exists during the tracking process. The first object mentioned here may refer to any object appearing in the target area.

[0045] Exemplarily, the monitoring system can extract the key points of the body of the first object based on the first monitoring image, thereby obtaining the skeleton information of the first object based on these key points. For example, if the first object is a person, the key points of the body of the first object can be key points of the human body, such as nose, left and right eyes, left and right ears, left and right shoulders, left and right elbows, left and right wrists, left and right hips, left and right knees, left and right hip joints, left and right knees, left and right ankles, and other key points of the human body.

[0046] S102: Based on the head key point information and the hip key point information in the skeleton information, obtain a position vector of the head of the first object relative to the hip.

[0047] The head key point information and the hip key point information are the positions of the head key points and the hip key points of the first object respectively.

[0048] For example, the head key point information may be the position coordinates of the center of the head joint point, and the hip key point information may be the position coordinates of the center of the hip joint point. The head key point information and the hip key point information may be calculated by the following formulas (1) and (2). The position vector of the head relative to the hip of the first object may be calculated by the following formula (3).

[0049] (1)

[0050] (2)

[0051] (3)

[0052] in, is the position coordinate of the center of the head joint, The position coordinates of the center of the hip joint. They are the position information of nose, left eye, right eye, left ear, right ear, left hip joint and right hip joint respectively. is the position vector of the head relative to the hips.

[0053] A position vector is a form of representation that describes the position of a point in space. Through a position vector, the position of a point in space and the distance relationship between points can be expressed. Therefore, the position vector of the head relative to the hip of the first object can reflect the relative position relationship between the head and the hip, as well as the distance relationship.

[0054] S103: Determine whether a suspected fall event occurs to the first object based on the position vector.

[0055] One possible implementation method is to decompose the position vector into a vertical component and a horizontal component. The vertical component refers to the projection length of the position vector on the vertical axis, which can reflect the change in the height of the object. The horizontal component refers to the projection length of the position vector on the horizontal axis, which can reflect the change in the width of the object. Generally speaking, when an object falls, its height and / or width will change. Therefore, these two components can be used to preliminarily determine whether a suspected fall event has occurred in the first object. For how to split components from a vector, please refer to the component splitting method of the prior art, which will not be elaborated on.

[0056] For example, if the component of the position vector in the vertical direction is greater than a first preset value, and / or the component in the horizontal direction is greater than a second preset value, it is determined that the first object is suspected of having a fall event. The first preset value and the second preset value can be obtained by analyzing the fall images of multiple sample objects collected offline. The sample objects mentioned here can be sample objects with different heights and different widths.

[0057] For example, if the component of the position vector in the vertical direction is greater than a first preset value, it is determined that the head position is lower than the hip position, and / or, if the component in the horizontal direction is greater than a second preset value, it is determined that the proportion in the horizontal direction exceeds a certain range, then it is considered that the body angle is too tilted, and it is determined that the first object is suspected of having a fall event.

[0058] Another possible implementation method is that a fall detection model trained using fall images of sample objects may be pre-installed on the monitoring system. Therefore, the position vector may be input into the fall detection model to obtain the result of whether a suspected fall event occurs to the first object.

[0059] S104: If a fall event is suspected to have occurred, determine whether an alarm condition is met based on a second monitoring image located after the first monitoring image.

[0060] The second monitoring image mentioned here may be a monitoring image that comes from the same image acquisition sensor as the first monitoring image and is acquired after the first monitoring image. In this implementation, the behavior and state of the first object can be further analyzed based on the second monitoring image after the suspected fall is determined to determine whether the conditions for an alarm after a fall are met. For example, if the posture of the first object is identified based on multiple consecutive second monitoring images and all meet the posture after a fall, it is determined that the alarm condition is met.

[0061] The second monitoring image may also be an image captured by other image acquisition sensors to obtain images of the first object from different perspectives. In this implementation, the images of the first object captured from multiple perspectives can be used to accurately analyze the body posture of the first object, thereby improving the accuracy of the analysis. Exemplarily, if the posture of the first object is identified as a posture after falling based on multiple consecutive second monitoring images of at least one monitoring perspective, it is determined that the alarm condition is met.

[0062] The second monitoring image may also include both of the above-mentioned images. In addition, in some embodiments, in addition to the above-mentioned second monitoring image, images of the first object collected and reported by other objects located in the target area using their own image acquisition sensors may also be obtained. In this implementation, such images may also be used as the second monitoring image.

[0063] Through the above method, it is possible to further determine whether the first subject actually falls down, thereby ensuring the accuracy of the subsequent output alarm information.

[0064] S105: If the conditions are met, output an alarm message indicating that the first subject has fallen.

[0065] Exemplarily, the monitoring system may output the alarm information to an output device in the target area. The output device may be, for example, a voice playback device located in the target area, thereby reminding relevant personnel in the target area to rescue the first object in a timely manner. The relevant personnel mentioned here may be, for example, staff in the target area.

[0066] The monitoring system may also send an alarm message to the terminal device of the target object in the target area, prompting the first object to fall. For example, the terminal device of the target object in the target area may be automatically dialed to prompt the first object to fall through a voice call. The target object mentioned here may be, for example, a specific staff member of a self-service outlet. In this way, it is possible to ensure that the alarm message is output to the relevant personnel in a timely and effective manner, thereby improving the timeliness and accuracy of the rescue of the first object.

[0067] Optionally, the monitoring system can further identify the injury degree of the first subject based on the first monitoring image and / or the second monitoring image. For example, the injury degree of the first subject can be analyzed based on one or more information reflecting the first subject's fall, such as the posture, movement, facial expression, etc. after the first subject falls. There is no limitation on how to analyze the injury degree of the first subject based on the above information, for example, it can be obtained by image recognition, or by other video or image analysis methods.

[0068] Exemplarily, when the first subject continues to maintain the posture unchanged after falling, it indicates that the first subject's fall injury is relatively serious. Alternatively, when the body part where the first subject fell is detected based on the first monitoring image and / or the second monitoring image, the degree of the first subject's fall injury can also be determined based on the involved body part. For example, in the case of falling on the head, it is determined that the fall injury is relatively serious.

[0069] If the first subject is a person, the monitoring system can optionally further analyze the injury degree of the first subject in combination with the user's identity information. For example, the monitoring system can extract the facial image of the first subject based on the first monitoring image and / or the second monitoring image, and then use the facial image to query the identity information of the first subject from the database. The identity information can reflect the age of the first subject, so that the injury degree of the first subject can be determined based on the age, the posture and / or the part of the body where the subject fell.

[0070] If the injury degree of the first subject is higher than the preset degree, indicating that the injury is more serious, then in this implementation method, while sending an alarm message to relevant personnel in the target area in the above-mentioned manner, an alarm message can also be sent to the emergency rescue platform so that the first subject can receive timely and effective rescue.

[0071] Optionally, in this implementation, the alarm information carries the location of the first subject and the degree of injury, so that the receiver can obtain relevant information of the first subject in a timely manner based on the alarm information, thereby improving the accuracy of rescue. Optionally, the contact information of the first subject can also be queried from the database based on the facial image of the first subject in the first surveillance image and / or the second surveillance image. If the contact information of the first subject can be queried, the above alarm information can also carry the contact information, thereby further improving the accuracy of rescue.

[0072] Optionally, in some embodiments, it is also possible to identify whether there is a second object associated with the first object in the target area. For example, it is possible to determine whether there is a second object associated with the first object based on an image analysis used in the process of tracking an object appearing in the target area based on a surveillance image of the target area.

[0073] For example, the interaction between the first object and other objects in the target area in the monitoring image used in the tracking process of the first object can be used to analyze whether there is a second object associated with the first object. The interaction behavior mentioned here can be, for example, a conversation behavior and / or a physical contact behavior.

[0074] If a second object associated with the first object is identified, a facial image of the second object is obtained based on the surveillance image containing the second image, and the contact information of the second object is queried from the database based on the facial image. If the contact information of the second object can be queried, an alarm message can be sent to the terminal device of the second object based on the contact information of the second object, so that the second object can be informed of the situation of the first object in a timely manner so that they can provide immediate help.

[0075] Optionally, in some embodiments, if there is no second object associated with the first object in the target area, it is also possible to query the database whether there is contact information of a third object associated with the first object. If the contact information of the third object can be found, an alarm message can be sent to the terminal device of the third object based on the contact information of the third object, so that the third object can be informed of the situation of the first object in a timely manner. The third object can be, for example, a personal relationship of the first object, or an emergency contact object of the first object, etc.

[0076] The monitoring method provided by the present application first identifies and tracks the target object in the monitoring image. Then, the skeletal information of the target object is obtained, and the head key point information and the hip key point information in the skeletal information of the target object are obtained by calculation. Based on the relative position of the head key point information and the hip key point information, it is determined whether the target object is suspected of having a fall event. After detecting a suspected fall event, it is possible to further determine whether the first object meets the alarm conditions, thereby ensuring the accuracy of the output alarm information. The method can monitor and respond to fall events in real time. After detecting a valid fall event, a corresponding alarm message is immediately issued to ensure that help is provided in the shortest time and the risk of injured persons is minimized.

[0077] The above content describes the monitoring system automatically monitoring the fall event based on the monitoring image and outputting the alarm information. In some embodiments, after identifying that the first object is suspected of having a fall event, the monitoring system can further determine whether the alarm condition is met by interacting with the first object. This interaction method can, for example, be implemented through Robotic Process Automation (RPA), which is not limited to this.

[0078] Figure 2 A schematic diagram of a method for taking measures after determining that an alarm condition is met is provided in an embodiment of the present application, such as Figure 2 As shown, after step S104, the method may further include:

[0079] S201: Construct a voice dialogue channel for interacting with a first object.

[0080] The voice conversation channel refers to a channel established between two or more communication entities for transmitting voice signals.

[0081] Exemplarily, the target area may be provided with a voice call component (eg, microphone + speaker), and thus the monitoring system may achieve the purpose of establishing a voice conversation channel for interacting with the first object by establishing a channel with the voice call component.

[0082] Optionally, as mentioned above, the monitoring system can also search for the contact information of the first object from the database, thereby establishing a voice conversation path with the terminal device of the first object through intelligent outbound calling. The contact information mentioned here can be a telephone number or a voice conversation path of a target application installed on the terminal device of the first object. Taking the banking scenario as an example, the target application mentioned here can be, for example, a banking application.

[0083] S202: Continuously play voice data to a first object in a target area through a voice dialogue channel.

[0084] Exemplarily, after determining that the alarm condition is met, the monitoring system can continuously play voice data to the first object in the target area through the voice dialogue channel. The voice data is used to ask the first object whether it needs help. For example, the voice data can be, for example, "Dear customer, do you need help?"

[0085] S203: Determine whether feedback data based on the voice data of the first object is received within a preset time period. If not, execute S204; if yes, execute S205.

[0086] For example, the monitoring system can continuously detect whether feedback data is received from the first object while continuously playing voice data to the first object in the target area. The feedback data can be, for example, any voice data that can express the current state of the first object, such as "I need help".

[0087] For example, the monitoring system may determine whether feedback data is received by determining whether a voice signal is received, or may perform voice recognition based on the received voice signal to determine whether the received feedback data is valid feedback data.

[0088] If no feedback data based on the voice data is received from the first subject within the preset time period, it means that the first subject may be in a coma or unable to communicate, and the degree of the fall injury may be serious. In this case, S204 can be directly executed.

[0089] If feedback data based on the voice data from the first object can be received within the preset time period, it indicates that the first object is currently in a wakeful state. In this manner, whether it is necessary to output an alarm message can be further determined based on the feedback data from the first object.

[0090] S204: Output alarm information.

[0091] S205: Obtain whether the first subject agrees to alarm through the voice dialogue channel. If yes, execute S204. If no, end.

[0092] Optionally, after outputting the alarm information, the monitoring system may continue to maintain the voice dialogue channel with the first subject until the first subject is rescued. In this way, the situation of the first subject can be obtained in a timely manner, and the rescue progress of the first subject can also be monitored, so as to evaluate whether it is necessary to continue to trigger the output of the alarm information, so that relevant personnel can rescue the first subject as soon as possible.

[0093] In this implementation, after the alarm information is output, if the first object determines that no subsequent rescue is needed based on its own situation, it can also provide feedback to the monitoring system through the voice dialogue channel to trigger the cancellation of the alarm information. That is, if the feedback data of the first object requesting to stop the alarm is received, feedback information indicating the withdrawal of the alarm information is output to the relevant personnel who have sent the alarm information. In this way, the waste of rescue resources can be reduced and the accuracy of rescue can be further improved.

[0094] The method provided by the present application continuously plays voice data to the first object in the target area through a voice dialogue channel after determining that the alarm conditions are met. If no feedback data based on the voice data is received from the first object within a preset time, an alarm message is output. After receiving the feedback data based on the voice data from the first object, the first object is asked whether it agrees to the alarm. The alarm is triggered after the user agrees to the alarm, and the feedback information of the alarm information is withdrawn if the user disagrees to the alarm. This method realizes an efficient and responsive monitoring method through voice interaction and feedback. It can automatically trigger an alarm when no one responds, and can also confirm the alarm demand through direct communication with the user. It not only ensures a quick response in the event of a fall, but also avoids the waste of resources caused by false alarms, thereby improving the accuracy of monitoring and user satisfaction.

[0095] The following is an example of how the monitoring system can automatically monitor fall events based on monitoring images. In the embodiment of the present application, monitoring fall events based on monitoring images can be mainly divided into four stages, the first stage is the target detection stage, the second stage is the human joint point detection stage, the third stage is the target tracking stage, and the fourth stage is the fall detection stage. The above four stages are described in detail below.

[0096] (1) Target detection stage

[0097] Target detection refers to the use of algorithms and models to analyze and process surveillance images to find and mark the target objects of interest. The human joint detection stage and target tracking stage are carried out on this basis, and the detection accuracy has a direct impact on the accuracy of subsequent detection. Therefore, it is very important to choose the appropriate target detection method according to the specific application scenario and requirements.

[0098] In a specific embodiment, after the object enters the target area, an image acquisition sensor (such as a camera, etc.) acquires the object appearing in the target area. The target detector acquires the monitoring image to perform target detection. Optionally, the target detection can be performed on the monitoring image by using an improved YOLOv5s algorithm to obtain the first object information.

[0099] Among them, YOLOv5s is a high-precision and fast target detection algorithm that can input monitoring images, perform forward reasoning, and output the detected target category and bounding box. It also integrates features at different scales and multi-level predictions, allowing YOLOv5s to accurately detect targets of different sizes.

[0100] Optionally, an embodiment of the present application proposes applying the loss function structured intersection over union (Soft Intersection over Union, SIoU) to the YOLOv5s network model and dataset to help accelerate the convergence of the model and improve training efficiency.

[0101] Among them, the SIoU loss function is an advanced loss function for bounding box regression in target detection, which comprehensively considers the alignment of shape, distance and aspect ratio to improve the convergence speed and prediction accuracy of the model.

[0102] Exemplarily, the SIoU loss function considers the distance between the center points of the real box and the predicted box, the overlapping area, and the aspect ratio, and adds angle loss to effectively reduce the total degrees of freedom of loss, so that the predicted box can be quickly moved to the nearest axis, accelerate training convergence, and improve the frame regression effect. Among them, the real box is a bounding box used to represent the actual position and size of the target object in the target detection task. The predicted box is the bounding box of the position and size of the target object predicted by the target detection model based on the input surveillance image.

[0103] In the SIoU loss function, the accuracy of the predicted box is measured by calculating the difference between the true box and the predicted box, including angle loss, distance loss, shape loss, and IoU loss, to guide the optimization during model training.

[0104] The angle loss is calculated by the following formula (4):

[0105] (4)

[0106] Figure 3 The following is a schematic diagram of the calculation principle of the SIoU loss function. The geometric meanings of the parameters in formula (4) are combined Figure 3 The following is an explanation.

[0107] in, Represents the angle between the connecting line of the center point of the real box and the predicted box and the horizontal direction, is the angle between the connecting line of the center points of the real box and the predicted box and the vertical direction. Note that in the use of formula (4), ,Right now , the convergence process at this time is to minimize Main, otherwise minimize . is the distance between the center point of the real frame and the detection frame, Indicates is the width of the rectangular frame formed by the diagonal lines, is the length of the rectangle.

[0108] Due to the addition of angle loss, the SIoU loss function greatly reduces the number of distance-related variables. and The size of SIoU makes the model's predicted box and true box close along the relevant axis, thereby accelerating the convergence speed. Therefore, SIoU is applied to the YOLOv5s network model and dataset to help the model accelerate convergence and improve training efficiency.

[0109] (2) Human joint detection stage

[0110] After target detection, the human skeleton information can be detected using a posture estimation algorithm. Pose estimation is a method for detecting and estimating the joint position and posture information of the human body from an image or video. Among them, posture estimation is divided into bottom-up posture estimation and top-down posture estimation. Bottom-up posture estimation is a method for estimating human posture by detecting and tracking joint vectors or feature points in an image. Top-down posture estimation is to identify the human body in the image through a target detection algorithm, and then detect the corresponding joint points for each detected target person, and perform more accurate posture refinement and optimization based on the detected overall posture.

[0111] In a specific embodiment, after target detection, a top-down high-resolution network (HRNet) posture estimation algorithm may be selected to obtain human skeleton information.

[0112] Among them, the HRNet posture estimation algorithm is often used to detect the joints of the human body and form a skeleton framework, which can accurately estimate the movements and postures of the human body, and output human skeleton information including the position and confidence of each joint of the body.

[0113] The network structure of HRNet adopts a parallel connection method and consists of multiple resolution sub-networks, each of which has its own feature extraction module and posture estimation module. These sub-networks work in parallel and fuse information through feature pyramids, thereby effectively retaining high-resolution features. Among them, feature pyramids are a technology used in computer vision to fuse features of different scales to capture global and local information of images and improve the accuracy of target detection and segmentation. During the training process, HRNet jointly trains by supervising features of different resolutions, allowing the network to make full use of multi-scale feature information.

[0114] The network structure of HRNet can be divided into four stages, and each stage is divided into two parts: the basic structure and the transition structure. The basic structure is responsible for extracting and fusing multi-scale features in each stage. The transition structure is responsible for transferring and adjusting features between stages to ensure that the network can efficiently process information of different scales.

[0115] In the infrastructure of each stage, residual units are used. The infrastructure of the first stage uses an hourglass structure, while the infrastructure of other stages uses a basic block structure.

[0116] In the first stage, the high-resolution subnet contains 4 residual units, which are composed of an hourglass structure with 64 channels. The hourglass residual unit contains two 1x1 convolutional layers and one 3×3 convolutional layer, which achieves the effect of first reducing the dimension and then increasing the dimension, with the aim of reducing the computational complexity.

[0117] In the second, third and fourth stages, each branch is composed of 4 basic block structure residual units, and the basic block structure is composed of two 3x3 convolutional layers. The feature information will be fused after being combined by the basic block structure. Among them, the fusion operation is divided into upsampling and downsampling. Upsampling is the conversion from low resolution to high resolution, and downsampling is the conversion from high resolution to low resolution. Its purpose is to exchange information between different branches.

[0118] In the transition part, the first stage uses a conversion layer to generate a branch that is downsampled twice. The fourth stage uses a single fusion layer to make the information of each branch intersect with each other. The second and third stages use a superposition of a conversion layer and a fusion layer to simultaneously interact with the information of different branches and generate downsampled branches.

[0119] The reason for this design is that there is only one branch in the first stage, so only the downsampling branch is generated, and the output module connected after the fourth stage does not need to generate branches. Each stage of HRNet generates a resolution branch with half the height and width and double the dimension based on all the resolution downsampling in the previous stage. As the network depth increases, the number of parallel resolutions also increases, and the top high-resolution branch is always retained.

[0120] HRNet's high-resolution feature representation can better capture detailed information, and the multi-resolution design enables the algorithm to take into account both global and local features. This allows HRNet to maintain a high accuracy rate in difficult situations such as complex backgrounds and posture occlusions.

[0121] (3) Target tracking stage

[0122] Object tracking refers to real-time tracking of one or more target objects in a video or image sequence. After the target detector detects the target, target tracking can continuously track the target, even if the target is blocked, the illumination changes, the scale changes, or the perspective changes in the video frame, the position and movement trajectory of the target can be accurately tracked.

[0123] In a specific embodiment, after performing target detection and human joint point detection, the objects appearing in the target area can be tracked according to a multi-target tracking algorithm. Optionally, a DeepSORT (Simple Online and Realtime Tracking with a Deep Association Metric) multi-target tracking algorithm based on deep learning can be selected to track the first object's frame.

[0124] Among them, the DeepSORT algorithm combines deep learning and the traditional target tracking algorithm SORT (Simple Online and Realtime Tracking). DeepSORT can detect targets in each frame of the image based on the target detector, and use multi-feature fusion technology to represent and describe the target. The algorithm introduces a cascade matching strategy and a new trajectory confirmation process to improve the frequent ID switching problem that occurs in the SORT algorithm during the tracking process. In terms of target identity management, DeepSORT maintains a global object identification manager that assigns a unique identification to each newly detected object in multiple frames. Accurate identity tracking of targets between different frames is achieved, avoiding the problem of target identity loss in SORT when targets cross.

[0125] For example, DeepSORT confirms the newly emerged trajectories by dividing them into confirmed and unconfirmed states.

[0126] For example, the newly appeared track is in an unconfirmed state, and it takes three consecutive successful matches to convert the unconfirmed state to the confirmed state. During the matching process, the time from the last match to the current moment of each track is recorded and recorded as A. The threshold Amax is set to represent the maximum time from the last match to the current moment of the track. When the matching time A of the track exceeds the set threshold Amax, it means that the track is terminated and deleted. Otherwise, it means that the track is matched successfully.

[0127] The steps of the DeepSORT multi-target tracking algorithm can be summarized as follows:

[0128] 1) Generate the detection box at time t through the target detector and update the trajectory set.

[0129] 2) Use the Kalman filter to predict the prediction box at time t+1.

[0130] 3) The object detector generates a detection box at time t+1.

[0131] 4) Use the Hungarian algorithm to calculate the best matching method, match the detection box and prediction box at time t+1, and update the trajectory set.

[0132] Then the above operation is repeated until the video frame ends.

[0133] The detection frame refers to the position and size of the first object identified by the target detector after processing the surveillance image at time t. For example, it includes the boundary coordinates of the first object. The prediction frame refers to the prediction of the position and state of the first object at time t+1 based on the Kalman filter. This prediction is based on the previously observed target state and motion model.

[0134] The Kalman filter is a recursive algorithm that statistically models the observations and states of the system, uses the observation information of the previous frame and the measurement of the current frame to predict the state of the next frame, and considers the measurement noise and model error to obtain the optimal estimate of the system state. The Hungarian algorithm is a classic algorithm for solving the allocation problem. It continuously modifies the edge weights in the cost matrix to find the best match that minimizes the overall cost.

[0135] (4) Fall detection stage

[0136] In the fall detection algorithm, the human body tilt angle is one of the key features, which can be used to determine whether the first object is in a state of falling. In general, when a person falls, the center line of the human body backbone will form a certain angle with the horizontal line of the ground. Once the angle is less than a certain range, the human body will lose balance and fall. Therefore, the human body tilt angle can be used as one of the feature values ​​of the first object to be detected.

[0137] In a specific embodiment, the skeleton information of the first object, i.e., the position information of the human joints, is obtained through the above three stages of identification and detection. The position information of the human joints includes, for example, the nose, left and right eyes, left and right ears, left and right shoulders, left and right elbows, left and right wrists, left and right hips, left and right knees, left and right hip joints, left and right knees, left and right ankles and other key points of the human body. The vectors between the corresponding body parts can be calculated through the positions of the joints, and then the inclination angle can be calculated through the angle between the vectors. Thus, the eigenvalue of the first object is obtained. The eigenvalue of the first object is input into the fall detection algorithm to determine whether the first object is in a state of falling.

[0138] For example, the key point information of the head and the key point information of the hips can be calculated by the above formulas (1) and (2). The position vector of the head of the first object relative to the hips can be calculated by the above formula (3).

[0139] Determine the size and direction of the position vector of the head relative to the hips. If the component in the vertical direction is greater than 0, that is, the head position is lower than the hip position, or the ratio in the horizontal direction exceeds a certain range, it is considered that the body angle is too large, that is, suspected to be a fall. The human body tilt angle is used as the input feature of the fall detection algorithm. The fall detection algorithm is based on the fusion of other features of the human body tilt angle, and through comprehensive analysis, detects whether the first object has fallen.

[0140] The method provided in this application monitors fall events in the target area by selecting appropriate machine learning algorithms and deep learning models. This method significantly improves the accuracy and real-time performance of the monitoring system detection.

[0141] The following is a specific example to illustrate how to implement fall monitoring. Figure 4 A flow chart of a method for implementing a fall monitoring method provided in an embodiment of the present application is shown as follows: Figure 4 As shown, the method may include the following steps:

[0142] S401, in a process of tracking an object appearing in a target area based on a monitoring image of the target area, obtaining skeleton information of the first object based on a first monitoring image in which the first object exists

[0143] S402: Based on the head key point information and the hip key point information in the skeleton information, obtain a position vector of the head of the first object relative to the hip.

[0144] S403: Determine whether a suspected fall event occurs to the first object based on the position vector.

[0145] S404: If a fall event is suspected to have occurred, determine whether an alarm condition is met based on a second monitoring image that is located after the first monitoring image.

[0146] S405: If the conditions are met, continuously play the voice data to the first object in the target area through the voice dialogue channel.

[0147] Exemplarily, after determining that the alarm condition is met, the monitoring system can continuously play voice data to the first object in the target area through the voice dialogue channel. The voice data is used to ask the first object whether it needs help. For example, the voice data can be, for example, "Dear customer, do you need help?"

[0148] S406: Determine whether feedback data based on the voice data from the first object is received within a preset time period. If not, execute S407; if yes, execute S408.

[0149] For example, the monitoring system can continuously detect whether feedback data is received from the first object while continuously playing voice data to the first object in the target area. The feedback data can be, for example, any voice data that can express the current state of the first object, such as "I need help".

[0150] For example, the monitoring system may determine whether feedback data is received by determining whether a voice signal is received, or may perform voice recognition based on the received voice signal to determine whether the received feedback data is valid feedback data.

[0151] If no feedback data based on the voice data is received from the first subject within the preset time period, it means that the first subject may be in a coma or unable to communicate, and the degree of the fall injury may be serious. In this case, S407 can be directly executed.

[0152] S407: Output alarm information.

[0153] Exemplarily, the monitoring system may output the alarm information to an output device in the target area. The output device may be, for example, a voice playback device located in the target area, thereby reminding relevant personnel in the target area to rescue the first object in a timely manner. The relevant personnel mentioned here may be, for example, staff in the target area.

[0154] S408: Obtain whether the first party agrees to alarm through the voice dialogue channel. If yes, execute S410. If no, end.

[0155] The method provided in the embodiment of the present application first identifies and tracks the target object in the monitoring image. Then, the skeletal information of the target object is obtained, and the relative position of the head position and the hip position in the skeletal information of the target object is obtained by calculation. Based on the position information of the head position and the hip position, it is determined whether the target object has a fall event. After the fall event is detected, corresponding countermeasures can be taken immediately to minimize the risk of injured persons. The method can monitor and respond to fall events in real time. After the fall event is detected, it is ensured that help is provided in the shortest time to minimize the risk of injured persons.

[0156] The above is the method of the embodiment of the present application. The device provided by the embodiment of the present application is described below.

[0157] Figure 5 A schematic diagram of a monitoring device provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the monitoring device includes: an acquisition module 501, a processing module 502 and an output module 503.

[0158] The acquisition module 501 is used to acquire skeleton information of the first object based on the first surveillance image in which the first object exists during the process of tracking the object appearing in the target area based on the surveillance image of the target area; and acquire the position vector of the head relative to the hip of the first object based on the head key point information and the hip key point information in the skeleton information;

[0159] The processing module 502 is used to determine whether a suspected fall event occurs to the first object based on the position vector; and if a suspected fall event occurs, determine whether an alarm condition is met based on a second monitoring image located after the first monitoring image;

[0160] The output module 503 is used to output alarm information that the first object has fallen down if the alarm condition is met.

[0161] Optionally, the processing module 502 is specifically configured to determine that the first object is suspected of having a fall event if a component of the position vector in the vertical direction is greater than a first preset value and / or a component in the horizontal direction is greater than a second preset value.

[0162] Optionally, the processing module 502 is specifically used to identify the posture of the first object based on multiple consecutive second monitoring images from multiple monitoring perspectives; if the posture of the first object is identified as meeting the posture after falling based on multiple consecutive second monitoring images from at least one monitoring perspective, it is determined that the alarm condition is met.

[0163] Optionally, the processing module 502 is also used to identify the degree of fall injury of the first object based on the second monitoring image after determining that the alarm condition is met; optionally, the output module 503 is specifically used to send an alarm message to the terminal device of the target object in the target area if the degree of fall injury of the first object is higher than a preset degree, and to send an alarm message to the emergency platform, the alarm message carrying the location of the first object and the degree of fall injury.

[0164] Optionally, the acquisition module 501 is also used to obtain the contact information of the second object from the database after the processing module 502 determines that the alarm condition is met, if a second object associated with the first object is identified in the process of tracking the object appearing in the target area based on the surveillance image of the target area; optionally, the output module 503 is also used to send an alarm message to the terminal device of the second object based on the contact information of the second object.

[0165] Optionally, the output module 503 is specifically used to build a voice dialogue channel for interacting with the first object after determining that the alarm condition is met; continuously play voice data to the first object in the target area through the voice dialogue channel; if no feedback data based on the voice data is received from the first object within a preset time period, output an alarm message.

[0166] Optionally, the output module 503 is specifically configured to obtain whether the first subject agrees to the alarm through a voice dialogue channel if feedback data based on the voice data is received from the first subject within a preset time period; if the first subject agrees, output the alarm information.

[0167] Optionally, the output module 503 is further configured to, after outputting the alarm information, output feedback information indicating that the alarm information is withdrawn if feedback data is received from the first object requesting to stop the alarm.

[0168] The monitoring device provided in the embodiment of the present application can be used to execute the aforementioned monitoring device method. Its implementation principle, process and beneficial effects can be found in the aforementioned embodiment and will not be repeated here.

[0169] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 6 As shown, the electronic device 600 may include: a memory 601 and a processor 602. Optionally, the electronic device may further include a transceiver 603, wherein the memory 601 and the processor 602 communicate; illustratively, the memory 601, the processor 602 and the transceiver 603 may communicate via a communication bus 604, the memory 601 is used to store a computer program, and the processor 602 executes the computer program to implement the method of the above embodiment.

[0170] Optionally, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc. The steps in the method embodiments disclosed in the present application may be directly implemented as being executed by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.

[0171] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, a method in any of the above method embodiments is implemented.

[0172] An embodiment of the present application also provides a computer program product, including a computer program, which implements the method in any of the above method embodiments when the computer program is executed by a processor.

[0173] All or part of the steps of the above-mentioned method embodiments can be completed by hardware related to program instructions. The above-mentioned program can be stored in a readable memory. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the above-mentioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid state drive, magnetic tape, floppy disk, optical disc and any combination thereof.

[0174] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0175] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0176] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0177] The embodiment of the present application also provides a monitoring system, which is used to execute the method provided by the aforementioned method embodiment. Exemplarily, the monitoring may include a controller and an image acquisition sensor. Among them, the sensor is used to acquire the monitoring image required to implement the method of the embodiment of the present application. The controller is used to execute the aforementioned method embodiment based on the data collected by the sensor to realize fall detection. Its implementation principle and technical effects are similar to those of the aforementioned method embodiment, and will not be repeated here.

[0178] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0179] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0180] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0181] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0182] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. Unless otherwise specified, the processor may be any appropriate hardware processor, such as CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0183] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0184] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0185] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0186] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A monitoring method, characterized in that: The method comprises: In a process of tracking an object appearing in a target area based on a monitoring image of the target area, acquiring skeleton information of the first object based on a first monitoring image in which the first object exists; Based on the head key point information and the hip key point information in the skeleton information, obtaining a position vector of the head relative to the hip of the first object; Based on the position vector, determining whether a suspected fall event occurs to the first object; If a fall event is suspected to have occurred, determining whether an alarm condition is met based on a second monitoring image located after the first monitoring image; If the conditions are met, an alarm message indicating that the first object has fallen is output.

2. The method according to claim 1, characterized in that The determining, based on the position vector, whether the first object has a fall event includes: If the component of the position vector in the vertical direction is greater than a first preset value, and / or the component of the position vector in the horizontal direction is greater than a second preset value, it is determined that the first object is suspected of having a fall event.

3. The method according to claim 1 or 2, characterized in that: The determining whether the alarm condition is met based on the second monitoring image located after the first monitoring image includes: recognizing a posture of the first object based on a plurality of continuous second monitoring images from a plurality of monitoring perspectives; If it is recognized based on a plurality of consecutive second monitoring images of at least one monitoring perspective that the posture of the first object satisfies the posture after falling, it is determined that the alarm condition is met.

4. The method according to claim 3, characterized in that After determining that the alarm condition is met, the method further includes: identifying the degree of injury of the first object caused by the fall based on the second monitoring image; The outputting the alarm information that the first object has fallen down comprises: If the injury degree of the first object is higher than a preset degree, an alarm message is sent to the terminal device of the target object in the target area, and an alarm message is sent to the emergency platform, wherein the alarm message carries the location of the first object and the injury degree.

5. The method according to claim 3, characterized in that: After determining that the alarm condition is met, the method further includes: If a second object associated with the first object is identified in the process of tracking the object appearing in the target area based on the surveillance image of the target area, contact information of the second object is obtained from a database; The outputting of the alarm information indicating that the first object has fallen down further includes: Based on the contact information of the second object, the alarm information is sent to the terminal device of the second object.

6. The method according to claim 1, characterized in that The method further comprises: After determining that the alarm condition is met, constructing a voice dialogue path for interacting with the first object; continuously playing voice data to a first object in the target area through the voice dialogue path; The outputting of the alarm information comprises: If no feedback data from the first object based on the voice data is received within a preset time period, the alarm information is output.

7. The method according to claim 6, characterized in that The outputting of the alarm information comprises: If feedback data from the first object based on the voice data is received within a preset time period, obtaining whether the first object agrees to send an alarm through the voice dialogue channel; If agreed, the alarm information is output.

8. The method according to claim 6 or 7, characterized in that: After outputting the alarm information, the method further includes: If feedback data indicating that the first object requires to stop the alarm is received, feedback information indicating that the alarm information is withdrawn is output.

9. A monitoring device, characterized in that: The monitoring device comprises: an acquisition module, configured to acquire skeleton information of the first object based on a first surveillance image in which the first object exists, during a process of tracking an object appearing in the target area based on the surveillance image of the target area; and to acquire a position vector of the head relative to the hip of the first object based on the head key point information and the hip key point information in the skeleton information; a processing module, configured to determine whether a suspected fall event occurs to the first object based on the position vector; and if a suspected fall event occurs, determine whether an alarm condition is met based on a second monitoring image located after the first monitoring image; The output module is used to output the alarm information that the first object has fallen down if the alarm condition is met.

10. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 8 when executed by a processor.

12. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 8 when being executed by a processor.

13. A monitoring system, characterized in that: The monitoring system is used to execute the method according to any one of claims 1 to 8.

Images