Rescue prompting method, device and equipment and storage medium
By analyzing surveillance images to detect the rescue behavior of a second party, and outputting prompts to address the rescue issues when pedestrians engage in dangerous behavior in public places, personal safety is improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG UNIVIEW TECH CO LTD
- Filing Date
- 2024-11-27
- Publication Date
- 2026-05-29
AI Technical Summary
In public places, pedestrians often lack timely assistance when they fall, have arguments, or engage in physical altercations, resulting in a lack of personal safety guarantees.
By analyzing the monitoring images, it can be detected whether the second object pays attention to and performs rescue actions. If not, a rescue prompt message is output to prompt the object to perform rescue actions or the monitoring image is saved.
It improves pedestrian safety and reduces personal risks through timely rescue alerts.
Smart Images
Figure CN122116456A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of video surveillance technology, and in particular to a rescue alert method, apparatus, device, and storage medium. Background Technology
[0002] In some public places, such as stations, squares, shopping malls, or senior communities, pedestrians often engage in dangerous behaviors such as falls, arguments, or physical conflicts, which makes it impossible to guarantee their safety and poses a certain risk to their personal safety.
[0003] Considering that if other pedestrians can provide timely assistance to the pedestrian when the aforementioned behavior occurs, the pedestrian's personal risk can be reduced to a certain extent, thereby improving the pedestrian's personal safety.
[0004] Therefore, how to enable other pedestrians to seek help in a timely manner is a technical problem that urgently needs to be solved in this field. Summary of the Invention
[0005] This application provides a rescue notification method, device, equipment, and storage medium to address the shortcomings of existing technologies in enabling other pedestrians to seek help in a timely manner, thereby reducing the personal risks to pedestrians to a certain extent and improving their personal safety.
[0006] This application provides a rescue notification method, including: If a first object is detected to be in need of rescue based on the collected monitoring images, the behavior state of a second object in the monitoring images is detected. The behavior state is used to characterize whether the second object has taken any rescue action against the first object. If the behavior state indicates that the second object does not take any rescue action for the first object, a first rescue prompt message is output. The first rescue prompt message is used to prompt the second object to perform a rescue action for the first object, and / or the monitoring image has been saved.
[0007] According to a rescue notification method provided in this application, detecting the behavioral state of a second object in the surveillance image includes: If it is determined that the second object has noticed the behavior to be rescued, the behavioral state of the second object in the surveillance image is detected.
[0008] According to a rescue notification method provided in this application, when it is determined that the second object has noticed the behavior to be rescued, the method detects the behavioral state of the second object in the monitoring image, including: Extract a first target feature from the second object in the surveillance image, wherein the first target feature includes at least one of facial expression features, emotion features, and action features; Based on the first target feature, determine the second object's attention result to the behavior to be rescued; If the attention result indicates that the second object has paid attention to the behavior to be rescued, the behavioral state of the second object in the monitoring image is detected.
[0009] According to a rescue notification method provided in this application, the first target feature includes facial expression features, emotional features, and action features. The step of determining the second object's attention to the rescued behavior based on the first target feature includes: The facial expression features, emotion features, and action features are input into an attention result recognition model. The attention result recognition model determines a first attention recognition result corresponding to the facial expression features, a second attention recognition result corresponding to the emotion features, and a third attention recognition result corresponding to the action features. The attention result recognition model is trained based on facial expression feature samples, emotion feature samples, and action feature samples and their respective attention recognition result labels. The facial expression feature samples, emotion feature samples, and action feature samples are collected when a first object sample in a first monitoring image sample exhibits a behavior requiring assistance, and a second object sample in the first monitoring image sample notices the behavior requiring assistance in the first object sample. The attention result is determined based on the first attention identification result, the second attention identification result, and the third attention identification result.
[0010] According to a rescue notification method provided in this application, detecting the behavioral state of a second object in the surveillance image includes: Extract a second target feature from the monitoring image of the second object, wherein the second target feature includes at least one of motion features, posture features, and environmental interaction features; Based on the second target feature, the behavioral state of the second object in the monitoring image is detected.
[0011] According to a rescue notification method provided in this application, the second target feature includes action features, posture features, and environmental interaction features. The step of detecting the behavioral state of a second object in the surveillance image based on the second target feature includes: The action features, posture features, and environmental interaction features are input into a rescue action recognition model. The rescue action recognition model determines a first rescue action recognition result corresponding to the action features, a second rescue action recognition result corresponding to the posture features, and a third rescue action recognition result corresponding to the environmental interaction features. The rescue action recognition model is trained based on action feature samples, posture feature samples, and environmental interaction feature samples and their corresponding rescue action recognition result labels. The action feature samples, posture feature samples, and environmental interaction feature samples are collected when a first object sample in a second monitoring image sample exhibits a rescue-required behavior, and a second object sample in the second monitoring image sample performs a rescue action on the first object sample. Based on the first rescue action recognition result, the second rescue action recognition result, and the third rescue action recognition result, the behavioral state of the second object is determined.
[0012] According to a rescue notification method provided in this application, the method further includes: When the behavior state indicates that the second object has taken a rescue action against the first object, a second rescue prompt message is output, which is used to prompt the second object that the monitoring image has been saved.
[0013] This application also provides a rescue alert device, comprising: The detection unit is used to detect the behavior state of a second object in the monitoring image when a first object is detected to be in need of rescue based on the collected monitoring image. The behavior state is used to characterize whether the second object has taken any rescue action against the first object. The output unit is configured to output a first rescue prompt message when the behavior state characterizes that the second object does not perform a rescue action on the first object. The first rescue prompt message is used to prompt the second object to perform a rescue action on the first object, and / or the monitoring image has been saved.
[0014] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the rescue prompting method as described above.
[0015] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the rescue prompting method as described above.
[0016] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the rescue prompting method as described above.
[0017] The rescue prompting method, apparatus, device, and storage medium provided in this application, when detecting a rescue-prone behavior of a first object based on a collected monitoring image, detects the behavioral state of a second object in the monitoring image; if the behavioral state indicates that the second object does not take any rescue action against the first object, outputs a first rescue prompt message. The first rescue prompt message is used to prompt the second object to perform a rescue action against the first object, and / or confirms that the monitoring image has been saved. By outputting the first rescue prompt message to the second object, it is possible to prompt the second object to seek help from the first object in a timely manner, which can reduce the personal risk to the first object to a certain extent, thereby improving the personal safety of the first object. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 This is a flowchart illustrating a rescue notification method provided in an embodiment of this application.
[0020] Figure 2 This is a flowchart illustrating a process for detecting the behavioral state of a second object in a surveillance image when it is determined that the second object has noticed the behavior to be rescued, as provided in an embodiment of this application.
[0021] Figure 3 This is a schematic diagram of another process for detecting the behavioral state of a second object in a surveillance image, provided as an embodiment of this application.
[0022] Figure 4 This is a flowchart illustrating another rescue notification method provided in an embodiment of this application.
[0023] Figure 5 This is a schematic diagram of a rescue alert device provided in an embodiment of this application.
[0024] Figure 6 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0025] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0026] In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone, where A and B can be singular or plural. In the textual description of this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.
[0027] The technical solutions provided in this application can be applied to public places such as stations, squares, shopping malls, or senior citizen communities. In these scenarios, pedestrians often engage in dangerous behaviors such as falls, arguments, or physical altercations, making their safety unprotected and posing a certain risk to their personal safety.
[0028] Considering that if other pedestrians can provide timely assistance to the pedestrian when the aforementioned behavior occurs, the pedestrian's personal risk can be reduced to a certain extent, thereby improving the pedestrian's personal safety.
[0029] To address the shortcomings of existing technologies in enabling other pedestrians to promptly seek assistance, this application provides an assistance prompting method. When a first object is detected to be in need of assistance based on a captured surveillance image, and if a second object in the surveillance image does not perform any assistance action towards the first object, a first assistance prompt message can be output. This prompt message serves as a reminder to the second object to perform assistance action towards the first object, and / or the surveillance image has been saved. By outputting the first assistance prompt message to the second object, it is possible to prompt the second object to promptly seek assistance from the first object, thereby reducing the personal risk to the first object and improving their personal safety to a certain extent.
[0030] It is understood that the subject of this method can be an electronic device such as a monitoring device, computer or server, or a rescue prompting device installed in the electronic device. The rescue prompting device can be implemented by software, hardware or a combination of both, and can be set according to actual needs.
[0031] The rescue notification method provided in this application will be described in detail below through several specific embodiments. It is understood that these specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.
[0032] Figure 1 This application provides a flowchart illustrating a rescue notification method. For example, please refer to... Figure 1 As shown, the rescue notification method may include: S101. If the first object is detected to be in need of rescue based on the collected monitoring image, the behavior state of the second object in the monitoring image is detected. The behavior state is used to characterize whether the second object has taken any rescue action against the first object.
[0033] For example, the behavior requiring assistance can be a dangerous behavior such as falling, arguing, or physical conflict, and the specific behavior can be set according to actual needs.
[0034] For example, in this embodiment of the application, when detecting whether a first object has a behavior requiring rescue based on the acquired surveillance image, behavior recognition technology can be used to analyze the behavior pattern of the first object in the surveillance image, and the behavior pattern obtained from the analysis can be used to detect whether the first object has a behavior requiring rescue; or, human detection and key point analysis technology can be used to perform human detection on the first object in the surveillance image, mark the coordinate position of the human body, and accurately locate the key points of the human body, such as the top of the head, facial features, neck, limbs and other major body parts joints, and the behavior requiring rescue can be detected based on the located key points of the human body; of course, other technologies can also be used, such as abnormal sound recognition, behavior analysis and abnormal detection technology or specific behavior pattern recognition technology, to detect whether the first object has a behavior requiring rescue. The specific settings can be made according to actual needs, and this embodiment of the application does not impose specific limitations.
[0035] Based on the above description, if the detection indicates that the first object is in a state requiring rescue, the behavior of the second object in the monitoring image can be further detected to determine whether the second object has taken any rescue action against the first object. If no rescue action is taken against the first object, then the following S102 is executed: S102. If the behavior status characterization indicates that the second object does not perform any rescue action on the first object, output the first rescue prompt information. The first rescue prompt information is used to prompt the second object to perform a rescue action on the first object and / or the monitoring image has been saved.
[0036] For example, the first rescue prompt message could be: "Please take action to request help in a timely manner; relevant surveillance evidence images have been saved"; or it could be: "Please actively participate in the rescue; relevant surveillance evidence images have been saved," etc. The specific message can be set according to actual needs.
[0037] For example, the first rescue prompt can be output via audio, or via audio plus alarm, for example, the speaker can repeatedly broadcast the first rescue prompt while the alarm light flashes, etc. The specific settings can be configured according to actual needs.
[0038] For example, when the first rescue notification is used to prompt the second party to perform rescue actions on the first party, the first rescue notification can prompt the second party to seek help from the first party in a timely manner, which can reduce the personal risk to the first party to a certain extent, thereby improving the personal safety of the first party. When the first rescue notification is used to prompt the second party to perform rescue actions on the first party, and the monitoring image has been saved, not only can the first rescue notification prompt the second party to seek help from the first party in a timely manner, but the fact that the monitoring image has been saved can also indicate to the second party that the monitoring equipment is operating normally, thereby reducing the second party's concerns about rescue liability. This allows the second party to provide timely rescue to the first party without bearing any liability risk, which can reduce the personal risk to the first party to a certain extent, thereby improving the personal safety of the first party.
[0039] As can be seen from the embodiments of this application, when the first object is detected to be in need of rescue based on the collected monitoring image, the behavioral state of the second object in the monitoring image is detected; if the behavioral state indicates that the second object does not take any rescue action for the first object, a first rescue prompt message is output. The first rescue prompt message is used to prompt the second object to perform a rescue action for the first object, and / or the monitoring image has been saved. In this way, by outputting the first rescue prompt message to the second object, the second object can be prompted to seek help from the first object in a timely manner, which can reduce the personal risk of the first object to a certain extent, thereby improving the personal safety of the first object.
[0040] Based on the above Figure 1In the illustrated embodiment, for example, in S101 above, when detecting the behavioral state of the second object in the monitoring image to determine whether the second object has taken any rescue action against the first object, it can be first determined whether the second object is aware of the rescue action. Only after determining that the second object is aware of the rescue action is the behavioral state of the second object in the monitoring image detected. This targeted detection of the behavioral state of the second object in the monitoring image, based on the determination that the second object is aware of the rescue action, avoids detecting the behavioral state of objects that are not aware of the rescue action. This not only effectively saves network resources but also improves the reliability of behavioral state detection.
[0041] For example, in the embodiments of this application, when it is determined that the second object has paid attention to the behavior to be rescued, the specific implementation of detecting the behavioral state of the second object in the monitoring image can be found below. Figure 2 The example shown.
[0042] Figure 2 This application provides a flowchart illustrating how to detect the behavioral state of a second object in a surveillance image when it is determined that the second object has become aware of the action requiring rescue. For example, please refer to [link to relevant documentation]. Figure 2 As shown, the method may include: S201. Extract the first target feature of the second object in the surveillance image. The first target feature includes at least one of facial expression features, emotion features, and action features.
[0043] For example, facial expression features can include happiness, surprise, hesitation, etc. Of course, it is also possible to continuously detect changes in the facial expression of the second subject in order to determine whether the second subject is paying attention to the behavior to be rescued through facial expression features.
[0044] For example, emotional characteristics can include calmness, worry, anxiety, tension, etc. Of course, the emotional changes of the second subject can also be continuously detected to determine whether the second subject is paying attention to the behavior to be rescued through emotional characteristics.
[0045] For example, action characteristics may include whether the person stops walking, whether they face or turn towards the first object, whether they move towards the first object, etc., so as to determine whether the second object pays attention to the behavior to be rescued through action characteristics.
[0046] For example, when extracting facial expression features of a second object in a surveillance image, one can use deep learning-based expression analysis learning, such as a convolutional neural network (CNN), to extract facial expression features of the second object in the surveillance image; or, one can use expression recognition technology to analyze the facial expression of the second object in the surveillance image to extract facial expression features of the second object, etc. The specific settings can be configured according to actual needs.
[0047] For example, when extracting the emotional features of a second object in a surveillance image, one can use deep learning-based sentiment analysis, such as a convolutional neural network, to extract the emotional features of the second object in the surveillance image; or, one can use emotion recognition technology to analyze the facial expressions, language, and behavior of the second object in the surveillance image to identify its emotional state and extract its emotional features. The specific settings can be configured according to actual needs.
[0048] For example, when extracting the action features of a second object in a surveillance image, one can use deep learning-based action analysis learning, such as a convolutional neural network, to extract the action features of the second object in the surveillance image; or, one can use multimodal behavior recognition technology to analyze the data of the second object in different modalities in the surveillance image, such as facial expressions, voice signals, physiological signals and action signals, to identify its action state and extract the action features of the second object. The specific settings can be configured according to actual needs.
[0049] Based on the above description, after extracting the first target feature of the second object in the surveillance image, the following S202 can be executed.
[0050] S202. Determine the attention outcome of the second object to the rescue behavior based on the characteristics of the first target.
[0051] The results of the monitoring include whether or not the behavior requiring assistance has been monitored.
[0052] Taking the first target feature as an example, which may include facial expression features, emotional features, and action features, in an embodiment of this application, when determining the attention result of the second object to the rescue behavior based on the first target feature, the facial expression features, emotional features, and action features can be input into the attention result recognition model. The attention result recognition model determines the first attention recognition result corresponding to the facial expression features, the second attention recognition result corresponding to the emotional features, and the third attention recognition result corresponding to the action features. And based on the first attention recognition result, the second attention recognition result, and the third attention recognition result, the attention result is determined.
[0053] Among them, the attention result recognition model is trained based on facial expression feature samples, emotion feature samples and action feature samples and their corresponding attention recognition result labels. The facial expression feature samples, emotion feature samples and action feature samples are collected when the first object sample in the first monitoring image sample has a behavior that needs to be rescued, and the second object sample in the first monitoring image sample pays attention to the behavior that needs to be rescued.
[0054] For example, the model for identifying the outcome can be a Bayesian network model or a deep learning model, which can be set according to actual needs.
[0055] Taking the Bayesian network model as an example of a model for identifying attention outcomes, a Bayesian network is a probabilistic graphical model that can represent the dependencies between variables and infer the attention outcome of the second object to the rescue behavior through conditional probability. The input variables of the Bayesian network model include facial expression features, emotion features, and action features, and the corresponding target output variable is: the attention outcome of the second object to the rescue behavior, including whether the rescue behavior has been noticed or not.
[0056] For example, when determining the first attention recognition result corresponding to facial expression features using a Bayesian network model, if the facial expression feature changes from a happy state to a surprised or hesitant state, then the corresponding first attention recognition result is determined to be a behavior requiring assistance. When determining the second attention recognition result corresponding to emotional features using a Bayesian network model, if the emotional feature changes from a calm state to a worried or anxious state, then the corresponding second attention recognition result is determined to be a behavior requiring assistance. When determining the third attention recognition result corresponding to action features using a Bayesian network model, if the action feature includes changing from a moving state to a standing state, or changing the direction of movement to move towards the first object, then the corresponding third attention recognition result is determined to be a behavior requiring assistance.
[0057] When training a Bayesian network model to determine the outcome of a second party's attention to rescue actions, structural learning algorithms, such as the K2 algorithm and PC algorithm, can be used to automatically learn the network structure from data samples, or the network structure can be manually designed based on domain knowledge. Suppose we manually design a simple Bayesian network structure: E->H; M->H; A->H; where (E, M, A) represent the input variables of the Bayesian network model, and H represents the target output variable. After designing the network structure, parameter learning algorithms, such as maximum likelihood estimation and Bayesian estimation, can be used to learn the conditional probabilities between variables from data samples. After learning the conditional probabilities between variables, inference can be performed based on the trained Bayesian network model to determine the outcome of the second party's attention to rescue actions.
[0058] For example, in this embodiment of the application, the data sample may include: facial expression feature samples, emotion feature samples, and action feature samples extracted from the first monitoring image sample, and a second object sample in the first monitoring image sample noting that the first object sample has a behavior requiring rescue; wherein, the facial expression feature samples, emotion feature samples, and action feature samples can be used as input variables of the Bayesian network model to be trained; the second object sample noting that the first object sample has a behavior requiring rescue can be used as the target output variable of the Bayesian network model to be trained, for training the Bayesian network model.
[0059] For example, the first attention identification result, the second attention identification result, and the third attention identification result can be represented by numerical values. If they are greater than or equal to a first preset value, it can be understood that the second object has paid attention to the behavior to be rescued; if they are less than the first preset value, it can be understood that the second object has not paid attention to the behavior to be rescued. The size of the first preset value can be set according to actual needs, and this application embodiment does not impose further limitations.
[0060] For example, in the embodiments of this application, when determining the attention result of the second object to the rescue behavior based on the first attention recognition result, the second attention recognition result, and the third attention recognition result, the attention result of the second object to the rescue behavior can be obtained by weighting the first attention recognition result, the second attention recognition result, and the third attention recognition result based on the respective weights of facial expression features, emotional features, and action features; or the maximum attention recognition result among the first attention recognition result, the second attention recognition result, and the third attention recognition result can be determined as the attention result of the second object to the rescue behavior, etc. The specific settings can be made according to actual needs, and this embodiment of the application does not impose further limitations.
[0061] If the second object's attention to the rescue behavior does not focus on the behavior to be rescued, the first target feature of the second object in the monitoring image can be extracted, and the attention result of the second object's attention to the rescue behavior can be determined based on the first target feature; conversely, if the second object's attention to the rescue behavior has focused on the behavior to be rescued, the following S203 is executed.
[0062] S203. When the result indicates that the second object has paid attention to the behavior to be rescued, detect the behavioral state of the second object in the monitoring image.
[0063] Based on the above description, when detecting the behavioral state of a second object in a surveillance image, the first target feature of the second object in the surveillance image can be extracted first, and the attention result of the second object to the rescue behavior can be determined based on the first target feature. If the attention result indicates that the second object has paid attention to the rescue behavior, then the behavioral state of the second object in the surveillance image can be detected. In this way, the behavioral state detection can be performed on the second object that has paid attention to the rescue behavior in a targeted manner, avoiding the detection of the behavioral state of objects that have not paid attention to the rescue behavior. This not only effectively saves network resources, but also improves the reliability of behavioral state detection.
[0064] Based on the above Figure 1 In the embodiment shown, when detecting the behavioral state of the second object in the monitoring image in S101 above, the specific implementation can be found in the following... Figure 3 The example shown.
[0065] Figure 3 For another flowchart illustrating the detection of the behavioral state of a second object in a surveillance image provided in this application embodiment, please refer to the example provided. Figure 3 As shown, the method may include: S301. Extract the second target features of the second object in the monitoring image. The second target features include at least one of action features, posture features, and environmental interaction features.
[0066] For example, action features may include whether the user takes out a mobile phone to make a call or send an emergency message (such as 110 or 120), whether the user makes a request for help action (such as waving or giving directions), or whether the user says keywords related to requesting help (such as "ambulance" or "police"), so as to determine whether the second object has taken any rescue action towards the first object.
[0067] For example, posture features may include whether the person bends over, extends their hand, etc., to determine whether the second person is paying attention to the behavior to be rescued.
[0068] For example, environmental interaction features may include whether the second object is looking for a first aid kit, whether it is looking for help from other pedestrians, or whether it is picking up or using surrounding objects, such as first aid kits or mobile phones, in order to determine whether the second object is taking any rescue action against the first object through environmental interaction features.
[0069] Understandably, within a time window, the action characteristics, posture characteristics, and environmental interaction characteristics of the second target can be analyzed to assess the pedestrian's reaction speed and duration, and to determine whether the second target performed a series of rescue actions on the first target after the first target's rescue behavior occurred.
[0070] For example, when extracting the pose features of a second object in a surveillance image, a pose estimation algorithm, such as a convolutional neural network or a recurrent neural network (RNN), can be used to extract the pose features of the second object in the surveillance image; alternatively, pose tracking techniques, such as Kalman filtering or particle filtering, can be used to track and smooth continuous pose data to extract the pose features of the second object in the surveillance image. The specific settings can be configured according to actual needs.
[0071] Based on the above description, after extracting the second target feature of the second object in the monitoring image, the following S302 can be executed.
[0072] S302. Based on the features of the second target, detect the behavioral state of the second object in the monitoring image.
[0073] Among them, the behavior state is used to determine whether the second object has taken any rescue action against the first object.
[0074] Taking the second target features as including action features, posture features, and environmental interaction features as an example, in an embodiment of this application, when detecting the behavioral state of a second object in a monitoring image based on the second target features, the action features, posture features, and environmental interaction features can be input into a rescue action recognition model. The rescue action recognition model determines the first rescue action recognition result corresponding to the action features, the second rescue action recognition result corresponding to the posture features, and the third rescue action recognition result corresponding to the environmental interaction features. Based on the first rescue action recognition result, the second rescue action recognition result, and the third rescue action recognition result, the behavioral state of the second object is determined.
[0075] Among them, the rescue action recognition model is trained based on action feature samples, posture feature samples and environmental interaction feature samples and their corresponding rescue action recognition result labels. The action feature samples, posture feature samples and environmental interaction feature samples are collected when the first object sample in the second monitoring image sample has a rescue behavior and the second object sample in the second monitoring image sample performs a rescue action on the first object sample.
[0076] For example, the rescue action recognition model can be a Bayesian network model or a deep learning model, which can be set according to actual needs.
[0077] Taking the rescue action recognition model as a Bayesian network model as an example, the input variables of this Bayesian network model include: action features, posture features and environmental interaction features, and the corresponding target output variable is: the behavior state of the second object, including whether the second object has a rescue action for the first object, or whether the second object does not have a rescue action for the first object.
[0078] For example, when determining the first rescue action recognition result corresponding to action features using a Bayesian network model, if the action features include moving towards the first object, taking out a mobile phone to make a call or send an emergency message (such as 110 or 120), making a request for help (such as waving or giving directions), or saying keywords related to requesting help (such as "ambulance" or "police"), then the corresponding first rescue action recognition result is determined to be that the second object has performed a rescue action on the first object. When determining the second rescue action recognition result corresponding to posture features using a Bayesian network model, if the posture features include bending over or extending a hand, then the corresponding second rescue action recognition result is determined to be that the second object has performed a rescue action on the first object. When determining the third rescue action recognition result corresponding to environmental interaction features using a Bayesian network model, if the environmental interaction features include looking for a first aid kit, looking for help from other pedestrians, or picking up or using surrounding objects, such as a first aid kit or mobile phone, then the corresponding third rescue action recognition result is determined to be that the second object has performed a rescue action on the first object.
[0079] When training the Bayesian network model used to determine the behavioral state of a second object, structural learning algorithms, such as the K2 algorithm and PC algorithm, can also be used to automatically learn the network structure from data samples, or the network structure can be manually designed based on domain knowledge. Suppose we manually design a simple Bayesian network structure: A->H; P->H; V->H; where (A, P, V) represent the input variables of the Bayesian network model, and H represents the target output variable of the Bayesian network model. After designing the network structure, parameter learning algorithms, such as maximum likelihood estimation and Bayesian estimation, can be used to learn the conditional probabilities between variables from data samples. After the conditional probabilities between variables are learned, inference can be performed based on the trained Bayesian network model to determine the behavioral state of the second object.
[0080] For example, in this embodiment of the application, the data sample may include: motion feature samples, posture feature samples, and environmental interaction feature samples extracted from the second monitoring image sample, as well as their respective corresponding rescue action recognition result labels; wherein, the motion feature samples, posture feature samples, and environmental interaction feature samples can be used as input variables of the Bayesian network model to be trained; the rescue action recognition result labels corresponding to the motion feature samples, posture feature samples, and environmental interaction feature samples can be used as target output variables of the Bayesian network model to be trained, for training the Bayesian network model.
[0081] For example, the first rescue action recognition result, the second rescue action recognition result, and the third rescue action recognition result can be represented by numerical values. If the result is greater than or equal to a second preset value, it can be understood that the second object performed a rescue action on the first object; if the result is less than the second preset value, it can be understood that the second object did not perform a rescue action on the first object. The size of the second preset value can be set according to actual needs, and this embodiment of the application does not impose further limitations.
[0082] For example, in the embodiments of this application, when determining the attention result of the second object to the rescue behavior based on the first rescue action recognition result, the second rescue action recognition result, and the third rescue action recognition result, the attention result of the second object to the rescue behavior can be determined by weighting the first rescue action recognition result, the second rescue action recognition result, and the third rescue action recognition result based on the respective weights of action features, posture features, and environmental interaction features; alternatively, the largest rescue action recognition result among the first rescue action recognition result, the second rescue action recognition result, and the third rescue action recognition result can be determined as the attention result of the second object to the rescue behavior, etc. The specific settings can be made according to actual needs, and this embodiment of the application does not impose further limitations.
[0083] Based on the above description, after detecting the behavioral state of the second object in the surveillance image, if the behavioral state indicates that the second object does not take any rescue action against the first object, a first rescue prompt message can be further output. This first rescue prompt message is used to prompt the second object to perform a rescue action against the first object, and / or to indicate that the surveillance image has been saved. By outputting the first rescue prompt message to the second object, it is possible to prompt the second object to seek help from the first object in a timely manner, which can reduce the personal risk to the first object to a certain extent, thereby improving the personal safety of the first object.
[0084] Based on any of the above embodiments, when the behavior status indicates that the second object has taken a rescue action for the first object, a second rescue prompt message is output. The second rescue prompt message is used to indicate to the second object that the monitoring image has been saved. In this way, by indicating that the monitoring image has been saved, the second object can request help from the first object without incurring liability risks, thereby reducing the personal risks of the first object to a certain extent and improving the personal safety of the first object.
[0085] For example, see Figure 4 As shown, Figure 4This is a flowchart illustrating another rescue notification method provided in this application embodiment. In this application embodiment, monitoring images can be acquired first, and the presence of a first object requiring rescue can be detected based on the monitoring images. If it is determined that the first object does not have any behavior requiring rescue, monitoring images can continue to be acquired. If it is determined that the first object has behavior requiring rescue, facial expression features, emotional features, and action features of a second object can be extracted, and whether the second object is aware of the behavior requiring rescue can be determined based on the extracted facial expression features, emotional features, and action features. If it is determined that the second object is not aware of the behavior requiring rescue, facial expression features, emotional features, and action features of the second object can continue to be extracted. If it is determined that the second object is aware of the behavior requiring rescue, action features, posture features, and environmental interaction features of the second object can be extracted, and whether the second object has taken any rescue action towards the first object can be determined based on the extracted action features, posture features, and environmental interaction features.
[0086] If it is determined that the second object does not take any rescue action against the first object, a first rescue prompt message can be output. The first rescue prompt message is used to prompt the second object to perform a rescue action against the first object, and / or the monitoring image has been saved. By outputting the first rescue prompt message to the second object, the second object can be prompted to seek help from the first object in a timely manner, which can reduce the personal risk to the first object to a certain extent and thus improve the personal safety of the first object.
[0087] If it is determined that the second object has taken rescue actions against the first object, a second rescue prompt message can be output. The second rescue prompt message is used to indicate to the second object that the monitoring image has been saved. In this way, by indicating that the monitoring image has been saved, the second object can request help from the first object without incurring liability risks, thereby reducing the personal risks to the first object to a certain extent and improving the personal safety of the first object.
[0088] The rescue notification device provided in this application is described below. The rescue notification device described below can be referred to in correspondence with the rescue notification method described above.
[0089] Figure 5 This is a schematic diagram of a rescue alert device provided in an embodiment of this application. For example, please refer to [link to relevant documentation]. Figure 5 As shown, the rescue alert device 50 may include: The detection unit 501 is used to detect the behavior state of a second object in the monitoring image when a first object is detected to be in need of rescue based on the collected monitoring image. The behavior state is used to characterize whether the second object has taken any rescue action against the first object. The output unit 502 is configured to output a first rescue prompt message when the behavior state characterizes that the second object does not perform a rescue action on the first object. The first rescue prompt message is used to prompt the second object to perform a rescue action on the first object, and / or the monitoring image has been saved.
[0090] For example, in this embodiment of the application, the detection unit 501 is used to detect the behavioral state of the second object in the monitoring image, including: If it is determined that the second object has noticed the behavior to be rescued, the behavioral state of the second object in the surveillance image is detected.
[0091] For example, in an embodiment of this application, the detection unit 501 is used to detect the behavioral state of the second object in the surveillance image when it is determined that the second object has paid attention to the behavior to be rescued, including: Extract a first target feature from the second object in the surveillance image, wherein the first target feature includes at least one of facial expression features, emotion features, and action features; Based on the first target feature, determine the second object's attention result to the behavior to be rescued; If the attention result indicates that the second object has paid attention to the behavior to be rescued, the behavioral state of the second object in the monitoring image is detected.
[0092] For example, in this embodiment of the application, the first target feature includes facial expression features, emotion features, and action features. The detection unit 501 is used to determine the attention result of the second object to the behavior to be rescued based on the first target feature, including: The facial expression features, emotion features, and action features are input into an attention result recognition model. The attention result recognition model determines a first attention recognition result corresponding to the facial expression features, a second attention recognition result corresponding to the emotion features, and a third attention recognition result corresponding to the action features. The attention result recognition model is trained based on facial expression feature samples, emotion feature samples, and action feature samples and their respective attention recognition result labels. The facial expression feature samples, emotion feature samples, and action feature samples are collected when a first object sample in a first monitoring image sample exhibits a behavior requiring assistance, and a second object sample in the first monitoring image sample notices the behavior requiring assistance in the first object sample. The attention result is determined based on the first attention identification result, the second attention identification result, and the third attention identification result.
[0093] For example, in this embodiment of the application, the detection unit 501 is used to detect the behavioral state of the second object in the monitoring image, including: Extract a second target feature from the monitoring image of the second object, wherein the second target feature includes at least one of motion features, posture features, and environmental interaction features; Based on the second target feature, the behavioral state of the second object in the monitoring image is detected.
[0094] For example, in this embodiment of the application, the second target feature includes action features, posture features, and environmental interaction features. The detection unit 501 is used to detect the behavioral state of the second object in the monitoring image based on the second target feature, including: The action features, posture features, and environmental interaction features are input into a rescue action recognition model. The rescue action recognition model determines a first rescue action recognition result corresponding to the action features, a second rescue action recognition result corresponding to the posture features, and a third rescue action recognition result corresponding to the environmental interaction features. The rescue action recognition model is trained based on action feature samples, posture feature samples, and environmental interaction feature samples and their corresponding rescue action recognition result labels. The action feature samples, posture feature samples, and environmental interaction feature samples are collected when a first object sample in a second monitoring image sample exhibits a rescue-required behavior, and a second object sample in the second monitoring image sample performs a rescue action on the first object sample. Based on the first rescue action recognition result, the second rescue action recognition result, and the third rescue action recognition result, the behavioral state of the second object is determined.
[0095] For example, in an embodiment of this application, the output unit 502 is further configured to: When the behavior state indicates that the second object has taken a rescue action against the first object, a second rescue prompt message is output, which is used to prompt the second object that the monitoring image has been saved.
[0096] The rescue prompting device 50 provided in this application embodiment can execute the technical solution of the rescue prompting method in any of the above embodiments. Its implementation principle and beneficial effects are similar to those of the rescue prompting method. Please refer to the implementation principle and beneficial effects of the rescue prompting method. It will not be repeated here.
[0097] Figure 6 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of this application, such as... Figure 6As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, communications interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a rescue prompt method. This method includes: when a first object is detected to be in need of rescue based on a acquired monitoring image, detecting the behavioral state of a second object in the monitoring image, the behavioral state indicating whether the second object has taken any rescue action against the first object; if the behavioral state indicates that the second object has not taken any rescue action against the first object, outputting a first rescue prompt message, the first rescue prompt message prompting the second object to perform a rescue action against the first object, and / or the monitoring image has been saved.
[0098] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0099] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the rescue prompt method provided by the above methods. The method includes: when a first object is detected to have a rescue-required behavior based on a collected monitoring image, detecting the behavior state of a second object in the monitoring image, the behavior state being used to characterize whether the second object has taken a rescue action for the first object; when the behavior state indicates that the second object has not taken a rescue action for the first object, outputting a first rescue prompt message, the first rescue prompt message being used to prompt the second object to perform a rescue action for the first object, and / or the monitoring image has been saved.
[0100] In another aspect, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program is implemented to perform the rescue prompting method provided by the above methods. The method includes: when a first object is detected to be in need of rescue based on a collected monitoring image, detecting the behavioral state of a second object in the monitoring image, the behavioral state being used to characterize whether the second object has taken a rescue action against the first object; when the behavioral state indicates that the second object has not taken a rescue action against the first object, outputting first rescue prompt information, the first rescue prompt information being used to prompt the second object to perform a rescue action against the first object, and / or the monitoring image has been saved.
[0101] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0102] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A rescue notification method, characterized in that, include: If a first object is detected to be in need of rescue based on the collected monitoring images, the behavior state of a second object in the monitoring images is detected. The behavior state is used to characterize whether the second object has taken any rescue action against the first object. If the behavior state indicates that the second object does not take any rescue action for the first object, a first rescue prompt message is output. The first rescue prompt message is used to prompt the second object to perform a rescue action for the first object and / or the monitoring image has been saved.
2. The rescue notification method according to claim 1, characterized in that, The detection of the behavioral state of the second object in the monitoring image includes: If it is determined that the second object has noticed the behavior to be rescued, the behavioral state of the second object in the surveillance image is detected.
3. The rescue notification method according to claim 2, characterized in that, The step of detecting the behavioral state of the second object in the surveillance image when it is determined that the second object has noticed the behavior to be rescued includes: Extract a first target feature from the second object in the surveillance image, wherein the first target feature includes at least one of facial expression features, emotion features, and action features; Based on the first target feature, determine the second object's attention result to the behavior to be rescued; If the attention result indicates that the second object has paid attention to the behavior to be rescued, the behavioral state of the second object in the monitoring image is detected.
4. The rescue notification method according to claim 3, characterized in that, The first target features include facial expression features, emotional features, and action features. The step of determining the second object's attention to the behavior requiring assistance based on the first target features includes: The facial expression features, emotion features, and action features are input into an attention result recognition model. The attention result recognition model determines a first attention recognition result corresponding to the facial expression features, a second attention recognition result corresponding to the emotion features, and a third attention recognition result corresponding to the action features. The attention result recognition model is trained based on facial expression feature samples, emotion feature samples, and action feature samples and their respective attention recognition result labels. The facial expression feature samples, emotion feature samples, and action feature samples are collected when a first object sample in a first monitoring image sample exhibits a behavior requiring assistance, and a second object sample in the first monitoring image sample notices the behavior requiring assistance in the first object sample. The attention result is determined based on the first attention identification result, the second attention identification result, and the third attention identification result.
5. The rescue notification method according to any one of claims 1-4, characterized in that, The detection of the behavioral state of the second object in the monitoring image includes: Extract a second target feature from the monitoring image of the second object, wherein the second target feature includes at least one of motion features, posture features, and environmental interaction features; Based on the second target feature, the behavioral state of the second object in the monitoring image is detected.
6. The rescue notification method according to claim 5, characterized in that, The second target features include action features, posture features, and environmental interaction features. The step of detecting the behavioral state of the second object in the surveillance image based on the second target features includes: The action features, posture features, and environmental interaction features are input into a rescue action recognition model. The rescue action recognition model determines a first rescue action recognition result corresponding to the action features, a second rescue action recognition result corresponding to the posture features, and a third rescue action recognition result corresponding to the environmental interaction features. The rescue action recognition model is trained based on action feature samples, posture feature samples, and environmental interaction feature samples and their corresponding rescue action recognition result labels. The action feature samples, posture feature samples, and environmental interaction feature samples are collected when a first object sample in a second monitoring image sample exhibits a rescue-required behavior, and a second object sample in the second monitoring image sample performs a rescue action on the first object sample. Based on the first rescue action recognition result, the second rescue action recognition result, and the third rescue action recognition result, the behavioral state of the second object is determined.
7. The rescue notification method according to any one of claims 1-4, characterized in that, The method further includes: When the behavior state indicates that the second object has taken a rescue action against the first object, a second rescue prompt message is output, which is used to prompt the second object that the monitoring image has been saved.
8. A rescue alert device, characterized in that, include: The detection unit is used to detect the behavior state of a second object in the monitoring image when a first object is detected to be in need of rescue based on the collected monitoring image. The behavior state is used to characterize whether the second object has taken any rescue action against the first object. The output unit is configured to output a first rescue prompt message when the behavior state characterizes that the second object does not perform a rescue action on the first object. The first rescue prompt message is used to prompt the second object to perform a rescue action on the first object, and / or the monitoring image has been saved.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the rescue prompting method as described in any one of claims 1 to 7.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the rescue prompting method as described in any one of claims 1 to 7.