Dangerous behavior early warning method and device, storage medium and electronic equipment

Abstract

The present disclosure provides a dangerous behavior warning method and device, a storage medium and an electronic device, and relates to the technical field of artificial intelligence. For example, a subject vector and an actual behavior vector are constructed in a Gaussian space according to a knowledge graph; the subject vector and the actual behavior vector are input into a pre-constructed model to output predicted behavior information; when the predicted behavior information includes dangerous behavior information, a warning signal is sent. The dangerous behavior warning method provided in the present disclosure can represent all predicted entity relationships in a vector form through a knowledge graph algorithm, and can predict in advance, actively detect the behavior of a care object in view of potential dangerous factors and possible dangerous behaviors, and avoid dangerous conditions in advance.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, and in particular, to a dangerous behavior early warning method and device, a storage medium, and an electronic device. BACKGROUND

[0002] With the gradual development of smart home and the maturity of various artificial intelligence technologies, smart home care plays an increasingly important role in daily life. The current mainstream home care includes two categories: one is wearable, and the current mature ones are smart watches, smart bracelets, etc., which mainly function in precise positioning, physical health monitoring, and one-key alarm; the other belongs to the home category, which changes due to demand, is mainly installed in the room, and can respond to unexpected situations under home conditions and timely warn and deliver signals.

[0003] In the prior art, the smart home care facility mostly adopts a hardware sensor alarm mode, which is a result of a posteriori method that often "knows" the dangerous situation "after the fact". It cannot actively detect the behavior of the care object, avoid dangerous conditions in advance, and detect potential dangerous factors and possible dangerous behaviors.

[0004] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0005] The present disclosure provides a dangerous behavior early warning method and device, a storage medium, and an electronic device, which at least partially overcome the problem that the hardware sensor alarm mode in the related art cannot avoid dangerous conditions in advance.

[0006] Other characteristics and advantages of the present disclosure will become apparent from the following detailed description, or will be learned by practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a dangerous behavior early warning method is provided, comprising: constructing a subject vector and an actual behavior vector in a Gaussian space according to a knowledge graph; inputting the subject vector and the actual behavior vector into a pre-constructed model to output predicted behavior information; and issuing a warning signal when the predicted behavior information includes dangerous behavior information.

[0008] In some embodiments, before the subject vector and the actual behavior vector are constructed in the Gaussian space according to the knowledge graph, the method further comprises: acquiring a target image video; and acquiring subject information and behavior information in the target image video through target detection.

[0009] In some embodiments, the constructing the subject vector and the actual behavior vector in the Gaussian space according to the knowledge graph comprises: initializing the subject information and the behavior information in the Gaussian space according to a knowledge graph algorithm, and constructing the subject vector and the actual behavior vector, wherein the initializing is setting the subject information and the behavior information to be subject to Gaussian distribution and unified in the same Gaussian space to construct corresponding vector representations.

[0010] In some embodiments, the pre-constructed model comprises: obtaining a pre-training sample set; constructing a vector representation for a sample tuple in the pre-training sample set; training the model according to the vector to determine the pre-constructed model.

[0011] In some embodiments, the constructing a vector representation for a sample tuple in the pre-training sample set comprises: extracting different meanings of the same behavior in the pre-training sample set; defining a standard normal distribution for the subject of the pre-training sample set; defining a Gaussian distribution to which the behavior appearing in the pre-training sample set is subject; defining a predicted behavior vector representing the predicted possible behavior; defining an actual behavior vector in the actual verification set representing the actual behavior for use in correcting parameters in the model training process; and constructing a training triple vector and a prediction triple vector, wherein the training triple vector comprises the subject vector, the actual behavior vector, and the behavior vector to be predicted, and the prediction triple vector comprises the subject vector, the actual behavior vector, and the predicted behavior vector.

[0012] In some embodiments, the training the model according to the vector comprises: using an optimizer to optimize the parameters, setting an accuracy threshold, training the model according to the training triple vector and the prediction triple vector; and saving the current model parameters when the model accuracy reaches the preset threshold.

[0013] In some embodiments, the method further comprises: training the model using the maximum likelihood estimation principle, performing Gibbs sampling on the CRP process for the non-parametric part to determine the probability representation of the parameters of the new component; constructing a model cost function according to the bulldozer distance; using an optimizer to optimize the objective function, and saving the optimized parameters as the initialization parameters of the later model.

[0014] According to another aspect of the present disclosure, a dangerous behavior warning device is also provided, comprising: a Gaussian space vector initialization module for constructing a subject vector and an actual behavior vector in a Gaussian space according to a knowledge graph; a behavior prediction module for inputting the subject vector and the actual behavior vector into a pre-constructed model and outputting predicted behavior information; and a warning module for issuing a warning signal when the predicted behavior information includes dangerous behavior information.

[0015] According to another aspect of the present disclosure, an electronic device is also provided, which comprises a processor, and a memory for storing executable instructions of the processor, wherein the processor is configured to execute the dangerous behavior warning method according to any one of the above aspects by executing the executable instructions.

[0016] According to another aspect of the present disclosure, a computer readable storage medium is also provided, which stores a computer program, and the computer program, when executed by a processor, implements the dangerous behavior warning method according to any one of the above aspects.

[0017] According to another aspect of the present disclosure, a computer program product is also provided, which comprises a computer program, and the computer program, when executed by a processor, implements the dangerous behavior warning method according to any one of the above aspects.

[0018] The dangerous behavior warning method provided in the embodiments of the present disclosure comprises: constructing a subject vector and an actual behavior vector in a Gaussian space according to a knowledge graph; inputting the subject vector and the actual behavior vector into a pre-constructed model to output predicted behavior information; and issuing a warning signal when the predicted behavior information comprises dangerous behavior information. The dangerous behavior warning method provided in the embodiments of the present disclosure converts all predicted entity relationships into vectors by using a knowledge graph algorithm, and can predict, avoid and detect potential dangerous factors and possible dangerous behaviors of a care object in advance.

[0019] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not intended to limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0020] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure. It is clear that the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0021] Figure 1 A schematic diagram showing the structure of a dangerous behavior warning system according to an embodiment of the present disclosure;

[0022] Figure 2 A flowchart showing a dangerous behavior warning method according to an embodiment of the present disclosure;

[0023] Figure 3 A flowchart showing a specific example of a dangerous behavior warning method according to an embodiment of the present disclosure;

[0024] Figure 4A flow chart showing a specific example of a dangerous behavior early warning method in an embodiment of the present disclosure;

[0025] Figure 5 A flow chart showing another specific example of a dangerous behavior early warning method in an embodiment of the present disclosure;

[0026] Figure 6 A structural example diagram of a dangerous behavior early warning method in an embodiment of the present disclosure is shown;

[0027] Figure 7 A schematic diagram of a dangerous behavior early warning device in an embodiment of the present disclosure is shown;

[0028] Figure 8 A structural block diagram of a computer device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0029] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the implementations set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the inventive aspects to those skilled in the art. Features described in the description, structures, or characteristics may be combined in any suitable manner in one or more implementations.

[0030] In addition, the accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and serve to explain the principles of the present disclosure. The same reference numbers in different drawings represent the same or similar elements.

[0031] The specific embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0032] Figure 1 An example application system architecture schematic diagram to which the dangerous behavior early warning method in an embodiment of the present disclosure can be applied is shown. As shown in the figure, the system architecture can include a terminal device 101, a network 102, and a server 103. Figure 1

[0033] The server 103 is configured to construct a subject vector and an actual behavior vector in a Gaussian space according to a knowledge graph; input the subject vector and the actual behavior vector into a pre-constructed model to output predicted behavior information; determine whether the predicted behavior information includes dangerous behavior information, and send the determination result to the terminal device 101 through the network 102. ​

[0034] The server 103 can also be configured to obtain a target image video captured by the terminal device 101, and obtain subject information and behavior information in the target image video through target detection.

[0035] The server 103 can also be configured to construct a subject vector and an actual behavior vector in a Gaussian space according to a knowledge graph, including initializing the subject information and the behavior information in the Gaussian space according to a knowledge graph algorithm, and constructing the subject vector and the actual behavior vector, wherein the initialization is to set the subject information and the behavior information to be subject to Gaussian distribution and unified in the same Gaussian space to construct corresponding vector representations.

[0036] The server 103 can also be configured to obtain a pre-training sample set, construct vector representations for sample tuples in the pre-training sample set, and train a model according to the vectors to determine a pre-constructed model.

[0037] The server 103 can also be configured to extract different meanings of the same behavior in the pre-training sample set, define a standard normal distribution for subjects in the pre-training sample set, define a Gaussian distribution to which behaviors in the pre-training sample set are subject, define a predicted behavior vector representing a predicted possible behavior, define an actual behavior vector in a verification set representing an actual behavior for correcting parameters in a model training process, and construct a training triple vector and a prediction triple vector, wherein the training triple vector includes the subject vector, the actual behavior vector, and a behavior vector to be predicted, and the prediction triple vector includes the subject vector, the actual behavior vector, and the predicted behavior vector.

[0038] The server 103 can also be configured to optimize parameters using an optimizer, set an accuracy threshold, train the model according to the training triple vector and the prediction triple vector, and save current model parameters when the accuracy of the model reaches a preset threshold.

[0039] The server 103 can also be configured to train the model using a maximum likelihood estimation principle, perform Gibbs sampling on a CRP process for a non-parametric part to determine a probability representation of parameters of a new component, construct a model cost function according to a bulldozer distance, optimize an objective function using an optimizer, save optimized parameters as initialization parameters of a later model.

[0040] The terminal device 101 can send a warning signal when receiving the dangerous behavior information sent by the server 103.

[0041] For example, when the terminal device (mobile phone) receives the dangerous behavior information sent by the server, a vibration or ringtone reminder is sent to achieve the effect of real-time warning.

[0042] The medium used by the network 102 to provide a communication link between the terminal device 101 and the server 103 can be a wired network or a wireless network.

[0043] Optionally, the wireless network or wired network described above uses standard communication techniques and / or protocols. The network is usually the Internet, but can also be any network, including but not limited to a Local Area Network (LAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), a mobile, wired or wireless network, a private network or any combination of virtual private networks). In some embodiments, techniques and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged through the network. In addition, all or some links can be encrypted using conventional encryption techniques such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec), etc. In other embodiments, custom and / or dedicated data communication techniques can be used instead of or in addition to the above data communication techniques.

[0044] The terminal device 101 can be various electronic devices, including but not limited to a smartphone, a tablet computer, a laptop computer, a desktop computer, a wearable device, an augmented reality device, a virtual reality device, a camera device, etc.

[0045] Optionally, the clients of the application programs installed in different terminal devices 101 are the same, or the clients of the same type of application programs based on different operating systems. Based on the difference in terminal platforms, the specific form of the client of the application program can also be different, for example, the application program client can be a mobile phone client, a PC client, etc.

[0046] The server 103 can be a server that provides various services, for example, a background management server that provides support for the device operated by the user using the terminal device 101. The background management server can analyze and process received request data, etc., and feed back the processing result to the terminal device.

[0047] Optionally, the server can be a stand-alone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, and the like, but is not limited thereto. The terminal and the server can be connected directly or indirectly through wired or wireless communication, which is not limited in the present application.

[0048] Compared with the traditional contact type care system, the system provided by the present disclosure can actively detect and successfully solve the problem that the person being cared for has a dangerous condition and cannot perform or trigger related rescue operations, thereby delaying treatment. The system of the present disclosure belongs to a non-contact type prediction alarm and can well prevent these problems.

[0049] The present disclosure proposes a system using vector representation, target recognition and behavior classification, which comprehensively considers each link in the whole system, optimizes different links based on existing market products, and uses vector representation to more accurately complete the task of computer processing of actual problems, real-time target detection to provide a better basis for later work, and behavior classification to better find the differences and potential correlations between different behaviors.

[0050] Those skilled in the art can know that, Figure 1 The number of terminal devices, networks and servers in the above system architecture is only illustrative, and any number of terminal devices, networks and servers can be provided according to actual needs. The present disclosure embodiments are not limited thereto.

[0051] Under the above system architecture, a dangerous behavior warning method is provided in the present disclosure embodiments, which can be executed by any electronic device with computing processing capability.

[0052] In some embodiments, the dangerous behavior warning method provided in the present disclosure embodiments can be executed by the terminal device of the above system architecture; in other embodiments, the dangerous behavior warning method provided in the present disclosure embodiments can be executed by the server in the above system architecture; in other embodiments, the dangerous behavior warning method provided in the present disclosure embodiments can be realized by the terminal device and the server in the above system architecture through interaction.

[0053] Figure 2 A flow chart of a dangerous behavior warning method in the present disclosure embodiments is shown in FIG. 1. Figure 2 As shown in FIG. 1, the dangerous behavior warning method provided in the present disclosure embodiments includes the following steps:

[0054] S202, constructing a subject vector and an actual behavior vector in a Gaussian space according to the knowledge graph.

[0055] It should be noted that the above knowledge graph (KG, Knowledge Graph) can be a form of knowledge representation, and knowledge is stored in triples. The triples are constructed in the form of (Head, Relation, Tail), and the interaction between entities (Entities) is modeled. The above Gaussian space can be a space without the parallel line axiom. The above vector can be a quantity with size and direction.

[0056] In one specific example, constructing a subject vector and an actual behavior vector in a Gaussian space according to a knowledge graph includes initializing subject information and behavior information in a Gaussian space according to a knowledge graph algorithm, constructing a subject vector and an actual behavior vector, wherein the initialization is to set the subject information and the behavior information to be subject to Gaussian distribution, and be unified in the same Gaussian space to construct the corresponding vector representation.

[0057] For example, the initialization of the entity and the behavior sets the entity and the behavior to be subject to Gaussian distribution, and is unified in the same Gaussian space to construct the mean and the covariance matrix of the corresponding vector representation, which are randomly selected from the uniform distribution, and are represented by the following formula (1) and formula (2):

[0058] p e ~ N(u e ,∑e);(1)

[0059] p a ~ N(u a ,∑a);(2)

[0060] Where p e represents the Gaussian distribution corresponding to the entity, u e represents the Gaussian distribution mean (set) subject to, and∑e is the variance. p a represents the Gaussian distribution corresponding to the behavior, u a represents the Gaussian distribution mean subject to, and∑a is the variance.

[0061] For each subject (individual) in the subject set, a vector representation subject to Gaussian distribution is constructed as the following formula (3):

[0062]

[0063] Where u e is the Gaussian distribution subject to the mean of a subject, is the Gaussian distribution with variance.

[0064] S204, input the subject vector and the actual behavior vector into the pre-constructed model, and output predicted behavior information.

[0065] It should be noted that the above behavior information corresponds to the subject, and indicates the predicted behavior that the subject is likely to have next.

[0066] S206, when the predicted behavior information includes dangerous behavior information, an early warning signal is sent.

[0067] It should be noted that the above early warning signal can be a signal that can notify and remind people, and can be divided into multiple levels of signals according to actual behavior.

[0068] For example, when the predicted behavior information belongs to a dangerous situation, a buzzer is set to warn and send an early warning signal to a mobile phone connected to the system, achieving the effect of real-time warning.

[0069] The dangerous behavior early warning method provided in the embodiment of the disclosure can convert all predicted entity relationship vectors through a knowledge graph algorithm, predict before the event occurs, and actively detect the behavior of the care object and avoid dangerous conditions in advance in view of potential dangerous factors and possible dangerous behaviors.

[0070] In one embodiment of the disclosure, as shown in Figure 3 The dangerous behavior early warning method provided in the embodiment of the disclosure can determine the subject information and behavior information in the target image through the following steps, and can convert the specific behavior of the person being cared for into information:

[0071] S302, acquiring a target image video;

[0072] S304, obtaining subject information and behavior information in the target image video through target detection.

[0073] In one embodiment of the disclosure, as shown in Figure 4 The dangerous behavior early warning method provided in the embodiment of the disclosure can pre-construct a model through the following steps, and can predict dangerous behavior based on a Gaussian space vector:

[0074] S402, acquiring a pre-training sample set;

[0075] S404, constructing a vector representation for a sample tuple in the pre-training sample set;

[0076] S406, training the model according to the vector to determine the pre-constructed model.

[0077] In one embodiment of the disclosure, as shown in Figure 5As shown, the dangerous behavior early warning method provided in the embodiments of the present disclosure can construct the vector representation through the following steps, and can predict the behavior of the person being cared for through the Gaussian space vector prediction:

[0078] S502, extracting different meanings of the same behavior in the pre-training sample set;

[0079] S504, defining a standard normal distribution for the subject of the pre-training sample set;

[0080] S506, defining the Gaussian distribution to which the behavior appearing in the pre-training sample set is subjected;

[0081] S508, defining a predicted behavior vector representing the predicted possible behavior;

[0082] S510, verifying the actual behavior vector in the actual verification set, representing the actual behavior, which is used to correct the parameters in the model training process;

[0083] S512, constructing a training triple vector and a prediction triple vector, wherein the training triple vector includes a subject vector, an actual behavior vector, and a behavior vector to be predicted, and the prediction triple vector includes a subject vector, an actual behavior vector, and a predicted behavior vector.

[0084] For example, for the sample tuple in the pre-training sample set, the vector representation is constructed as follows:

[0085] The Chinese restaurant process CRP is used to extract different meanings of the same behavior, as shown in the following formula (4):

[0086] π r,m ~ cRP (β); (4)

[0087] For example, the old person at home also walks forward, and the dangerous degree of this behavior is different at different time periods and different locations. The different meanings of one behavior are described by the Chinese restaurant process CRP, which is a Gaussian mixture model.

[0088] For the subject of the pre-training sample, a standard normal distribution is defined, as shown in the following formula (5):

[0089]

[0090] Here, the subject, i.e. the person, is defined as a probability distribution to describe different states of the person. For example, in this example, the probability distribution can be understood as all possible different states of the same old person at different times and different locations. For example, u e represents an old person getting up at night.

[0091] The Gaussian distribution to which the appearing behavior is subjected is defined as shown in the following formula (6):

[0092]

[0093] Similarly, a Gaussian distribution fitting estimation is also made for the behavior, and it is considered that different behaviors have different representations in different environments and semantics, where u a may be embodied as a big step taken by the old person.

[0094] A behavior embedding vector u predict is defined, representing the predicted possible behavior, referred to as u p , as shown in the following formula (7):

[0095]

[0096] Here, u p may be understood in this context as predicting the behavior that the old person may have in the next moment or the next stage through u e and u a , normal walking or falling; and u a2 may be understood in this context as the actual behavior of the old person, normal walking or falling, and then comparing the predicted behavior with the actual behavior for training.

[0097] The final correct behavior in the actual verification set is u a2 , representing the actual behavior, which is used to correct the parameters in the model training process, as shown in the following formula (8):

[0098]

[0099] The training triple is constituted as: (u e , u a , u a2 ), and the prediction triple is: (u e , u a , u predict ).

[0100] In an example, training the model according to the vectors includes: using an optimizer to optimize the parameters, setting an accuracy threshold, training the model according to the training triple vector and the prediction triple vector; and when the accuracy of the model reaches the preset threshold, saving the current model parameters.

[0101] For example, the model is trained according to existing samples, an Adam optimizer is used to optimize the parameters, an accuracy threshold of 85% is set, and when the accuracy of the model reaches the standard, the current model parameters are saved. During the training of the model, a regularization term is added to the vector to prevent overfitting, and different regularization strategies are used for the mean and covariance, respectively.

[0102] The present disclosure realizes the prediction of the behavior of the person being cared for by predicting the behavior of the person being cared for through Gaussian space vector prediction, and optimizing by continuous learning to select a more suitable vector representation to fit the actual situation, thereby realizing the prediction of the behavior of the person being cared for, predicting the occurrence in advance before the danger occurs, and greatly reducing the occurrence of the danger.

[0103] In one example, the above-mentioned dangerous behavior early warning method further comprises: training the model by using the maximum likelihood estimation principle, performing Gibbs sampling on the CRP process for the non-parametric part to determine the probability representation of the parameters of the new component; constructing a model cost function according to the bulldozer distance; optimizing the objective function by using an optimizer, saving the optimized parameters as the initialization parameters of the later model.

[0104] For example, the model is trained by using the maximum likelihood estimation principle, Gibbs sampling is performed on the CRP process for the non-parametric part to obtain the probability representation of the parameters of the new component, as shown in the following formula (9):

[0105]

[0106] The newly defined parameter component is updated iteratively in the training process.

[0107] The probability representation is taken as the current posterior probability, and a model cost function is constructed according to the Wasserstein Distance (equivalent to the bulldozer distance), as shown in the following formula (10):

[0108]

[0109] The SGD optimizer is used to optimize the objective function, and the optimized parameters are saved as the initialization parameters of the later prediction model.

[0110] Overall model accuracy calculation and measurement index:

[0111] For the overall model: the model includes two types of positive and negative samples.

[0112] Positive sample: a certain action detection is successfully completed and responds to subsequent measures.

[0113] Negative sample: unable to complete a certain action detection and respond to subsequent measures.

[0114] TP: the system retrieves the occurrence of the action and timely completes the subsequent response measures (correct identification).

[0115] In this example, it is expressed as: predicting the action and responding in time.

[0116] FP: the system retrieves the occurrence of the action, but actually does not detect it (one type of error identification).

[0117] In this case, it means that dangerous behavior actions were predicted but did not actually occur.

[0118] FN: The system did not retrieve the action, but a dangerous behavior actually occurred (second type of false recognition).

[0119] In this case, the dangerous behavior occurred but was not actually detected.

[0120] TN: The system did not detect any action, and no dangerous behavior actually occurred (correct identification).

[0121] In this case, the dangerous behavior did not occur and the situation was not actually detected.

[0122] Accuracy: The ratio of the number of correctly classified samples to the total number of samples, that is: (TP+TN) / (ALL).

[0123] Precision rate: the ratio of the number of correctly retrieved samples to the total number of retrieved samples, that is: TP / (TP+FP).

[0124] Recall rate: the ratio of the number of samples correctly retrieved to the number of samples that should have been retrieved, that is: TP / (TP+FN).

[0125] Comprehensive indicators: More often, we hope to refer to both precision and recall at the same time, but we do not want to simply calculate the accuracy rate as in the case of accuracy. In this case, a new indicator F-Score is introduced to comprehensively consider various indicators, as shown in the following formula (11):

[0126]

[0127] Among them, β is used to adjust the weight. When β = 1, the two have the same weight, referred to as F1-Score. If you think the accuracy is more important, then reduce β. If you think the recall is more important, then increase β.

[0128] The dangerous behavior warning method provided by the present invention vectorizes all predicted entity relationships through a knowledge graph algorithm. Before an unexpected situation occurs, it comprehensively considers the characteristics of the person and the behavior to issue a warning before the danger occurs. The dangerous behavior warning has a better ability to avoid emergencies, minimizes the harm to the person being cared for, fully explores the multiple attributes of different objects, and the potential meanings of different behaviors, and can make predictions before things happen so that corresponding countermeasures can be taken.

[0129] The present disclosure digitizes actual problems by mapping them into Gaussian space, facilitating accurate prediction and processing by the model. The entire system is efficient, unique, convenient, and practical. The detection behavior and the prediction behavior are defined as belonging to the same Gaussian distribution, and the subject is defined as a Gaussian distribution.

[0130] The disclosure proposes a vector representation based on Gaussian distribution, according to which the mean covariance of the distribution is further processed to better fit the complexity of the actual situation, change the original single model performance, and effectively improve the accuracy and practicality of the early warning system.

[0131] The disclosure uses Wasserstein Distance to measure the similarity between Gaussian distributions, calculates the minimum average distance moved from entities and behaviors to the final behavior, and is more suitable for distance measurement in Gaussian space.

[0132] The disclosure monitors the motion behavior of the detected person, the data initialization part first models the entity of the cared object, constructs the entity representation in the Gaussian distribution space, detects and classifies the behavior of the cared object, and constructs the spatial vector representation. Then, according to the constructed probability function subject to Gaussian distribution, set the vector representation of different objects and behaviors, perform behavior prediction, and continuously optimize the vector representation through the prediction result. Finally, according to the trained vector representation, the dangerous situation in the actual scene is predicted and timely warning is given, effectively reducing the possibility of accidents.

[0133] Figure 6 An example structure diagram of a dangerous behavior early warning method in an embodiment of the disclosure is shown, as shown in Figure 6 The Gaussian space 600 constructed in the actual scene includes subjects 611, 612, and 613, behaviors 621, 622, and 623, and all subjects and behaviors are converted into a prediction mechanism of model 633 in Gaussian space. There are different subjects on the position point, different behaviors on the position point, and behavior triplets are constructed. The Wasserstein Distance is used as the cost function, and the optimal result is found by optimizing the cost function.

[0134] The disclosure focuses on considering the characteristics of the characters and the characteristics of the behaviors before the accident occurs, and warning before the danger occurs. The dangerous behavior early warning has better avoidance ability for emergency situations, maximally reduces the harm to the cared person, fully excavates multiple attributes of different objects and potential meanings of different behaviors, and can predict before the event occurs. Take appropriate measures in advance.

[0135] The knowledge graph algorithm is applied to behavior prediction in real life as a warning device for dangerous behavior. The knowledge graph algorithm vectorizes all entity relationships in prediction, and uses as short a vector as possible to comprehensively depict the entity relationship, thereby saving the difficulty of modeling the subject and improving the calculation speed. The prediction based on the knowledge graph can make the model have a high accuracy rate through training of the training set, thereby ensuring the highest accuracy rate that can be achieved under the conditions of low complexity and high calculation speed. In the present disclosure, the detected behavior and the predicted behavior belong to the same class of Gaussian distribution, and the subject is defined as a class of Gaussian distribution. The Wasserstein Distance is used to measure the similarity between Gaussian distributions, which is more suitable for distance measurement in Gaussian space, making the modeling calculation more accurate and the result more reliable.

[0136] Based on the same inventive concept, the present disclosure also provides a dangerous behavior warning device, as described in the following embodiments. Since the principle of solving the problem of the device embodiment is similar to that of the above-mentioned method embodiment, the implementation of the device embodiment can be referred to the implementation of the above-mentioned method embodiment, and the repeated parts will not be described again.

[0137] Figure 7 A schematic diagram of a dangerous behavior warning device in the present disclosure is shown, as shown in Figure 6 The device includes a Gaussian space vector initialization module 71, a behavior prediction module 72, a warning module 73, a real-time monitoring module 74, and a model construction module 75.

[0138] The Gaussian space vector initialization module 71 is configured to construct a subject vector and an actual behavior vector in a Gaussian space according to a knowledge graph.

[0139] The behavior prediction module 72 is configured to input the subject vector and the actual behavior vector into a pre-constructed model and output predicted behavior information.

[0140] The warning module 73 is configured to issue a warning signal when the predicted behavior information includes dangerous behavior information.

[0141] In one example, the above-mentioned dangerous behavior warning device further includes a real-time monitoring module 74 configured to acquire a target image video and obtain subject information and behavior information in the target image video through target detection.

[0142] In one example, the Gaussian space vector initialization module 71 is further configured to initialize the subject information and the behavior information in a Gaussian space according to a knowledge graph algorithm, and construct a subject vector and an actual behavior vector. The initialization is to set the subject information and the behavior information to be subject to Gaussian distribution and unified in the same Gaussian space, and to construct a corresponding vector representation.

[0143] In an example, the dangerous behavior early warning device described above further comprises a model construction module 75 configured to obtain a pre-training sample set; construct a vector representation for a sample tuple in the pre-training sample set; train a model according to the vector to determine a pre-constructed model.

[0144] In an example, the model construction module 75 described above is further configured to: extract different meanings of the same behavior in the pre-training sample set; define a standard normal distribution for a subject in the pre-training sample set; define a Gaussian distribution to which a behavior in the pre-training sample set is subject to; define a predicted behavior vector representing a predicted possible behavior; define an actual behavior vector in a validation set representing an actual behavior used to correct parameters in a model training process; and construct a training triple vector and a prediction triple vector, wherein the training triple vector comprises a subject vector, an actual behavior vector, and a behavior vector to be predicted, and the prediction triple vector comprises a subject vector, an actual behavior vector, and a predicted behavior vector.

[0145] In an example, the model construction module 75 described above is further configured to: use an optimizer to optimize parameters, set an accuracy threshold, and train the model according to the training triple vector and the prediction triple vector; and save current model parameters when the accuracy of the model reaches a preset threshold.

[0146] In an example, the model construction module 75 described above is further configured to: train the model using a maximum likelihood estimation principle, perform Gibbs sampling on a non-parametric part with respect to a CRP process to determine a probability representation of parameters of a new component; construct a model cost function according to a bulldozer distance; use an optimizer to optimize an objective function, and save optimized parameters as initialization parameters of a later model.

[0147] It should be noted that the Gaussian space vector initialization module 71, the behavior prediction module 72, and the early warning module 73 correspond to S202-S206 in the method embodiment, and the modules have the same examples and application scenarios as the corresponding steps, but are not limited to the content disclosed in the above method embodiment. It should be noted that the modules as part of the device can be executed in a computer system such as a set of computer executable instructions.

[0148] Those skilled in the art can understand that each aspect of the present disclosure can be implemented as a system, a method or a program product. Therefore, each aspect of the present disclosure can be embodied as a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" herein.

[0149] The electronic device 800 according to this embodiment of the present disclosure will be described below with reference to Figure 8 Figure 8 ​The electronic device 800 shown is merely one example and should not be taken as limiting the scope of functionality or use of embodiments of the disclosure.

[0150] As shown in Figure 8 The electronic device 800 is in the form of a general computing device. Components of the electronic device 800 can include, but are not limited to, the at least one processing unit 810 described above, the at least one storage unit 820 described above, and a bus 830 connecting the different system components, including the storage unit 820 and the processing unit 810.

[0151] The storage unit stores program code that can be executed by the processing unit 810, so that the processing unit 810 performs the steps described in the above "Exemplary Methods" section according to various exemplary embodiments of the disclosure.

[0152] For example, the processing unit 810 can perform the following steps of the above method embodiments:

[0153] Constructing a subject vector and an actual behavior vector in a Gaussian space according to a knowledge graph;

[0154] Inputting the subject vector and the actual behavior vector into a pre-constructed model to output predicted behavior information;

[0155] When the predicted behavior information includes dangerous behavior information, issuing a warning signal.

[0156] For example, the processing unit 810 can perform the following steps of the above method embodiments:

[0157] Obtaining a target image video;

[0158] Obtaining subject information and behavior information in the target image video through target detection.

[0159] For example, the processing unit 810 can perform the following steps of the above method embodiments: initializing the subject information and the behavior information in a Gaussian space according to a knowledge graph algorithm, constructing a subject vector and an actual behavior vector, wherein the initialization is to set the subject information and the behavior information to be subject to Gaussian distribution, and unified in the same Gaussian space to construct corresponding vector representations.

[0160] For example, the processing unit 810 can perform the following steps of the above method embodiments:

[0161] Obtaining a pre-training sample set;

[0162] Constructing vector representations for sample tuples in the pre-training sample set;

[0163] Training a model according to the vectors to determine a pre-constructed model.

[0164] For example, the processing unit 810 can perform the following steps of the above-mentioned method embodiments:

[0165] extracting different meanings of the same behavior in the pre-training sample set;

[0166] defining a standard normal distribution for the subject of the pre-training sample set;

[0167] defining a Gaussian distribution to which the behaviors appearing in the pre-training sample set are subject to;

[0168] defining a predicted behavior vector representing the predicted possible behaviors;

[0169] actual behavior vectors in the actual verification set, representing actual behaviors, are used to correct parameters in the model training process;

[0170] constructing a training triple vector and a prediction triple vector, wherein the training triple vector includes a subject vector, an actual behavior vector and a to-be-predicted behavior vector, and the prediction triple vector includes a subject vector, an actual behavior vector and a predicted behavior vector.

[0171] For example, the processing unit 810 can perform the following steps of the above-mentioned method embodiments: using an optimizer to optimize parameters, setting an accuracy threshold, and training the model according to the training triple vector and the prediction triple vector;

[0172] When the accuracy of the model reaches a preset threshold, the current model parameters are saved.

[0173] For example, the processing unit 810 can perform the following steps of the above-mentioned method embodiments: using the maximum likelihood estimation principle to train the model, performing Gibbs sampling on the CRP process for the non-parametric part, and determining the probability representation of the parameters of the new component;

[0174] According to the distance of the bulldozer, a model cost function is constructed;

[0175] using an optimizer to optimize the objective function, and saving the optimized parameters as initialization parameters for the later model.

[0176] The storage unit 820 can include a readable medium in the form of a volatile storage unit, such as a random access memory (RAM) 8201 and / or a cache memory unit 8202, and can further include a read-only memory (ROM) 8203.

[0177] The storage unit 820 also can include a program / utility 8204 having a set of programs / modules 8205, each of which performs one or more of the processes to be executed by the processing unit(s) 810, including the implementation of the network environment as described in each or some combination of the above examples.

[0178] The bus 830 can represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration bus, a processor or local bus using any of a variety of bus architectures, and the like.

[0179] The electronic device 800 also can communicate with one or more external devices 840 such as a keyboard or pointing device, a Bluetooth device, etc.; other devices such as printers, scanners, etc.; and / or various types of networks including a local area network (LAN), a wide area network (WAN), and / or the Internet, etc. using the network adapter 860. The communication can be facilitated by way of the input / output (I / O) interface 850. In this manner, the electronic device 800 can obtain information from and / or transmit information to other devices. The electronic device 800 can also include one or more communication interfaces, such as a wireless communication interface, a Bluetooth® interface, a USB interface, and the like.

[0180] From the above description of embodiments, it is manifest that the example embodiments described herein can be implemented in software and / or in hardware, including but not limited to the requisite hardware and / or software of a computing device, such as a personal computer, server, terminal device, or network device, etc. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (which can be a CD-ROM, U disk, mobile hard disk, etc.) or network, and includes a number of instructions to enable a computing device (which can be a personal computer, server, terminal device, or network device, etc.) to perform the methods according to the embodiments of the present disclosure.

[0181] In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer program product, which includes a computer program that, when executed by a processor, implements the above-mentioned dangerous behavior warning method.

[0182] In the example embodiments of the present disclosure, a computer readable storage medium is also provided, which can be a readable signal medium or a readable storage medium. A program product is stored thereon, which can implement the method of the present disclosure. In some possible implementations, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for causing the terminal device to perform the steps described in the above "example method" section according to various example embodiments of the present disclosure when the program product is run on the terminal device.

[0183] More specific examples of the computer readable storage medium in the present disclosure can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination of the foregoing.

[0184] In the present disclosure, the computer readable storage medium can include a data signal carried in the baseband or as a part of a carrier wave propagating through the program codes, in which the readable program codes are borne. Such a propagating data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal, or any appropriate combination of the foregoing. The readable signal medium can also be any readable medium other than the readable storage medium, which can send, propagate or transmit programs for use by or in connection with an instruction execution system, apparatus or device.

[0185] Optionally, the program codes contained on the computer readable storage medium can be transmitted by any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any appropriate combination of the foregoing.

[0186] In specific implementation, the program codes for performing the operations of the present disclosure can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, C++, etc., and a conventional procedural programming language such as "C" language or similar programming languages. The program codes can be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case involving a remote computing device, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, through the Internet by connecting to an Internet service provider).

[0187] It should be noted that, although several modules or units of the devices for action execution are mentioned in the above detailed description, such division is not mandatory. Indeed, according to embodiments of the disclosure, features and functionalities of two or more modules or units described above can be embodied in one module or unit. Conversely, features and functionalities of one module or unit described above can be further divided into embodied by multiple modules or units.

[0188] Furthermore, although the various steps of the methods in the disclosure are described in a particular order in the drawings, this is not required or implied as to the particular order of execution of the steps, nor is it required that all of the steps shown be executed to achieve the desired result. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, one step can be broken into multiple steps, etc.

[0189] From the above description of the embodiments, those skilled in the art will readily perceive that the example embodiments described herein can be implemented by software and / or by software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, U disk, mobile hard disk, etc.) or network, and includes several instructions to make a computing device (which can be a personal computer, server, mobile terminal, or network device, etc.) execute the method according to the embodiments of the disclosure.

[0190] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. The disclosure is intended to cover any variations, uses, or adaptations of the disclosure following the general principles thereof and including such departures from the present disclosure that come within known use or custom in the art to which the disclosure pertains. The specification and examples are to be regarded as illustrative only, and the true scope and spirit of the disclosure are indicated by the appended claims.

Claims

1. A dangerous behavior warning method characterized by, The method comprises: acquiring a target image video; obtaining subject information and behavior information in the target image video through target detection; initializing the subject information and the behavior information in a Gaussian space according to a knowledge graph algorithm, and constructing a subject vector and an actual behavior vector; inputting the subject vector and the actual behavior vector into a pre-constructed model to output predicted behavior information; when the predicted behavior information includes dangerous behavior information, issuing a warning signal; wherein the pre-constructed model comprises: acquiring a pre-training sample set; constructing a vector representation for a sample tuple in the pre-training sample set; training a model according to the vector to determine a pre-constructed model; wherein the construction of the vector representation for the sample tuple in the pre-training sample set comprises: extracting different meanings of the same behavior in the pre-training sample set; defining a standard normal distribution for the subject of the pre-training sample set; defining a Gaussian distribution to which the behaviors appearing in the pre-training sample set are subject; defining a predicted behavior vector representing the predicted possible behaviors; an actual behavior vector in an actual verification set representing the actual behaviors used to correct parameters in the model training process; constructing a training triple vector and a prediction triple vector, wherein the training triple vector comprises a subject vector, an actual behavior vector and a behavior vector to be predicted, and the prediction triple vector comprises a subject vector, an actual behavior vector and a predicted behavior vector.

2. The dangerous behavior warning method according to claim 1, characterized by, The initialization is to set the subject information and the behavior information to be subject to Gaussian distribution and unified in the same Gaussian space to construct corresponding vector representations.

3. The dangerous behavior warning method according to claim 1, characterized by, The training of the model according to the vector comprises: using an optimizer to optimize parameters, setting an accuracy threshold, and training the model according to the training triple vector and the prediction triple vector; when the model accuracy reaches a preset threshold, saving the current model parameters.

4. The dangerous behavior warning method according to claim 3, characterized in that: The method further comprises: training the model using the maximum likelihood estimation principle, performing Gibbs sampling on the non-parametric part for the CRP process to determine the probability representation of the parameters of the new components; constructing a model cost function according to the distance of the bulldozer; using an optimizer to optimize the objective function, saving the optimized parameters as the initialization parameters of the later model.

5. A dangerous behavior warning device, characterized by, The method comprises: a real-time monitoring module for acquiring a target image video; obtaining subject information and behavior information in the target image video through target detection; a Gaussian space vector initialization module for initializing the subject information and the behavior information in a Gaussian space according to a knowledge graph algorithm, and constructing a subject vector and an actual behavior vector; a behavior prediction module for inputting the subject vector and the actual behavior vector into a pre-constructed model to output predicted behavior information; a warning module for issuing a warning signal when the predicted behavior information includes dangerous behavior information; A model construction module is configured to acquire a pre-training sample set, construct a vector representation for a sample tuple in the pre-training sample set, train a model according to the vector, and determine a pre-constructed model. The construction of the vector representation for the sample tuple in the pre-training sample set includes extracting different meanings of the same behavior in the pre-training sample set, defining a standard normal distribution for a subject of the pre-training sample set, defining a Gaussian distribution to which a behavior appearing in the pre-training sample set is subject, defining a predicted behavior vector representing a predicted possible behavior, defining an actual behavior vector in an actual verification set representing an actual behavior for use in correcting parameters in a model training process, and constructing a training triple vector and a prediction triple vector. The training triple vector includes a subject vector, an actual behavior vector, and a behavior-to-be-predicted vector, and the prediction triple vector includes a subject vector, an actual behavior vector, and a predicted behavior vector.

6. An electronic device, comprising: Comprise: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to implement the dangerous behavior early warning method of any one of claims 1-4 via execution of the executable instructions.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the dangerous behavior early warning method of any one of claims 1-4.

Images