Multi-source heterogeneous target object risk assessment and early warning method and device

Abstract

The invention relates to the technical field of data processing, and discloses a multi-source heterogeneous target object risk assessment and early warning method and device. The method comprises the following steps: constructing a multi-modal risk early warning model based on historical dynamic information of a target object by using a preset deep learning model; inputting the current dynamic information of the target object into the multi-modal danger early warning model, and predicting multi-modal danger early warning information; constructing an influence factor scale based on the risk research data of the target object, and obtaining evaluation result data of the influence factor scale; and training a preset machine learning algorithm model through the multi-modal risk early warning information and the evaluation result data to obtain a multi-modal fusion risk assessment model, and predicting risk assessment information of the target object through the multi-modal fusion risk assessment model. According to the method, based on the multi-source heterogeneous data, the risk of the target object is comprehensively evaluated and predicted by adopting the multi-modal deep learning algorithm, and accurate prediction of risk evaluation information of the target object is realized.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a multi-source heterogeneous target object risk assessment and early warning method and device. Background Art

[0002] In today's complex and ever-changing socio-economic environment, target objects are facing an increasing number of dangerous challenges. These risks are often multi-dimensional and cross-domain in nature, making it difficult to effectively evaluate and warn through single-dimensional data or models.

[0003] Traditional hazard assessment methods mainly rely on expert experience, statistical analysis of historical data and other means. The assessors have a heavy workload and are difficult to operate. They require assessors to have relatively rich experience and are greatly influenced by the assessors' subjective consciousness. They lack standardized processes and scientific analysis. Not only are they inefficient, but they also find it difficult to fully capture the multimodal characteristics of hazards and their dynamic changes, making it difficult to accurately identify and warn of hazards. Summary of the invention

[0004] In view of this, the purpose of the present invention is to overcome the deficiencies in the prior art and to provide a multi-source heterogeneous target object hazard assessment and early warning method and device.

[0005] The present invention provides the following technical solutions: In a first aspect, an embodiment of the present disclosure provides a multi-source heterogeneous target object risk assessment and early warning method, the method comprising: Collect historical dynamic information of the target object, and build a multimodal hazard warning model based on the historical dynamic information and using a preset deep learning model; Collecting current dynamic information of the target object, and inputting the current dynamic information into the multimodal hazard warning model, and predicting the multimodal hazard warning information of the target object through the multimodal hazard warning model; Collecting risk research data of the target object, constructing an impact factor scale based on the risk research data, and obtaining evaluation result data of the impact factor scale uploaded by the user; A preset machine learning algorithm model is trained by using the multimodal risk warning information and the evaluation result data to obtain a multimodal fusion risk assessment model, and the risk assessment information of the target object is predicted by using the multimodal fusion risk assessment model.

[0006] Optionally, the historical dynamic information includes historical negative event text data, historical physiological punch-in data, historical sleep video data, historical meeting video data and historical call audio data; the current dynamic information includes current negative event text data, current physiological punch-in data, current sleep video data, current meeting video data and current call audio data; the multimodal hazard warning information includes negative event warning information, physiological punch-in warning information, sleep warning information, meeting warning information and call warning information.

[0007] Optionally, the predicting the multimodal danger warning information of the target object by using the multimodal danger warning model includes: Acquire a preset danger level of the target object, and predict a current impact level of the current negative event text data of the target object according to the multimodal danger warning model; According to the current impact level and preset danger level of the target object, and in combination with preset warning rules, negative event warning information of the target object is generated.

[0008] Optionally, generating the negative event warning information of the target object according to the current impact level and the preset danger level of the target object and in combination with preset warning rules includes: Determine whether the current impact level is greater than a first preset impact threshold; if the current impact level is less than or equal to the first preset impact threshold, determine whether the preset danger level is greater than a first preset danger threshold; if the preset danger level is greater than the first preset danger threshold, generate the negative event warning information; If the current impact level is greater than the first preset impact threshold, then determine whether the current impact level is greater than the second preset impact threshold; if the current impact level is less than or equal to the second preset impact threshold, then determine whether the preset danger level is greater than the second preset danger threshold; if the preset danger level is greater than the second preset danger threshold, then generate the negative event warning information, wherein the first preset impact threshold is less than the second preset impact threshold, and the first preset danger threshold is greater than the second preset danger threshold; If the current impact level is greater than the second preset impact threshold, determine whether the current impact level is greater than the third preset impact threshold; if the current impact level is greater than the third preset impact threshold, generate the negative event warning information, wherein the second preset impact threshold is less than the third preset impact threshold.

[0009] Optionally, the predicting the multimodal risk warning information of the target object by using the multimodal risk warning model further includes: Constructing a physiological clock-in status norm based on the historical physiological clock-in data, and using the multimodal risk warning model to monitor the current physiological clock-in data of the target object in real time; When it is monitored that the current physiological clock-in data of the target object deviates from the physiological clock-in state norm, the physiological clock-in warning information is generated.

[0010] Optionally, the predicting the multimodal risk warning information of the target object by using the multimodal risk warning model further includes: Using the multimodal risk warning model to monitor in real time the number of tossing and turning and the number of leaving the bed in the current sleep video data of the target object; A sleep baseline of the target object is established based on the monitoring results of a preset time period, and the sleep warning information is generated when it is monitored that the number of tossing and getting out of bed of the target object deviates from the sleep baseline.

[0011] Optionally, the predicting the multimodal risk warning information of the target object by using the multimodal risk warning model further includes: The multimodal risk warning model is used to extract the current posture features and the current facial features in the current meeting video data of the target object, and the extracted current posture features and the current facial features are analyzed to obtain the current body movement amplitude and the current emotional state stability of the target object; If the current body movement amplitude exceeds a preset amplitude threshold, and / or the current emotional state stability exceeds a preset stability threshold, the meeting warning information is generated.

[0012] Optionally, the predicting the multimodal risk warning information of the target object by using the multimodal risk warning model further includes: Extracting current audio features from the current call audio data of the target object using the multimodal risk warning model, and analyzing the extracted current audio features; When the similarity between the current audio feature and the preset dangerous feature exceeds a preset similarity threshold, the call warning information is generated.

[0013] Optionally, constructing an impact factor scale based on the risk research data includes: Determine the scale dimensions of the impact factor scale according to the risk research data, and design the evaluation items of the impact factor scale and their corresponding scoring methods based on the scale dimensions; Constructing a first impact factor scale according to the scale dimensions, the evaluation items and the scoring method, and obtaining first evaluation result data of the first impact factor scale uploaded by the user; Eliminate invalid items in the evaluation items according to the first evaluation result data, generate a second impact factor scale, and obtain second evaluation result data of the second impact factor scale uploaded by the user; The confidence of the second impact factor scale is tested according to the second evaluation result data, and if the confidence exceeds a preset confidence threshold, the impact factor scale is generated.

[0014] In a second aspect, an embodiment of the present disclosure provides a multi-source heterogeneous target object risk assessment and early warning device, the device comprising: A construction module is used to collect historical dynamic information of the target object, and to construct a multimodal hazard warning model based on the historical dynamic information and using a preset deep learning model; A collection module, used for collecting current dynamic information of the target object, and inputting the current dynamic information into the multimodal hazard warning model, and predicting the multimodal hazard warning information of the target object through the multimodal hazard warning model; An acquisition module, used to collect risk research data of the target object, construct an impact factor scale based on the risk research data, and obtain evaluation result data of the impact factor scale uploaded by the user; The prediction module is used to train a preset machine learning algorithm model through the multimodal risk warning information and the evaluation result data to obtain a multimodal fusion risk assessment model, and predict the risk assessment information of the target object through the multimodal fusion risk assessment model.

[0015] Beneficial effects of this application: The multi-source heterogeneous target object risk assessment and early warning method provided in the embodiment of the present application collects historical dynamic information of the target object, builds a multimodal risk early warning model based on the historical dynamic information and using a preset deep learning model; collects current dynamic information of the target object, and inputs the current dynamic information into the multimodal risk early warning model, and predicts the multimodal risk early warning information of the target object through the multimodal risk early warning model; collects risk research data of the target object, builds an impact factor scale based on the risk research data, and obtains the evaluation result data of the impact factor scale uploaded by the user; trains a preset machine learning algorithm model through the multimodal risk early warning information and the evaluation result data to obtain a multimodal fusion risk assessment model, and predicts the risk assessment information of the target object through the multimodal fusion risk assessment model. The present application is based on multi-source heterogeneous data, and uses a multimodal deep learning algorithm to comprehensively evaluate and predict the risk of the target object, so as to achieve accurate prediction of the risk assessment information of the target object.

[0016] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work. In each of the drawings, similar components are numbered similarly.

[0018] Figure 1 A flowchart of a multi-source heterogeneous target object risk assessment and early warning method provided in an embodiment of the present application is shown; Figure 2 A flowchart of another multi-source heterogeneous target object risk assessment and early warning method provided in an embodiment of the present application is shown; Figure 3 A schematic diagram of the structure of a multi-source heterogeneous target object risk assessment and early warning device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0020] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. In contrast, when an element is referred to as being "directly on" another element, there is no intermediate element. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0021] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0022] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of the template are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0024] Example 1 like Figure 1 FIG. 1 is a flowchart of a multi-source heterogeneous target object risk assessment and early warning method in an embodiment of the present application. The multi-source heterogeneous target object risk assessment and early warning method provided in an embodiment of the present application includes the following steps: Step S110, collecting historical dynamic information of the target object, and building a multimodal hazard warning model based on the historical dynamic information and using a preset deep learning model.

[0025] In this embodiment, the dynamic information of the target object includes negative event text data, physiological clock-in data (heart rate, blood pressure, respiratory rate, face temperature, emotion, head movement, voice and other information), sleep video data, meeting video data and call audio data. The multimodal risk warning model includes negative event warning sub-model, physiological clock-in warning sub-model, sleep warning sub-model, meeting warning sub-model and call warning sub-model. Specifically, the construction method of each sub-model is as follows: (1) Based on the historical negative event text data and their corresponding impact levels, a negative event warning sub-model is constructed using preset text analysis technology; (2) Based on the historical physiological clock-in data collected by the physiological clock-in machine, use statistical methods (such as mean and standard deviation) to construct the physiological clock-in status norm, select a suitable regression model (such as linear regression, random forest regression, etc.), train the regression model based on the historical physiological clock-in data and the physiological clock-in status norm, and generate the physiological clock-in warning sub-model; (3) Obtain a public sleep video dataset, and optimize the preset human recognition model and preset activity detection model of the target object based on the historical sleep video data of the target object collected by the monitoring equipment every day during the sleeping period through a deep learning model and an optical flow algorithm (the optical flow algorithm combined with deep learning can more effectively identify and analyze motion changes in the video), and obtain a sleep warning sub-model; (4) Obtain a public meeting video dataset, conduct preliminary training on the deep learning model based on the public meeting video dataset, so that the model has basic human motion and facial expression recognition capabilities, and then use the historical meeting video data of the target object collected by the meeting system camera to further optimize the deep learning model to obtain the meeting warning sub-model; (5) Obtain a public call audio dataset and obtain the preset danger level of the target object (derived from the diagnosis of a psychologist or counselor). Use the danger level as a label and build a call warning sub-model based on the public call audio dataset, historical call audio data, historical physiological check-in data (including physiological indicators such as breathing and heart rate), preset danger level labels and machine learning algorithms (when the data is insufficient to fully train the model, use the public AVEC depression dataset as a supplement to enhance the generalization ability of the model).

[0026] The above-mentioned sub-models combine the concepts of big data analysis, deep learning, artificial intelligence and mental health management, which can effectively improve the management efficiency of the target objects, ensure the physical and mental health of the target objects, facilitate subsequent more intelligent and comprehensive predictions of the target objects, and realize full-dimensional and full-time risk management.

[0027] Step S120, collecting current dynamic information of the target object, and inputting the current dynamic information into the multimodal hazard warning model, and predicting the multimodal hazard warning information of the target object through the multimodal hazard warning model.

[0028] Furthermore, after the current dynamic information of the target object is collected, the current dynamic information is input into the multimodal risk warning model constructed in step S110, and the multimodal risk warning information of the target object is predicted by the multimodal risk warning model, including negative event warning information, physiological clock-in warning information, sleep warning information, meeting warning information and call warning information. Specifically, the generation method of each warning information is as follows: (1) Negative event warning refers to analyzing the target object's negative event text data and using text analysis technology to warn the target object of the negative event impact level: first, obtain the preset danger level of the target object that has been pre-evaluated, and predict the impact level of the target object's current negative event text data based on the negative event warning sub-model. Then, based on the target object's current impact level and the preset danger level, and combined with the preset warning rules, generate the target object's negative event warning information; Preferably, the preset warning rules are as follows: ① First, determine whether the current impact level is greater than the first preset impact threshold (that is, determine whether the current impact level is a slight impact); if the current impact level is less than or equal to the first preset impact threshold, then determine whether the preset danger level is greater than the first preset danger threshold (that is, determine whether the preset danger level is extremely high); if the preset danger level is greater than the first preset danger threshold, generate negative event warning information; ② If the current impact level is greater than the first preset impact threshold, then determine whether the current impact level is greater than the second preset impact threshold (that is, determine whether the current impact level is a general impact); if the current impact level is less than or equal to the second preset impact threshold, then generate negative event warning information. ③ If the current impact level is greater than the second preset impact threshold, it is determined whether the preset risk level is greater than the second preset risk threshold (i.e., it is determined whether the preset risk level is intermediate). If the preset risk level is greater than the second preset risk threshold, a negative event warning message is generated; ③ If the current impact level is greater than the second preset impact threshold, it is determined whether the current impact level is greater than the third preset impact threshold (i.e., it is determined whether the current impact level is severe). If the current impact level is greater than the third preset impact threshold, a negative event warning message is generated (wherein the first preset impact threshold is less than the second preset impact threshold, the first preset risk threshold is greater than the second preset risk threshold, and the second preset impact threshold is less than the third preset impact threshold); (2) Physiological clock-in warning refers to collecting the target object's heart rate, blood pressure, respiratory rate, facial temperature, emotions, head movements, voice and other data through the physiological clock-in machine, and using the physiological clock-in warning sub-model to detect abnormalities in the target object's various physiological indicators, and initiate a warning in real time: first, the physiological clock-in warning model is used to monitor the target object's current physiological clock-in data in real time, and then when it is detected that the target object's current physiological clock-in data significantly deviates from the physiological clock-in state norm, the physiological clock-in warning information is automatically generated; (3) Sleep warning refers to obtaining the target object's activity information during the night rest period by analyzing the monitoring video, and issuing a warning for the target object's abnormal sleep condition in combination with the time-series sequence results: first, the sleep warning sub-model is used to analyze the movement changes in the target object's current sleep video data, and the number of tossing and turning (as an indicator for evaluating sleep quality) and the number of times leaving the bed (special attention is paid to the behavior of frequently leaving the bed) are calculated. Then, based on the monitoring results of the preset time period, the target object's sleep baseline is established, including the normal number of tossing and turning and the normal number of times leaving the bed. The current monitoring data is compared with the baseline in real time. When the target object's tossing and turning and leaving the bed are monitored to deviate from the sleep baseline (such as a surge in the number of tossing and leaving the bed, frequent leaving the bed, etc.), a sleep warning message is generated; (4) Meeting warning refers to recording the target object's performance during the meeting through a camera, and identifying its body movements and emotions in combination with an algorithm model, so as to achieve a warning of the dangers caused by dangerous behaviors and abnormal emotional expressions during the meeting: first, the meeting warning sub-model is used to extract the current posture features and current facial features (including body movement amplitude, joint angles, facial expression changes, etc.) in the current meeting video data of the target object, and the extracted current posture features and current facial features are comprehensively analyzed to obtain the current body movement amplitude and current emotional state stability of the target object; if the current body movement amplitude exceeds a preset amplitude threshold (such as rapid movement, excessive tension, etc.), and / or the current emotional state stability exceeds a preset stability threshold (such as anger, anxiety, sleepiness, etc.), a meeting warning message is generated; (5) Call warning refers to collecting the audio files of the target object when making a call through a microphone, and combining the algorithm model to identify its abnormal emotions, so as to achieve a dangerous warning for the target object with abnormal emotions on the phone: when the target object makes a call, the relevant user will first monitor in real time and preliminarily evaluate whether there is any abnormality in the content of the conversation. If no obvious abnormality is detected, the call warning sub-model will be used to extract the current audio features (such as pitch, speaking speed, volume change, emotional characteristics, heart rate variability, breathing rate change) in the target object's current call audio data, and analyze the extracted current audio features; if the similarity between the current audio features and the preset dangerous features (such as audio features of abnormal emotions) exceeds the preset similarity threshold, a call warning information is generated.

[0029] The above method combines the target object's multi-dimensional audio and video, text information, and multi-scenario meeting information to conduct an inspection and evaluation, which is more objective, comprehensive, and scientific. It intelligently assists users to quickly evaluate the current status of the target object, promptly detect anomalies and issue warnings, and assist users to evaluate whether the danger of the target object has reached the level that requires human intervention and immediate evaluation, and start paying attention to the target object in advance to effectively reduce the danger. Audio and video data analysis based on deep learning methods can automatically conduct long-term and time-series comprehensive analysis of the performance of a large number of target objects, and more accurately and conveniently distinguish target objects of different levels of danger.

[0030] Step S130, collecting risk research data of the target object, constructing an impact factor scale based on the risk research data, and obtaining evaluation result data of the impact factor scale uploaded by the user.

[0031] For the target objects for which multimodal hazard warning information has been generated, the impact factor scale will be used for evaluation. Figure 2 As shown, step S130 includes: Step S131, determining the scale dimensions of the impact factor scale according to the risk research data, and designing the evaluation items of the impact factor scale and the corresponding scoring method based on the scale dimensions; Step S132, constructing a first impact factor scale according to the scale dimensions, the evaluation items and the scoring method, and obtaining first evaluation result data of the first impact factor scale uploaded by the user; Step S133, eliminating invalid items in the evaluation items according to the first evaluation result data, generating a second impact factor scale, and obtaining second evaluation result data of the second impact factor scale uploaded by the user; Step S134: testing the confidence of the second impact factor scale according to the second evaluation result data; if the confidence exceeds a preset confidence threshold, generating the impact factor scale.

[0032] The steps for constructing the impact factor scale are as follows: (1) Use literature research, survey research and other risk research data to determine the scale dimensions of the impact factor scale, and design assessment items and their corresponding scoring methods based on the scale dimensions. For example, when the specific application scenario is a regulatory environment, such as in a prison, the literature research mainly searches for domestic and foreign research related to the risk assessment of criminals in prison, providing a theoretical basis for the determination of the scale dimensions; the survey research is to obtain first-hand information on the risk assessment of criminals in prison from the front-line police officers in the prison, so that the compiled scale is both in line with international standards and in line with the actual situation of local prisons; (2) Construct the first impact factor scale based on the scale dimensions, evaluation items and scoring methods, extract representative samples of the target objects for prediction, and obtain the first evaluation result data of the first impact factor scale uploaded by users. After the initial test, analyze the quality of the evaluation items. The analysis is divided into qualitative analysis and quantitative analysis: qualitative analysis mainly investigates and analyzes the popularity, accuracy, and ambiguity of the evaluation item language, while quantitative analysis analyzes the difficulty (popularity) and discrimination of the evaluation items; (3) Based on the analysis results of the previous step, modify the items according to the qualitative analysis results, and eliminate invalid items with low discrimination according to the quantitative analysis results to form a second impact factor scale. Draw a representative sample of the target object for prediction (the sample size of this test must be larger than the first time), and obtain the second evaluation result data of the second impact factor scale uploaded by the user. Based on the second evaluation result data, test the confidence (i.e. reliability and validity) of the second impact factor scale. If the confidence exceeds the preset confidence threshold, it proves that the effect is ideal. At this time, a formal impact factor scale is formed and further subjected to practical testing. The evaluation result data of the impact factor scale uploaded by the user is obtained; if the confidence does not exceed the preset confidence threshold, it proves that the effect is not ideal, and it is necessary to revise the evaluation items, conduct item analysis, and confidence test again until it is ideal.

[0033] The above method quantifies expert experience, adopts a data-driven approach combined with expert experience, comprehensively incorporates the influencing factors of the target object's risk assessment, and combines the target object's psychology and behavior. The assessment is relatively comprehensive and non-redundant, and realizes the transformation of the target object's risk assessment from subjective experience to artificial intelligence, formulates personalized transformation measures, and thus reduces individual risk.

[0034] Step S140, inputting the multimodal hazard warning information and the evaluation result data into a preset machine learning algorithm model, and predicting the hazard assessment information of the target object through the preset machine learning algorithm model.

[0035] Understandably, select a preset machine learning algorithm model with interpretability, such as decision tree, linear regression, logistic regression, generalized learning vector quantization (GMLVQ), etc., use multimodal hazard warning information and evaluation result data to train the model, and obtain a multimodal fusion hazard assessment model suitable for hazard assessment. During the training process, methods such as cross-validation can be used to evaluate the performance and stability of the model.

[0036] Preferably, a trained multimodal fusion risk assessment model is used to predict the real-time multimodal risk warning information and evaluation result data of the target object to obtain risk assessment information of different levels (such as high risk, medium risk, and low risk) of the target object.

[0037] The above method predicts the individual risk of the target object in advance based on artificial intelligence and big data technology methods, which can assist users in daily assessment work based on scales, interviews and personal experience. The identified risk assessment information is conducive to the implementation of targeted assistance and education for the target object.

[0038] The multi-source heterogeneous target object risk assessment and early warning method provided in the embodiment of the present application collects historical dynamic information of the target object, builds a multimodal risk early warning model based on the historical dynamic information and using a preset deep learning model; collects current dynamic information of the target object, and inputs the current dynamic information into the multimodal risk early warning model, and predicts the multimodal risk early warning information of the target object through the multimodal risk early warning model; collects risk research data of the target object, builds an impact factor scale based on the risk research data, and obtains the evaluation result data of the impact factor scale uploaded by the user; trains a preset machine learning algorithm model through the multimodal risk early warning information and the evaluation result data to obtain a multimodal fusion risk assessment model, and predicts the risk assessment information of the target object through the multimodal fusion risk assessment model. The present application is based on multi-source heterogeneous data, and uses a multimodal deep learning algorithm to comprehensively evaluate and predict the risk of the target object, so as to achieve accurate prediction of the risk assessment information of the target object.

[0039] Example 2 like Figure 3 FIG. 1 is a schematic diagram of a multi-source heterogeneous target object risk assessment and early warning device 300 in an embodiment of the present application, wherein the device comprises: A construction module 310 is used to collect historical dynamic information of the target object, and to construct a multimodal risk warning model based on the historical dynamic information and using a preset deep learning model; The acquisition module 320 is used to acquire the current dynamic information of the target object, and input the current dynamic information into the multimodal hazard warning model, and predict the multimodal hazard warning information of the target object through the multimodal hazard warning model; The acquisition module 330 is used to collect the risk research data of the target object, construct an impact factor scale based on the risk research data, and obtain the evaluation result data of the impact factor scale uploaded by the user; The prediction module 340 is used to train a preset machine learning algorithm model through the multimodal risk warning information and the evaluation result data to obtain a multimodal fusion risk assessment model, and predict the risk assessment information of the target object through the multimodal fusion risk assessment model.

[0040] The multi-source heterogeneous target object hazard assessment and early warning device provided in the embodiment of the present application is based on multi-source heterogeneous data and uses a multi-modal deep learning algorithm to comprehensively assess and predict the hazard of the target object, thereby achieving accurate prediction of the target object hazard assessment information.

[0041] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow chart, and the combination of boxes in the structure diagram and / or the flow chart, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0042] In addition, the functional modules or units in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0043] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium can be a non-volatile storage medium or a volatile storage medium. For example, the storage medium can be: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes.

[0044] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A multi-source heterogeneous target object risk assessment and early warning method, characterized in that: The method comprises: Collect historical dynamic information of the target object, and build a multimodal hazard warning model based on the historical dynamic information and using a preset deep learning model; Collecting current dynamic information of the target object, and inputting the current dynamic information into the multimodal hazard warning model, and predicting the multimodal hazard warning information of the target object through the multimodal hazard warning model; Collecting risk research data of the target object, constructing an impact factor scale based on the risk research data, and obtaining evaluation result data of the impact factor scale uploaded by the user; A preset machine learning algorithm model is trained by using the multimodal risk warning information and the evaluation result data to obtain a multimodal fusion risk assessment model, and the risk assessment information of the target object is predicted by using the multimodal fusion risk assessment model.

2. The multi-source heterogeneous target object risk assessment and early warning method according to claim 1 is characterized in that: The historical dynamic information includes historical negative event text data, historical physiological punch-in data, historical sleep video data, historical meeting video data and historical call audio data; the current dynamic information includes current negative event text data, current physiological punch-in data, current sleep video data, current meeting video data and current call audio data; the multimodal hazard warning information includes negative event warning information, physiological punch-in warning information, sleep warning information, meeting warning information and call warning information.

3. The multi-source heterogeneous target object risk assessment and early warning method according to claim 2 is characterized in that: The predicting the multimodal hazard warning information of the target object by using the multimodal hazard warning model includes: Acquire a preset danger level of the target object, and predict a current impact level of the current negative event text data of the target object according to the multimodal danger warning model; According to the current impact level and preset danger level of the target object, and in combination with preset warning rules, negative event warning information of the target object is generated.

4. The multi-source heterogeneous target object risk assessment and early warning method according to claim 3 is characterized in that: The generating of negative event warning information of the target object according to the current impact level and the preset danger level of the target object and in combination with the preset warning rules includes: Determine whether the current impact level is greater than a first preset impact threshold; if the current impact level is less than or equal to the first preset impact threshold, determine whether the preset danger level is greater than a first preset danger threshold; if the preset danger level is greater than the first preset danger threshold, generate the negative event warning information; If the current impact level is greater than the first preset impact threshold, then determine whether the current impact level is greater than the second preset impact threshold; if the current impact level is less than or equal to the second preset impact threshold, then determine whether the preset danger level is greater than the second preset danger threshold; if the preset danger level is greater than the second preset danger threshold, then generate the negative event warning information, wherein the first preset impact threshold is less than the second preset impact threshold, and the first preset danger threshold is greater than the second preset danger threshold; If the current impact level is greater than the second preset impact threshold, determine whether the current impact level is greater than the third preset impact threshold; if the current impact level is greater than the third preset impact threshold, generate the negative event warning information, wherein the second preset impact threshold is less than the third preset impact threshold.

5. The multi-source heterogeneous target object risk assessment and early warning method according to claim 2 is characterized in that: The method of predicting the multimodal hazard warning information of the target object by using the multimodal hazard warning model further includes: Constructing a physiological clock-in status norm based on the historical physiological clock-in data, and using the multimodal risk warning model to monitor the current physiological clock-in data of the target object in real time; When it is monitored that the current physiological clock-in data of the target object deviates from the physiological clock-in state norm, the physiological clock-in warning information is generated.

6. The multi-source heterogeneous target object risk assessment and early warning method according to claim 2 is characterized in that: The method of predicting the multimodal hazard warning information of the target object by using the multimodal hazard warning model further includes: Using the multimodal risk warning model to monitor in real time the number of tossing and turning and the number of leaving the bed in the current sleep video data of the target object; A sleep baseline of the target object is established based on the monitoring results of a preset time period, and the sleep warning information is generated when it is monitored that the number of tossing and getting out of bed of the target object deviates from the sleep baseline.

7. The multi-source heterogeneous target object risk assessment and early warning method according to claim 2 is characterized in that: The method of predicting the multimodal hazard warning information of the target object by using the multimodal hazard warning model further includes: The multimodal risk warning model is used to extract the current posture features and the current facial features in the current meeting video data of the target object, and the extracted current posture features and the current facial features are analyzed to obtain the current body movement amplitude and the current emotional state stability of the target object; If the current body movement amplitude exceeds a preset amplitude threshold, and / or the current emotional state stability exceeds a preset stability threshold, the meeting warning information is generated.

8. The multi-source heterogeneous target object risk assessment and early warning method according to claim 2 is characterized in that: The method of predicting the multimodal hazard warning information of the target object by using the multimodal hazard warning model further includes: Extracting current audio features from the current call audio data of the target object using the multimodal risk warning model, and analyzing the extracted current audio features; When the similarity between the current audio feature and the preset dangerous feature exceeds a preset similarity threshold, the call warning information is generated.

9. The multi-source heterogeneous target object risk assessment and early warning method according to claim 1, characterized in that: The constructing of the impact factor scale based on the risk research data includes: Determine the scale dimensions of the impact factor scale according to the risk research data, and design the evaluation items of the impact factor scale and their corresponding scoring methods based on the scale dimensions; Constructing a first impact factor scale according to the scale dimensions, the evaluation items and the scoring method, and obtaining first evaluation result data of the first impact factor scale uploaded by the user; Eliminate invalid items in the evaluation items according to the first evaluation result data, generate a second impact factor scale, and obtain second evaluation result data of the second impact factor scale uploaded by the user; The confidence of the second impact factor scale is tested according to the second evaluation result data, and if the confidence exceeds a preset confidence threshold, the impact factor scale is generated.

10. A multi-source heterogeneous target object risk assessment and early warning device, characterized in that: The device comprises: A construction module is used to collect historical dynamic information of the target object, and to construct a multimodal hazard warning model based on the historical dynamic information and using a preset deep learning model; A collection module, used for collecting current dynamic information of the target object, and inputting the current dynamic information into the multimodal hazard warning model, and predicting the multimodal hazard warning information of the target object through the multimodal hazard warning model; An acquisition module, used to collect risk research data of the target object, construct an impact factor scale based on the risk research data, and obtain evaluation result data of the impact factor scale uploaded by the user; The prediction module is used to train a preset machine learning algorithm model through the multimodal risk warning information and the evaluation result data to obtain a multimodal fusion risk assessment model, and predict the risk assessment information of the target object through the multimodal fusion risk assessment model.

Images