Personnel risk assessment method based on multi-modal data fusion

Abstract

The invention discloses a multi-modal data fusion-based personnel risk assessment method. The multi-modal data fusion-based personnel risk assessment method comprises the steps of obtaining positioning data, video data and physiological data of a to-be-assessed person; according to the video data and the physiological data, respectively determining a behavior risk probability and a physiological risk probability of the to-be-assessed person; according to the positioning data, determining an area where a to-be-assessed person is located and a positioning risk probability; according to the determined area where the to-be-assessed person is located, dynamic weight values of the positioning risk probability, the physiological risk probability and the behavior risk probability of the to-be-assessed person are determined respectively; and according to the determined dynamic weight values of the positioning risk probability, the physiological risk probability and the behavior risk probability, calculating the risk value of the to-be-assessed person.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a personnel risk evaluation method based on multi-modal data fusion. BACKGROUND

[0002] In a high-risk chemical scene, real-time and accurate personnel safety closed-loop management is required in a strong corrosive and flammable and explosive environment. The single alarm currently has the following technical pain points: (1) high single-mode alarm misjudgment rate: the positioning system (such as the boundary crossing alarm) cannot identify the physiological state (such as syncope), the video monitoring has a high misjudgment rate in the smoke / shading scene, and the physiological monitoring (such as the heart rate alarm) ignores the environmental risk (such as the leakage area); (2) serious response delay: the traditional system has a long alarm response time, which cannot meet the requirements of the high-risk scene; (3) data island effect: the positioning / video / physiological data are not associated, and the composite risk (such as running in the leakage area + sudden increase in heart rate) has a low detection rate.

[0003] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0004] The main purpose of the present application is to provide a personnel risk evaluation method based on multi-modal data fusion, which aims to solve the above technical problems in the prior art.

[0005] To achieve the above purpose, the present application provides a personnel risk evaluation method based on multi-modal data fusion, which comprises:

[0006] obtaining positioning data, video data and physiological data of a personnel to be evaluated;

[0007] determining a behavior risk probability and a physiological risk probability of the personnel to be evaluated according to the video data and the physiological data, respectively;

[0008] determining a region where the personnel to be evaluated is located and a positioning risk probability according to the positioning data;

[0009] determining dynamic weight values of the positioning risk probability, the physiological risk probability and the behavior risk probability of the personnel to be evaluated according to the region where the personnel to be evaluated is located;

[0010] calculating a risk value of the personnel to be evaluated according to the dynamic weight values of the positioning risk probability, the physiological risk probability and the behavior risk probability.

[0011] Preferably, in the personnel risk assessment method based on multi-modal data fusion, in the step of calculating the risk value of the personnel to be evaluated according to the dynamic weight values of the determined positioning risk probability, physiological risk probability and behavior risk probability, the calculation formula is:

[0012] R = a · R loc + b · R beh + g · d(t) · R phy ;

[0013] Wherein, R is the risk value of the personnel to be evaluated;

[0014] a is the weight of the positioning risk probability;

[0015] b is the weight of the behavior risk probability;

[0016] g is the weight of the physiological risk probability;

[0017] R loc is the positioning risk probability;

[0018] R beh is the behavior risk probability;

[0019] R phy is the physiological risk probability;

[0020] d(t) is a time factor, and t is the continuous staying time of the personnel to be evaluated.

[0021] Preferably, in the personnel risk assessment method based on multi-modal data fusion, the dynamic weight values of the positioning risk probability, physiological risk probability and behavior risk probability of the personnel to be evaluated are determined according to the determined area where the personnel to be evaluated is located, comprising:

[0022] According to the determined area where the personnel to be evaluated is located, the area risk level of the area where the personnel to be evaluated is located is determined;

[0023] According to the determined area risk level, the dynamic weight values of the positioning risk probability, physiological risk probability and behavior risk probability corresponding to the area risk level are determined.

[0024] Preferably, in the personnel risk assessment method based on multi-modal data fusion, the area risk level of the area where the personnel to be evaluated is located is determined according to the determined area where the personnel to be evaluated is located, comprising:

[0025] According to the determined area where the personnel to be evaluated is located, the historical dangerous events and corresponding dangerous levels in the area are obtained;

[0026] According to the obtained historical dangerous events and corresponding dangerous levels, the area risk level of the area where the personnel to be evaluated is located is calculated, and the calculation formula is as follows:

[0027]

[0028] The risk level of the area;

[0029] x1, x2, ..., x n The danger levels corresponding to danger events 1, 2, ..., n are:

[0030] n represents the number of dangerous events within a preset time period in the region;

[0031] f1, f2, ..., f n They are x1, x2, ..., x n The corresponding weights.

[0032] Preferably, in the method for assessing personnel risk based on multimodal data fusion, determining the dynamic weight values ​​of the location risk probability, physiological risk probability, and behavioral risk probability corresponding to the determined regional risk level includes:

[0033] When the determined regional risk level is less than the first threshold, the dynamic weight value α1 of the location risk probability is greater than the dynamic weight value β1 of the behavioral risk probability, which is greater than the dynamic weight value γ1 of the physiological risk probability.

[0034] When the determined regional risk level is greater than the first threshold and less than the second threshold, the dynamic weight value γ2 of the physiological risk probability is greater than the dynamic weight value α2 of the location risk probability and the dynamic weight value β2 of the behavioral risk probability.

[0035] When the determined regional risk level is greater than the second threshold, the dynamic weight value of physiological risk probability is greater than the dynamic weight value of location risk probability α3 + the dynamic weight value of behavioral risk probability β3, and γ3 > γ2.

[0036] Preferably, in the method for assessing personnel risk based on multimodal data fusion, the formula for calculating the risk value of the person to be assessed based on the determined dynamic weight values ​​of location risk probability, physiological risk probability, and behavioral risk probability includes the following formula:

[0037]

[0038] δ0 is the weight value of the person to be evaluated when entering the area and staying for 0 hours.

[0039] Preferably, in the method for assessing personnel risk based on multimodal data fusion, the formula for calculating the risk value of the person to be assessed based on the determined dynamic weight values ​​of location risk probability, physiological risk probability, and behavioral risk probability includes the following formula:

[0040]

[0041] Among them, R phy For physiological risk probability;

[0042] HRV is the real-time heart rate variability coefficient.

[0043] k is the sensitivity coefficient to small changes in physiological quantities;

[0044] θ represents the risk threshold of the area where the person to be assessed is located.

[0045] Preferably, in the method for assessing personnel risk based on multimodal data fusion, the physiological risk probability is determined based on the degree of eye fatigue; the method further includes:

[0046] Based on video data, determine the open and closed state of the eyes of the person to be evaluated within a preset time period;

[0047] The vibration signals of the muscles around the eyes are acquired, and an evaluation index for assessing the degree of eye fatigue is output based on a pre-trained evaluation model.

[0048] The fatigue level of the person being evaluated is determined based on the obtained evaluation indicators and the opening and closing status of the eyes within a preset time.

[0049] Determine the probability of physiological risk based on the level of fatigue of the person being assessed;

[0050] The evaluation model is as follows:

[0051]

[0052] Where x(n) represents the nth sample value of the vibration signal;

[0053] w(n) is the residual value of white noise;

[0054] p is the order of the AR model;

[0055] a k It is the k-th coefficient of the evaluation model, i.e., the parameter to be determined.

[0056] To achieve the above objectives, the present invention provides a computer device, the computer device comprising:

[0057] At least one processor; and,

[0058] A memory communicatively connected to the at least one processor; wherein,

[0059] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the aforementioned personnel risk assessment method based on multimodal data fusion.

[0060] To achieve the above objectives, the present invention provides a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the above-described method for assessing personnel risk based on multimodal data fusion.

[0061] The present invention has at least the following beneficial effects:

[0062] This invention provides a method for acquiring location data, video data, and physiological data of a person to be assessed; determining the behavioral risk probability and physiological risk probability of the person to be assessed based on the video data and physiological data, respectively; determining the location area and location risk probability of the person to be assessed based on the location data; determining the dynamic weight values ​​of the location risk probability, physiological risk probability, and behavioral risk probability of the person to be assessed based on the determined location area; and calculating the risk value of the person to be assessed based on the determined dynamic weight values ​​of the location risk probability, physiological risk probability, and behavioral risk probability. This enables scenario-based risk perception, overcomes data silo bottlenecks, improves the accuracy and timeliness of composite risk identification and early warning, and reduces the incidence of major accidents.

[0063] Furthermore, this invention transforms multimodal data into accurate risk decisions through dynamic weight allocation and environmental coupling calculation. Attached Figure Description

[0064] Figure 1 A schematic diagram of one embodiment of the personnel risk assessment method based on multimodal data fusion provided by the present invention;

[0065] Figure 2 This is a schematic diagram of an embodiment of the computer device provided by the present invention.

[0066] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0067] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0068] In this embodiment of the invention, the term "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0069] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0070] In this embodiment of the invention, the term "multiple" refers to two or more, and other quantifiers are similar.

[0071] In this invention, unless otherwise stated, directional terms such as "upper," "lower," "top," and "bottom" are generally used in relation to the direction shown in the accompanying drawings, or in relation to the vertical, perpendicular, or gravitational direction of the component itself; similarly, for ease of understanding and description, "inner" and "outer" refer to the inner and outer contours of each component itself, but the above directional terms are not intended to limit this invention.

[0072] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details are presented in the embodiments of the present invention to facilitate a better understanding of the invention. However, the technical solutions claimed in the present invention can be implemented even without these technical details and various variations and modifications based on the following embodiments. The division of the following embodiments is for ease of description and should not constitute any limitation on the specific implementation of the present invention. The various embodiments can be combined with and referenced by each other without contradiction.

[0073] To address the aforementioned issues, this implementation method relates to a personnel risk assessment method based on multimodal data fusion, which can be applied to computer equipment.

[0074] The following describes the implementation details of a personnel risk assessment method based on multimodal data fusion according to an embodiment of the present invention. The following implementation details are provided for ease of understanding only and are not necessary for implementing this solution. Figure 1The diagram illustrates the personnel risk assessment method based on multimodal data fusion of the present invention.

[0075] In step S100, the location data, video data, and physiological data of the person to be evaluated are acquired. It should be noted that the location data of the person to be evaluated refers to their current location coordinates. These coordinates can be obtained from the device carried by the person; alternatively, they can be calculated from Bluetooth beacon information and GPS signals scanned by the device.

[0076] The video data of the person to be evaluated can be videos of the person to be evaluated taken by a camera device. The behavioral characteristics of the person to be evaluated are assessed through video data. Taking a chemical plant environment as an example, the behavioral characteristics of the person to be evaluated include missing a safety helmet, falling to the ground, running, etc.

[0077] The physiological data of the person being evaluated can be physiological characteristics related to that person, such as their heart rate characteristics. In some implementations, the physiological data of the person being evaluated is primarily assessed using the heart rate variability coefficient (HRV), which is calculated by acquiring the heart rate through a device worn by the person being evaluated.

[0078] In step S200, the behavioral risk probability and physiological risk probability of the person to be assessed are determined based on the video data and physiological data, respectively. It should be noted that the criteria for judging risky behavior differ in different scenarios. Therefore, the specific scenario must be considered when determining the behavioral risk probability. The criteria for judging behavioral risk probability vary in different scenarios. In some implementations, the behavioral risk probability of the person to be assessed can be determined based on video data and the area where the person is located. For example, failing to wear protective equipment (safety helmet, gloves, etc.) as required in a production workshop would be considered a high behavioral risk probability; while in a general location, the corresponding behavioral risk probability is generally lower.

[0079] The behavioral risk probability is determined based on behavioral patterns identified from video data. For example, if the person being assessed is detected to be missing a helmet, the behavioral risk probability is 0.7; if the person being assessed is detected to have fallen, the behavioral risk probability is 0.9; and if the person being assessed is detected to be running, the behavioral risk probability is 0.4. Of course, the behavioral risk probability can be more accurately assessed by combining it with the area where the person being assessed is located. For example, when the person being assessed is detected to be running, the corresponding behavioral risk probability will be significantly different in different areas. The behavioral risk probability will differ greatly between running in an escape zone and running in a high-risk area; the behavioral risk probability will be significantly lower when running in an escape zone. Therefore, in some embodiments, step S200 further includes determining the area where the person being assessed is located based on location data; and determining the behavioral risk probability of the person being assessed based on video data and the area where the person being assessed is located.

[0080] Physiological data typically include basic vital signs data (such as heart rate, blood pressure, breathing rate, etc.), blood-related physiological data (such as blood glucose, blood lipids, complete blood count, etc.), and neuromuscular physiological data (such as electroencephalogram, electromyography, skin electrical activity, etc.).

[0081] Similarly, the criteria for determining the probability of physiological risk differ in different scenarios. In some implementations, the probability of physiological risk can be based on the heart rate variability coefficient. In other implementations, it can be based on the physiological fatigue coefficient, such as the degree of eye fatigue. Of course, in some implementations, the probability of physiological risk also needs to be evaluated in conjunction with the time factor.

[0082] Taking the determination of physiological risk probability based on the degree of human eye fatigue as an example, the method also includes determining the opening and closing state of the eyes of the person to be evaluated within a preset time period based on video data; acquiring vibration signals of the muscles around the eyes; outputting evaluation indicators for evaluating the degree of eye fatigue based on a pre-trained evaluation model; determining the degree of fatigue of the person to be evaluated based on the obtained evaluation indicators and the opening and closing state of the eyes within a preset time period; and finally determining the physiological risk probability based on the degree of fatigue of the person to be evaluated.

[0083] The evaluation model is as follows:

[0084]

[0085] Where x(n) represents the nth sample value of the vibration signal;

[0086] w(n) is the residual value of white noise;

[0087] p is the order of the AR model;

[0088] a kIt is the k-th coefficient of the evaluation model, i.e., the parameter to be determined.

[0089] When the eyes are open for a preset time, the weight of the evaluation indicator is lower; when the eyes are closed for a preset time, the weight of the evaluation indicator is higher. In some implementations, the weight of the evaluation indicator can be defined as 0.3 when the eyes are open for a preset time, and as 0.8 when the eyes are closed for a preset time. In other implementations, the weight can also be defined in conjunction with the area where the person being evaluated is located. For example, in high-risk areas where the person being evaluated needs to concentrate highly, the weight of the evaluation indicator will be relatively higher, meaning the fatigue level of the person being evaluated will be higher; in normal areas where the person being evaluated does not need to concentrate highly, the weight of the evaluation indicator will be relatively lower, meaning the fatigue level of the person being evaluated will be lower. The higher the fatigue level of the person being evaluated, the higher the probability of physiological risk; the lower the fatigue level of the person being evaluated, the lower the probability of physiological risk.

[0090] In step S300, the location of the person to be assessed and the probability of location risk are determined based on the location data. The location data determines the area type corresponding to the person's current location, such as a regular area, a high-risk area, or a leakage incident area. When the person to be assessed is in a leakage incident area, the probability of location risk is relatively high; when the person is in a high-risk area, the probability of location risk is generally high; and when the person is in a regular area, the probability of location risk is relatively low.

[0091] In some implementations, the location risk probability can be defined based on electronic fence rules, according to the different areas.

[0092] For example, when the person to be assessed crosses the boundary in a low-risk area, the corresponding location risk probability is 0.2; when the person to be assessed stays in a high-risk area, the corresponding location risk probability is 0.8; when the person to be assessed is at rest and has an abnormal heart rate, the location risk probability is 1.0.

[0093] In step S400, based on the determined area where the person to be assessed is located, dynamic weight values ​​for the location risk probability, physiological risk probability, and behavioral risk probability of the person to be assessed are determined respectively.

[0094] It should be noted that different regions may have different assessments of the risk probability of various data.

[0095] In some implementations, data can be directly located to determine the area type corresponding to the area where the person being assessed is currently located, such as a regular area, a high-risk area, or a leak incident area. Different area types correspond to different dynamic weight values ​​for each risk probability. In other implementations, the risk level of the current area can be determined through real-time calculation, and then the dynamic weight values ​​for each risk probability can be further determined based on the risk level. Specifically, calculating the risk level of the current area may include steps S410 and S420.

[0096] In step S410, the regional risk level of the area where the person to be assessed is located is determined based on the identified area. Step S410 includes obtaining historical hazardous events and their corresponding risk levels within the identified area; and calculating the regional risk level of the area where the person to be assessed is located based on the obtained historical hazardous events and their corresponding risk levels. The calculation formula is as follows:

[0097]

[0098] The risk level of the area;

[0099] x1, x2, ..., x n The danger levels corresponding to danger events 1, 2, ..., n are:

[0100] n represents the number of dangerous events within a preset time period in the region;

[0101] f1, f2, ..., f n They are x1, x2, ..., x n The corresponding weights.

[0102] In some implementations, the risk level for a region can be determined separately based on the type of risk, thus making the determination of the probability of each risk more accurate.

[0103] In step S420, based on the determined regional risk level, dynamic weight values ​​for the location risk probability, physiological risk probability, and behavioral risk probability corresponding to the regional risk level are determined.

[0104] In some implementations, when the determined regional risk level is less than a first threshold, the dynamic weight value α1 of the location risk probability > the dynamic weight value β1 of the behavioral risk probability > the dynamic weight value γ1 of the physiological risk probability; when the determined regional risk level is greater than the first threshold and less than a second threshold, the dynamic weight value γ2 of the physiological risk probability > the dynamic weight value α2 of the location risk probability > the dynamic weight value β2 of the behavioral risk probability; when the determined regional risk level is greater than the second threshold, the dynamic weight value of the physiological risk probability > the dynamic weight value α3 of the location risk probability + the dynamic weight value β3 of the behavioral risk probability, γ3 > γ2.

[0105] In other implementations, a mapping relationship can be established between each area type and the dynamic weight values ​​of the location risk probability, physiological risk probability, and behavioral risk probability. For example, in a normal area, where location data is dominant, the dynamic weights of the corresponding location risk probability, physiological risk probability, and behavioral risk probability are α, γ, and β, respectively, where α is 0.5, β is 0.3, and γ is 0.2; in a high-risk area, the physiological weight is increased, with α being 0.3, β being 0.2, and γ being 0.5; and in a leakage accident area, the physiological weight is further increased, with α being 0.1, β being 0.1, and γ being 0.8.

[0106] Furthermore, it should be noted that the dynamic weighting value of physiological risk probability is not only related to the region type or region risk level, but also often to time. For example, the degree of fatigue of the person being assessed needs to be considered in conjunction with the duration of the fatigue to determine the magnitude of their risk probability.

[0107] In step S500, the risk value of the person to be assessed is calculated based on the dynamic weight values ​​of the determined location risk probability, physiological risk probability, and behavioral risk probability.

[0108] It should be noted that the formula for calculating the risk value of the person being assessed is as follows:

[0109] R = α·R loc +β·R beh +γ·δ(t)·R phy ;

[0110] Where R is the risk value of the person to be assessed;

[0111] α represents the weight of the probability of location risk;

[0112] β is the weight of the probability of behavioral risk;

[0113] γ is the weight of the physiological risk probability;

[0114] R loc To determine the probability of risk;

[0115] Rbeh Probability of behavioral risk;

[0116] R phy For physiological risk probability;

[0117] δ(t) is the time factor, where t is the duration of the person being assessed. Taking the physiological risk probability based on the heart rate variability coefficient as an example, t can be the duration of the heart rate abnormality. The longer t is, the higher the weight of the corresponding physiological risk probability.

[0118] The formula for calculating the time factor is as follows:

[0119]

[0120] δ0 is the weight value of the person to be evaluated when entering the area and staying for 0 hours.

[0121] Taking the physiological risk probability based on the heart rate variability coefficient as an example, the formula for calculating the physiological risk probability is:

[0122]

[0123] Among them, R phy For physiological risk probability;

[0124] HRV is the real-time heart rate variability coefficient.

[0125] k is the sensitivity coefficient to small changes in physiological quantities;

[0126] θ represents the risk threshold of the area where the person to be assessed is located.

[0127] In addition, the method also includes step S600, which sends corresponding early warning measures based on the calculated risk value of the person to be assessed.

[0128] It is important to note that the calculated risk values ​​of the personnel to be assessed are normalized, and then the response is tiered according to the risk value. Different calculated risk values ​​correspond to different early warning measures. For example, if the calculated risk value is between 0 and 0.3, the corresponding early warning measure is no action; if the calculated risk value is between 0.3 and 0.6, the corresponding early warning measure is a pre-alarm, pushing a screenshot of the risk location to the safety officer and avoiding false alarms that could interfere with the control room; if the calculated risk value is between 0.6 and 0.8, the corresponding early warning measure is a Level 1 alarm, automatically broadcasting a voice warning (e.g., "Please evacuate from Zone B"), using audible and visual alarms to adapt to high-noise environments; if the calculated risk value is greater than 0.8, the corresponding early warning measure is a Level 2 alarm, activating the emergency plan and sending escape information to the individual's terminal. It is important to note that the escape route should avoid the leakage diffusion area.

[0129] This invention provides a method for acquiring location data, video data, and physiological data of a person to be assessed; determining the behavioral risk probability and physiological risk probability of the person to be assessed based on the video data and physiological data, respectively; determining the location area and location risk probability of the person to be assessed based on the location data; determining the dynamic weight values ​​of the location risk probability, physiological risk probability, and behavioral risk probability of the person to be assessed based on the determined location area; and calculating the risk value of the person to be assessed based on the determined dynamic weight values ​​of the location risk probability, physiological risk probability, and behavioral risk probability. This enables scenario-based risk perception, overcomes data silo bottlenecks, improves the accuracy and timeliness of composite risk identification and early warning, and reduces the incidence of major accidents.

[0130] To achieve the above objectives, the present invention also provides a computer device, such as... Figure 2 As shown, the computer device includes at least one processor 701; and a memory 702 communicatively connected to at least one processor 701; wherein the memory 702 stores instructions executable by at least one processor 701, the instructions being executed by at least one processor 701 to enable at least one processor 701 to perform the aforementioned personnel risk assessment method based on multimodal data fusion.

[0131] The memory 702 and processor 701 are connected via a bus, which can include any number of interconnecting buses and bridges. The bus connects various circuits of one or more processors 701 and memory 702 together. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 701 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 701.

[0132] Processor 701 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 702 can be used to store data used by processor 701 during operation.

[0133] To achieve the above objectives, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor 701, implements the above-described method for assessing personnel risk based on multimodal data fusion.

[0134] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0135] Obviously, the embodiments described above are merely some, not all, embodiments of the present invention. Based on the embodiments of the present invention, those skilled in the art can make other variations or modifications without creative effort, and all such variations or modifications should fall within the scope of protection of the present invention.

Claims

1. A method for assessing personnel risk based on multimodal data fusion, characterized in that, include: Obtain location data, video data, and physiological data of the person to be evaluated; Based on the video data and physiological data, the behavioral risk probability and physiological risk probability of the person to be assessed are determined respectively. Based on the location data, the location of the person to be assessed and the probability of location risk are determined; Based on the determined area where the person to be assessed is located, dynamic weight values ​​for the location risk probability, physiological risk probability, and behavioral risk probability of the person to be assessed are determined respectively. The risk value of the person to be assessed is calculated based on the dynamic weight values ​​of the determined location risk probability, physiological risk probability, and behavioral risk probability.

2. The personnel risk assessment method based on multimodal data fusion as described in claim 1, characterized in that, In the step of calculating the risk value of the person to be assessed based on the determined dynamic weight values ​​of the location risk probability, physiological risk probability, and behavioral risk probability, the calculation formula is as follows: R=α·R loc +β·R beh +γ·δ(t)·R phy ; Where R is the risk value of the person to be assessed; α represents the weight of the probability of location risk; β is the weight of the probability of behavioral risk; γ is the weight of the physiological risk probability; R loc To determine the probability of risk; R beh Probability of behavioral risk; R phy For physiological risk probability; δ(t) is the time factor, where t is the duration of stay of the person being evaluated.

3. The personnel risk assessment method based on multimodal data fusion as described in claim 2, characterized in that, The step of determining the dynamic weight values ​​of the location risk probability, physiological risk probability, and behavioral risk probability of the person to be assessed based on the determined area of ​​the person to be assessed includes: Based on the identified area where the person to be assessed is located, determine the regional risk level of that area. Based on the determined regional risk level, dynamic weight values ​​for the location risk probability, physiological risk probability, and behavioral risk probability corresponding to the regional risk level are determined.

4. The personnel risk assessment method based on multimodal data fusion as described in claim 3, characterized in that, The step of determining the regional risk level of the region where the person to be assessed is located, based on the region where the person to be assessed is located, includes: Based on the identified area where the person to be assessed is located, obtain the historical hazardous events and corresponding hazard levels within that area; Based on the historical hazardous events and their corresponding hazard levels, the regional risk level of the area where the person to be assessed is located is calculated using the following formula: The risk level of the area; x1, x2, ..., x n The danger levels corresponding to danger events 1, 2, ..., n are: n represents the number of dangerous events within a preset time period in the region; f1, f2, ..., f n They are x1, x2, ..., x n The corresponding weights.

5. The personnel risk assessment method based on multimodal data fusion as described in claim 3, characterized in that, The step of determining dynamic weight values ​​for the location risk probability, physiological risk probability, and behavioral risk probability corresponding to the determined regional risk level includes: When the determined regional risk level is less than the first threshold, the dynamic weight value α1 of the location risk probability is greater than the dynamic weight value β1 of the behavioral risk probability, which is greater than the dynamic weight value γ1 of the physiological risk probability. When the determined regional risk level is greater than the first threshold and less than the second threshold, the dynamic weight value γ2 of the physiological risk probability is greater than the dynamic weight value α2 of the location risk probability and the dynamic weight value β2 of the behavioral risk probability. When the determined regional risk level is greater than the second threshold, the dynamic weight value of physiological risk probability is greater than the dynamic weight value of location risk probability α3 + the dynamic weight value of behavioral risk probability β3, and γ3 > γ2.

6. The personnel risk assessment method based on multimodal data fusion as described in claim 2, characterized in that, The formula for calculating the risk value of the person to be assessed based on the determined dynamic weight values ​​of location risk probability, physiological risk probability, and behavioral risk probability includes the following formula for calculating the time factor: δ0 is the weight value of the person to be evaluated when entering the area and staying for 0 hours.

7. The personnel risk assessment method based on multimodal data fusion as described in claim 2, characterized in that, The formula for calculating the risk value of the person to be assessed based on the determined dynamic weight values ​​of location risk probability, physiological risk probability, and behavioral risk probability is as follows: Among them, R phy For physiological risk probability; HRV is the real-time heart rate variability coefficient; k is the sensitivity coefficient to small changes in physiological quantities; θ represents the risk threshold of the area where the person to be assessed is located.

8. The personnel risk assessment method based on multimodal data fusion as described in claim 1, characterized in that, The physiological risk probability is determined based on the degree of eye fatigue. The method further includes: Based on video data, determine the open and closed state of the eyes of the person to be evaluated within a preset time period; The vibration signals of the muscles around the eyes are acquired, and an evaluation index for assessing the degree of eye fatigue is output based on a pre-trained evaluation model. The fatigue level of the person being evaluated is determined based on the obtained evaluation indicators and the opening and closing status of the eyes within a preset time. Determine the probability of physiological risk based on the level of fatigue of the person being assessed; The evaluation model is as follows: Where x(n) represents the nth sample value of the vibration signal; w(n) is the residual value of white noise; p is the order of the AR model; a k It is the k-th coefficient of the evaluation model, i.e., the parameter to be determined.

9. A computer device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the personnel risk assessment method based on multimodal data fusion as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the personnel risk assessment method based on multimodal data fusion as described in any one of claims 1 to 8.

Images