Dangerous source identification reasoning device and method based on image identification

Abstract

The invention discloses a dangerous source identification and reasoning device and method based on image identification. The device comprises a data acquisition and analysis module, a dangerous object extraction module, a dangerous source reasoning module and an early warning processing module. The data acquisition and analysis module collects and preprocesses the identified image data, stores the image data to a basic data set, and then transmits the data to the dangerous object extraction module; the dangerous object extraction module constructs a dangerous object comparison library, compares the basic data set with the dangerous object comparison library, and fuses attributes, confidence, coordinates and rectangular frame data of dangerous objects to form a dangerous object data set; the danger source reasoning module is used for extracting data of objects in the danger data set in a time period tn, calculating an area change trend and a direction change trend in the time period tn, reasoning and calculating a time point when danger occurs again, analyzing danger types and transmitting the danger types to the early warning processing module; and the early warning processing module is used for establishing a danger level, carrying out judgment, emergency pushing and emergency processing on the danger level, and providing and processing information of related departments. According to the invention, danger occurrence data can be provided for advanced early warning and emergency processing, efficient and rapid safety management and control are improved, and the possibility of occurrence of dangerous events is reduced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a hazard source identification and reasoning device and method based on image recognition. Background Art

[0002] Machine vision technology has developed rapidly in recent years. With the continuous improvement of technology, the fields of application are also expanding, and it has had a profound impact on many industries. Currently, those involved include but are not limited to visual navigation, autonomous driving, medical care, security, etc.

[0003] In the industrial field, machine vision technology is widely used in processing positioning, sorting, measurement, defect detection, environmental monitoring and other aspects. Machine vision technology is constantly improving the intelligence of production, and it also plays an important role in some dangerous working environments or occasions that are difficult to complete manually. It is also widely used in the field of hazardous materials control. Intelligent vision technology is used to achieve real-time monitoring and early warning of hazardous chemicals, improve the management quality of hazardous materials, and effectively promote the healthy and stable development of the industry. With the widespread application of machine vision technology in various fields, it not only improves the rapid development and safety of society, but also reduces labor costs and the possibility of misjudgment. This shows that the importance of machine vision technology will become higher and higher and its applicability will become wider and wider.

[0004] With the continuous development and progress of society, there are more and more dangerous objects in all walks of life, and the resulting security pressure is also increasing. At present, machine vision technology has been fully adopted in the field of security monitoring, and has been applied in personnel, behavior, environment, patrol, production, etc. The monitoring environment that has been established and applied is optimized and used, and the image data processed by machine vision is analyzed and processed again, and the results of image recognition are further optimized to infer various potential safety hazards or developments in the future, so as to improve the intelligence level of security monitoring and the efficiency and ability of emergency response to security prevention. Summary of the invention

[0005] Purpose of the invention: The present invention provides a hazard source identification and reasoning device and method based on image recognition, which can provide hazard occurrence data for advance warning and emergency processing, improve efficient and rapid safety management and control, and reduce the possibility of dangerous events.

[0006] Technical solution: The hazard source identification and reasoning device based on image recognition described in the present invention includes: a data acquisition and analysis module, a hazard extraction module, a hazard source reasoning module and an early warning processing module; the data acquisition and analysis module collects and preprocesses the image data it identifies, saves it to a basic data set, and then transmits it to the hazard extraction module; the hazard extraction module builds a hazard comparison library, compares the basic data set with it, and fuses the attributes, confidence, coordinates and rectangular frame data of the hazard to form a hazard data set; the hazard source reasoning module calculates the time period t of the object in the hazard data set n Extract the data within and calculate the time period t n Based on the changing trend of the internal area and direction, the time point when the danger will occur again is inferred and calculated, the type of danger is analyzed, and the information is transmitted to the early warning processing module; the early warning disposal module establishes the danger level, determines the degree of danger, pushes emergency messages, handles emergencies, and provides information to relevant departments.

[0007] Accordingly, a hazard source identification reasoning method based on image recognition includes the following steps:

[0008] Step 1: Receive image data through the interface in the data acquisition and analysis module, and identify and collect the image data;

[0009] Step 2: preprocess all collected data, the main purpose of which is to exclude objects with low probability of existing in the image bounding box;

[0010] Step 3: Through the comparison calculation method of the dangerous object extraction module, the object attribute T in the basic data set is compared and matched with the constructed dangerous object comparison library, and the attribute T, confidence C, center position O(u, v) of the successfully matched dangerous object and the length x width y data of the object rectangular box are combined to form a dangerous object data set;

[0011] Step 4: Calculate and infer the change trend of the dangerous objects. By calculating the area change and direction change of the dangerous objects in the dangerous object data set, according to the constructed dangerous object relationship model, calculate the meeting time of two or more related dangerous objects in the dangerous object relationship model, and infer the time node when the major danger occurs;

[0012] Step 5: Calculate the degree of danger and plan the danger level based on the equivalent of the dangerous substance and the distribution of the surrounding population. According to the danger level, analyze the control measures and provide emergency treatment plans.

[0013] Furthermore, in step 1, the image data is identified and collected, and the data of all objects are extracted, including the attribute T, the confidence C, the center position O(u, v), and the length x, width y of the object's rectangular box.

[0014] Furthermore, in step 2, in time period tn Extract N frames of data for the object, calculate the standard deviation σ of all confidence levels C, and when the standard deviation σ is between 0 and β, compare the average value μ with the set comparison value C0 to determine the data close to the real object, and store the object data in the basic data set;

[0015] Standard deviation calculation formula:

[0016]

[0017] Where: σ is the standard deviation; C n is the value of the nth confidence level C; μ is the average value of confidence level C; N is the total number of frames, which is equal to the total number of confidence levels C, n; β is the set discreteness calibration value; first calculate the average value μ and standard deviation σ, and judge the discreteness by β-σ>0. If the result is less than 0, it is judged to be a misrecognition; if the result is greater than 0, then use μ-C0>0 to judge whether it is data close to the real object. If the result is greater than 0, put the object data into the basic data set. If multiple objects are recognized, perform the above calculations in sequence.

[0018] Furthermore, in step 3, the object attribute T in the basic data set is compared and matched with the constructed dangerous object comparison library through the comparison calculation method of the dangerous object extraction module, and the attribute T, confidence C, center position O(u, v) of the successfully matched dangerous object and the length x width y data of the object rectangular frame are combined to form a dangerous object data set, which specifically includes the following steps:

[0019] Step 31: Based on the provisions of the national standard "List of Dangerous Goods" (GB12268), a large amount of initial basic data is collected for the nine major categories of dangerous goods, and the initial basic data is divided into nine regions according to the nine major categories and the regions are marked to obtain a basic dangerous goods comparison library;

[0020] Step 32: Collect and obtain the public attributes of dangerous goods, classify them according to the types of dangerous goods, compare, supplement and enrich the corresponding 9 major categories of dangerous goods, and then make final numerical annotations according to the specific attributes of the specific area, so that the corresponding attributes T n becomes a fixed value Q n , in order to optimize, improve and expand the basic hazardous material comparison library, continuously obtain the properties of hazardous materials for multiple verifications to determine whether the comparison and identification are successful; if there are unidentified hazardous materials, they will be classified and labeled again, and this process will be repeated to finally obtain a more complete hazardous material comparison library;

[0021] Step 33, Dangerous object extraction: Object attributes are extracted from the basic data set in turn, and the attribute T of the object is matched and labeled with the dangerous object comparison library. The matching calculation formula is as follows to determine whether T exists in the dangerous object comparison library, and finally form a dangerous object data set:

[0022]

[0023] Where: Q is the hazard identification index value; T n is the matching annotation value of object n in the basic dataset; Q1, Q2, ..., Q n is the attribute annotation value of the dangerous object in the dangerous object comparison library. When the attribute of the object is equal to the attribute annotation value of the dangerous object in the dangerous object comparison library, the identification index value will be equal to 1, which means that object n is a dangerous object. This will be repeated until all objects in the basic data set are calculated above, and then all object data with identification index values ​​equal to 1 are extracted to form a dangerous object data set.

[0024] Furthermore, in step 4, the area change is calculated as follows:

[0025]

[0026] Where: S n is the time period t n S0 is the area of ​​the dangerous object at the initial moment, and the area is calculated according to the above formula using the length x width y of the dangerous object rectangle; k is the area growth or decay index of the dangerous object, which determines the rate at which the area changes over time; t n is the time period; e is the base of the natural logarithm, approximately equal to 2.71828; by n The dangerous object data between the two is taken, and the index k is calculated according to the above calculation formula. If k is positive, the area increases with time; if k is negative, the area decreases with time; if k is zero, the area remains unchanged, and the area value that changes with time is calculated by combining the index;

[0027] By obtaining the length and width of the rectangular frame of the dangerous object in time period t, we know the areas S1 = x1·y1 and S2 = x2·y2 at the specific start time t1 and end time t2, and find the specific value of k;

[0028] If there is hazardous object 1 and its associated hazardous object 2 in the hazardous object relationship database, then after the above calculation, judgment method 1 is used for judgment. Otherwise, judgment method 2 is used to judge hazardous object 1 alone to determine whether it is a hazard source.

[0029] Further, judgment method 1 is as follows:

[0030] k1>0 or k2>0, calculate the direction change;

[0031] It is considered that the hazard source has weakened, and the time when the area is equal to 0 is calculated by the area change, and then step 5 is carried out;

[0032] If k1=0 and k2=0, it is not considered as a hazard source;

[0033] Judgment method 2 is as follows:

[0034] If k1>0, then hazardous object 1 is regarded as a hazardous source and the direction change calculation is performed;

[0035] If k1<0, then hazardous object 1 is considered as a weakened hazard source, and the time when the area is equal to 0 is calculated by the area change, and then step 5 is performed;

[0036] If k1=0, dangerous object 1 is not considered as a hazard source.

[0037] Furthermore, the direction change is calculated as follows:

[0038]

[0039] Where: D n is the dangerous object in time period t n D0 is the angle of change of the direction of the dangerous object at the initial moment; K is the direction change rate index of the dangerous object, which can be positive (indicating increase) or negative (indicating decrease) according to the actual situation; t n is the time period; e is the base of the natural logarithm, approximately equal to 2.71828; by n The dangerous object data value is taken, and the direction change angle and index K are calculated according to the above formula n If K is positive, it means that the hazard source direction change index increases positively over time; if K is negative, it decreases in the opposite direction over time; if K is zero, the direction remains unchanged at the initial angle, and the direction angle value that changes over time is calculated in combination with the index;

[0040] Assume that the centers of the dangerous object at time points t1 and t2 are O1(u1, v1) and O2(u2, v2) respectively, and then calculate the angle of change from the origin O to point O1 and point O2. Assume that this angle is θ, the vector The modulus lengths of vector OO1 and vector OO2 are and The angle θ between the two vectors is given by It is calculated that, therefore, the angle θ is obtained by Calculate and convert the result from radians to degrees to get Thus, the data at time point t3 is obtained, and the specific value of direction D3 is obtained by combining the data at time point t1, and then the specific value of K is calculated to calculate the direction change;

[0041] If there are two objects in the dangerous object data set, the direction change angles D1 of dangerous object 1 and dangerous object 2 are calculated respectively.n and D2 n , if both are positive or both are negative, it means the same direction, and it is possible to approach unilaterally; if one is positive and the other is negative, it means the opposite direction, and it is possible to approach quickly; then the intersection time is calculated;

[0042] The intersection time is calculated as follows:

[0043]

[0044]

[0045] Where: r is the distance from the origin of the dangerous object to the rectangular frame; A n min is the area of ​​the dangerous object covering the lower left corner coordinate; A n max is the upper right corner coordinate of the area covered by the dangerous object; t n The time period data is brought into the above formula to calculate the maximum and minimum coordinates of the area coverage.

[0046]

[0047]

[0048]

[0049]

[0050] In contrast, the calculated coordinates are at t n Whether there is any intersection within the time period is determined and calculated using judgment method 3.

[0051] Further, judgment method 3: comparison with A1 n min and A1 n Is there any overlap between the two coordinate ranges of maxA2 n min and A2 n max. If the area coverage coordinate data of dangerous object 1 is equal to the area coverage coordinate data of dangerous object 2, then calculate the specific value of the intersection time t. If the specific time t can be calculated, there is an intersection and it is determined to be a major hazard source. Otherwise, there is no intersection between the dangerous objects, but it is only a hazard source. After completing the calculation of the intersection time t, transmit the data of the hazard source and the intersection time t to step 5 to analyze the specific hazard level.

[0052] Furthermore, in step 5, the hazard level classification index is calculated based on the hazard relationship model, and the specific calculation formula is as follows;

[0053] The hazard level index is calculated as follows:

[0054]

[0055] Where: R is the hazard level classification index; α is the correction coefficient for personnel, which is determined according to the population distribution around the hazard source (if the attribute is identified as a person, the number is calculated statistically, if not, it is 1); ψ1, ψ2, ..., ψ n S1 is the correction coefficient corresponding to the hazard source. According to the 9 categories specified in the "List of Dangerous Goods", different values ​​are assigned to it when constructing the dangerous goods comparison library. These coefficients reflect the degree of danger of different hazard sources; n , S2 n , ..., S n is the t of each hazard source n The actual area of ​​the time period; P1, P2, ..., Pn are the critical quantities corresponding to each hazard source, which are assigned different values ​​when constructing the hazardous material comparison library. The critical quantity refers to a specified threshold value. When the threshold value is exceeded, the existence of the hazard source is determined to be a major hazard source;

[0056] According to the specific value of the danger level classification index R, the danger level classification is completed, and emergency push is planned according to its danger level.

[0057] Beneficial effect: Compared with the prior art, the present invention has the following significant advantages: it collects and analyzes image data analyzed by machine vision image analysis, can analyze the specific dangerous objects in the image, and then use the hazard source identification reasoning method to infer the changing trend of the dangerous objects, and analyze whether there is a hazard source based on the reasoning, and finally provide the evolution data of the hazard source and the disposal method of the hazard source, providing hazard occurrence data for advance warning and emergency processing, thereby improving efficient and rapid safety management and reducing the possibility of dangerous incidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of the device structure of the present invention.

[0059] Figure 2 This is a schematic diagram of the dangerous substance relationship network of the present invention.

[0060] Figure 3 This is a schematic diagram of emergency push planning based on the danger level of the present invention. DETAILED DESCRIPTION

[0061] like Figure 1As shown, a hazard source identification and reasoning device based on image recognition includes: a data acquisition and analysis module, a hazard extraction module, a hazard source reasoning module and an early warning processing module; the data acquisition and analysis module collects and preprocesses the image data it recognizes, saves it to a basic data set, and then transmits it to the hazard extraction module; the hazard extraction module builds a hazard comparison library, compares the basic data set with it, and fuses the attributes, confidence, coordinates and rectangular frame data of the hazard to form a hazard data set; the hazard source reasoning module calculates the time period t of the object in the hazard data set n Extract the data within and calculate the time period t n Based on the changing trend of the internal area and direction, the time point when the danger will occur again is inferred and calculated, the type of danger is analyzed, and the information is transmitted to the early warning processing module; the early warning disposal module establishes the danger level, determines the degree of danger, pushes emergency messages, handles emergencies, and provides information to relevant departments.

[0062] Accordingly, a hazard source identification reasoning method based on image recognition includes the following steps:

[0063] Step 1: Receive image data through the interface in the data acquisition and analysis module, identify and collect the image data, and extract the data of all objects, including attributes T, confidence C, center position O(u, v), and length x, width y of the object's rectangular box.

[0064] Step 2: Preprocess all collected data, the main purpose of which is to exclude objects with low probability of existing in the image bounding box. n N frames of data are extracted for the object, and the standard deviation σ of all confidence levels C is calculated. When the standard deviation σ is between 0 and β, the average value μ is compared with the set comparison value C0 to determine the data close to the real object, and the object data is stored in the basic data set.

[0065] Formula 1-1: Standard deviation calculation formula

[0066]

[0067] Where: σ is the standard deviation; C n is the value of the nth confidence level C; μ is the average value of confidence level C; N is the total number of frames, which is equal to the total number of confidence levels C, n; β is the set discreteness calibration value. First, calculate the average value μ, and then use formula 1-1 to calculate the standard deviation σ, and determine the discreteness by β-σ>0. If the result is less than 0, it can be determined that it is a misidentification. If the result is greater than 0, then use μ-C0>0 to determine whether it is data close to the real object. If the result is greater than 0, the object data is placed in the basic data set. If multiple objects are identified, the above calculations can be performed in sequence.

[0068] Step 3: Through the comparative calculation method of the hazardous object extraction module, the object attributes T in the basic data set are compared and matched with the constructed hazardous object comparison library, and the attributes T, confidence C, center position O(u, v) of the successfully matched hazardous objects and the length x width y data of the object rectangular box are combined to form a hazardous object data set.

[0069] The dangerous goods comparison library is based on the national standard "Dangerous Goods List" (GB12268). Through the first step of data collection and the second step of data comparison and enrichment, all dangerous goods are formed into a dangerous goods comparison library. The specific steps are as follows:

[0070] Step 1: There are 9 categories in the national standard "List of Dangerous Goods". A large amount of initial basic data is collected for each of the 9 categories of dangerous goods. The initial basic data is divided into 9 regions according to the 9 categories and the regions are marked to obtain a basic dangerous goods comparison library, such as Figure 2 shown.

[0071] Step 2: Collect and obtain the public attributes of dangerous goods, classify them according to the types of dangerous goods, compare, supplement and enrich the corresponding 9 major categories of dangerous goods, and then make the final numerical annotation according to the specific attributes of the specific area, so that the corresponding attributes T n becomes a fixed value Q n , in order to optimize, improve and expand the basic hazardous material comparison library. The properties of hazardous materials are continuously obtained for multiple verifications to determine whether the comparison and identification are successful. If there are unidentified hazardous materials, they will be classified and labeled again. This process is repeated to finally obtain a more complete hazardous material comparison library.

[0072] Dangerous object extraction mainly involves extracting object attributes from the basic data set in turn, matching and annotating the object's attribute T with the dangerous object comparison library. The matching calculation formula is shown below to determine whether T exists in the dangerous object comparison library, and finally form a dangerous object data set.

[0073] Formula 1-2:

[0074]

[0075] Where: Q is the hazard identification index value; T n is the matching annotation value of object n in the basic dataset; Q1, Q2, ..., Q n is the attribute annotation value of the dangerous object in the dangerous object comparison library. When the attribute of the object is equal to the attribute annotation value of the dangerous object in the dangerous object comparison library, the identification index value will be equal to 1, indicating that object n is a dangerous object. This is repeated until all objects in the basic data set are calculated above, and then all object data with identification index values ​​equal to 1 are extracted to form a dangerous object data set.

[0076] Step 4: Calculate and infer the changing trend of the dangerous objects. By calculating the area change and direction change of the dangerous objects in the dangerous object data set, according to the constructed dangerous object relationship model, calculate the time when two or more related dangerous objects in the dangerous object relationship model meet, and infer the time node when the major danger occurs. Before calculating and inferring, it is necessary to obtain the specific data of dangerous object 1 at time t1 and time t2, including the length x width y of the rectangular box containing the dangerous object, which are x1, y1, x2, y2 respectively, and the center position O1 (u1, v1) and O2 (u2, v2) are used for the following reasoning calculations.

[0077] The main conditions for building a dangerous substance relationship network are based on the dangers caused by the isolation, hazards, evolution and other changes of dangerous sources stipulated in the "Regulations on the Safety Management of Hazardous Chemicals", "Work Safety Law of the People's Republic of China" and "Regulations on the Safety Management of Radioactive Materials Transport". The related dangerous substance relationship network is constructed to support the following calculation reasoning, as follows Figure 1 shown.

[0078] As shown in the figure above, flammable and explosive items cannot be close to open flames, otherwise they will explode violently, which is a major hazard source. Or some chemicals cannot be exposed to open flames, otherwise they will burn and release toxic gases. Or flammable and explosive items cannot be in contact with toxic substances, which are prone to explosions and air pollution. All items that intersect and produce huge dangers need to be associated to determine whether there is a possibility of evolving into a major hazard source.

[0079] The formulas for calculating the area change and direction change of dangerous objects are shown in Formula 1-3 and Formula 1-4. The formula for calculating the intersection time after the dangerous object relationship is determined is shown in Formula 1-5, which is used to infer whether there is a gathering time for multiple related dangerous objects and to determine the evolution of the hazard source.

[0080] Formula 1-3: The area change is calculated as follows

[0081]

[0082] Where: S n is the time period t n S0 is the area of ​​the dangerous object at the initial moment, and the area is calculated according to the above formula using the length x width y of the dangerous object rectangle; k is the area growth or decay index of the dangerous object, which determines the rate at which the area changes over time; t n is the time period; e is the base of the natural logarithm, which is approximately equal to 2.71828. n The dangerous object data between the two values ​​is taken, and the index k is calculated according to the above calculation formula. If k is positive, the area increases over time; if k is negative, the area decreases over time; if k is zero, the area remains unchanged. Then the area value that changes over time is calculated by combining the index.

[0083] By obtaining the length and width of the rectangular frame of the dangerous object in time period t, we can know the areas S1=x1·y1 and S2=x2·y2 at the specific starting time points t1 and ending time points t2. Substitute them into formula 3 to find the specific value of k.

[0084] If there is hazardous object 1 and its associated hazardous object 2 in the hazardous object relationship database, then after the above calculation, judgment method 1 is used for judgment. Otherwise, judgment method 2 is used to judge hazardous object 1 alone to determine whether it is a hazard source.

[0085] Judgment method 1 is as follows:

[0086] k1>0 or k2>0, perform calculation according to formula 1-4;

[0087] It is considered that the hazard source has weakened, and the time when the area is equal to 0 is calculated by formula 1-3, and then step 5 is carried out;

[0088] If k1=0, k2=0, it is not considered as a hazard source.

[0089] Judgment method 2 is as follows:

[0090] If k1>0, then hazardous substance 1 is regarded as a hazardous source and formula 1-4 is used;

[0091] If k1<0, then hazardous object 1 is considered as a weakened hazard source, and the time when the area is equal to 0 is calculated by formula 1-3, and then step 5 is performed;

[0092] If k1=0, dangerous object 1 is not considered as a hazard source.

[0093] Formula 1-4: The direction change is calculated as follows

[0094]

[0095] Where: D n is the dangerous object in time period t n D0 is the angle of change of the direction of the dangerous object at the initial moment; K is the direction change rate index of the dangerous object, which can be positive (indicating growth) or negative (indicating decrease) according to the actual situation; t n is the time period; e is the base of the natural logarithm, which is approximately equal to 2.71828. n The dangerous object data value is taken, and the direction change angle and index K are calculated according to the above formula n If K is positive, it means that the hazard source direction change index increases positively over time; if K is negative, it decreases in the opposite direction over time. If K is zero, the direction remains unchanged at the initial angle. Combined with the index, the direction angle value that changes over time is calculated.

[0096] Suppose the centers of the dangerous object at two time points t1 and t2 are O1(u1, v1) and O2(u2, v2), respectively, and then calculate the angle of change from the origin O to O1 and O2. Assuming this angle is θ, the vector The modulus lengths of vector OO1 and vector OO2 are and The angle θ between the two vectors can be expressed by It is calculated that, therefore, the angle θ can be obtained by Calculate and convert the result from radians to degrees to get Thus, the data at time point t3 is obtained, and the specific value of direction D3 is obtained by combining it with the data at time point t1, and then substituted into formula 1-4 to calculate the specific value of K for calculating the direction change.

[0097] If there are two objects in the dangerous object data set, the direction change angles D1 of dangerous object 1 and dangerous object 2 are calculated respectively. n and D2 n If both are positive or negative, it means the direction is forward and it is possible to approach unilaterally; if one is positive and the other is negative, it means the direction is opposite and it is possible to approach quickly; then use formulas 1-5 and 1-6 for calculation.

[0098] Formula 1-5, 1-6: The intersection time is calculated as follows

[0099]

[0100]

[0101] Where: r is the distance from the origin of the dangerous object to the rectangular frame; A n main is the area of ​​the dangerous object covering the lower left corner coordinate; A n max is the upper right corner coordinate of the area covered by the dangerous object; t n The time period data is brought into the above formula to calculate the maximum and minimum coordinates of the area coverage.

[0102]

[0103]

[0104]

[0105]

[0106] In contrast, the calculated coordinates are at t nWhether there is any intersection within the time period is determined and calculated using judgment method 3.

[0107] Judgment method 3: Comparison with A1 n main and A1 n Is there any overlap between the two coordinate ranges of maxA2 n main and A2 n max. If the area coverage coordinate data of dangerous object 1 is equal to the area coverage coordinate data of dangerous object 2, then calculate the specific value of the intersection time t. If the specific time t can be calculated, there is an intersection and it is determined to be a major hazard source. Otherwise, there is no intersection between the dangerous objects, but it is only a hazard source. After completing the calculation of the intersection time t, transmit the data of the hazard source and the intersection time t to step 5 to analyze the specific hazard level.

[0108] Step 5: Calculate the degree of danger and the planned danger level based on the equivalent of dangerous objects and the distribution of surrounding population. According to the danger level, analyze the means of control and provide emergency treatment plans. Calculate the danger level classification index based on the danger relationship model. The specific calculation formula is as follows.

[0109] Formula 1-11: The hazard level index is calculated as follows

[0110]

[0111] Where: R is the hazard level classification index; α is the correction coefficient for personnel, which is determined according to the population distribution around the hazard source (if the attribute is identified as a person, the number is calculated statistically, if not, it is 1); ψ1, ψ2, ..., ψ n S1 is the correction coefficient corresponding to the hazard source. According to the 9 categories specified in the "List of Dangerous Goods", different values ​​are assigned to it when constructing the dangerous goods comparison library. These coefficients reflect the degree of danger of different hazard sources; n , S2 n , ..., S n is the t of each hazard source n The actual area of ​​the time period; P1, P2, ..., Pn are the critical quantities corresponding to each hazard source, and different values ​​are assigned to them when constructing the hazardous material comparison library. The critical quantity refers to a specified threshold value. When the threshold value is exceeded, the existence of the hazard source is determined to be a major hazard source.

[0112] According to the specific value of the hazard level classification index R, the hazard level classification is completed, and the emergency push is planned according to the hazard level, such as Figure 3 shown.

Claims

1. A hazard source identification and reasoning device based on image recognition, characterized in that: include: Data collection and analysis module, hazardous material extraction module, hazard source reasoning module and early warning processing module; The data acquisition and analysis module collects and preprocesses the image data it recognizes, saves it to the basic data set, and then transmits it to the dangerous object extraction module; the dangerous object extraction module builds a dangerous object comparison library, compares the basic data set with it, and fuses the attributes, confidence, coordinates and rectangular frame data of the dangerous object to form a dangerous object data set; the dangerous source reasoning module calculates the time period t of the object in the dangerous object data set n Extract the data within and calculate the time period t n The change trend of the internal area and direction is calculated to infer the time point when the danger will occur again, analyze the type of danger, and transmit it to the early warning processing module; The early warning and disposal module establishes danger levels, determines the degree of danger, pushes emergency notifications, handles emergencies, and provides information to relevant departments.

2. A reasoning method based on the hazard source identification reasoning device based on image recognition as claimed in claim 1, characterized in that: The steps include: Step 1: Receive image data through the interface in the data acquisition and analysis module, and identify and collect the image data; Step 2: preprocess all collected data, the main purpose of which is to exclude objects with low probability of existing in the image bounding box; Step 3: Through the comparison calculation method of the dangerous object extraction module, the object attribute T in the basic data set is compared and matched with the constructed dangerous object comparison library, and the attribute T, confidence C, center position O(u, v) of the successfully matched dangerous object and the length x width y data of the object rectangular box are combined to form a dangerous object data set; Step 4: Calculate and infer the change trend of the dangerous objects. By calculating the area change and direction change of the dangerous objects in the dangerous object data set, according to the constructed dangerous object relationship model, calculate the meeting time of two or more related dangerous objects in the dangerous object relationship model, and infer the time node when the major danger occurs; Step 5: Calculate the degree of danger and plan the danger level based on the equivalent of the dangerous substance and the distribution of the surrounding population. According to the danger level, analyze the control measures and provide emergency treatment plans.

3. The hazard source identification reasoning method based on image recognition as claimed in claim 2 is characterized in that: In step 1, the image data is identified and collected to extract the data of all objects, including the attribute T, the confidence C, the center position O(u, v), and the length x, width y of the object's rectangular box.

4. The hazard source identification reasoning method based on image recognition as claimed in claim 2 is characterized in that: In step 2, during time period t n Extract N frames of data for the object, calculate the standard deviation σ of all confidence levels C, and when the standard deviation σ is between 0 and β, compare the average value μ with the set comparison value C0 to determine the data close to the real object, and store the object data in the basic data set; Standard deviation calculation formula: Where: σ is the standard deviation; C n is the value of the nth confidence level C; μ is the average value of confidence level C; N is the total number of frames, which is equal to the total number of confidence levels C, n; β is the set discreteness calibration value; first calculate the average value μ and standard deviation σ, and judge the discreteness by β-σ>0. If the result is less than 0, it is judged to be a misrecognition; if the result is greater than 0, then use μ-C0>0 to judge whether it is data close to the real object. If the result is greater than 0, put the object data into the basic data set. If multiple objects are recognized, perform the above calculations in sequence.

5. The hazard source identification and reasoning method based on image recognition as claimed in claim 2 is characterized in that: In step 3, the object attribute T in the basic data set is compared and matched with the constructed dangerous object comparison library through the comparison calculation method of the dangerous object extraction module, and the attribute T, confidence C, center position O(u, v) of the successfully matched dangerous object and the length x width y data of the object rectangular box are combined to form a dangerous object data set, which specifically includes the following steps: Step 31: Based on the provisions of the national standard "List of Dangerous Goods" (GB12268), a large amount of initial basic data is collected for the nine major categories of dangerous goods, and the initial basic data is divided into nine regions according to the nine major categories and the regions are marked to obtain a basic dangerous goods comparison library; Step 32: Collect and obtain the public attributes of dangerous goods, classify them according to the types of dangerous goods, compare, supplement and enrich the corresponding 9 major categories of dangerous goods, and then make final numerical annotations according to the specific attributes of the specific area, so that the corresponding attributes T n becomes a fixed value Q n , in order to optimize, improve and expand the basic hazardous material comparison library, continuously obtain the properties of hazardous materials for multiple verifications to determine whether the comparison and identification are successful; if there are unidentified hazardous materials, they will be classified and labeled again, and this process will be repeated to finally obtain a more complete hazardous material comparison library; Step 33, Dangerous object extraction: Object attributes are extracted from the basic data set in turn, and the attribute T of the object is matched and labeled with the dangerous object comparison library. The matching calculation formula is as follows to determine whether T exists in the dangerous object comparison library, and finally form a dangerous object data set: Where: Q is the hazard identification index value; T n is the matching annotation value of object n in the basic dataset; Q1, Q2, ..., Q n is the attribute annotation value of the dangerous object in the dangerous object comparison library. When the attribute of the object is equal to the attribute annotation value of the dangerous object in the dangerous object comparison library, the identification index value will be equal to 1, which means that object n is a dangerous object. This will be repeated until all objects in the basic data set are calculated above, and then all object data with identification index values ​​equal to 1 are extracted to form a dangerous object data set.

6. The hazard source identification reasoning method based on image recognition as claimed in claim 2 is characterized in that: In step 4, the area change is calculated as follows: Where: S n is the time period t n S0 is the area of ​​the dangerous object at the initial moment, and the area is calculated according to the above formula using the length x width y of the dangerous object rectangle; k is the area growth or decay index of the dangerous object, which determines the rate at which the area changes over time; t n is the time period; e is the base of the natural logarithm, approximately equal to 2.71828; by n The dangerous object data between the two is taken, and the index k is calculated according to the above calculation formula. If k is positive, the area increases with time; if k is negative, the area decreases with time; if k is zero, the area remains unchanged, and the area value that changes with time is calculated by combining the index; By obtaining the length and width of the rectangular frame of the dangerous object in time period t, we know the areas S1 = x1·y1 and S2 = x2·y2 at the specific start time t1 and end time t2, and find the specific value of k; If there is hazardous object 1 and its associated hazardous object 2 in the hazardous object relationship database, then after the above calculation, judgment method 1 is used for judgment. Otherwise, judgment method 2 is used to judge hazardous object 1 alone to determine whether it is a hazard source.

7. The hazard source identification reasoning method based on image recognition as claimed in claim 6 is characterized in that: Judgment method 1 is as follows: k1>0 or k2>0, calculate the direction change; It is considered that the hazard source has weakened, and the time when the area is equal to 0 is calculated by the area change, and then step 5 is carried out; If k1=0 and k2=0, it is not considered as a hazard source; Judgment method 2 is as follows: If k1>0, then hazardous object 1 is regarded as a hazardous source and the direction change calculation is performed; If k1 is less than 0, then the dangerous object 1 is regarded as a weakened dangerous source, and the time when the area is equal to 0 is calculated by the area change, and then the step 5 is processed; If k1=0, dangerous object 1 is not considered as a hazard source.

8. The hazard source identification and reasoning method based on image recognition as claimed in claim 2 is characterized in that: The direction change is calculated as follows: Where: D n is the dangerous object in time period t n D0 is the angle of change of the direction of the dangerous object at the initial moment; K is the direction change rate index of the dangerous object, which can be positive or negative according to the actual situation; t n is the time period; e is the base of the natural logarithm, approximately equal to 2.71828; by n The dangerous object data value is taken, and the direction change angle and index K are calculated according to the above formula n If K is positive, it means that the hazard source direction change index increases positively over time; if K is negative, it decreases in the opposite direction over time; if K is zero, the direction remains unchanged at the initial angle, and the direction angle value that changes over time is calculated in combination with the index; Assume that the centers of the dangerous object at time points t1 and t2 are O1(u1, v1) and O2(u2, v2) respectively, and then calculate the angle of change from the origin O to point O1 and point O2. Assume that this angle is θ, the vector The modulus lengths of vector OO1 and vector OO2 are and The angle θ between the two vectors is given by It is calculated that, therefore, the angle θ is obtained by Calculate and convert the result from radians to degrees to get Thus, the data at time point t3 is obtained, and the specific value of direction D3 is obtained by combining the data at time point t1, and then the specific value of K is calculated to calculate the direction change; If there are two objects in the dangerous object data set, the direction change angles D1 of dangerous object 1 and dangerous object 2 are calculated respectively. n and D2 n , if both are positive or both are negative, it means the same direction, and it is possible to approach unilaterally; if one is positive and the other is negative, it means the opposite direction, and it is possible to approach quickly; then the intersection time is calculated; The intersection time is calculated as follows Where: r is the distance from the origin of the dangerous object to the rectangular frame; A n min is the area of ​​the dangerous object covering the lower left corner coordinate; A n max is the upper right corner coordinate of the area covered by the dangerous object; t n The time period data is brought into the above formula to calculate the maximum and minimum coordinates of the area coverage. In contrast, the calculated coordinates are at t n Whether there is any intersection within the time period is determined and calculated using judgment method 3.

9. The hazard source identification and reasoning method based on image recognition as claimed in claim 8, characterized in that: Judgment method 3: Compare with A1 n min and A1 n Is there any overlap between the max two coordinate ranges A2 n min and A2 n max. If the area coverage coordinate data of dangerous object 1 is equal to the area coverage coordinate data of dangerous object 2, then calculate the specific value of the intersection time t. If the specific time t can be calculated, there is an intersection and it is determined to be a major hazard source. Otherwise, there is no intersection between the dangerous objects, but it is only a hazard source. After completing the calculation of the intersection time t, transmit the data of the hazard source and the intersection time t to step 5 to analyze the specific hazard level.

10. The hazard source identification and reasoning method based on image recognition as claimed in claim 2, characterized in that: In step 5, the hazard level classification index is calculated based on the hazard relationship model, and the specific calculation formula is as follows; The hazard level index is calculated as follows: Where: R is the hazard level classification index; α is the correction coefficient for personnel, which is determined according to the population distribution around the hazard source. If the attribute is identified as a person, the number is calculated statistically, and if not, it is 1; ψ1, ψ2, ..., ψ n S1 is the correction coefficient corresponding to the hazard source. According to the 9 categories specified in the "List of Dangerous Goods", different values ​​are assigned to it when constructing the dangerous goods comparison library. These coefficients reflect the degree of danger of different hazard sources; n , S2 n , ..., S n is the t of each hazard source n The actual area of ​​the time period; P1, P2, ..., Pn are the critical quantities corresponding to each hazard source, which are assigned different values ​​when constructing the hazardous material comparison library. The critical quantity refers to a specified threshold value. When the threshold value is exceeded, the existence of the hazard source is determined to be a major hazard source; According to the specific value of the danger level classification index R, the danger level classification is completed, and emergency push is planned according to its danger level.

Images