Computer vision based lng terminal risk scenario identification platform and method

Abstract

The present application relates to a computer vision-based LNG receiving station risk scenario identification platform and method. The LNG receiving station risk scenario unit is used for identifying risk scenarios based on typical receiving station function partitions and risk scenarios based on hazard categories; the image data unit is used for acquiring and preprocessing image data; the computer vision identification model unit is used for managing, training and verifying models; the result output unit is used for instantaneously saving the identified risk scenarios, image data, time, monitoring device name and number, data path number according to the computer vision-identified models, and outputting specific type alarm signals for taking targeted risk response measures. The recognized risk information is outputted in time for early warning, thereby solving problems such as low reliability, low intelligent level and great human subjective factor influence of artificial monitoring means.

Description

TECHNICAL FIELD

[0001] The present application relates to liquefied natural gas (LNG) receiving station computer vision identification technology, in particular to a LNG receiving station risk scene identification platform and method based on computer vision. BACKGROUND

[0002] Video image monitoring is an effective monitoring and early warning means for potential risk scenes. At present, the production area of the LNG receiving station basically realizes full coverage of video monitoring, but the identification of risk scenes in the video monitoring image still relies on manual means, especially for the prevention and control of personnel unsafe behaviors which account for a high proportion in the direct causes of safety accidents, which mainly relies on conventional management means (such as safety training, management system, emergency plan and drilling) and manual supervision (on-site or video monitoring, safety inspection) in the operation site. This method needs a large amount of manpower, has low automation degree, has high requirements for personnel supervision level, knowledge and experience, responsibility, etc., cannot realize real-time hidden danger early warning, and has uncertain factors for risk control. The current automatic image recognition technology gradually applied in other fields can realize automatic identification of specific types of personnel unsafe behaviors to replace manual methods, but the problem is that it does not target the actual operation characteristics of the LNG receiving station, cannot cover all types of risk scenes in the LNG receiving station, and still has a large gap with the actual engineering situation. In order to realize effective identification and timely early warning of potential risk scenes in the LNG receiving station relying on existing basis and intelligent means, and to ensure the safe and stable operation of the project, the above technical problems need to be solved.

[0003] The applicant's literature search results show that only a scholar has established a full-digital industrial video system in the LNG receiving station (Shen Duxiaodong, Application of full-digital industrial video system in LNG receiving station), but it does not realize automatic identification of artificial intelligence computer vision, and there is no risk scene identification platform and method based on computer vision technology in the LNG receiving station. SUMMARY

[0004] In view of the above problems, the purpose of the present application is to solve the problems that the current LNG receiving station still relies on manual means for identification of risk scenes in video monitoring images, needs a large amount of manpower, has low automation degree, has high requirements for personnel supervision level, knowledge and experience, responsibility, etc., cannot realize real-time hidden danger early warning, and has uncertain factors for risk control; the current automatic image recognition technology gradually applied in other fields does not target the actual operation characteristics of the LNG receiving station, cannot cover all types of risk scenes in the LNG receiving station, and still has a large gap with the actual engineering situation, by providing a LNG receiving station risk scene identification platform and method based on computer vision.

[0005] To achieve the above object, the present application adopts the following technical solutions:

[0006] The LNG receiving station risk scenario identification platform based on computer vision comprises:

[0007] The LNG receiving station risk scenario unit is used for identifying risk scenarios based on typical receiving station function partitions and risk scenarios based on hazard categories.

[0008] The image data unit is used for acquiring and preprocessing image data.

[0009] The computer vision identification model unit is used for managing, training and verifying models.

[0010] The result output unit is used for instantaneously saving the identified risk scenarios, image data, time, monitoring device name and number, data path number according to the model identified by computer vision, and outputting a specific type of alarm signal for taking targeted risk response measures.

[0011] The risk scenarios based on typical receiving station function partitions comprise determining risk scenarios according to typical partitions of receiving stations and their functions, environment and facility device characteristics, and the typical partitions of receiving stations comprise a wharf area, an LNG storage tank area, a process device area, a tank car loading area, a public engineering and auxiliary production area, an office standby area and a perimeter area.

[0012] The risk scenarios based on hazard categories comprise human unsafe behavior risk scenarios, unsafe state of object risk scenarios and environmental risk scenarios.

[0013] The human unsafe behavior risk scenarios comprise behavior scenarios leading to potential risk consequences due to subjective or objective factors of personnel, including mis-touching and mis-operation, violating command and operation, violating regulations and behaviors, and not being equipped with protection, not being monitored, and not responding to actions in emergency conditions.

[0014] The unsafe state of object risk scenarios comprise scenarios leading to potential risk consequences due to objective existence or state of objects not meeting expectations, including risk scenarios generated due to defects of objects, lack of risk protection barriers, space positions or motion states not meeting expectations.

[0015] The environmental risk scenarios comprise scenarios leading to potential risk consequences due to natural climate environment or operation environment not meeting expectations.

[0016] The image data unit comprises an image data acquisition subunit and an image data preprocessing subunit. The image data acquisition subunit acquires a video stream from a video monitoring center hard disk video recorder or from a front-end image acquisition device through an RTSP protocol or a GB 28181 protocol in a video stream pulling manner, and realizes image acquisition through decoding and frame extraction or realizes the function of image acquisition through direct photo grabbing. The image data preprocessing subunit selects and extracts features of image data in combination with a recognition scene, adds feature labels, discards image data that does not meet the requirements, randomly groups image data after adding feature labels, takes 70% of the image data as a training group and 30% of the image data as a test group, divides samples of the training group image data, divides positive samples or negative samples according to labels and a scene, and adds sample labels.

[0017] The computer vision recognition model unit comprises a model management subunit. After the model management subunit loads a computer vision recognition model of a specific risk scene, the model management subunit executes a recognition command for target image data, outputs a result, and unloads the model. The entire process takes an instruction layer as an entrance, executes a model loading instruction, and performs model file parameter verification in a business layer. If the verification fails, a model loading failure indication is returned. If the verification succeeds, the model file of the business layer is loaded into a system execution layer, a model loading success indication is returned, the target image data is executed, and a result signal is output. The instruction layer is taken as an entrance, a model unloading instruction is executed, and model state verification is performed in the business layer. If the verification state is that the model is not loaded, a model unloading success indication is directly returned. If the verification state is that the model is loaded, the execution layer model is unloaded and the process execution file is deleted, and a model unloading success indication is returned.

[0018] The computer vision recognition model unit comprises a model training and verification subunit, the model training and verification subunit is configured to perform convolutional neural network training on the test group image data set after grouping and adding annotations, apply the parameter model after iteration to the test group image data set, and verify the model recognition accuracy by a subunit based on an input layer, a convolution operation layer, a pooling sampling layer, a full connection layer and an output layer to form a convolutional neural network model structure, and import the test group image data set and feature annotation information into the neural network input layer; local feature perception of the image data is realized through convolution operation, and the obtained feature map is subjected to pooling sampling processing, the average value is calculated to realize dimension reduction of the feature map, the feature map is input into the full connection layer to form a one-dimensional feature map; the one-dimensional feature map of the full connection layer is classified, the Soft-max classifier is used to calculate the maximum likelihood probability of different categories to obtain the geometric feature parameters of the image data; a network loss function is defined, the residual value between each layer and the expected value is calculated, the algorithm iteration operation is performed on the weight update, the test group image data set is applied to the model after iteration to verify the accuracy; if the accuracy does not meet the requirements, more training data samples are imported for relearning, and the model parameters are adjusted and re-iterated until the accuracy meets the requirements, and the application model is formed.

[0019] The LNG receiving station risk scenario recognition method based on computer vision comprises risk scenario feature recognition and definition and scenario image recognition.

[0020] The risk scenario feature recognition and definition comprises defining the image features of the risk scenario by four elements of a main object, a secondary object, environment and background information and feature relationship description, and comprises the following steps.

[0021] An element layer is established to determine the content of the main object, the secondary object, the environment and the background information in the image layer.

[0022] A state layer is established to determine the state of the main object and the secondary object, and define the environment in which the scenario is located according to the environment and background information.

[0023] A feature relationship layer is established to determine the relationship features between the main object and the secondary object, and the relationship features between the objects and the scenario environment, and define the risk scenario based on the above features.

[0024] Scenario image recognition; and

[0025] The scenario image recognition comprises image grayscale algorithm, image feature recognition algorithm, image segmentation algorithm and image enhancement algorithm, and the specific algorithms are selected and combined according to the risk scenario.

[0026] The present application has the following advantages due to the above technical solutions.

[0027] The computer vision-based LNG receiving station risk scenario identification platform and method are based on the production and operation characteristics of the LNG receiving station, the main risk scenario areas and categories of the LNG receiving station are combed and defined, a typical risk scenario model is proposed, and an image data, computer vision identification, result output unit is constructed, thereby forming a method for carrying out computer vision risk identification and outputting identification results based on the video monitoring data of the LNG receiving station. By defining the image features of different risk scenarios of the LNG receiving station by four elements, i.e., a main object, a secondary object, environmental and background information, and a feature relationship description, and the association thereof, the technical problem of the application of the artificial intelligence computer vision identification algorithm in the LNG receiving station risk scenario identification is effectively solved. The platform and method proposed in the application can maximize the use of the high-definition video monitoring system already deployed in the LNG receiving station, import video stream data, and automatically identify the risk scenarios in real time by applying the model, and output the identified risk information in time to give an early warning, thereby solving the problems of low reliability, low intelligent level, and great influence of human subjective factors of the artificial monitoring means. BRIEF DESCRIPTION OF DRAWINGS

[0028] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The detailed description is made with reference to the accompanying drawings.

[0029] Figure 1 FIG. 1 is a flowchart of a computer vision-based LNG receiving station risk scenario identification platform and method;

[0030] Figure 2 FIG. 2 is a schematic diagram of the unit composition of the LNG receiving station risk scenario;

[0031] Figure 3 FIG. 3 is a schematic diagram of the composition and flow of the image data unit; and

[0032] Figure 4 FIG. 4 is a schematic diagram of the composition and application example of the risk scenario feature identification and definition module. DETAILED DESCRIPTION

[0033] Exemplary embodiments of the present application will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present application can be more thoroughly understood and so that the scope of the present application can be completely conveyed to those skilled in the art.

[0034] The technical scheme adopted by the present application to solve its technical problems is as follows: a LNG receiving station risk scene identification platform and method based on computer vision, comprising:

[0035] The LNG receiving station computer vision identification platform and the risk scene identification method;

[0036] The LNG receiving station computer vision identification platform comprises a LNG receiving station risk scene unit, an image data unit, a computer vision identification model unit and a result output unit.

[0037] The risk scene identification method uses a risk scene feature identification and definition module and a scene image identification module.

[0038] The LNG receiving station risk scene unit comprises a risk scene based on typical receiving station functional partition and a risk scene based on hazard category.

[0039] The risk scene based on typical receiving station functional partition is determined according to the typical partition of the receiving station (including the wharf area, the LNG storage tank area, the process device area, the tank car loading area, the public engineering and auxiliary production area, the office standby area and the perimeter area) and the characteristics of the functions, environment and facility devices thereof.

[0040] The risk scene based on hazard category comprises a human unsafe behavior risk scene, a material unsafe state risk scene and an environmental risk scene.

[0041] The human unsafe behavior risk scene is a behavior scene that leads to potential risk consequences due to subjective or objective factors of personnel, including actions such as "not to be executed but executed", such as misoperation, illegal command and operation, violation of regulations, and actions such as "should be executed but not executed", such as not equipped with protection, not monitored, and not responded in emergency situations.

[0042] The material unsafe state risk scene is a scene that leads to potential risk consequences due to the objective existence or state of the material not meeting the expectation, including risk scenes caused by defects of the material itself, lack of risk protection barriers, space position or motion state not meeting the expectation.

[0043] The environmental risk scene is a scene that leads to potential risk consequences due to the natural climate environment or work environment not meeting the expectation.

[0044] The image data unit comprises an image data acquisition subunit and an image data preprocessing subunit.

[0045] The image data acquisition subunit is a functional unit for acquiring video streams from video monitoring center hard disk video recorders and other devices or from front-end image acquisition devices through RTSP, GB 28181 and other protocols in a video stream pulling manner, and achieving image acquisition through decoding and frame extraction or directly grabbing photos;

[0046] The image data preprocessing subunit includes the following specific steps:

[0047] (1) Feature selection and extraction are performed on image data in combination with recognition scenes, feature labels are added, and image data that does not meet the requirements is discarded;

[0048] (2) The image data with added feature labels is randomly grouped, 70% of the image data is used as a training group, and 30% of the image data is used as a test group;

[0049] (3) The training group image data is divided into samples, and is divided into positive samples or negative samples according to the labels and scenes, and sample labels are added;

[0050] The computer vision recognition model unit includes a model management subunit, a model training and verification subunit;

[0051] The model management subunit is a whole process of loading a computer vision recognition model for a specific risk scene, executing a recognition command for target image data, outputting a result, and unloading the model, and includes the following specific steps:

[0052] (1) Taking the instruction layer as an entry, a model loading instruction is executed, and a model file parameter check is performed in the business layer;

[0053] (2) If the check fails, a model loading failure indication is returned; if the check succeeds, the model file in the business layer is loaded into the system execution layer, a model loading success indication is returned, a recognition is performed on the target image data, and a result signal is output;

[0054] (3) Taking the instruction layer as an entry, a model unloading instruction is executed, and a model state check is performed in the business layer;

[0055] (4) If the check state is that the model is not loaded, a model unloading success indication is directly returned; if the check state is that the model is loaded, the execution layer model is unloaded and the process execution file is deleted, and a model unloading success indication is returned;

[0056] The model training and verification subunit is a subunit for training a convolutional neural network on a test group image data set that has been grouped and added with annotations, applying a parameter model after iteration to the test group image data set to verify the recognition accuracy of the model, and includes the following specific steps:

[0057] (1) Based on the input layer, convolution operation layer, pooling sampling layer, full connection layer and output layer, a convolution neural network model structure is constructed, and the test group image data set and its feature annotation information are imported into the neural network input layer;

[0058] (2) The local feature of the image data is realized by convolution operation, and the obtained feature map is processed by pooling sampling, the average value is calculated to realize the dimension reduction of the feature map, and the feature map is input into the full connection layer to form a one-dimensional feature map;

[0059] (3) The one-dimensional feature map of the full connection layer is classified, the Soft-max classifier is used to calculate the maximum likelihood probability of different categories, and the geometric feature parameters of the image data are obtained;

[0060] (4) The network loss function is defined, the residual value between each layer and the expected value is calculated, the algorithm iteration operation is performed on the weight update, and after the iteration is completed, the test group image data set is applied to the model to verify the precision;

[0061] (5) If the precision does not meet the requirements, more training data samples are imported to return to step (2) for relearning, and the model parameters are adjusted and iterated again, until the precision meets the requirements, and the application model is formed;

[0062] The result output unit is to save the risk scene type, associated image, time, monitoring equipment name and number, data path number and other information found after computer vision recognition of the actual LNG receiving station video monitoring data, and output a specific type of alarm signal to take targeted risk response measures;

[0063] The risk scene feature recognition and definition module is a method module for defining the image features of the risk scene by four types of elements, i.e. primary object (PO), secondary object (SO), environmental and background information (EBI) and characteristic relation description (CRD), which includes the following specific steps:

[0064] (1) Establish the element layer to determine the content of the primary object, secondary object and environmental and background information in the image layer;

[0065] (2) Establish the state layer to determine the state of the primary object and secondary object, and define the environment of the scene according to the environmental and background information;

[0066] (3) Establishing a feature relationship layer, determining the relationship characteristics between the main object and the secondary object, and the relationship characteristics between the object and the scene environment, and defining the risk scene based on the above characteristics;

[0067] The scene image recognition module includes image gray algorithm, image feature recognition algorithm, image segmentation algorithm and image enhancement algorithm. According to the risk scene, specific algorithms are selected and combined for application.

[0068] The image gray algorithm is to make the R, G and B values (representing the brightness of red, green and blue three primary colors) of each pixel in the RGB model (additive color mixing model) equal, that is, R=G=B=x, so that the color shows a gray color, thereby obtaining a gray image more suitable for feature recognition, wherein x is a gray value ranging from 0 to 255.

[0069] The image feature recognition algorithm is to select and analyze the features such as color distribution information, circularity, spectral characteristics, sharp corners, centroid movement, area change, edge characteristics, and similarity and stability between multiple images, which are comprehensively used as the basis for identifying certain features of the image.

[0070] The image segmentation algorithm is to let the set R represent the entire image area, divide R into n non-empty subsets R1, R2, R3…Rn that meet the following conditions, so as to convert the original image into a more abstract form suitable for higher layer image recognition and understanding.

[0071] (1)

[0072] (2) For all i and j, i≠j, there are

[0073] (3) For i=1,2,3……,n, there are P(R i ) = TRUE;

[0074] (4) For i≠j, there are P(R i ∪R j ) = FALSE;

[0075] (5) For i=1,2,3……,n, R i is a connected region.

[0076] In the above formula, P is a logical predicate for all elements in the set R, representing an empty set.

[0077] The image enhancement algorithm is to perform histogram equalization processing on the gray image, uniformly distribute the original histogram gray value, increase the dynamic range of pixel gray value, enhance the overall contrast effect of the image, and at the same time, since the number of gray scale occurrences does not change, the original information structure of the image can be ensured unchanged.

[0078] A computer vision-based LNG receiving station risk scenario identification platform and method according to some embodiments, comprising:

[0079] LNG receiving station computer vision identification platform and risk scenario identification method;

[0080] As shown in Figure 1 , the LNG receiving station computer vision identification platform comprises an LNG receiving station risk scenario unit, an image data unit, a computer vision identification model unit, and a result output unit, wherein the LNG receiving station risk scenario unit serves as the basis for risk scenario division and definition, and supports the establishment of the computer vision identification model, and the image data unit serves as the data source interface for the model training data set and the test data set;

[0081] The risk scenario identification method comprises a risk scenario feature recognition and definition module and a scenario image recognition module;

[0082] As shown in Figure 2 , the LNG receiving station risk scenario unit comprises risk scenarios based on typical receiving station functional zoning and risk scenarios based on hazard categories, which complement each other to cover the LNG receiving station risk scenarios;

[0083] As shown in Figure 2 , the risk scenarios based on typical receiving station functional zoning are determined according to the typical zoning of the receiving station (including the wharf area, the LNG storage tank area, the process device area, the tank car loading area, the public engineering and auxiliary production area, the office standby area, and the perimeter area) and the characteristics of its functions, environment, and facility devices. Different functional zones have different focus risk scenarios, for example, for the tank top area of the LNG storage tank area, due to the associated high-altitude environment, the unsafe behavior of the tank top operator not effectively wearing a safety belt is one of the key risk scenarios in this area.

[0084] As shown in Figure 2 , the risk scenarios based on hazard categories include human unsafe behavior risk scenarios, unsafe state of object risk scenarios, and environmental risk scenarios;

[0085] The human unsafe behavior risk scenario is a behavior scenario that leads to potential risk consequences due to subjective or objective factors of personnel, including "should not be executed but executed" actions such as misoperation, illegal command and operation, and violation of regulations, and "should be executed but not executed" actions such as lack of protection, lack of monitoring, and lack of response in emergency situations;

[0086] An unsafe state risk scenario of an object refers to a scenario in which a potential risk consequence is caused due to an objective existence or a state of the object not meeting an expectation, including a risk scenario caused due to a defect of the object itself, a lack of a risk protection barrier, a space position or a motion state not meeting an expectation;

[0087] An environmental risk scenario refers to a scenario in which a potential risk consequence is caused due to a natural climate environment or an operation environment not meeting an expectation;

[0088] As shown in FIG. 1, the image data unit includes an image data acquisition subunit and an image data preprocessing subunit. Figure 3

[0089] As shown in FIG. 2, the image data acquisition subunit refers to a functional unit that acquires a video stream in a video stream pulling manner from a video monitoring center hard disk video recorder or the like or through an RTSP, GB 28181 or the like from a front-end image acquisition device, and acquires an image through decoding and frame extraction or directly captures a photo to achieve an image acquisition purpose. Figure 3

[0090] The image data preprocessing subunit includes the following specific steps:

[0091] (1) Feature selection and extraction are performed on image data in combination with a recognition scenario, feature labels are added, and image data that does not meet a requirement is discarded;

[0092] (2) The image data to which the feature labels are added is randomly grouped, 70% of the image data is used as a training group, and 30% of the image data is used as a test group;

[0093] (3) Sample division is performed on the training group image data, the image data is divided into positive samples or negative samples according to labels and a scenario, and sample labels are added;

[0094] As shown in FIG. 3, the computer vision recognition model unit includes a model management subunit and a model training and verification subunit. Figure 1 The model management subunit refers to an entire process in which a computer vision recognition model for a specific risk scenario is loaded, a recognition command is executed on target image data, a result is output, and the model is unloaded, and includes the following specific steps:

[0095] (1) An instruction layer is used as an entry, a model loading instruction is executed, and a model file parameter is verified in a business layer;

[0096] (2) If the verification fails, a model loading failure indication is returned; if the verification succeeds, the model file in the business layer is loaded into a system execution layer, a model loading success indication is returned, a recognition is performed on the target image data, and a result signal is output;

[0097]

[0098] ​​​(3) Taking the instruction layer as an entrance, executing the model unloading instruction, and performing model state checking in the business layer;

[0099] (4) If the checking state is that the model is not loaded, directly returning a model unloading success indication; if the checking state is that the model is loaded, unloading the execution layer model and deleting the process execution file, and then returning a model unloading success indication;

[0100] The model training and verification subunit, i.e., a subunit for training a convolutional neural network on a test group image data set after grouping and adding annotations, applying a parameter model after iteration to the test group image data set to verify the model recognition accuracy, includes the following specific steps:

[0101] (1) Taking an input layer, a convolution operation layer, a pooling sampling layer, a full connection layer, and an output layer as a basis to constitute a convolutional neural network model structure, and importing a test group image data set and its feature annotation information into the neural network input layer;

[0102] (2) Realizing local feature perception of image data through convolution operation, and performing pooling sampling processing on the obtained feature map to calculate the average value to realize dimension reduction of the feature map, inputting the feature map into the full connection layer to form a one-dimensional feature map;

[0103] (3) Classifying the one-dimensional feature map of the full connection layer, calculating the maximum likelihood probability of different categories by using a Soft-max classifier to obtain geometric feature parameters of the image data;

[0104] (4) Defining a network loss function, commonly used loss functions include a negative log likelihood loss function, a mean square error loss function, etc., calculating residual values between each layer and an expected value, and performing algorithm iteration operation on weight updates, and after iteration, applying the test group image data set to the model to verify the accuracy. Residual For example, the calculation formula is as follows:

[0105]

[0106] In the formula, a i Input value of each layer node of the network;

[0107] f′() Derivative of the activation function;

[0108] (5) If the accuracy does not meet the requirements, returning to step (2) for relearning by importing more training data samples, adjusting and reiterating the model parameters, until the accuracy meets the requirements, and forming an application model;

[0109] The result output unit is configured to save the risk scene type, associated image, time, monitoring device name and number, data channel number and the like discovered after computer vision recognition of the actual LNG receiving station video monitoring data, and output a specific type of alarm signal, so as to take risk response measures in a targeted manner.

[0110] As shown in Figure 4 The risk scene feature recognition and definition module is a method module for defining the image features of a risk scene by four types of elements, i.e., a primary object (PO), a secondary object (SO), environmental and background information (EBI) and characteristic relation description (CRD), and includes the following specific steps:

[0111] (1) Establish an element layer to determine the content of the primary object, the secondary object and the environmental and background information in the image layer;

[0112] (2) Establish a state layer to determine the state of the primary object and the secondary object, and define the environment in which the scene is located according to the environmental and background information;

[0113] (3) Establish a characteristic relation layer to determine the relation characteristics between the primary object and the secondary object, and the relation characteristics between the objects and the scene environment, and define the risk scene based on the above characteristics;

[0114] The scene image recognition module includes an image gray-scale algorithm, an image feature recognition algorithm, an image segmentation algorithm and an image enhancement algorithm, and the specific algorithms are selected and combined according to the risk scene and applied;

[0115] The image gray-scale algorithm is to make the R, G and B values (representing the brightness of red, green and blue three primary colors) of each pixel in the RGB model (additive color mixing model) equal, i.e., R=G=B=x, so that the color is shown in a gray color, thereby obtaining a gray image more suitable for feature recognition, wherein x is a gray value ranging from 0 to 255. The three-component brightness in the color image is weighted and averaged to obtain a more reasonable gray image, and the weighted average gray-scale algorithm formula is as follows considering the sensitivity of the human eye to different colors:

[0116] f(i,j) = 0.30R(i,j) + 0.58G(i,j) + 0.12B(i,j)

[0117] Image feature recognition algorithm is to select and analyze the color distribution information, circularity, spectral characteristics, sharp corners, centroid movement, area change, edge characteristics and similarity and stability between multiple images, etc. contained in the image, which are comprehensively used as the basis for recognizing certain features of the image. Taking circularity as an example, it is used to measure the similarity to a circle and is often used to measure the regularity of the shape of the object in the image as a criterion for identifying a specific risk scene. The circularity is the ratio of the square of the perimeter of the figure boundary to the area thereof, and the calculation formula is as follows:

[0118]

[0119] In the formula, L S is the perimeter of the figure boundary;

[0120] A S is the area of the figure;

[0121] After the image is processed, it can be regarded as a binary image representing the maximum area, the pixel of the bright area is 1, the pixel of the black area is 0, the area is the total count of the internal pixels after the maximum area is filled, and the perimeter is obtained by the boundary. The maximum circularity is 1, which represents the prototype; the smaller the circularity is, the more irregular the object is;

[0122] The image segmentation algorithm is to divide the set R representing the entire image area into n non-empty subsets R1, R2, R3…Rn satisfying the following conditions, so as to convert the original image into a more abstract form suitable for higher layer image recognition and understanding.

[0123] When the gray value of the background in the image is obviously different from the target area, the target area and the background can be segmented by setting a reasonable threshold value, and the best threshold value can be obtained by iteration.

[0124] The image enhancement algorithm is to perform histogram equalization on the gray image, to uniformly distribute the original histogram gray value, to increase the dynamic range of the pixel gray value, to enhance the overall contrast effect of the image, and to ensure that the original information structure of the image is unchanged due to the unchanged frequency of the gray level. The frequency of the pixel of the gray level in the digital image, i.e. the normalized histogram formula, is as follows:

[0125]

[0126] In the formula, n k is the number of pixels of the gray level r k ; and

[0127] N is the total number of pixels in the image;

[0128] L is the total number of possible gray levels of the pixel;

[0129] The histogram equalization is performed on the image, and a discrete form corresponding to the histogram equalization can be expressed as:

[0130]

[0131] Mapping each pixel in the input image to the output image can obtain the equalized histogram, and the image enhancement algorithm is realized.

[0132] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A computer vision based LNG terminal risk scenario identification platform, characterized in that, The LNG receiving station risk scenario unit is used for identifying risk scenarios based on typical receiving station functional partitions and risk scenarios based on hazard categories. The image data unit is used for acquiring image data and pre-processing. The computer vision recognition model unit is used for managing, training and verifying models. The result output unit is used for instantaneously saving the identified risk scenarios, image data, time, monitoring device name and number, data path number according to the computer vision recognized model, and outputting specific type alarm signals for taking targeted risk response measures. The computer vision-based LNG receiving station risk scenario identification includes risk scenario feature recognition and definition and scenario imaging identification, wherein The risk scenario feature recognition and definition includes defining image features of a risk scenario by four elements of main objects, secondary objects, environment and background information, and feature relationship description, including the following steps: An element layer is established to determine the content of main objects, secondary objects, environment and background information at the image level. A state layer is established to determine the state of main objects and secondary objects, and to define the environment where the scenario is located according to the environment and background information. A feature relationship layer is established to determine the relationship features between main objects and secondary objects, and the relationship features between objects and the scenario environment, and to define the risk scenario based on the above features; and The scenario imaging identification includes image grayscale algorithm, image feature recognition algorithm, image segmentation algorithm and image enhancement algorithm, and specific algorithms are selected and combined for application according to the risk scenario. The risk scenarios based on typical receiving station functional partitions include determining risk scenarios according to typical partitions of receiving stations and their functions, environment and facility device characteristics, wherein the typical partitions of receiving stations include wharf area, LNG storage tank area, process device area, tank car loading area, public engineering and auxiliary production area, office standby area and perimeter area. The risk scenarios based on hazard categories include human unsafe behavior risk scenarios, unsafe state of object risk scenarios and environmental risk scenarios.

2. The computer vision based LNG receiving terminal risk scenario identification platform of claim 1, wherein, The human unsafe behavior risk scenarios include behavior scenarios that cause potential risk consequences due to subjective or objective factors of personnel, including misoperation, illegal command and operation, violation of regulations, and lack of protection, monitoring and response in emergency situations.

3. The computer vision based LNG receiving terminal risk scenario identification platform of claim 1, wherein, The unsafe state of object risk scenarios include scenarios that cause potential risk consequences due to the objective existence or state of objects not meeting expectations, including risk scenarios caused by defects of objects, lack of risk protection barriers, space position or motion state not meeting expectations.

4. The computer vision based LNG receiving terminal risk scenario identification platform of claim 3, wherein, The environmental risk scenarios include scenarios that cause potential risk consequences due to natural climate environment or work environment not meeting expectations.

5. The computer vision based LNG receiving terminal risk scenario identification platform of claim 3, wherein, The image data unit includes:

6. The computer vision based LNG receiving terminal risk scenario identification platform of claim 3, wherein, ​ 7. The computer vision based LNG receiving terminal risk scenario identification platform of claim 1, wherein, ​ The image data acquisition subunit acquires a video stream from a video monitoring center hard disk video recorder in a video stream pulling manner or from a front-end image acquisition device through an RTSP protocol or a GB 28181 protocol, and realizes image acquisition through decoding and frame extraction or realizes image acquisition through direct photo grabbing; The image data preprocessing subunit selects and extracts features from image data in combination with a recognition scene, adds feature labels, discards image data that does not meet requirements, randomly groups image data after the feature labels are added, 70% of the image data is used as a training group, 30% of the image data is used as a test group, divides samples of the training group image data, divides positive samples or negative samples according to labels and scenes, and adds sample labels.

8. The computer vision based LNG receiving terminal risk scenario identification platform of claim 1, wherein, The computer vision recognition model unit includes a model management subunit, which loads a computer vision recognition model for a specific risk scene, executes a recognition command for target image data, outputs a result, and unloads the model after the whole process, takes an instruction layer as an entrance, executes a model loading instruction, and performs model file parameter verification in a business layer: if verification fails, a model loading failure indication is returned; if verification succeeds, the model file of the business layer is loaded into a system execution layer, a model loading success indication is returned, and a recognition is performed on the target image data to output a result signal; takes an instruction layer as an entrance, executes a model unloading instruction, and performs model state verification in a business layer: if the verification state is that the model is not loaded, a model unloading success indication is directly returned; if the verification state is that the model is loaded, the execution layer model is unloaded and the process execution file is deleted, and a model unloading success indication is returned.

9. The computer vision based LNG receiving terminal risk scenario identification platform of claim 1, wherein, The computer vision recognition model unit includes a model training and verification subunit, which trains a convolutional neural network for a test group image data set after grouping and adding annotations, applies a parameter model after iteration to the test group image data set to verify the precision of the model, takes an input layer, a convolution operation layer, a pooling sampling layer, a full connection layer and an output layer as a basis to constitute a convolutional neural network model structure, imports the test group image data set and its feature label information into the neural network input layer; local feature perception of the image data is realized through convolution operation, and the obtained feature map is subjected to pooling sampling processing, the average value is calculated to realize dimension reduction of the feature map, the feature map is input into the full connection layer to form a one-dimensional feature map; the one-dimensional feature map of the full connection layer is classified, the Soft-max classifier is used to calculate the maximum likelihood probability of different categories to obtain geometric feature parameters of the image data; a network loss function is defined, residual values between each layer and an expected value are calculated, algorithm iteration operation is performed on weight updates, the test group image data set is applied to the model after iteration to verify the precision; if the precision does not meet the requirements, more training data samples are imported for relearning, and the model parameters are adjusted and re-iterated until the precision meets the requirements to form an application model.

Images