Intelligent supervision method and device for petrochemical engineering construction site

Abstract

The invention relates to an intelligent supervision method and device for a petrochemical engineering construction site, and the method comprises the steps: collecting a monitoring video of the construction site, carrying out the image quality evaluation of each frame of image in the monitoring video, and recovering the images with the image quality not reaching the standard, and obtaining a processed monitoring video; analyzing the processed monitoring video based on a pre-established deep learning network to obtain analysis result data; and performing data integration on the analysis result data based on a pre-established PIM workstation, and displaying the data. In order to solve the problems that the amount of information is large, but no unified information exchange and delivery standard exists, data integration is carried out on analysis result data based on the pre-established PIM workstation, a PIM technology is adopted to carry out unification of a large amount of information, convenience and reliability of enterprise construction project management are improved, and the operation benefits of enterprises can be improved.

Description

Technical Field

[0001] The present invention relates to an intelligent supervision method and device for a petrochemical engineering construction site. Background Art

[0002] With the development of information technology, a large amount of information is electronic and digital. Especially for petrochemical engineering construction projects of petrochemical enterprises, the projects involve a large amount of information with various types, and there is no certain standard for information exchange and delivery. As a result, petrochemical enterprises need to spend a great deal of manpower, material resources and financial resources to integrate this information, and there are also inconsistencies and incompleteness in information integration, which in turn causes problems such as project management chaos and insufficient risk control ability.

[0003] Moreover, the data in the construction process of petrochemical engineering construction projects is different every day. The form of data generation is dynamic and updated rapidly. If information is updated manually in real time, firstly, the cost is huge, and secondly, the error rate of manually collecting and inputting monitoring information is high, and inaccurate uploaded data will lead to problems such as project management chaos and insufficient risk control ability. Summary of the Invention

[0004] In order to analyze the on-site information in the construction process of petrochemical engineering construction projects and realize intelligent analysis and supervision of the construction site, the present invention provides an intelligent supervision method and device for a petrochemical engineering construction site. The technical solutions proposed by the present invention are as follows:

[0005] In a first aspect, the present invention provides an intelligent supervision method for a petrochemical engineering construction site, including:

[0006] Collecting monitoring videos of the construction site, evaluating the image quality of each frame of the monitoring videos, and restoring the images with unqualified image quality to obtain processed monitoring videos;

[0007] Analyzing the processed monitoring videos based on a pre-established deep learning network to obtain analysis result data;

[0008] Integrating and displaying the analysis result data based on a pre-established PIM workstation.

[0009] In one or some embodiments, the images with unqualified image quality are restored by the following method:

[0010] Constructing a Gaussian model with the image with unqualified image quality as the center point, and for each pixel point of the image with unqualified image quality:

[0011] Input the pixel value of the pixel point and the pixel values of the corresponding pixel points of the pixel point in multiple frames of images before and after, respectively, into the Gaussian model in chronological order to obtain corresponding multiple model output values;

[0012] Based on the multiple model output values, determine the weighting coefficients of the pixel points of each frame of image respectively;

[0013] According to the weighting coefficients of the pixel points of each frame of image, use the following formula to perform weighted summation on the pixel value of the pixel point and the pixel values of the corresponding pixel points of the pixel point in multiple frames of images before and after to obtain the restored pixel value of the pixel point:

[0014]

[0015] In the formula, x′ i is the restored pixel value, d j is the weighting coefficient of the pixel point of the jth frame of image, x i is the pixel value of the pixel point of the ith frame of image, M is the number of subsequent images, and N is the number of previous images.

[0016] In one or some embodiments, determine the weighting coefficients of the pixel points of each frame of image through the following formula:

[0017]

[0018] In the formula, d i is the weighting coefficient of the pixel point of the ith frame of image, y i is the ith model output value; wherein, the Gaussian model is:

[0019]

[0020] In the formula, μ is the mean value of the pixel values of the same pixel point in the current frame of image and multiple frames of images before and after, σ is the standard deviation of the pixel values of the same pixel point in the current frame of image and multiple frames of images before and after, and x i is the pixel value of the pixel point of the ith frame of image.

[0021] In one or some embodiments, perform image quality evaluation on each frame of image in the surveillance video through the following method:

[0022] For each frame of image in the surveillance video, fuse the previous frame image of the current frame image with a preset background image to obtain a fused image, and subtract the fused image from the current frame image to obtain a difference image;

[0023] Perform image quality evaluation based on the difference image.

[0024] In one or some embodiments, the previous frame image of the current frame image and a preset background image are fused to obtain a fused image in the following manner:

[0025] The pixel values of the pixel points at the same position in the previous frame image of the current frame image and the preset background image are weighted and summed according to a preset proportional weight to obtain a fused image.

[0026] In one or some embodiments, the deep learning network includes a first CNN model, an LSTM, and a second CNN model;

[0027] The processed surveillance video is analyzed based on the pre-established deep learning network to obtain analysis result data, including:

[0028] Feature extraction is respectively performed on each frame image in the processed surveillance video based on the first CNN model to obtain corresponding surveillance features;

[0029] The surveillance features of all images are input into the LSTM model in chronological order, and the feature sequence is predicted and corrected based on the LSTM model to obtain a predicted and corrected feature sequence;

[0030] Feature analysis is performed on the corrected surveillance features based on the second CNN model to obtain analysis result data.

[0031] In one or some embodiments, the analysis result data includes construction progress, whether there is a construction anomaly, the location of the construction anomaly, the construction worker number, and the manager label;

[0032] The analysis result data is integrated and displayed based on the pre-established PIM workstation, including:

[0033] The construction progress, whether there is a construction anomaly, the location of the construction anomaly, the construction worker number, and the manager label are input into a report in a preset format to generate a standard form for visual display.

[0034] In one or some embodiments, the method further includes:

[0035] Obtain a three-dimensional sand table of the construction site, compare the analysis result data with the scene data in the three-dimensional sand table to obtain a comparison result.

[0036] In one or some embodiments, the image quality evaluation based on the difference image includes:

[0037] Determine the mean square error of the pixel values of all pixel points in the difference image, and determine whether the mean square error is within a preset range;

[0038] If so, the image quality meets the standard; if not, the image quality does not meet the standard.

[0039] In a second aspect, the present invention provides an intelligent supervision device for a petrochemical engineering construction site, including:

[0040] An evaluation module, configured to collect a monitoring video of the construction site, evaluate the image quality of each frame of the monitoring video, and restore the images with unqualified image quality to obtain a processed monitoring video;

[0041] An analysis module, configured to analyze the processed monitoring video based on a pre-established deep learning network to obtain analysis result data;

[0042] An integration module, configured to integrate and display the analysis result data based on a pre-established PIM workstation.

[0043] In a third aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the intelligent supervision method for a petrochemical engineering construction site as described in the first aspect.

[0044] In a fourth aspect, the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0045] The memory is used to store a computer program;

[0046] The processor is configured to implement the intelligent supervision method for a petrochemical engineering construction site as described in the fourth aspect when executing the program stored on the memory.

[0047] Based on the above technical solutions, the beneficial effects of the present invention compared with the prior art are:

[0048] The intelligent supervision method for petrochemical engineering construction sites provided by the present invention aims at the characteristics of fast data update during the construction process of petrochemical engineering construction projects and the problems of low efficiency and high error rate in manual data collection. The present invention uses video monitoring to collect the monitoring videos of the construction site in real time, evaluates the image quality of each frame of the monitoring video, and restores the images with unqualified image quality, which can retain the original features of the images. Then, based on the pre-established deep learning network, the restored monitoring video is analyzed to obtain the analysis result data. By means of artificial intelligence, the analysis result data is automatically obtained, with a low error rate and high efficiency. Moreover, by analyzing the restored monitoring video, the accuracy and reliability of intelligent analysis at the construction site are improved. Then, based on the pre-established PIM workstation, the analysis result data is integrated, and the PIM technology is used to unify a large amount of information, which improves the convenience and reliability of enterprise construction project management and can improve the operating efficiency of the enterprise.

[0049] Other features and advantages of the present invention will be described in the following description, and, in part, will be obvious from the description or learned by implementing the present invention. The objectives and other advantages of the present invention are achieved and obtained by the structures specifically pointed out in the description, claims, and drawings.

[0050] To make the above objectives, features, and advantages of the present invention more obvious and understandable, the following specifically provides preferred embodiments and, in conjunction with the accompanying drawings, details are described as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0052] Figure 1 It is a schematic flowchart of the intelligent supervision method for petrochemical engineering construction sites provided by the embodiments of the present invention;

[0053] Figure 2 It is a schematic diagram of the image sequence for image restoration provided by the embodiments of the present invention;

[0054] Figure 3 It is a schematic structural diagram of the deep learning network provided by the embodiments of the present invention;

[0055] Figure 4 It is a schematic structural diagram of the intelligent supervision device for petrochemical engineering construction sites provided by the embodiments of the present invention;

[0056] Figure 5 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0057] Hereinafter, exemplary embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art.

[0058] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present invention. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0059] The inventor found in work that with the development of information technology, a large amount of information is electronic and digital, especially for petrochemical engineering construction projects in petrochemical enterprises. Such projects involve a large amount of information and various types, but there is no certain standard for information exchange and delivery. Moreover, there are problems such as fast data update in petrochemical engineering construction projects, low efficiency and high error rate of data collection means, which cannot meet the inventor's expectations. After further research and development, the inventor made the present invention.

[0060] Embodiment 1

[0061] An embodiment of the present invention provides an intelligent supervision method for a petrochemical engineering construction site, which combines video monitoring means on the basis of PIM technology to realize intelligent analysis and supervision of the construction site. Referring to Figure 1 as shown, the specific steps include:

[0062] S101. Collect the monitoring video of the construction site, evaluate the image quality of each frame of the monitoring video, and restore the images with unqualified image quality to obtain the processed monitoring video;

[0063] Set up monitoring cameras at the construction site, and collect the monitoring video of the construction site through the monitoring cameras. Send the monitoring video to the server background for preprocessing: including operations such as denoising and enhancement. In addition, for some images with poor quality, the frame image can also be restored, or the image with poor quality can be directly deleted. For the above-mentioned denoising and enhancement processing methods, those skilled in the art can refer to the specific descriptions in the prior art, and will not be specifically limited here.

[0064] S102. Analyze the processed surveillance video based on a pre-established deep learning network to obtain analysis result data;

[0065] The server background uses artificial intelligence to analyze the surveillance video to obtain analysis result data. Input the processed surveillance video into the deep learning network for analysis to obtain analysis result data. The analysis result data includes information such as construction progress, whether there are construction anomalies, construction anomaly locations, construction worker numbers, and management personnel labels. The model structure of the deep learning network is CNN + LSTM + CNN. Based on the first CNN model, feature extraction is performed on each frame of the processed surveillance video respectively, and a surveillance feature is obtained for each frame of the image, resulting in corresponding surveillance features; the surveillance features (feature sequences) of all images are input into the LSTM model in chronological order, and the LSTM model is used to predict and correct the surveillance features to obtain corrected surveillance features (prediction and correction feature sequences); based on the second CNN model, feature analysis is performed on the corrected surveillance features to obtain analysis result data. The model structure of the deep learning network is referred to Figure 3 as shown. The first CNN model includes three convolutional layers, four pooling layers, and one fully connected layer, and its structure is convolutional layer - pooling layer - convolutional layer - pooling layer - pooling layer - convolutional layer - pooling layer - fully connected layer; the second CNN model includes three convolutional layers, four pooling layers, and one fully connected layer, and its structure is convolutional layer - pooling layer - pooling layer - convolutional layer - pooling layer - convolutional layer - pooling layer - fully connected layer. The above deep learning network is trained through the following method: The training set includes many frames of surveillance images. Input the training set images into the model shown in Figure 3 to train the model until the output of the model converges, terminate the training, and obtain the trained model, that is, the above deep learning network.

[0066] S103. Integrate and display the analysis result data based on a pre-established PIM workstation.

[0067] The above PIM workstation can specifically refer to the description in the prior art, send the analysis result data to the Product Information Management (PIM) workstation, and the PIM workstation performs data integration and unifies the standards. The PIM workstation combines the on-site usage data obtained by the PIM workstation to perform three-dimensional visualization display of the construction site. Input the construction progress, whether there are construction anomalies, the location of construction anomalies, the construction worker numbers, and the management staff numbers into a report in a preset format to generate a standard form, input the above analysis result data into the PIM workstation, and call the visualization tool of PIM for visualization display. Specifically, the generated data form can be made into a report and directly exhibited in the form of a report, such as a histogram. The present invention can unify information with a large amount of data, automatically collect on-site information, automatically extract important information for analysis, reduce the data error rate, improve data accuracy, and improve the accuracy and reliability of intelligent analysis at the construction site.

[0068] The present invention relates to the field of electronic information technology. Aiming at the characteristics of fast data update during the construction process of petrochemical engineering construction projects, and the problems of low efficiency and high error rate in the means of manual data collection, the present invention adopts the method of video monitoring to collect the monitoring video of the construction site in real time, evaluate the image quality of each frame of the monitoring video, and restore the images with unqualified image quality, which can retain the original features of the images. Then, based on a pre-established deep learning network, the restored monitoring video of the images is analyzed to obtain analysis result data. By means of artificial intelligence, the analysis result data is automatically obtained, with a low error rate and high efficiency. Moreover, by analyzing the restored monitoring video of the images, the accuracy and reliability of intelligent analysis at the construction site are improved. Aiming at the problem that although there is a large amount of information, there is no unified information exchange and delivery standard, based on a pre-established PIM workstation, the analysis result data is integrated, and the PIM technology is used to unify a large amount of information, improving the convenience and reliability of enterprise construction project management and the business efficiency of the enterprise.

[0069] In an optional embodiment, the method further includes:

[0070] Obtain a three-dimensional sand table of the construction site, compare the analysis result data with the scene data in the three-dimensional sand table, and obtain a comparison result.

[0071] Construct a three-dimensional sand table of the construction site through the PIM model, then select a time in the sand table to display the current progress, and monitor the current progress: compare the analysis result data with the scene data in the three-dimensional sand table to realize intelligent analysis by comparing the three-dimensional model with the actual video scene. The process of constructing the three-dimensional sand table of the construction site can refer to the description in the prior art and will not be elaborated here.

[0072] In an alternative embodiment, the image quality of each frame of the monitoring video in step S101 is evaluated as follows:

[0073] For each frame of the monitoring video, the previous frame image of the current frame image is fused with a preset background image to obtain a fused image, and the current frame image is subtracted from the fused image to obtain a difference image; the image quality is evaluated based on the difference image.

[0074] The evaluation method for the image quality in the monitoring video is as follows: the background difference method is used to obtain the difference image. Specifically, the previous frame image of the current frame image is fused with the preset background image to obtain the background image for background difference operation, that is, the above-mentioned fused image. The specific fusion method is to perform weighted summation on the pixel values of the pixel points at the same position according to a preset proportional weight. Then, the current frame image is subtracted from the fused image to obtain the difference image. If the mean square error of the difference image is within a preset range (1 - 150), it is determined that the quality of this frame of image meets the standard. If the mean square error is not within this range, it is determined that the quality of the current frame of image is poor and needs to be restored or directly deleted. In this embodiment, the pixel values of the pixel points at the same position are weighted and summed according to the proportional weights of 0.6 for the previous frame image and 0.4 for the preset background image to obtain the above-mentioned fused image.

[0075] In an alternative embodiment, the images with unqualified image quality are restored as follows:

[0076] A Gaussian model is constructed with the image with unqualified image quality as the center point. For each pixel point of the image with unqualified image quality: the pixel value of the pixel point and the pixel values of the corresponding pixel points in multiple previous and subsequent frames of the image are respectively input into the Gaussian model in chronological order to obtain corresponding multiple model output values; based on the multiple model output values, the weighted coefficients of the pixel points of each frame of image are respectively determined; according to the weighted coefficients of the pixel points of each frame of image, the following formula is used to perform weighted summation on the pixel value of the pixel point and the pixel values of the corresponding pixel points in multiple previous and subsequent frames of the image to obtain the restored pixel value of the pixel point:

[0077]

[0078] In the formula, x′ i is the restored pixel value, d j is the weighted coefficient of the pixel point of the jth frame of image, x i is the pixel value of the pixel point of the ith frame of image, M is the number of subsequent images, and N is the number of previous images.

[0079] The adopted image restoration method can be: fusing the first N frames of images and the last M frames of images of the image to be restored, where N = 2, 3, 4 and M = 3, 4, 5. The fusion method is: constructing a Gaussian model with the current frame image (the image to be restored) as the center point, that is, taking a certain pixel point of the current frame image as the center point of the Gaussian model, and inputting the pixel value of this pixel point, the first N pixel points corresponding to this pixel point in the first N frames of images respectively, and the last M pixel points corresponding to this pixel point in the last M frames of images respectively (a total of M + N + 1 values) into the Gaussian model in chronological order, and correspondingly obtaining M + N + 1 output values (which have a one-to-one correspondence with the image pixel points). Calculate the weighting coefficient of each pixel point based on the output value of the Gaussian model. Specifically:

[0080]

[0081] In the formula, d i is the weighting coefficient of the pixel point of the i-th frame image, and y i is the i-th model output value; where the Gaussian model is:

[0082]

[0083] In the formula, μ is the mean value of the pixel values of the same pixel point in the current frame image and multiple frames of images before and after, σ is the standard deviation of the pixel values of the same pixel point in the current frame image and multiple frames of images before and after, and x i is the pixel value of the pixel point of the i-th frame image.

[0084] Perform weighted summation on the pixel values of the corresponding pixel points to obtain the restored pixel value of the pixel point of the current frame image. Referring to Figure 2 as shown, the current frame is the image to be restored. First, obtain the first 2 frames and the last 2 frames, and then combine the current frame and sort them according to the shooting time to obtain an image sequence. Input the pixel values corresponding to the same pixel point in this image sequence into the Gaussian model in turn to obtain 5 model output values.

[0085] If the image quality does not meet the standard, through the fusion of the first N frames and the last M frames, while retaining the original features of the image, the influence of the fusion of the first N frames and the last M frames on this image is considered, so that the fused image has high fidelity and high accuracy. Calculating the weighting coefficient of the pixel point through the Gaussian model conforms to the law of the influence degree of different images on the current image: the images closer to this image have a greater influence on this image (the closer, the more similar, and the weighting coefficient should be larger). By weighting and restoring in this way, the reliability of image restoration is improved.

[0086] The present invention can unify information with a large amount of data, automatically collect on-site information, automatically extract important information for analysis, reduce the data error rate, improve data accuracy, and improve the accuracy and reliability of intelligent analysis at the construction site.

[0087] Embodiment 2

[0088] Based on the PIM technology and combined with video monitoring means, intelligent analysis and supervision of the construction site are realized. The specific method is as follows:

[0089] Step 1: Install monitoring cameras at the construction site, and collect the monitoring videos of the construction site through the monitoring cameras.

[0090] Step 2: Send the monitoring videos to the server background, and the server background uses artificial intelligence to analyze the monitoring videos to obtain the analysis result data.

[0091] S1: Preprocess the monitoring videos: including operations such as denoising and enhancement. In addition, for some images with poor quality, image restoration operations can also be performed on this frame of image, or directly delete this image with poor quality.

[0092] The evaluation method for the image quality in the monitoring videos is as follows: The background difference method is used to obtain the difference image (the background image for background difference operation is obtained by fusing the previous frame image of the current frame image with the preset background image, and the specific fusion method can be to perform weighted summation on the pixel values of the same position pixel points). If the mean square error of the difference image is within a certain range (1 - 150), it is determined that the quality of this frame of image meets the standard. If the mean square error is not within this range, it is determined that the quality of the current frame of image is poor and needs to be restored, or directly deleted.

[0093] The image restoration method adopted can be: fuse the first N frame images and the last M frame images of the image to be restored, N = 2, 3, 4, M = 3, 4, 5. The fusion method is: construct a Gaussian model with the current frame image (the image to be restored) as the center point, that is, take a certain pixel point of the current frame image as the center point of the Gaussian model, and input the pixel value of this pixel point and the pixel values of the first N pixel points corresponding to this pixel point in the first N frame images and the last M pixel points corresponding to this pixel point in the last M frame images (a total of M + N + 1 values) into the Gaussian model in chronological order, and correspondingly obtain M + N + 1 output values (there is a one-to-one correspondence with the image pixel points). Calculate the weighting coefficient of each pixel point based on the output values of the Gaussian model. Specifically:

[0094]

[0095] In the formula, d iis the weighting coefficient of the pixel point of the i-th frame image, y i is the output value of the i-th model; among them, the Gaussian model is:

[0096]

[0097] In the formula, μ is the mean value of the pixel values of the same pixel point in the current frame image and multiple frames of images before and after, σ is the standard deviation of the pixel values of the same pixel point in the current frame image and multiple frames of images before and after, and x i is the pixel value of the pixel point of the i-th frame image.

[0098] Perform weighted summation on the pixel values of the corresponding pixel points to obtain the restored pixel value of the pixel point of the current frame image:

[0099]

[0100] In the formula, x′ i is the restored pixel value.

[0101] The current frame is the image to be restored. First, obtain the first N frames and the last M frames, and then combine the current frame and sort them according to the shooting time to obtain an image sequence.

[0102] S2: Input the preprocessed monitoring image into the deep learning network. The model structure of the deep learning network is a CNN model. For a single CNN model, the CNN model directly learns the analysis result data based on the monitoring video, including information such as whether there is a construction anomaly, the location of the construction anomaly, the construction worker number, and the manager label. Among them, the CNN includes 3 convolutional layers, 2 pooling layers, and a fully connected layer, and its structure is convolutional layer - pooling layer - convolutional layer - pooling layer - convolutional layer - fully connected layer.

[0103] Step 3: Send the analysis result data to the PIM workstation. The PIM workstation performs data integration and unifies the standards. The PIM workstation combines the on-site data obtained by the PIM workstation to perform a three-dimensional visualization display of the construction site.

[0104] Construct a three-dimensional sand table of the construction site through the PIM model, and then select the time in the sand table to display the current progress. Monitor the current progress: Compare the analysis result data with the scene data in the three-dimensional sand table to realize the intelligent analysis by comparing the three-dimensional model and the actual video scene.

[0105] Example 3

[0106] Based on the PIM technology and combined with the means of video monitoring, realize the intelligent analysis and supervision of the construction site. The specific method is:

[0107] Step 1: Obtain the monitoring video by shooting the construction site with a 720 drone.

[0108] Step 2: Send the surveillance video to the server background. The server background analyzes the surveillance video by means of artificial intelligence to obtain the analysis result data.

[0109] S1: Preprocess the surveillance video: including operations such as denoising and enhancement. In addition, for some images with poor quality, image restoration operations can also be performed on this frame of image, or directly delete this image with poor quality.

[0110] The evaluation method for the image quality in the surveillance video is as follows: Obtain the difference image by using the background difference method (the background image for background difference operation is obtained by fusing the previous frame image of the current frame image and the preset background image, and the specific fusion method can be to perform weighted summation on the pixel values of the same-position pixel points). If the mean square error of the difference image is within a certain range (1 - 150), it is determined that the quality of this frame of image meets the standard. If the mean square error is not within this range, it is determined that the quality of the current frame of image is poor and restoration operations need to be performed or directly deleted.

[0111] The image restoration method adopted can be: fuse the first N frame images and the last M frame images of the image to be restored, where N = 2, 3, 4 and M = 3, 4, 5. The fusion method is: construct a Gaussian model with the current frame image (the image to be restored) as the center point, that is, take a certain pixel point of the current frame image as the center point of the Gaussian model, and input the pixel value of this pixel point and the pixel values of the first N pixel points corresponding to this pixel point in the first N frame images and the pixel values of the last M pixel points corresponding to this pixel point in the last M frame images (a total of M + N + 1 values) into the Gaussian model in chronological order, and correspondingly obtain M + N + 1 output values (there is a one-to-one correspondence with the image pixel points). Calculate the weighting coefficient of each pixel point based on the output value of the Gaussian model. Specifically:

[0112]

[0113] In the formula, d i is the weighting coefficient of the pixel point of the i-th frame image, and y i is the i-th model output value; where the Gaussian model is:

[0114]

[0115] In the formula, μ is the mean value of the pixel values of the same pixel point in the current frame image and multiple front and rear frame images, σ is the standard deviation of the pixel values of the same pixel point in the current frame image and multiple front and rear frame images, and x i is the pixel value of the pixel point of the i-th frame image

[0116] Perform weighted summation on the pixel values of the corresponding pixel points to obtain the restored pixel value of the pixel point in the current frame image:

[0117]

[0118] In the formula, x i ′ is the restored pixel value.

[0119] The current frame is the image to be restored. First, obtain the first N frames and the last M frames, and then combine the current frame and sort them according to the shooting time to obtain an image sequence.

[0120] S2: Input the preprocessed surveillance image into the deep learning network. The model structure of the deep learning network is a CNN model. For a single CNN model, the CNN model directly learns the analysis result data based on the surveillance video, including information such as whether there is a construction anomaly, the location of the construction anomaly, the construction worker number, the manager label, etc. Among them, the CNN includes 3 convolutional layers, 2 pooling layers and a fully connected layer, and its structure is convolutional layer - pooling layer - convolutional layer - pooling layer - convolutional layer - fully connected layer.

[0121] Step 3: Send the analysis result data to the PIM workstation. The PIM workstation performs data integration and unifies the standards. The PIM workstation combines the on-site data obtained by the PIM workstation to perform three-dimensional visualization display of the construction site.

[0122] Construct a three-dimensional sand table of the construction site through the PIM model, and then select the time in the sand table to display the current progress. Monitor the current progress: Compare the analysis result data with the scene data in the three-dimensional sand table to realize intelligent analysis by comparing the three-dimensional model with the actual video scene.

[0123] Example 4

[0124] The embodiment of the present invention provides an intelligent supervision device for a petrochemical engineering construction site. Referring to Figure 4 as shown, it includes:

[0125] An evaluation module 201, configured to collect the surveillance video of the construction site, evaluate the image quality of each frame of the image in the surveillance video, and restore the image with unqualified image quality to obtain the processed surveillance video;

[0126] An analysis module 202, configured to analyze the processed surveillance video based on a pre-established deep learning network to obtain analysis result data;

[0127] An integration module 203, configured to perform data integration and display on the analysis result data based on a pre-established PIM workstation.

[0128] The intelligent supervision device for petrochemical engineering construction sites provided by the embodiments of the present invention has a similar implementation principle and technical effect to any of the foregoing method embodiments, and will not be elaborated here.

[0129] Embodiment 5

[0130] The embodiments of the present invention provide a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the intelligent supervision method for petrochemical engineering construction sites as described in any of the foregoing method embodiments.

[0131] The computer-readable storage medium may be included in the device / apparatus described in the above embodiments; or it may exist alone without being assembled into the device / apparatus. The above computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the method according to the embodiments of the present invention is implemented.

[0132] According to the embodiments of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: portable computer disks, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, device, or device.

[0133] Embodiment 6

[0134] The present invention provides an electronic device. Referring to Figure 5 as shown, it includes a processor 111, a communication interface 112, a memory 113, and a communication bus 114. Among them, the processor 111, the communication interface 112, and the memory 113 complete mutual communication through the communication bus 114.

[0135] The memory 113 is used to store a computer program.

[0136] When the processor 111 executes the program stored on the memory 113, it implements the intelligent supervision method for petrochemical engineering construction sites as described in any of the foregoing method embodiments.

[0137] The electronic device provided by the embodiments of the present invention has a similar implementation principle and technical effect to any of the foregoing method embodiments, and will not be elaborated here.

[0138] The above-mentioned memory 113 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. The memory 113 has a storage space for program codes for executing any of the method steps in the above-mentioned method. For example, the storage space for program codes may include respective program codes for implementing the respective steps in the above method. These program codes may be read from or written into one or more computer program products. These computer program products include program code carriers such as hard disks, optical discs (CDs), memory cards, or floppy disks. Such computer program products are usually portable or fixed storage units. The storage unit may have a storage segment or a storage space, etc., arranged similarly to the memory 113 in the above-mentioned electronic device. The program codes may be compressed in an appropriate form, for example. Generally, the storage unit includes a program for executing the method steps according to the embodiments of the present invention, that is, codes that can be read by, for example, the processor 111. When these codes are run by the electronic device, they cause the electronic device to execute the respective steps in the method described above.

[0139] Each embodiment in this specification is described in a progressive manner, and the same or similar parts among the embodiments may be referred to each other. Each embodiment focuses on the differences from other embodiments. It should be noted that, without conflict, the embodiments and features in the present invention may be combined with each other. The present invention is not limited to any single aspect, nor to any single embodiment, nor to any arbitrary combination and / or permutation of these aspects and / or embodiments. Each aspect and / or embodiment of the present invention may be used alone, or in combination with one or more other aspects and / or other embodiments.

[0140] Finally, it should be noted that the above-mentioned embodiments are only specific embodiments of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting them. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or make equivalent replacements for some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. An intelligent supervision method for petrochemical engineering construction sites, characterized in that, it includes: Collecting the monitoring videos of the construction site, evaluating the image quality of each frame of the monitoring videos, and restoring the images with unqualified image quality to obtain the processed monitoring videos; Analyzing the processed monitoring videos based on a pre-established deep learning network to obtain analysis result data; Integrating and displaying the analysis result data based on a pre-established PIM workstation.

2. The intelligent supervision method for petrochemical engineering construction sites according to claim 1, characterized in that, The images with unqualified image quality are restored in the following manner: Constructing a Gaussian model with the image with unqualified image quality as the center point, and for each pixel point of the image with unqualified image quality: Inputting the pixel value of the pixel point and the pixel values of the corresponding pixel points in multiple frames of images before and after the pixel point into the Gaussian model in chronological order to obtain corresponding multiple model output values; Based on the multiple model output values, respectively determining the weighting coefficients of the pixel points of each frame of image; According to the weighting coefficients of the pixel points of each frame of image, the pixel value of the pixel point and the pixel values of the corresponding pixel points in multiple frames of images before and after the pixel point are weighted and summed through the following formula to obtain the restored pixel value of the pixel point: where x i i is the restored pixel value, d j is the weighting coefficient of the pixel point of the j-th frame image, x i is the pixel value of the pixel point of the i-th frame image, M is the number of subsequent images, and N is the number of previous images.

3. The intelligent supervision method for petrochemical engineering construction sites according to claim 2, characterized in that, The weighting coefficients of the pixel points of each frame of image are determined through the following formula: where d i is the weighting coefficient of the pixel point of the i-th frame image, and y i is the output value of the i-th model; wherein, the Gaussian model is: Where μ is the mean value of the pixel values of the same pixel point in the current frame image and multiple frames of images before and after, σ is the standard deviation of the pixel values of the same pixel point in the current frame image and multiple frames of images before and after, and x i is the pixel value of the pixel point in the i-th frame image.

4. The intelligent supervision method for petrochemical engineering construction sites according to claim 1, characterized in that, The image quality of each frame of the monitoring videos is evaluated in the following manner: For each frame of the monitoring videos, fusing the previous frame image of the current frame image with a preset background image to obtain a fused image, and subtracting the fused image from the current frame image to obtain a difference image; Based on the difference image, the image quality is evaluated.

5. The intelligent supervision method for petrochemical engineering construction sites according to claim 4, characterized in that, The previous frame image of the current frame image and the preset background image are fused to obtain a fused image in the following manner: The pixel values of the pixel points at the same positions in the previous frame image of the current frame image and the preset background image are weighted and summed according to a preset proportional weight to obtain a fused image.

6. The intelligent supervision method for petrochemical engineering construction sites according to claim 1, characterized in that, The deep learning network includes a first CNN model, an LSTM, and a second CNN model; The analysis of the processed monitoring videos based on the pre-established deep learning network to obtain analysis result data includes: Based on the first CNN model, respectively extracting features from each frame of the processed monitoring videos to obtain corresponding monitoring features; Inputting the monitoring features of all images into the LSTM model in chronological order, and predicting and correcting the feature sequence based on the LSTM model to obtain a predicted and corrected feature sequence; Performing feature analysis on the corrected monitoring features based on the second CNN model to obtain analysis result data.

7. The intelligent supervision method for petrochemical engineering construction sites according to claim 1, wherein, the analysis result data includes construction progress, whether there is construction abnormality, construction abnormality location, construction personnel number, and management personnel label; the data integration and display of the analysis result data based on a pre-established PIM workstation includes: Inputting the construction progress, whether there is construction abnormality, construction abnormality location, construction personnel number, and management personnel label into a report form in a preset format to generate a standard form for visual display.

8. The intelligent supervision method for petrochemical engineering construction sites according to claim 1, wherein, the method further includes: Obtaining a three-dimensional sand table of the construction site, comparing the analysis result data with the scene data in the three-dimensional sand table to obtain a comparison result.

9. The intelligent supervision method for petrochemical engineering construction sites according to claim 4, wherein, the image quality evaluation based on the difference image includes: Determining the mean square error of the pixel values of all pixel points in the difference image, and judging whether the mean square error is within a preset range; If so, the image quality meets the standard; if not, the image quality does not meet the standard.

10. An intelligent supervision device for petrochemical engineering construction sites, wherein, it includes: An evaluation module, configured to collect monitoring videos of the construction site, perform image quality evaluation on each frame of the monitoring videos, and restore the images with unqualified image quality to obtain processed monitoring videos; An analysis module, configured to analyze the processed monitoring videos based on a pre-established deep learning network to obtain analysis result data; An integration module, configured to perform data integration and display on the analysis result data based on a pre-established PIM workstation.

11. A computer-readable storage medium, on which a computer program is stored, wherein, when the program is executed by a processor, it implements the intelligent supervision method for petrochemical engineering construction sites according to any one of claims 1-9.

12. An electronic device, wherein, it includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus; The memory is used to store a computer program; The processor, when executing the program stored on the memory, implements the intelligent supervision method for petrochemical engineering construction sites according to any one of claims 1-9.

Images