LDAR file quality assessment method based on multi-source data fusion

Abstract

The present application discloses a LDAR archiving quality assessment method based on multi-source data fusion, the method comprising: obtaining original images of target industrial enterprise equipment and pipeline components and a marked image to be analyzed; determining an image quality assessment result of the original image using a preset clarity evaluation model; extracting pixel features of the original image according to the image quality assessment result; extracting attribute features of the target industrial enterprise equipment and pipeline components in the original image, and performing feature fusion on the pixel features and the attribute features to obtain target features; matching the equipment component tag name corresponding to the target feature from a preset feature database, and performing image tagging on the original image based on the equipment component tag name to obtain a target marked image; comparing the target marked image with the marked image to be analyzed to obtain a tagging error rate of the marked image to be analyzed, and determining a target assessment result based on the tagging error rate, thereby improving the assessment efficiency of LDAR archiving.

Description

Technical Field

[0001] The present application belongs to the field of environmental protection technology, and in particular, relates to a LDAR file quality assessment method based on multi-source data fusion. Background Art

[0002] Leak Detection And Repair (LDAR) is a powerful tool for the unorganized control of volatile organic compounds (VOCs) in industrial enterprises, and is also an important means to achieve refined VOCs governance. It achieves the purpose of reducing VOCs leakage emissions by implementing quantitative detection of the leakage concentration of equipment components, discovering leakage points and repairing them in a timely manner. The identification of project sealing points is the core work of LDAR filing. Filing specifically refers to the process of locating and encoding sealing points and establishing archives based on the identification of controlled ranges and controlled points, collecting information such as the device, area, type, and accessibility of the sealing points. Filing work is the basis for the subsequent on-site implementation of LDAR, and it is also the most critical step to ensure the smooth implementation of LDAR work.

[0003] With the increasing emphasis on the control of volatile organic compounds, equipment leak detection and repair technology has been increasingly widely used, but chaos in the implementation process has also become increasingly apparent. Therefore, it is necessary to accurately control the quality of LDAR implementation and conduct a comprehensive assessment of the integrity, accuracy and compliance of LDAR filing, thereby providing a solid foundation for subsequent leak detection and repair work.

[0004] The traditional LDAR evaluation method often relies on manual sampling. However, due to the large number of sealing points, this method requires a lot of time and manpower, and the evaluation efficiency is low.

[0005] Therefore, how to improve the efficiency of filing and evaluation of industrial enterprise equipment and pipeline components is a technical problem that needs to be solved urgently. Summary of the invention

[0006] The present application provides an LDAR file quality assessment method based on multi-source data fusion, which can quickly and efficiently assess the files established by enterprises in the file creation process, thereby improving the efficiency of file creation and assessment of industrial enterprise equipment and pipeline components.

[0007] On the one hand, an embodiment of the present application provides an LDAR archiving quality assessment method based on multi-source data fusion, the method comprising: obtaining an original image of the target industrial enterprise equipment and pipeline components and a marked image to be analyzed corresponding to the original image; the marked image to be analyzed is an image obtained by marking the name of the target industrial enterprise equipment and pipeline components on the original image during the archiving link of the equipment leakage detection and repair process; determining the image quality assessment result of the original image using a preset clarity evaluation model; extracting pixel features of the original image based on the image quality assessment result; the pixel features represent the features of the pixels occupied by the target industrial enterprise equipment and pipeline components in the original image; extracting the attribute features of the target industrial enterprise equipment and pipeline components in the original image, and performing feature fusion on the pixel features and the attribute features to obtain the target features; matching the equipment component label name corresponding to the target feature from a preset feature database, and labeling the original image based on the equipment component label name to obtain the target label image; the preset feature database records the correspondence between the preset target feature and the preset equipment component label name; comparing the target label image with the labeled image to be analyzed to obtain the labeling error rate of the labeled image to be analyzed, and determining the target assessment result based on the labeling error rate.

[0008] In an exemplary embodiment, after obtaining the target marked image, the method further includes:

[0009] Constructing a target three-dimensional model corresponding to the target industrial enterprise equipment and pipeline components according to the target marked image, wherein the target three-dimensional model is marked with the equipment component mark name;

[0010] The target three-dimensional model is compared with the labeled image to be analyzed to obtain the labeling error rate of the labeled image to be analyzed.

[0011] In an exemplary embodiment, extracting pixel features of the original image includes:

[0012] Performing pixel clustering processing on the original image to obtain the center point position of the target industrial enterprise equipment and pipeline components in the original image;

[0013] Determine the pixel value at the center point position and the frequency value of the pixel value at the center point position as the main color feature;

[0014] Obtaining a gray level co-occurrence matrix corresponding to the original image;

[0015] Determine the energy value and contrast of the original image based on the gray level co-occurrence matrix to obtain texture features;

[0016] The main color feature and the texture feature are determined as the pixel features.

[0017] In an exemplary embodiment, extracting attribute features of the target industrial enterprise equipment and pipeline components in the original image includes:

[0018] Acquire the size features of the target industrial enterprise equipment and pipeline components in the original image;

[0019] Determine, based on the original image, the three-dimensional contours corresponding to the target industrial enterprise equipment and pipeline components;

[0020] Performing contour curvature extraction processing based on the three-dimensional contour to obtain shape features of the target industrial enterprise equipment and pipeline components;

[0021] The size feature and the shape feature are determined as the attribute features.

[0022] In an exemplary embodiment, the performing of contour curvature extraction processing based on the three-dimensional contour to obtain shape features of the target industrial enterprise equipment and pipeline components includes:

[0023] Dividing the three-dimensional contour in the horizontal direction to obtain the outer edge of the contour;

[0024] Sampling the outer edge of the contour according to a preset sampling interval to obtain a plurality of contour key points;

[0025] Determine the sine value corresponding to each of the contour key points on the contour outer edge line, and determine the curvature value of the contour outer edge line according to the sine value;

[0026] An average value of the curvature values ​​is determined as the shape feature of the target industrial enterprise equipment and pipeline components.

[0027] In an exemplary embodiment, comparing the target marked image with the marked image to be analyzed to obtain a marking error rate of the marked image to be analyzed includes:

[0028] Compare the target marked image with the marked image to be analyzed to obtain the amount of erroneous marking in the marked image to be analyzed; the amount of erroneous marking is the sum of the amount of missed marking, the amount of multiple marking, the amount of wrong marking and the amount of wrong marking position;

[0029] The ratio between the amount of erroneous labeling and the number of the device component label names labeled in the target label image is determined as the labeling error rate.

[0030] In an exemplary embodiment, the process of constructing the preset clarity evaluation model includes:

[0031] Acquire training images of multiple industrial enterprise equipment and pipeline components to obtain a training image set, wherein the training images are annotated with clarity level labels;

[0032] The initial clarity evaluation model is trained according to the training image set to generate the preset clarity evaluation model.

[0033] On the other hand, the present application also provides an LDAR archiving quality assessment device based on multi-source data fusion, the device comprising: an image acquisition module, used to acquire an original image of a target industrial enterprise equipment and pipeline components and a marked image to be analyzed corresponding to the original image; the marked image to be analyzed is an image obtained by marking the name of the target industrial enterprise equipment and pipeline components on the original image during the archiving link of the equipment leakage detection and repair process;

[0034] An image quality assessment module, used to determine an image quality assessment result of the original image using a preset clarity assessment model;

[0035] A pixel feature determination module, used to extract pixel features of the original image according to the image quality assessment result; the pixel features represent features of pixels occupied by the target industrial enterprise equipment and pipeline components in the original image;

[0036] An attribute feature determination module is used to extract the attribute features of the target industrial enterprise equipment and pipeline components in the original image, and perform feature fusion on the pixel features and the attribute features to obtain target features;

[0037] An image marking module is used to match the device component marking name corresponding to the target feature from a preset feature database, and mark the original image based on the device component marking name to obtain a target marked image; the preset feature database records the correspondence between the preset target feature and the preset device component marking name;

[0038] The marking comparison module is used to compare the target marked image with the marked image to be analyzed, obtain the marking error rate of the marked image to be analyzed, and determine the target evaluation result based on the marking error rate.

[0039] On the other hand, the present application also provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the LDAR archiving quality assessment method based on multi-source data fusion as described above.

[0040] On the other hand, the present application also provides a computer storage medium, which stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to implement the LDAR archiving quality assessment method based on multi-source data fusion as described above.

[0041] On the other hand, the present application also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device component reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device component executes to implement the LDAR archiving quality assessment method based on multi-source data fusion as described above.

[0042] The method provided by this application has the following technical effects:

[0043] In the embodiment of the present application, the original image of the target industrial enterprise equipment and pipeline components in the enterprise LDAR file and the marked image to be analyzed corresponding to the original image are first obtained, and then the clarity evaluation model is used to automatically determine whether the file image is a clear image that can be edited point by point. Then, the pixel features of the original image and the attribute features of the target industrial enterprise equipment and pipeline components therein are further extracted, and the target features are obtained after the features of the two are fused; then the equipment component label name corresponding to the target feature is matched from the preset feature database, and the original image is labeled based on the equipment component label name to obtain the target labeled image; finally, the target labeled image is compared with the labeled image to be analyzed to obtain the labeling error rate of the labeled image to be analyzed, and the target evaluation result is determined based on the labeling error rate. The method of the present application can automatically analyze the images uploaded by the enterprise for file creation, and obtain comprehensive and accurate features for describing the target industrial enterprise equipment and pipeline components contained in the original image by extracting the multi-dimensional features of the original image, thereby accurately matching the equipment component label names existing in the original image, so that the evaluation of enterprise file creation does not rely on manual review, but instead achieves fast and accurate enterprise LDAR file review through an accurate intelligent image analysis process, effectively improving the evaluation efficiency of LDAR file creation. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1This is an application environment diagram of the LDAR archiving quality assessment method based on multi-source data fusion provided in an embodiment of the present application.

[0046] Figure 2 It is a flow chart of the LDAR file quality assessment method based on multi-source data fusion provided in an embodiment of the present application.

[0047] Figure 3 It is a schematic diagram of the process of extracting pixel features of the original image provided in an embodiment of the present application.

[0048] Figure 4 It is a schematic diagram of a process for extracting attribute features of target industrial enterprise equipment and pipeline components in an original image provided by an embodiment of the present application.

[0049] Figure 5 It is a schematic diagram of the process of determining shape features provided in an embodiment of the present application.

[0050] Figure 6 It is a structural block diagram of the LDAR file quality assessment system based on multi-source data fusion provided in an embodiment of the present application.

[0051] Figure 7 It is a structural diagram of an LDAR archiving quality assessment device based on multi-source data fusion provided in an embodiment of the present application.

[0052] Figure 8 This is a hardware structure block diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0054] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or device components.

[0055] Figure 1 This is an application environment diagram of the LDAR archiving quality assessment method based on multi-source data fusion provided in an embodiment of the present application.

[0056] like Figure 1 As shown, the application environment may include at least a server 01 and a terminal 02. In an optional embodiment, the server 01 may be used to extract multiple dimensional features from the original image and to establish a three-dimensional model corresponding to the target industrial enterprise equipment and pipeline components, etc. The server 01 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. In an optional embodiment, the terminal 02 may be used to receive the archives uploaded by the enterprise for archiving the target industrial enterprise equipment and pipeline components, and send the archive data to the server 01 for archiving evaluation; the terminal 02 may also be used to display the target evaluation results. Specifically, the terminal 02 may include but is not limited to electronic devices such as smart phones, desktop computers, tablet computers, laptops, smart speakers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, smart wearable devices, vehicle-mounted terminals, smart TVs, etc.; it may also be software running on the above electronic devices, such as applications, applets, etc. The operating system running on the electronic device in the embodiment of the present application may include but is not limited to Android system, IOS system, linux, windows, etc.

[0057] Figure 2 It is a flow chart of the LDAR file quality assessment method based on multi-source data fusion provided in an embodiment of the present application.

[0058] S201: Acquire original images of equipment and pipeline components of a target industrial enterprise and marked images to be analyzed corresponding to the original images;

[0059] Among them, the marked image to be analyzed is an image obtained by marking the names of the target industrial enterprise equipment and pipeline components on the original image during the archiving stage of the equipment leak detection and repair (LDAR) process.

[0060] The target industrial enterprise equipment and pipeline components are equipment and pipeline components that carry volatile organic compounds (VOCs) materials. These equipment and pipeline components have sealing points that may leak volatile organic compounds, such as pumps, compressors (shaft seals), mixers (shaft seals), valves, pressure relief devices (safety valves), sampling connection systems, open ends or open pipelines, flanges and connectors (threaded connections), etc.

[0061] Leak detection and repair is a systematic project that uses conventional or unconventional detection methods to detect or check sealing points, and take effective measures to repair leakage points within a certain period of time to control material leakage in the entire industrial production process. In this process, enterprises need to establish archives for the information of the sealing points involved in accordance with the standards. Among them, the archive content includes the process flow charts, pipeline instrumentation diagrams, material balance sheets, process operating procedures, original images of the equipment and pipeline components of the target industrial enterprise uploaded by the enterprise, and the original images edited by points, that is, the marked images to be analyzed after the sealing points are marked. By evaluating the errors or omissions in the marked images to be analyzed, it is possible to quickly determine whether the enterprise's archive is accurate, so as to take corresponding early warning and treatment measures.

[0062] S203: Determine the image quality evaluation result of the original image by using a preset clarity evaluation model;

[0063] After obtaining the original images of the target industrial enterprise's equipment and pipeline components, first determine whether the clarity of the original image meets the conditions, that is, whether normal point editing can be performed. Specifically, the preset clarity evaluation model can be used to perform clarity evaluation processing on the original image to obtain the clarity level of the original image. Then compare the clarity level with the preset clarity threshold. If the clarity level is greater than or equal to the preset clarity threshold, the image quality assessment result of the LDAR file corresponding to the industrial enterprise project is that the point editing is normal and the image clarity is normal; if the clarity level is less than the preset clarity threshold, the image quality assessment result of the LDAR file corresponding to the industrial enterprise project is that the point editing is abnormal and the image is blurred, and a prompt message is displayed. The prompt message is used to prompt the enterprise to re-edit the point and re-shoot the image before archiving.

[0064] In another implementation, the clarity level can also be compared with a preset reference interval. If the clarity level is within the preset reference interval, the image quality assessment result of the LDAR file is that the point editing is normal and the image clarity is normal; if the clarity feature is less than the preset reference interval, the image quality assessment result of the LDAR file is that the point editing is abnormal and the image is blurred.

[0065] S205: extracting pixel features of the original image according to the image quality evaluation result;

[0066] The pixel features represent the features of the pixels occupied by the target industrial enterprise equipment and pipeline components in the original image.

[0067] When the image quality assessment result indicates that the point editing is normal and the image clarity is normal, the image content of the original image can be further analyzed to obtain the target features of the target industrial enterprise equipment and pipeline components. The features extracted in the embodiment of the present application include the features of the pixels occupied by the target industrial enterprise equipment and pipeline components in the original image, such as the features of the center point pixels of the target industrial enterprise equipment and pipeline components in the original image.

[0068] S207: extracting attribute features of the target industrial enterprise equipment and pipeline components in the original image, and performing feature fusion on the pixel features and the attribute features to obtain target features;

[0069] In an embodiment of the present application, attribute features of the target industrial enterprise equipment and pipeline components can also be extracted based on the original image. The attribute features represent the inherent properties of the target industrial enterprise equipment and pipeline components, including shape features, size features, etc.

[0070] By fusing the above pixel features and attribute features, target features can be obtained, which can accurately and comprehensively describe the content of the original image, so as to facilitate subsequent use in determining the label names corresponding to the target industrial enterprise equipment and pipeline components contained in the original image.

[0071] S209: matching a device component label name corresponding to the target feature from a preset feature database, and marking the original image based on the device component label name to obtain a target labeled image;

[0072] The preset feature database records the correspondence between preset target features and preset device component tag names.

[0073] The preset feature database records the preset target features corresponding to the preset industrial enterprise equipment and pipeline components and the corresponding preset equipment component tag names. The similarity between the preset target features and the target features is calculated, and the similarities are sorted from high to low. The tag name corresponding to the preset target feature in the front ranking is determined as the equipment component tag name of the target industrial enterprise equipment and pipeline components.

[0074] In one implementation, the equipment component tag name may be a unique number of the equipment and pipeline components of the target industrial enterprise. For example, the number M21N0101FVL0 may be used to represent a valve in the first zone of a reactor.

[0075] Then, the above equipment component mark names are marked at the locations of the target industrial enterprise equipment and pipeline components in the original image to obtain the target marked image.

[0076] It should be noted that there may be multiple equipment component tag names or only one equipment component tag name in the target tag image, and the number of equipment component tag names corresponds to the number of target industrial enterprise equipment and pipeline components in the original image.

[0077] S211: Compare the target marked image with the marked image to be analyzed to obtain a marking error rate of the marked image to be analyzed, and determine a target evaluation result based on the marking error rate.

[0078] By comparing the target marked image with the marked image to be analyzed, the errors or omissions of the marks in the marked image to be analyzed can be determined, thereby determining the evaluation results for archiving.

[0079] In the embodiment of the present application, the original image of the target industrial enterprise equipment and pipeline components in the enterprise LDAR file and the marked image to be analyzed corresponding to the original image are first obtained, and then the clarity evaluation model is used to automatically determine whether the file image is a clear image that can be edited point by point. Then, the pixel features of the original image and the attribute features of the target industrial enterprise equipment and pipeline components therein are further extracted, and the target features are obtained after the features of the two are fused; then the equipment component label name corresponding to the target feature is matched from the preset feature database, and the original image is labeled based on the equipment component label name to obtain the target labeled image; finally, the target labeled image is compared with the labeled image to be analyzed to obtain the labeling error rate of the labeled image to be analyzed, and the target evaluation result is determined based on the labeling error rate. The method of the present application can automatically analyze the images uploaded by the enterprise for file creation, and obtain comprehensive and accurate features for describing the target industrial enterprise equipment and pipeline components contained in the original image by extracting the multi-dimensional features of the original image, thereby accurately matching the equipment component label names existing in the original image, so that the evaluation of enterprise file creation does not rely on manual review, but instead achieves fast and accurate enterprise LDAR file review through an accurate intelligent image analysis process, effectively improving the evaluation efficiency of LDAR file creation.

[0080] It should be noted that in the actual production process, not all industrial enterprise equipment and pipeline components contain volatile organic materials. Therefore, before evaluating the clarity of the archived image, it is necessary to first determine the necessity of archiving. The process includes: obtaining the material information corresponding to the target industrial enterprise equipment and pipeline components; the material information represents the component information of the materials contained in the target industrial enterprise equipment and pipeline components and the content information corresponding to the component information; according to the component information and the content information, determine the mass proportion of volatile organic compounds contained in the material; when the mass proportion is greater than or equal to the preset content threshold, determine that the target industrial enterprise equipment and pipeline components meet the archiving conditions; otherwise, determine that the target industrial enterprise equipment and pipeline components do not meet the archiving conditions.

[0081] The materials corresponding to the equipment and pipeline components of industrial enterprises include the raw materials, intermediate products, final products and various additives involved. Component information refers to the type of each component in the material; content information refers to the mass proportion of each component. By obtaining the photos, bill of materials, process flow chart and material balance sheet of LDAR file, the mass proportion of each component of the material of the equipment and pipeline components of the target industrial enterprise can be obtained, which can be recorded as zl i , where i represents the number of the component type, i=1, 2, 3...I, and I is a positive integer. According to different production processes, the proportion of volatile organic compounds in each component can be calculated and recorded as VOCs i , where i represents the number of the component type, i=1, 2, 3...I, and I is a positive integer.

[0082] The mass percentage of volatile organic compounds contained in the material can be calculated by the following formula:

[0083]

[0084] Wherein, yl represents the mass proportion of volatile organic compounds in the equipment and pipeline components of the target industrial enterprise, zl i Indicates the mass proportion of each component in the material, VOCs i Indicates the proportion of volatile organic compounds in each component.

[0085] It should be noted that the preset content threshold can be specifically 10%, and the preset content threshold can be set according to the actual application standard. In the embodiment of the present application, the mass proportion of volatile organic matter of the target industrial enterprise equipment and pipeline components is first determined to determine the necessity of archiving, so as to avoid the unnecessary subsequent image analysis process of archiving industrial enterprise equipment and pipeline components that do not meet the archiving conditions. On the one hand, it saves computing resources, and on the other hand, it also makes the archiving evaluation process standardized.

[0086] In one embodiment, when the target industrial enterprise equipment and pipeline components meet the above-mentioned archiving conditions, the image quality assessment step can be performed. In this application, a preset clarity evaluation model is used to quickly implement image quality assessment. Specifically, the construction process of the preset clarity evaluation model may include: obtaining training images of multiple industrial enterprise equipment and pipeline components to obtain a training image set, wherein the training images are annotated with clarity level labels; according to the training image set, the initial clarity evaluation model is trained to generate the preset clarity evaluation model.

[0087] In one implementation, the training images may include multi-angle images of industrial enterprise equipment and pipeline components in the industrial site collected by an image acquisition device, and may also include standard images of industrial enterprise equipment and pipeline components in process drawings or pipeline instrumentation diagrams, and each training image is annotated with a clarity level label. Random noise may also be added to the above-collected training images, and clarity level labels may be annotated to the images obtained after adding random noise, thereby expanding the training image set. The training images are then input into the initial clarity evaluation model for clarity evaluation processing to obtain a predicted clarity level; based on the difference between the predicted clarity level and the clarity level label, a loss value is determined, and the constructed initial clarity evaluation model is trained according to the loss value until the training end condition is met, and the initial clarity evaluation model at the end of the training is determined as the preset clarity evaluation model.

[0088] After the clarity assessment is qualified, it is necessary to further evaluate the image marking to prevent the company from mislabeling or missing labels. In the embodiment of the present application, intelligent labeling is achieved by extracting pixel features and attribute features of equipment and pipeline components of the target industrial enterprise from the original image, so as to facilitate the subsequent comparison and evaluation of the labeled images to be analyzed uploaded by the enterprise.

[0089] Figure 3 : is a schematic diagram of a process for extracting pixel features of the original image provided by an embodiment of the present application. In one embodiment, the extraction of pixel features of the original image may include:

[0090] S301: performing pixel clustering processing on the original image to obtain the center point positions of the target industrial enterprise equipment and pipeline components in the original image;

[0091] Randomly select M initial cluster centers, calculate the Euclidean distance from each pixel in the original image to each initial cluster center, redistribute each pixel to the group of the cluster center with the smallest Euclidean distance, calculate the average value of the pixels in all groups, and use the pixel where the average value is located as the new cluster center. Repeat the steps of updating the cluster center until the change value of the cluster center reaches the set threshold or reaches the preset number of iterations. The cluster center after the iteration is determined as the center point position of the subject in the original image.

[0092] S303: Determine the pixel value at the center point position and the frequency value of the occurrence of the pixel value at the center point position as the main color feature;

[0093] Get the pixel values ​​of the M cluster centers after iteration, that is, the RGB values. Count the frequency of the RGB values ​​of each cluster center in the original image, recorded as pRGB M , the frequency value represents the pixel ratio of the subject corresponding to the cluster center in the original image. By introducing the pixel value of the center point position and the frequency value of the occurrence of the pixel value of the center point position, the characteristics of the subject corresponding to the target industrial enterprise equipment and pipeline components in the original image can be determined, so as to facilitate subsequent feature matching and determine the tag name.

[0094] S305: Obtaining a gray level co-occurrence matrix corresponding to the original image;

[0095] The Gray Level Co-occurrence Matrix (GLCM) is determined based on the grayscale image corresponding to the original image. Therefore, the original image needs to be converted into a grayscale image first. In a color image, three channels, red, green and blue, are usually included. A grayscale image is an image that removes color information and retains only brightness information. In one embodiment, different weights can be assigned to each color channel according to the different sensitivities of the human eye to different color channels to convert them into grayscale values. In one example, the weighting coefficient of the red channel (R) can be 0.299, the weighting coefficient of the green channel (G) can be 0.587, and the weighting coefficient of the blue channel (B) can be 0.114. The grayscale value corresponding to each pixel can be 0.299×R+0.587×G+0.114×B. The grayscale value is then normalized so that the grayscale value is adjusted to the [0,255] interval to obtain a grayscale image corresponding to the original image.

[0096] The gray-level co-occurrence matrix can be used to characterize the texture features of the original image. In one embodiment, the ... 2 Then count the number of occurrences of each (a, j) value, arrange them into a square matrix, and then use the total number of (a, j) occurrences to normalize them into the probability of occurrence P(a, j), thus forming a two-dimensional grayscale co-occurrence matrix. Among them, (dx, dy) is the distance difference value, which represents the relative position relationship of the two pixels in the grayscale co-occurrence matrix. For example, (1, 0) means that the point pair formed by the pixel points is in the horizontal direction, and (1, 1) means that the point pair formed by the pixel points is in the right diagonal direction.

[0097] S307: Determine the energy value and contrast of the original image based on the gray level co-occurrence matrix to obtain texture features;

[0098] The gray level co-occurrence matrix describes the texture characteristics of an image by counting the conditions in which two pixels in an image that maintain a certain distance and direction each have a certain gray level. The energy value represents the uniformity of gray level distribution, and the contrast value represents the depth of texture.

[0099] In one implementation, the energy value can be calculated by the following formula:

[0100]

[0101] Where N represents the energy value, and P(a, j) represents the probability of (a, j) appearing. The energy value reflects the uniformity of the grayscale distribution and the coarseness of the texture of the image. When the element values ​​in the grayscale co-occurrence matrix are distributed more evenly (such as images with severe noise), the energy value is small. When the element values ​​in the grayscale co-occurrence matrix are concentrated in certain areas (such as images with regular textures), the energy value is large.

[0102] In one implementation, the contrast ratio can be calculated by the following formula:

[0103]

[0104] Where D represents contrast, P(a, j) represents the probability of (a, j) appearing, and aj represents the difference in the grayscale value of the point pair formed by two pixels. Contrast describes the degree of contrast between different grayscale levels in the grayscale co-occurrence matrix, reflecting the clarity of the image and the depth of the texture grooves. When the element values ​​in the grayscale co-occurrence matrix are mainly concentrated near the diagonal (that is, there are more pixel pairs with similar grayscale values), the contrast is small, indicating that the grayscale changes in the image are relatively gentle, the texture is shallow, and the image appears blurred. When the element values ​​in the grayscale co-occurrence matrix are far away from the diagonal distribution (that is, there are more pixel pairs with large grayscale value differences), the contrast is large, indicating that the grayscale changes in the image are more drastic, the texture is deep, and the image appears clear.

[0105] In the embodiment of the present application, the gray-level co-occurrence matrix is ​​used to determine the energy value and contrast, so that the pixel features can contain the texture information in the original image to facilitate subsequent feature matching and accurately determine the tag name.

[0106] S309: Determine the main color feature and the texture feature as the pixel feature.

[0107] The embodiment of the present application comprehensively considers the pixel features of the original image, including the main color features and the texture features, so that the pixel features can fully and accurately describe the features of the main content of the original image, thereby improving the accuracy and credibility of subsequent feature matching.

[0108] It should be noted that, before extracting the pixel features of the original image, the original image may be preprocessed, including removing noise, adjusting brightness and contrast, etc., to further improve the image clarity of the original image.

[0109] Figure 4 1 is a flow chart of extracting the attribute features of target industrial enterprise equipment and pipeline components in the original image provided by an embodiment of the present application. In one embodiment, extracting the attribute features of the target industrial enterprise equipment and pipeline components in the original image may include:

[0110] S401: Acquire the size features of the target industrial enterprise equipment and pipeline components in the original image;

[0111] The size features include information such as the length, width, height, surface area, and volume of the target industrial enterprise equipment and pipeline components. In one implementation, the target industrial enterprise equipment and pipeline components present in the original image can be detected by a preset target detection algorithm or image segmentation algorithm. Since different types of industrial enterprise equipment and pipeline components often have different specifications, the size features of the industrial enterprise equipment and pipeline components present in the original image can be accurately output by the target detection algorithm or the image segmentation algorithm.

[0112] S403: Determine the three-dimensional contours corresponding to the target industrial enterprise equipment and pipeline components according to the original image;

[0113] In one implementation, a three-dimensional model corresponding to the equipment and pipeline components of the target industrial enterprise can be established based on the original image and the process flow diagram or the pipeline instrument diagram to obtain a three-dimensional contour.

[0114] In another implementation, a three-dimensional model corresponding to the target industrial enterprise equipment and pipeline components may be obtained based on laser scanning three-dimensional imaging technology to determine the three-dimensional contour.

[0115] S405: performing contour curvature extraction processing based on the three-dimensional contour to obtain shape features of the target industrial enterprise equipment and pipeline components;

[0116] Curvature describes the degree of curvature of an object's surface and can be used to represent the shape of target industrial enterprise equipment and pipeline components. The contour curvature extraction process first needs to extract the object's contour line from the 3D reconstructed model, and then perform curvature analysis along the contour line to obtain shape features.

[0117] S407: Determine the size feature and the shape feature as the attribute feature.

[0118] In the embodiment of the present application, by extracting the size characteristics and shape characteristics of the target industrial enterprise equipment and pipeline components as attribute characteristics, the attribute characteristics can comprehensively and accurately characterize the target industrial enterprise equipment and pipeline components to improve the accuracy of the subsequent matching process.

[0119] Figure 5 It is a schematic diagram of the process of determining shape features provided in an embodiment of the present application. Figure 5 Can be seen as Figure 4 A specific example of the method shown. In one embodiment, performing contour curvature extraction processing based on the three-dimensional contour to obtain the shape features of the target industrial enterprise equipment and pipeline components may include:

[0120] S501: Divide the three-dimensional contour in the horizontal direction to obtain the outer edge of the contour;

[0121] In one implementation, a three-dimensional contour may be divided using a plurality of horizontal planes, and then the outer edge of the contour formed by the horizontal planes and the three-dimensional contour is placed in a two-dimensional coordinate system for subsequent processing.

[0122] S503: Sampling the outer edge of the contour according to a preset sampling interval to obtain a plurality of contour key points;

[0123] For each outer edge of the contour, the preset sampling interval may be 1 cm, that is, the outer edge of the contour is sampled at intervals of 1 cm to obtain a plurality of contour key points. The present application does not limit the specific value of the preset sampling interval.

[0124] S505: Determine the sine value corresponding to each of the contour key points on the contour outer edge line, and determine the curvature value of the contour outer edge line according to the sine value;

[0125] The curvature value is the curvature value of the plane contour, which is used to represent the shape of the plane contour. The contour key points are numbered, and the corresponding sine value of each contour key point on the outer edge of the contour can be expressed as sin (θp / 2), where the subscript p is the number of each contour key point, p = 1, 2, 3...P, and P is the total number of contour key points; θp is the angle between each contour key point and the polyline obtained by connecting the two adjacent contour key points, and the value range of θp is .

[0126] For each outer edge of the contour, the curvature value of the plane contour can be calculated by the following formula:

[0127]

[0128] Among them, zxz represents the curvature value of the plane contour, P is the total number of contour key points, and θp is the angle of the polyline obtained by connecting each contour key point with two adjacent contour key points.

[0129] S507: Determine the average value of the curvature value as the shape feature of the target industrial enterprise equipment and pipeline components.

[0130] The plane contour curvature value of each contour outer edge line obtained by dividing each horizontal plane is counted, and its average value is determined as the shape feature.

[0131] In an embodiment of the present application, the sine value of the contour key point in each contour outer edge is used to represent the curvature, and the average value of the curvature value is used to describe the shape of the three-dimensional contour, thereby accurately characterizing the shape of the target industrial enterprise equipment and pipeline components.

[0132] In one embodiment, comparing the target marked image with the marked image to be analyzed to obtain the marking error rate of the marked image to be analyzed may include: comparing the target marked image with the marked image to be analyzed to obtain the amount of erroneous marking in the marked image to be analyzed; the amount of erroneous marking is the sum of the amount of missing markings, the amount of multiple markings, the amount of mismarking and the amount of marking position errors; and determining the ratio between the amount of erroneous markings and the number of the device component marking names marked in the target marked image as the marking error rate.

[0133] The target marked image is obtained by marking the image after intelligent image analysis and processing, while the marked image to be analyzed is marked by the enterprise itself. By comparing the two marks one by one, it is possible to determine the company's missed markings, including missed pump flushing pipelines, missed instrument pressure pipelines and pressure transmitters, missed cold insulation and thermal insulation coated flanges, missed control valve cores, etc.; it is also possible to determine the company's multiple markings, including multiple open pipeline rear flanges, multiple valve cover flanges, etc.; it is also possible to determine the company's wrong markings, including wrongly marked safety valves connected to the pipeline network, etc.; at the same time, it is also possible to determine the inaccurate point position markings.

[0134] Furthermore, the labeling error rate can be calculated as follows:

[0135]

[0136] Among them, Zl represents the labeling error rate, Z1 represents the labeling omission amount, Z2 represents the labeling over-scalar amount, Z3 represents the labeling mis-scalar amount, Z4 represents the labeling position error amount, and Z represents the number of device component label names marked in the target labeling image.

[0137] In an embodiment of the present application, by comparing the target marked image with the marks in the marked image to be analyzed one by one, the error amounts such as mislabeling, missing labeling, multiple labeling, and positional errors of the marks in the marked image to be analyzed are determined, thereby quickly achieving comprehensive marking analysis and improving the accuracy and efficiency of file creation and evaluation.

[0138] In one embodiment, after obtaining the target marked image, the method provided in the present application may further include: constructing a target three-dimensional model corresponding to the target industrial enterprise equipment and pipeline components based on the target marked image, wherein the target three-dimensional model is marked with the equipment component marking name; comparing the target three-dimensional model with the marked image to be analyzed to obtain the marking error rate of the marked image to be analyzed.

[0139] Based on the two-dimensional target mark image, the embodiment of the present application further converts the two-dimensional image into a three-dimensional model, which more clearly and intuitively displays the device components and the connection between the device components, thereby facilitating the reviewer to view or edit the point information.

[0140] In one embodiment, when the tag error rate is greater than a preset reference threshold, the target evaluation result can be determined to be unqualified for file creation, and an early warning message is issued, the early warning message is used to instruct the target enterprise to conduct an error tag check on the target industrial enterprise equipment and pipeline components; when the tag error rate is less than or equal to the preset reference threshold, the target evaluation result is determined to be qualified for file creation. The above embodiment can quickly determine whether the enterprise file creation is qualified, and issue an early warning message when it is unqualified so that the enterprise can make timely rectifications, thereby reducing the possibility of volatile organic compound leakage in the target industrial enterprise equipment and pipeline components.

[0141] In one embodiment, the preset clarity evaluation model can also be generated by first extracting multiple clarity sub-features of the original image, and then using the feature vectors formed by the multiple clarity sub-features as a feature training set, each feature vector corresponding to a clarity level label; and training the initial clarity evaluation model according to the feature training set to generate the preset clarity evaluation model.

[0142] Specifically, by performing image discreteness analysis on the original image, the discreteness of the pixel distribution in the original image can be obtained, which is recorded as the first clarity sub-feature; by performing pixel point gradient analysis, the distribution of highly acute angles and low acute angles of the blocks in the original image can be obtained, which is recorded as the second clarity sub-feature; by performing image texture change analysis, the texture change in the original image can be obtained, which is recorded as the third clarity sub-feature.

[0143] The first definition sub-feature can be obtained by fusing the pixel difference feature and the color distribution feature.

[0144] In one implementation, obtaining the pixel difference feature of the original image may include: obtaining the standard deviation of the pixel value distribution in the original image; traversing the pixel points in the original image in turn, and for each current pixel point traversed, determining the absolute value of the pixel difference between the current pixel point and the previous pixel point to obtain a pixel difference value, and summing the pixel difference values ​​to obtain a total pixel difference value; based on the standard deviation and the total pixel difference value, determining the pixel difference feature. The standard deviation can indicate the uniformity of the distribution of pixel values; the total pixel difference value can indicate the degree of difference in pixel values. In one implementation, the standard deviation and the total pixel difference value can be weightedly summed to obtain the pixel difference feature. In this way, the pixel difference feature can accurately characterize the pixel value distribution feature of the original image, making the feature extraction more comprehensive.

[0145] In one implementation, obtaining the color distribution feature may include: dividing the original image into a plurality of tiles to obtain a tile set; traversing the tiles in the tile set in sequence, and determining the color distribution mean corresponding to each traversed current tile for each traversed current tile; wherein the color distribution mean includes the brightness mean, the red and green component mean, and the yellow and blue component mean; determining the color distribution feature of the original image based on the color distribution mean; constructing a preset graph based on the pixel difference feature and the color distribution feature, and determining the perimeter of the preset graph as the first clarity sub-feature. Specifically, the original image may be evenly divided to obtain a preset number of tiles; traversing the tiles in sequence, and recording the brightness mean of the traversed current tile as lj. n , the mean of the red and green components is recorded as hj n , the mean value of the yellow-blue component is recorded as hz n , where n represents the traversal order of the current tile in the tile set. Therefore, the color distribution characteristics of the original image can be calculated by the following formula:

[0146]

[0147] Among them, sb represents the color distribution characteristics, lj n represents the mean brightness of the nth block, hj n Represents the mean value of the red and green components of the nth block, hz n represents the mean value of the yellow-blue component of the nth block, N represents the number of blocks, A1 represents the average value of the brightness of the N blocks, A2 represents the mean value of the red and green components of the brightness of the N blocks, and A3 represents the mean value of the yellow-blue component of the N blocks.

[0148] The color distribution feature indicates the uniformity of color distribution of the color channel of the original image. In the embodiment of the present application, the perimeter of the constructed parallelogram is determined as the first clarity sub-feature, so that the first clarity sub-feature can integrate the uniformity of pixel value distribution, the difference and the uniformity of color distribution, so as to fully characterize the discreteness of the pixel distribution of the original image.

[0149] In one embodiment, the pixel change gradient corresponding to each pixel point in the current image block can be obtained, and the average pixel change gradient of the current image block can be determined based on the pixel change gradient corresponding to each pixel point; if the average pixel change gradient is greater than or equal to a preset gradient threshold, the current image block is determined as a high-acute-angle image block; if the average pixel change gradient is less than the preset gradient threshold, the current image block is determined as a low-acute-angle image block; and the second clarity sub-feature is determined according to the number of the high-acute-angle images blocks and the number of the low-acute-angle images blocks in the image block set.

[0150] In one implementation, the pixels of the current image block are traversed, and for each traversed current pixel, the first pixel and the second pixel adjacent to the current pixel in the horizontal direction are obtained; the pixel difference between the first pixel and the second pixel is determined as the horizontal gradient component of the current pixel; the third pixel and the fourth pixel adjacent to the current pixel in the vertical direction are obtained; the pixel difference between the third pixel and the fourth pixel is determined as the vertical gradient component of the current pixel; based on the horizontal gradient component and the vertical gradient component, the pixel change gradient of the current pixel is determined. The pixel change gradient of the current pixel can be calculated by the following formula:

[0151]

[0152] Among them, G represents the pixel change gradient of the current pixel, Dx represents the horizontal gradient component of the current pixel, and Dy represents the vertical gradient component of the current pixel. The average pixel change gradient of the current block is the average value of the pixel change gradients of all pixels in the current block. By integrating the gradient changes in the horizontal and vertical directions and considering the gradient information in different directions, the basis for judging whether a block belongs to a high-sharp-angle block or a low-sharp-angle block can be made more accurate.

[0153] In addition to the first and second definition sub-features, Figure 3 The texture features in are used as the third clarity sub-features, and the above clarity sub-features are formed into feature vectors, so as to train the initial clarity evaluation model so that the model outputs accurate clarity levels.

[0154] Figure 6 : is a structural block diagram of the LDAR file quality assessment system based on multi-source data fusion provided in the embodiment of the present application. Figure 6As shown, the file building and evaluation system may include the above-mentioned server 01 and system platform 620. The server 01 includes a pre-file analysis unit 611, a marking unit 612, a comparison unit 613 and an effect evaluation unit 614. The pre-file analysis unit 611 is used to determine the necessity of LDAR file building; and to evaluate the clarity of the LDAR file image. The marking unit 612 is used to extract multi-dimensional features from the original image of the target industrial enterprise equipment and pipeline components to obtain the target features of the target industrial enterprise equipment and pipeline components. The target feature is matched with the preset feature database to obtain the tag name and mark the original image accordingly to obtain the target marked image, which is then sent to the comparison unit 613; the marking unit 612 is also used to determine and mark the three-dimensional model of the target industrial enterprise equipment and pipeline components; the comparison unit 613 is used to compare and analyze the received target marked image or the constructed three-dimensional model with the marked photo of the enterprise LDAR file to obtain the error number value of the enterprise LDAR file, and send it to the effect evaluation unit 614; the effect evaluation unit 614 is used to analyze the error results of the enterprise LDAR file, obtain corresponding early warning and processing measures, so that the enterprise can conduct targeted inspections of the industrial enterprise equipment and pipeline components to reduce the possibility of leakage of volatile organic compounds.

[0155] Figure 7 Schematic diagram of the structure of the LDAR file quality assessment device based on multi-source data fusion provided in the embodiment of the present application. Figure 7 As shown, the device 700 includes:

[0156] The image acquisition module 701 is used to acquire the original image of the target industrial enterprise equipment and pipeline components and the marked image to be analyzed corresponding to the original image; the marked image to be analyzed is an image obtained by marking the name of the target industrial enterprise equipment and pipeline components on the original image during the archiving stage of the equipment leakage detection and repair process;

[0157] An image quality assessment module 702 is used to determine an image quality assessment result of the original image using a preset clarity assessment model;

[0158] The pixel feature determination module 703 is used to extract the pixel features of the original image according to the image quality assessment result; the pixel features represent the features of the pixels occupied by the target industrial enterprise equipment and pipeline components in the original image;

[0159] The attribute feature determination module 704 is used to extract the attribute features of the target industrial enterprise equipment and pipeline components in the original image, and perform feature fusion on the pixel features and the attribute features to obtain target features;

[0160] The image marking module 705 is used to match the device component marking name corresponding to the target feature from the preset feature database, and mark the original image based on the device component marking name to obtain a target marked image; the preset feature database records the correspondence between the preset target feature and the preset device component marking name;

[0161] The labeling comparison module 706 is used to compare the target labeling image with the labeling image to be analyzed, obtain the labeling error rate of the labeling image to be analyzed, and determine the target evaluation result based on the labeling error rate.

[0162] In some embodiments, the apparatus 700 may further include:

[0163] A target three-dimensional model construction module is used to construct a target three-dimensional model corresponding to the target industrial enterprise equipment and pipeline components according to the target marked image, and the target three-dimensional model is marked with the equipment component mark name;

[0164] The model comparison module is used to compare the target three-dimensional model with the marked image to be analyzed to obtain the marking error rate of the marked image to be analyzed.

[0165] In some embodiments, the pixel feature determination module may include:

[0166] A pixel clustering submodule, used to perform pixel clustering processing on the original image to obtain the center point position of the target industrial enterprise equipment and pipeline components in the original image;

[0167] A pixel feature determination submodule, used to determine the pixel value at the center point position and the frequency value of the occurrence of the pixel value at the center point position as the main color feature;

[0168] A gray level co-occurrence matrix acquisition submodule is used to obtain the gray level co-occurrence matrix corresponding to the original image;

[0169] A texture feature determination submodule, used to determine the energy value and contrast of the original image based on the gray level co-occurrence matrix to obtain texture features;

[0170] The pixel feature determination submodule is used to determine the main color feature and the texture feature as the pixel feature.

[0171] In some embodiments, the attribute feature determination module may include:

[0172] A size feature acquisition submodule, used to acquire size features of the target industrial enterprise equipment and pipeline components in the original image;

[0173] A three-dimensional contour determination submodule, used to determine the three-dimensional contours corresponding to the target industrial enterprise equipment and pipeline components according to the original image;

[0174] A contour curvature extraction submodule, used to perform contour curvature extraction processing based on the three-dimensional contour to obtain shape features of the target industrial enterprise equipment and pipeline components;

[0175] The attribute feature determination submodule is used to determine the size feature and the shape feature as the attribute feature.

[0176] In some embodiments, the contour curvature extraction submodule may include:

[0177] A contour outer edge determination unit, used for dividing the three-dimensional contour in a horizontal direction to obtain a contour outer edge;

[0178] A key point sampling unit, used for sampling the outer edge of the contour according to a preset sampling interval to obtain a plurality of contour key points;

[0179] a curvature value determining unit, used to determine the sine value corresponding to each of the contour key points on the contour outer edge line, and determine the curvature value of the contour outer edge line according to the sine value;

[0180] The shape feature determination unit is used to determine the average value of the curvature value as the shape feature of the target industrial enterprise equipment and pipeline components.

[0181] In some embodiments, the marker comparison module may include:

[0182] The error labeling amount determination submodule is used to compare the target labeling image with the labeling image to be analyzed to obtain the error labeling amount in the labeling image to be analyzed; the error labeling amount is the sum of the labeling omission amount, the labeling excess amount, the labeling misalignment amount and the labeling position error amount;

[0183] The marking error rate determination submodule is used to determine the ratio between the amount of erroneous marking and the number of the device component marking names marked in the target marking image as the marking error rate.

[0184] In some embodiments, the apparatus 700 may further include:

[0185] A training image acquisition module is used to acquire training images of multiple industrial enterprise equipment and pipeline components to obtain a training image set, wherein the training images are annotated with clarity level labels;

[0186] The model training module is used to train the initial clarity evaluation model according to the training image set to generate the preset clarity evaluation model.

[0187] The device and method embodiments in the described device embodiments are based on the same inventive concept.

[0188] An embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the method provided in the above method embodiment.

[0189] An embodiment of the present application also provides a computer storage medium, which can be set in a terminal to store at least one instruction or at least one program related to a method provided in the above method embodiment for implementing a method embodiment, and the at least one instruction or at least one program is loaded and executed by the processor to implement the method provided in the above method embodiment.

[0190] The embodiment of the present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device component reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device component executes to implement the method provided in the above method embodiment.

[0191] Optionally, in the embodiment of the present application, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in the present embodiment, the storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0192] The memory described in the embodiment of the present application can be used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, application programs required for functions, etc.; the data storage area may store data created according to the use of the device components, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0193] The method provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Taking running on a server as an example, Figure 8 is a hardware structure block diagram of a server provided in an embodiment of the present application. Figure 8 As shown, still taking server 01 as an example, the server 01 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 810 (the central processing unit 810 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 830 for storing data, and one or more storage media 820 (such as one or more mass storage device components) for storing application programs 823 or data 822. Among them, the memory 830 and the storage medium 820 can be short-term storage or permanent storage. The program stored in the storage medium 820 may include one or more modules, each of which may include a series of instruction operations on the server. Furthermore, the central processing unit 810 may be configured to communicate with the storage medium 820 to execute a series of instruction operations in the storage medium 820 on the server 01. Server 01 may also include one or more power supplies 860, one or more wired or wireless network interfaces 850, one or more input and output interfaces 840, and / or one or more operating systems 821, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0194] The input / output interface 840 may be used to receive or send data via a network. The specific example of the network may include a wireless network provided by a communication provider of the server 01. In one example, the input / output interface 840 includes a network adapter (Network Interface Controller, NIC), which may be connected to other network device components through a base station so as to communicate with the Internet. In one example, the input / output interface 840 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0195] It can be understood by those skilled in the art that Figure 8 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 8 More or fewer components as shown, or with Figure 8 Different configurations shown.

[0196] It can be seen from the embodiments of the LDAR filing quality assessment method, device, electronic device and storage medium based on multi-source data fusion provided by the present application that the method of the present application can automatically analyze the images uploaded by the enterprise for filing, and obtain comprehensive and accurate features for describing the target industrial enterprise equipment and pipeline components contained in the original image by extracting the multi-dimensional features of the original image, thereby accurately matching the equipment component label names existing in the original image, so that the assessment of enterprise filing does not rely on manual review, but instead achieves fast and accurate enterprise LDAR filing review through an accurate intelligent image analysis process, effectively improving the assessment efficiency of LDAR filing.

[0197] It should be noted that the above-mentioned sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above-mentioned specific embodiments of this specification are described. Other embodiments are within the scope of the attached claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0198] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment component, and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0199] A person skilled in the art will appreciate that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0200] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A LDAR file quality assessment method based on multi-source data fusion, characterized in that: include: Acquire original images of target industrial enterprise equipment and pipeline components and marked images to be analyzed corresponding to the original images; The marked image to be analyzed is an image obtained by marking the name of the target industrial enterprise equipment and pipeline components on the original image during the archiving stage of the equipment leakage detection and repair process; Determine the image quality evaluation result of the original image by using a preset clarity evaluation model; According to the image quality assessment result, the original image is subjected to pixel clustering processing to obtain the center point position of the target industrial enterprise equipment and pipeline components in the original image; the pixel value of the center point position and the frequency value of the occurrence of the pixel value are determined as the main color feature; Determine the energy value and contrast of the original image based on the gray level co-occurrence matrix corresponding to the original image to obtain texture features; Determining the main color feature and the texture feature as pixel features; Acquire the size features of the target industrial enterprise equipment and pipeline components in the original image; determine the three-dimensional contours corresponding to the target industrial enterprise equipment and pipeline components according to the original image; perform contour curvature extraction processing based on the three-dimensional contour to obtain shape features; Determine the size feature and the shape feature as attribute features, and perform feature fusion on the pixel feature and the attribute feature to obtain a target feature; Matching the device component label name corresponding to the target feature from a preset feature database, and marking the original image based on the device component label name to obtain a target labeled image; the preset feature database records the correspondence between the preset target feature and the preset device component label name; The target marked image is compared with the marked image to be analyzed to obtain a marking error rate of the marked image to be analyzed, and a target evaluation result is determined based on the marking error rate.

2. The method according to claim 1, characterized in that After obtaining the target marked image, the method further includes: Constructing a target three-dimensional model corresponding to the target industrial enterprise equipment and pipeline components according to the target marked image, wherein the target three-dimensional model is marked with the equipment component mark name; The target three-dimensional model is compared with the labeled image to be analyzed to obtain the labeling error rate of the labeled image to be analyzed.

3. The method according to claim 1, characterized in that The contour curvature extraction process is performed based on the three-dimensional contour to obtain the shape features of the target industrial enterprise equipment and pipeline components, including: Dividing the three-dimensional contour in the horizontal direction to obtain the outer edge of the contour; Sampling the outer edge of the contour according to a preset sampling interval to obtain a plurality of contour key points; Determine the sine value corresponding to each of the contour key points on the contour outer edge line, and determine the curvature value of the contour outer edge line according to the sine value; An average value of the curvature values ​​is determined as the shape feature of the target industrial enterprise equipment and pipeline components.

4. The method according to claim 1, characterized in that: The step of comparing the target marked image with the marked image to be analyzed to obtain a marking error rate of the marked image to be analyzed includes: Compare the target marked image with the marked image to be analyzed to obtain the amount of erroneous marking in the marked image to be analyzed; the amount of erroneous marking is the sum of the amount of missed marking, the amount of multiple marking, the amount of wrong marking and the amount of wrong marking position; The ratio between the amount of erroneous labeling and the number of the device component label names labeled in the target label image is determined as the labeling error rate.

5. The method according to claim 1, characterized in that The construction process of the preset clarity evaluation model includes: Acquire training images of multiple industrial enterprise equipment and pipeline components to obtain a training image set, wherein the training images are annotated with clarity level labels; The initial clarity evaluation model is trained according to the training image set to generate the preset clarity evaluation model.

6. A LDAR file quality assessment device based on multi-source data fusion, characterized in that: include: An image acquisition module, used to acquire original images of target industrial enterprise equipment and pipeline components and marked images to be analyzed corresponding to the original images; The marked image to be analyzed is an image obtained by marking the name of the target industrial enterprise equipment and pipeline components on the original image during the archiving stage of the equipment leakage detection and repair process; An image quality assessment module, used to determine an image quality assessment result of the original image using a preset clarity assessment model; A pixel feature determination module is used to perform pixel point clustering processing on the original image according to the image quality assessment result to obtain the center point position of the target industrial enterprise equipment and pipeline components in the original image; and determine the pixel value of the center point position and the frequency value of the occurrence of the pixel value as the main color feature; Determine the energy value and contrast of the original image based on the gray level co-occurrence matrix corresponding to the original image to obtain texture features; Determining the main color feature and the texture feature as pixel features; The attribute feature determination module is used to obtain the size features of the target industrial enterprise equipment and pipeline components in the original image; determine the three-dimensional contour corresponding to the target industrial enterprise equipment and pipeline components according to the original image; perform contour curvature extraction processing based on the three-dimensional contour to obtain shape features; Determine the size feature and the shape feature as attribute features, and perform feature fusion on the pixel feature and the attribute feature to obtain a target feature; An image marking module is used to match the device component marking name corresponding to the target feature from a preset feature database, and mark the original image based on the device component marking name to obtain a target marked image; the preset feature database records the correspondence between the preset target feature and the preset device component marking name; The marking comparison module is used to compare the target marked image with the marked image to be analyzed, obtain the marking error rate of the marked image to be analyzed, and determine the target evaluation result based on the marking error rate.

7. An electronic device, characterized in that: The electronic device includes: a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the LDAR archiving quality assessment method based on multi-source data fusion as described in any one of claims 1-5.

8. A computer storage medium, characterized in that: The computer storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the LDAR archiving quality assessment method based on multi-source data fusion as described in any one of claims 1-5.

Images