Data acquisition method and equipment based on extension line, medium and product

Abstract

The invention relates to the technical field of data processing, and discloses a data acquisition method and device based on an extension cord, a medium and a product, and the method comprises the steps: collecting scale data of a sensor group through a preprocessing unit, and determining a data preprocessing period; the preprocessing unit is connected with the sensor group deployed on the pipeline through an extension line; collecting data of all the sensors in the preprocessing period, establishing a to-be-analyzed data set, and judging whether redundant data exists in the to-be-analyzed data set or not according to the data type and the data value; establishing a data matrix for the same type of data in the to-be-analyzed data group after the redundant data is removed according to sensor positions, and determining abnormal data and abnormal data types; and comparing the abnormal data and the abnormal data type as feature data with a historical abnormal database, and determining a data acquisition result according to a comparison result. The intelligent level and reliability of the oil and gas pipeline monitoring system are improved, potential safety hazards are reduced, and the ability to deal with emergencies is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a data acquisition method, device, medium and product based on an extension line. Background Art

[0002] In the modern industrial field, pipeline systems are widely used in industries such as oil, gas, chemicals and water treatment. Over time, pipelines are affected by corrosion, wear and other external factors, causing their wall thickness to gradually decrease, which may cause safety hazards such as leakage and explosion. Therefore, in modern oil and gas pipeline monitoring systems, the accuracy and real-time nature of data collection are crucial.

[0003] However, current traditional data collection methods often rely on static sensors and manual inspections, and face a series of problems. First, the distribution and coverage of sensors are limited, resulting in incomplete data collection and failure to reflect the actual status of the pipeline in a timely manner. Secondly, the data collection cycle is not flexible enough and often cannot adapt to the rapid changes in pipeline operation, which greatly reduces the effectiveness of real-time monitoring. Furthermore, when traditional methods process large amounts of data, redundant data is often not effectively eliminated, resulting in redundant data and increasing the complexity of subsequent analysis. These problems not only affect the safety and efficiency of the pipeline, but may also lead to serious accidents such as oil leaks and explosions. Summary of the invention

[0004] In view of this, the present invention proposes a data collection method, device, medium and product based on an extension line, aiming to solve the problems of incomplete current data collection, huge data volume, inability to effectively eliminate redundant data, increased data processing time and affecting pipeline safety.

[0005] In a first aspect, the present invention proposes a data acquisition method based on an extension line, comprising: collecting scale data of a sensor group through a preprocessing unit, and determining a data preprocessing period according to the scale data; connecting the preprocessing unit to the sensor group deployed on the pipeline through an extension line; collecting data of all sensors in the sensor group within the preprocessing period, establishing a data group to be analyzed, judging whether there is redundant data in the data group to be analyzed according to the data type and data value of the collected sensor data, and removing the redundant data; establishing a data matrix according to the sensor position for the same type of data in the data group to be analyzed after removing the redundant data, performing abnormal data judgment on the data matrix, determining the abnormal data and determining the abnormal data type; comparing the abnormal data and the abnormal data type as feature data with the historical abnormal data in the historical abnormal database, determining the data acquisition result according to the comparison result, and sending the data acquisition result to the processing unit;

[0006] Among them, determining abnormal data and determining the type of abnormal data includes: arbitrarily determining a center point (x0, y0) in the data matrix and determining the neighborhood radius; calculating the neighborhood average value according to the data values ​​of each coordinate point within the neighborhood radius; when the difference between the data value of the center point and the neighborhood average value is greater than the data threshold, it is determined that the data value of the center point is abnormal and is determined to be abnormal data; according to the sensor type of the abnormal data, the abnormal data type is obtained; the neighborhood average value is calculated by the following formula:

[0007]

[0008] Among them, P avg represents the average value of the neighborhood, e represents the neighborhood radius, (x0, y0) represents the coordinates of the center point, and P(x, y) represents the data value of the point (x, y) on the data matrix.

[0009] Optionally, determining the data preprocessing cycle according to the scale data includes: comparing the scale data with a plurality of preset scale data respectively, and determining the data preprocessing cycle according to the comparison results.

[0010] Optionally, the multiple preset scale data include a first preset scale data and a second preset scale data, and the first preset scale data is smaller than the second preset scale data; the scale data is compared with the multiple preset scale data respectively, and the data preprocessing period is determined according to the comparison results, including: when the scale data is less than or equal to the first preset scale data, the data preprocessing period is determined to be the first preset data processing period; when the scale data is greater than the first preset scale data and less than or equal to the second preset scale data, the data preprocessing period is determined to be the second preset data processing period; when the scale data is greater than the second preset scale data, the data preprocessing period is determined to be the third preset data processing period; the first preset data processing period is smaller than the second preset data processing period, and the second preset data processing period is smaller than the third preset data processing period.

[0011] Optionally, judging whether there is redundant data in the data group to be analyzed based on the data type and data value, and eliminating the redundant data, including: arranging the data of the same sensor in the data group to be analyzed in time series to form a data chain; when there are two or more data chains of the same type of data, and the data similarity between the two or more data chains is greater than a preset threshold, judging that there is redundant data; when it is judged that there is redundant data, obtaining the data mean within the data preprocessing period based on the data value in the data group to be analyzed, comparing each data of each data chain in the two or more data chains with the data mean, obtaining the data difference, retaining the data chain with the largest data difference, and eliminating the remaining data chains.

[0012] Optionally, the abnormal data and the abnormal data type are used as feature data to be compared with historical abnormal data in a historical abnormal database, and the data collection result is determined based on the comparison result, including: when there is historical abnormal data of the same abnormal data type as the abnormal data in the historical abnormal database, cluster analysis is used to classify the abnormal data to determine a first data collection result; the first data collection result includes: risk type or risk valuation; when there is no historical abnormal data of the same abnormal data type as the abnormal data in the historical abnormal database, a second data collection result is determined based on the abnormal data type and the abnormal data; the second data collection result includes: risk valuation, abnormal data and abnormal data type.

[0013] Optionally, cluster analysis is used to classify the abnormal data to determine the first data collection result, including: S1: screening out historical abnormal data with the same abnormal data type in the historical abnormality database, and establishing a cluster analysis library; S2: initializing K centroids in the cluster analysis library, assigning the abnormal data to the nearest centroid, and forming K clusters; S3: recalculating the centroid of each cluster; S4: repeating S2 and S3 until the centroid no longer changes or repeating a preset number of times to obtain at least one cluster; S5: determining the first data collection result based on the clustering result of the abnormal data.

[0014] Optionally, the first data collection result is determined based on the clustering result of the abnormal data, including: when the abnormal data is classified into at least one cluster of the cluster analysis library, the historical risk type of the cluster to which the abnormal data belongs is used as the risk type of the abnormal data; when the abnormal data is not classified into at least one cluster of the cluster analysis library, the risk valuation is determined based on the abnormal data type and the abnormal data.

[0015] Optionally, the centroid is calculated as follows:

[0016]

[0017] Among them, Ck represents the centroid of the kth cluster, |Mk| represents the number of data in the kth cluster, and Ti represents the feature vector of the i-th data.

[0018] Optionally, a risk valuation is determined based on the type of abnormal data and the abnormal data, including:

[0019]

[0020] Among them, R represents risk valuation, n represents the number of abnormal data types, Ej represents the data value of the j-th abnormal data, Sj represents the standard value of the j-th abnormal data, Wj represents the weight of the j-th abnormal data type, and αj represents the nonlinear weight index of the j-th abnormal data type.

[0021] In a second aspect, a data acquisition device based on an extension line is provided, comprising: a communication unit and a processing unit; the communication unit is used to obtain ultrasonic echo signals of multiple detections of a plurality of pipeline samples through an ultrasonic sensor; the ultrasonic sensor is arranged at a predetermined position of the pipeline sample; the communication unit is also used to obtain specification parameters of the pipeline sample, the specification parameters including pipeline sample thickness and pipeline sample density; the processing unit is used to perform noise reduction processing on the ultrasonic echo signal, and determine the wave velocity of the ultrasonic wave corresponding to the ultrasonic echo signal in the pipeline sample according to the ultrasonic echo signal after noise reduction processing and the pipeline sample thickness; the communication unit is also used to obtain the detection temperature of the pipeline sample, and correct the pipeline sample density of the pipeline sample according to the detection temperature to obtain the corrected density; the processing unit is also used to determine the acoustic impedance of the pipeline sample according to the corrected density and wave velocity, and determine the acoustic characteristic parameters of the pipeline sample according to the acoustic impedance, and establish an ultrasonic sensor system model based on the acoustic characteristic parameters; the ultrasonic sensor system model is used to detect the pipeline quality of the detected pipeline by detecting the acoustic characteristic parameters of the detected pipeline.

[0022] In a third aspect, a data acquisition device based on an extension cord is provided, comprising a memory and a processor; the memory is used to store computer execution instructions, and the processor is connected to the memory through a bus; when the data acquisition device based on the extension cord is running, the processor executes the computer execution instructions stored in the memory, so that the data acquisition device based on the extension cord executes the data acquisition method based on the extension cord described in the first aspect.

[0023] The data acquisition device based on the extension line can be a network device, or a part of the device in the network device, such as a chip system in the network device. The chip system is used to support the network device to implement the functions involved in the first aspect and any possible implementation thereof, for example, to obtain, determine, and send the data and / or information involved in the above-mentioned data acquisition method based on the extension line. The chip system includes a chip, and may also include other discrete devices or circuit structures.

[0024] In a fourth aspect, a computer-readable storage medium is provided, the computer-readable storage medium comprising computer execution instructions, and when the computer execution instructions are run on a computer, the computer executes the data collection method based on the extension line described in the first aspect.

[0025] In a fifth aspect, a computer program product is also provided, which includes computer instructions. When the computer instructions are run on an extension line-based data acquisition device, the extension line-based data acquisition device executes the extension line-based data acquisition method as described in the first aspect above.

[0026] It should be noted that the above-mentioned computer instructions can be stored in whole or in part on a computer-readable storage medium. The computer-readable storage medium can be packaged together with the processor of the data acquisition device based on the extension line, or it can be packaged separately from the processor of the data acquisition device based on the extension line, and the embodiments of the present application are not limited to this.

[0027] The description of the second, third, fourth and fifth aspects of the present application can refer to the detailed description of the first aspect.

[0028] In the embodiments of the present application, the name of the data acquisition device based on the extension line does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear with other names. For example, the receiving unit may also be called a receiving module, a receiver, etc. As long as the functions of each device or functional module are similar to those of the present application, they belong to the scope of the claims of the present application and their equivalent technologies.

[0029] This application realizes efficient and real-time data collection and processing by setting up a preprocessing unit connected to the extension line of the sensor group deployed on the pipeline. The data preprocessing cycle is flexibly determined according to the scale data of the sensor group, so as to adapt to the dynamic changes in pipeline operation and ensure the timeliness and comprehensiveness of data collection. Secondly, by establishing a data group to be analyzed and eliminating redundant data, the data redundancy is significantly reduced and the accuracy of data analysis is improved. Then, the data matrix is ​​used to judge the abnormal data, which can not only quickly identify potential risks, but also compare with the historical abnormal database to ensure accurate handling of abnormal situations. It improves the intelligence level and reliability of the oil and gas pipeline monitoring system, reduces safety hazards, and enhances the ability to respond to emergencies. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0031] Figure 1 A structural diagram of a data acquisition system provided in an embodiment of the present application;

[0032] Figure 2 A schematic diagram of the hardware structure of a data acquisition device provided in an embodiment of the present application;

[0033] Figure 3 A flow chart of a data collection method provided in an embodiment of the present application;

[0034] Figure 4A flowchart of another data collection method provided in an embodiment of the present application;

[0035] Figure 5 A flowchart of another data collection method provided in an embodiment of the present application;

[0036] Figure 6 A flowchart of another data collection method provided in an embodiment of the present application;

[0037] Figure 7 A flowchart of another data collection method provided in an embodiment of the present application;

[0038] Figure 8 A structural schematic diagram of a data acquisition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0040] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0041] In order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second", etc. are used to distinguish the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first", "second", etc. are not limiting the quantity and execution order.

[0042] In modern industry, pipeline systems are widely used in many key areas such as oil, natural gas, chemicals and water treatment. However, as time goes by, pipeline systems are inevitably affected by corrosion, wear and external environmental factors, resulting in a continuous reduction in wall thickness, which in turn leads to serious safety hazards such as leakage and explosion. Therefore, the efficiency of oil and gas pipeline monitoring systems and the real-time accuracy of data collection are particularly important.

[0043] At present, many traditional pipeline monitoring methods rely on fixed static sensors and manual inspections, which have several significant problems. First, sensors are usually unevenly distributed, resulting in a limited monitoring range, which cannot fully reflect the actual operating status of the pipeline and easily ignores the risks of certain key areas. Secondly, the time period of data collection lacks flexibility and cannot respond to rapid changes in pipeline operation in a timely manner, resulting in a lag in real-time monitoring. Furthermore, when processing and analyzing large amounts of data, traditional methods often fail to effectively eliminate redundant data, resulting in information redundancy, further increasing the complexity and cost of data analysis. These deficiencies not only reduce the efficiency of the monitoring system, but also significantly increase the safety risks of the pipeline, which may lead to catastrophic consequences.

[0044] Therefore, an innovative data acquisition method based on extension lines is urgently needed to overcome the limitations of existing technologies. This method can optimize the layout of sensors, improve the flexibility and comprehensiveness of data acquisition, ensure the effectiveness of real-time monitoring, and thus provide reliable protection for the safe operation of pipelines.

[0045] In this case, an embodiment of the present application provides a data collection method based on an extension line, including: collecting scale data of a sensor group through a preprocessing unit, and determining a data preprocessing period according to the scale data; the preprocessing unit is connected to the sensor group deployed on the pipeline through an extension line; collecting data of all sensors in the sensor group within the preprocessing period, establishing a data group to be analyzed, judging whether there is redundant data in the data group to be analyzed according to the data type and data value of the collected sensor data, and eliminating the redundant data; establishing a data matrix for the same type of data in the data group to be analyzed after eliminating the redundant data according to the sensor position, performing abnormal data judgment on the data matrix, determining the abnormal data and determining the abnormal data type; comparing the abnormal data and the abnormal data type with the historical abnormal data in the historical abnormal database as feature data, determining the data collection result according to the comparison result, and sending the data collection result to the processing unit.

[0046] As can be seen from the above, this application realizes efficient and real-time data collection and processing by setting up a preprocessing unit connected to the extension line of the sensor group deployed on the pipeline. The data preprocessing cycle is flexibly determined according to the scale data of the sensor group, so as to adapt to the dynamic changes in the pipeline operation and ensure the timeliness and comprehensiveness of data collection. Secondly, by establishing a data group to be analyzed and eliminating redundant data, the data redundancy is significantly reduced and the accuracy of data analysis is improved. Then, the data matrix is ​​used to judge the abnormal data, which can not only quickly identify potential risks, but also compare with the historical abnormal database to ensure accurate handling of abnormal situations. The intelligence level and reliability of the oil and gas pipeline monitoring system are improved, the safety hazards are reduced, and the ability to respond to emergencies is enhanced.

[0047] The above-mentioned data acquisition method based on extension lines can be applied to a data acquisition system. Figure 1 Figure 2 shows the structural diagram of the data acquisition system. Figure 1 As shown, the data acquisition system includes: a data acquisition pipeline 101, a sensor group 102, a data acquisition device 103 and a storage server 104. The sensor group 102 includes a plurality of sensors. The data acquisition device 103 is connected to each sensor in the sensor group 102 via an extension line 105.

[0048] Optionally, the pipeline 101 for data collection may be pipelines of different types, such as a natural gas pipeline, a crude oil pipeline, a refined oil pipeline, etc.

[0049] The sensors in the sensor group 102 are used to collect data of the pipeline 101 to be collected, such as parameters such as thickness and temperature.

[0050] Optionally, the sensors in the above-mentioned sensor group 102 may include ultrasonic sensors, temperature sensors, vibration sensors, flow sensors, and pressure sensors. Ultrasonic sensors are sensors that convert ultrasonic signals into other energy signals (usually electrical signals), and are usually used to detect parameters such as the thickness of the object being detected. Temperature sensors are usually used to detect the temperature of the object being detected. Vibration sensors are usually used to detect the vibration state of the object being detected. Flow sensors are usually used to detect the flow rate generated when the fluid flows in the object being detected. Pressure sensors are usually used to detect the pressure of the object being detected.

[0051] In the embodiment of the present application, the sensors in the sensor group 102 can emit detection signals to the data collection pipeline 101 to be collected, and obtain the detection data (i.e., the data collected by the sensors) of the data collection pipeline 101 to be collected. Then, the sensors in the sensor group 102 can send the collected data to the data collection device 103 through the extension line 105.

[0052] The extension line 105 is mainly used to connect the sensor group 102 deployed on the data collection pipeline 101 to the data collection device 103. This allows the sensors in the sensor group 102 to be flexibly placed at a suitable position in the data collection pipeline 101, while ensuring that the data is transmitted to the data collection device 103 for subsequent data processing.

[0053] Optionally, since the environment of the pipeline 101 to be collected data may be complex, the extension line 105 needs to have good corrosion resistance. If it is in a pipeline scene such as petroleum and chemical industry, it may be exposed to various corrosive chemicals, so its outer skin material may need to use corrosion-resistant rubber or plastic.

[0054] Optionally, in order to ensure the quality of signal transmission, the conductive core material inside the extension cable 105 should have good conductivity and a good shielding layer to reduce the impact of external electromagnetic interference on the sensor signal. For example, in the field of industrial automation, when there are large motors and other equipment nearby, electromagnetic interference is more serious, and a well-shielded extension cable can effectively reduce signal distortion.

[0055] Optionally, the interfaces at both ends of the extension line 105 need to match the output interface of the sensor group 102 and the input interface of the data acquisition device 103. Common interfaces include aviation plug interfaces, waterproof sealing interfaces, etc. In outdoor pipeline monitoring environments, waterproof sealing interfaces can prevent rainwater, dew, etc. from entering the interfaces and causing problems such as short circuits.

[0056] Optionally, the length of the extension line 105 needs to be determined according to the pipeline layout of the pipeline 101 to be data collected and the actual distance between the sensor group 102 and the data acquisition device 103. In a large pipeline network, such as a city water supply pipeline monitoring system, a longer extension line may be required to span different pipeline areas and connect sensors and data acquisition devices 103 distributed in various places. However, an extension line that is too long may also cause signal attenuation, so it needs to be comprehensively considered during design, and if necessary, a signal amplifier or other equipment can be added to compensate for signal loss.

[0057] The data collection device 103 may receive raw data collected by the sensors in the sensor group 102 through the extension line, and determine the data collection result according to the data collected by the sensors in the sensor group 102 .

[0058] Optionally, the data acquisition device 103 may include a preprocessing unit. The preprocessing unit is connected to the sensor group 102 deployed on the data acquisition pipeline 101 through an extension line 105, and is used to collect scale data of the sensor group 102 and determine a data preprocessing cycle according to the scale data.

[0059] Optionally, the data collection device 103 may further include a processing unit. After determining the data collection result, the data collection result may be sent to the processing unit so that the processing unit can also make a reasonable risk assessment and respond quickly.

[0060] Optionally, the processing unit may also be a processing unit of a device other than the data acquisition device 103, which is not limited in this embodiment of the present application.

[0061] Optionally, the physical device of the data acquisition device 103 may be a server, a terminal, or other types of electronic devices, which is not limited in the embodiments of the present application.

[0062] Optionally, the terminal may be at least one of a smart phone, a smart watch, a desktop computer, a laptop computer, a virtual reality terminal, an augmented reality terminal, a wireless terminal, a laptop computer and the like.

[0063] Optionally, the above-mentioned server can be a server in a server cluster (consisting of multiple servers), or a chip in the server, or a system on a chip in the server, or can be implemented by a virtual machine (VM) deployed on a physical machine, which is not limited in the embodiments of the present application.

[0064] The storage server 104 is used to store various types of data in the embodiment of the present application, such as the data collection results obtained by the data collection device 103 from the data collection pipeline 101, the historical anomaly database used to determine the data collection results, etc.

[0065] The basic hardware structure of the data acquisition device 103 includes Figure 2 The data acquisition device shown in FIG. Figure 2 Taking the data acquisition device shown in the figure as an example, the hardware structure of the data acquisition device 103 is introduced.

[0066] like Figure 2 , which is a schematic diagram of a hardware structure of a data acquisition device provided in an embodiment of the present application. The data acquisition device includes a processor 21, a memory 22, a communication interface 23, and a bus 24. The processor 21, the memory 22, and the communication interface 23 can be connected via a bus 24.

[0067] The processor 21 is the control center of the data acquisition device, which can be a processor or a general term for multiple processing elements. For example, the processor 21 can be a general-purpose central processing unit (CPU) or other general-purpose processors. Among them, the general-purpose processor can be a microprocessor or any conventional processor.

[0068] As an embodiment, the processor 21 may include one or more CPUs, for example Figure 2 CPU 0 and CPU 1 are shown in .

[0069] The memory 22 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0070] In a possible implementation, the memory 22 may exist independently of the processor 21, and the memory 22 may be connected to the processor 21 via a bus 24 to store instructions or program codes. When the processor 21 calls and executes the instructions or program codes stored in the memory 22, the data acquisition method provided in the following embodiments of the present application can be implemented.

[0071] In the embodiment of the present application, for the data acquisition device 103, the software programs stored in the memory 22 are different, so the functions implemented by the data acquisition device 103 are different. The functions performed by each device will be described in conjunction with the following flowchart.

[0072] In another possible implementation, the memory 22 may also be integrated with the processor 21 .

[0073] The communication interface 23 is used for connecting the data acquisition device with other devices via a communication network, and the communication network may be Ethernet, wireless access network, wireless local area network (WLAN), etc. The communication interface 23 may include a receiving unit for receiving data and a sending unit for sending data.

[0074] The bus 24 may be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0075] It should be pointed out that Figure 2 The structure shown in the figure does not constitute a limitation on the data acquisition device, except Figure 2 In addition to the components shown, the data acquisition device may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0076] The data collection method provided in the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0077] The data collection method provided in the embodiment of the present application is applied to Figure 1 The data acquisition device 103 in the data acquisition system shown in FIG. Figure 3 As shown, the data collection method includes:

[0078] S301. The data acquisition device acquires scale data of the sensor group through a preprocessing unit, and determines a data preprocessing cycle according to the scale data.

[0079] The preprocessing unit is connected to the sensor group deployed on the pipeline through an extension line.

[0080] Depend on Figure 1 It can be known that the sensors in the above-mentioned sensor group may include ultrasonic sensors, temperature sensors, vibration sensors, flow sensors and pressure sensors. Ultrasonic sensors are sensors that convert ultrasonic signals into other energy signals (usually electrical signals), and are usually used to detect parameters such as the thickness of the object being detected. Temperature sensors are usually used to detect the temperature of the object being detected. Vibration sensors are usually used to detect the vibration state of the object being detected. Flow sensors are usually used to detect the flow rate generated when the fluid flows in the object being detected. Pressure sensors are usually used to detect the pressure of the object being detected.

[0081] The sensors in the sensor group are flexibly arranged in key areas of the pipeline to collect more comprehensive data.

[0082] The preprocessing unit in the data acquisition device is responsible for collecting the scale data of the sensor group and intelligently determining the data preprocessing cycle based on the distribution and characteristics of these data, thereby ensuring real-time monitoring under different operating conditions.

[0083] The scale data of the sensor group may include data such as the number and category of sensors in the sensor group.

[0084] In some embodiments, the data preprocessing period is positively correlated with the scale data of the sensor group. That is, the smaller the scale data of the sensor group, the smaller the data preprocessing period. Correspondingly, the larger the scale data of the sensor group, the larger the data preprocessing period.

[0085] S302, the data acquisition device acquires data from all sensors in the sensor group within the preprocessing period, establishes a data group to be analyzed, determines whether there is redundant data in the data group to be analyzed based on the data type and data value of the acquired sensor data, and removes the redundant data.

[0086] Specifically, within a certain preprocessing period, the data acquisition device can collect data from all sensors to establish a data group to be analyzed. By analyzing the data type and data value, it is determined whether there is redundant data and the redundant data is removed.

[0087] In some embodiments, the data acquisition device may use some common data processing algorithms and threshold settings to ensure that only valid data is retained in the data after elimination, thereby improving the quality of the data and reducing the burden of subsequent analysis.

[0088] S303, the data acquisition device establishes a data matrix according to the sensor position for the same type of data in the data group to be analyzed after removing redundant data, performs abnormal data judgment on the data matrix, determines the abnormal data and determines the type of abnormal data.

[0089] Specifically, after eliminating redundant data, the data acquisition device can establish a data matrix for the same type of data according to the sensor position. Using the data matrix to judge abnormal data, it is possible to analyze the trend and fluctuation of the data, determine which data points are abnormal, and further identify the type of abnormal data.

[0090] In some embodiments, the data acquisition device may combine statistical analysis and machine learning techniques to determine abnormal data in a data matrix to improve recognition accuracy.

[0091] S304: The data collection device compares the abnormal data and the abnormal data type as feature data with the historical abnormal data in the historical abnormal database, determines the data collection result according to the comparison result, and sends the data collection result to the processing unit.

[0092] Specifically, the data acquisition device can use the identified abnormal data and abnormal data types as feature data to compare with the historical abnormality database. This comparison process can help determine whether the current data collection results are consistent with the historical data, so as to timely discover potential risks or faults.

[0093] Optionally, after the data acquisition device determines the comparison result, it can analyze the comparison result to obtain an analysis result. Subsequently, the analysis result can be determined as the data acquisition result.

[0094] Optionally, after the data acquisition device determines the comparison result, it may also directly determine the comparison result as the data acquisition result.

[0095] As can be seen from the above, through flexible sensor layout and intelligent preprocessing mechanism, pipeline status data can be obtained comprehensively and timely, and the accuracy and reliability of data can be improved through redundant elimination and abnormal judgment technology. Not only does it reduce data noise in monitoring, but it also improves the recognition rate of abnormal situations, thereby reducing potential safety hazards and ensuring the safe and stable operation of oil and gas pipelines. This embodiment improves the intelligence level of the monitoring system and provides more reliable data support for decision-making.

[0096] In some embodiments of the present application, Figure 3 ,like Figure 4 As shown, in the above S303, the data acquisition device performs abnormal data judgment on the data matrix, determines the abnormal data and determines the type of abnormal data, specifically including:

[0097] S401. The data acquisition device arbitrarily determines a center point in the data matrix and determines a neighborhood radius.

[0098] Specifically, for the data collection device, a center point (x0, y0) can be arbitrarily determined in the data matrix as the center point, that is, the center of the circle, and e can be used as the neighborhood radius. Optionally, the value of e can be set according to manual experience.

[0099] S402, the data acquisition device calculates the neighborhood average value according to the data values ​​of each coordinate point within the neighborhood radius. In some embodiments of the present application, the neighborhood average value is calculated by the following formula:

[0100]

[0101] Among them, P avg represents the average value of the neighborhood, e represents the neighborhood radius, (x0, y0) represents the coordinates of the center point, and P(x, y) represents the data value of the point (x, y) on the data matrix.

[0102] S403: When the difference between the data value of the center point and the average value of the neighborhood is greater than the data threshold, the data acquisition device determines that the data value of the center point is abnormal and determines it as the abnormal data.

[0103] Specifically, the selection of the center point and the setting of the neighborhood radius constitute the local environment of the data, so that the method can identify local anomalies that may be concealed in the overall data, that is, the data acquisition device can identify outliers through local data analysis. The data acquisition device can effectively reflect the normal fluctuation range of the data by calculating the average of the data values ​​in the neighborhood, thereby more accurately judging outliers.

[0104] It is understandable that by introducing the comparison mechanism of the neighborhood average value, the outliers in the data matrix can be accurately identified, avoiding the misjudgment that may be caused by simple global threshold judgment. At the same time, this method is highly adaptable and can dynamically adjust the judgment criteria according to the local characteristics of the data, thereby improving the accuracy and flexibility of monitoring. It further improves the safety of the oil and gas pipeline monitoring system.

[0105] S404: The data acquisition device obtains the abnormal data type according to the sensor type of the abnormal data.

[0106] Specifically, data acquisition equipment records abnormal data and its type, which can provide an important basis for subsequent processing and analysis.

[0107] In some embodiments of the present application, Figure 4 ,like Figure 5 As shown, in the above S301, the method in which the data acquisition device determines the data preprocessing cycle according to the scale data specifically includes:

[0108] S501. The data acquisition device compares the scale data with a plurality of preset scale data respectively, and determines the data preprocessing cycle according to the comparison results.

[0109] In some embodiments, the plurality of preset scale data may include first preset scale data and second preset scale data, wherein the first preset scale data is smaller than the second preset scale data.

[0110] The data acquisition device may compare the scale data with the first preset scale data and the second preset scale data respectively, and determine the data preprocessing period according to the comparison result.

[0111] Specifically, when the scale data is less than or equal to the first preset scale data, the data preprocessing cycle is determined to be the first preset data processing cycle; when the scale data is greater than the first preset scale data and less than or equal to the second preset scale data, the data preprocessing cycle is determined to be the second preset data processing cycle; when the scale data is greater than the second preset scale data, the data preprocessing cycle is determined to be the third preset data processing cycle; the first preset data processing cycle is less than the second preset data processing cycle, and the second preset data processing cycle is less than the third preset data processing cycle.

[0112] For example, it is assumed that the first preset scale data is 50, and the second preset scale data is 100. The first preset data processing cycle is 1 minute, the second preset data processing cycle is 5 minutes, and the third preset data processing cycle is 10 minutes.

[0113] Case 1: When the scale data is 40, since 40 is less than or equal to the first preset scale data 50, the data preprocessing period is determined to be 1 minute.

[0114] Case 2: When the scale data is 75, 75 is greater than the first preset scale data 50 and less than or equal to the second preset scale data 100, and the data preprocessing period is determined to be 5 minutes.

[0115] Case 3: When the scale data is 120, 120 is greater than the second preset scale data 100, and the data preprocessing period is determined to be 10 minutes.

[0116] It is understandable that the technical means of threshold and hierarchical processing are adopted. By comparing the scale data with two preset scale data, the data preprocessing cycle can be flexibly adjusted according to the change of data. The hierarchical processing method ensures that the frequency of data collection can respond to the actual situation of the pipeline in a timely manner under different conditions. The dynamic change of scale data is used to optimize the responsiveness of the monitoring system. According to the actual collected scale data, by comparing it with the pre-set scale data threshold, it is decided under what circumstances to perform more frequent or sparse data processing.

[0117] In some embodiments of the present application, Figure 5 ,like Figure 6 As shown, in the above S302, the data acquisition device collects data from all sensors in the sensor group within the preprocessing period, establishes a data group to be analyzed, and determines whether there is redundant data in the data group to be analyzed according to the data type and data value of the collected sensor data, and removes the redundant data, specifically including:

[0118] S601. The data acquisition device arranges the data of the same sensor in the data group to be analyzed in time sequence to form a data chain.

[0119] Specifically, the data acquisition device can arrange the collected data of the same sensor in chronological order to form a data chain. This process ensures the temporal consistency of the data and facilitates subsequent comparison and analysis.

[0120] S602: When there are two or more data links of the same type of data, and the data similarity between the two or more data links is greater than a preset threshold, the data acquisition device determines that there is redundant data.

[0121] Optionally, the preset threshold may be 0.5. Of course, in practical applications, the preset threshold may also be flexibly adjusted according to manual experience.

[0122] Optionally, similarity can be calculated using methods such as Euclidean distance and cosine similarity to ensure accurate identification of data similarity.

[0123] S603. When it is determined that redundant data exists, the data acquisition device obtains the data mean within the data preprocessing period based on the data values ​​in the data group to be analyzed, and compares each data of each data chain in two or more data chains with the data mean to obtain the data difference, retains the data chain with the largest data difference, and eliminates the remaining data chains.

[0124] Specifically, after determining that there is redundant data, the data acquisition device can calculate the mean of all data in the preprocessing cycle. Subsequently, the data in each data chain is compared with the data mean to obtain the data difference. By comparing the size of the difference, the data chain with the largest data difference can be retained, and the remaining redundant data chains can be eliminated. As can be seen from the above, by analyzing the time series of the same sensor data, the redundant data is identified by similarity determination, thereby ensuring the uniqueness and validity of the data. The construction of the data chain enables the time series information to be retained, and by calculating the similarity between the data chains, the redundant data can be effectively identified. By comparing the difference, the chain with the largest difference from the mean is retained, which can effectively ensure the retention of representative and more effective data. Invalid or duplicate data is effectively eliminated, and more representative data chains are retained. This not only reduces the interference of data redundancy on subsequent analysis, but also improves the efficiency of data utilization, making the monitoring system more efficient and reliable when processing complex data. It improves the safety and real-time response capability of oil and gas pipeline monitoring, and provides a more accurate decision-making basis for pipeline management.

[0125] In some embodiments of the present application, Figure 6 ,like Figure 7 As shown, in the above S304, the data collection device compares the abnormal data and the abnormal data type as feature data with the historical abnormal data in the historical abnormal database, and the method for determining the data collection result according to the comparison result specifically includes:

[0126] S701. When there is historical abnormal data of the same abnormal data type as the abnormal data in the historical abnormality database, the data collection device classifies the abnormal data by clustering analysis to determine a first data collection result.

[0127] The first data collection result includes risk type or risk valuation.

[0128] Optionally, the data acquisition device can classify the current abnormal data through clustering algorithms such as K-means and DBSCAN to determine the risk type or risk valuation of the abnormal data.

[0129] Optionally, after determining the risk type or risk valuation (ie, the first data collection result), the data collection device may transmit the risk type or risk valuation to the processing unit.

[0130] S702: When there is no historical abnormal data of the same abnormal data type as the abnormal data in the historical abnormality database, the data collection device determines a second data collection result according to the abnormal data type and the abnormal data.

[0131] The second data collection result includes: risk valuation abnormal data and abnormal data type.

[0132] Specifically, if there is no historical data with the same abnormal data type in the historical abnormality database, the data acquisition device can independently calculate the risk valuation based on the abnormal data type and its specific value, and transmit the risk valuation, abnormal data and abnormal data type to the processing unit.

[0133] As can be seen from the above, data acquisition equipment can use the similarity of historical data to enhance the risk assessment of current data. In addition, through cluster analysis, current abnormal data can be compared with historical data to find potential risk patterns. Secondly, data acquisition equipment can identify similar abnormal situations, and then use the cluster characteristics of historical data to more accurately classify and value the risks of new data. In the absence of historical records, data acquisition equipment can independently judge the risk based on the type and value of the current abnormal data, ensuring that even in the absence of historical records, the processing unit can make reasonable risk assessments and respond quickly.

[0134] Thirdly, data collection equipment can effectively identify potential risk types and valuations by combining historical abnormal data for cluster analysis, thus improving the accuracy and reliability of risk assessment. It not only optimizes the response mechanism to abnormal situations, but also enhances the data collection and processing capabilities of the processing unit in the detection system.

[0135] In some embodiments of the present application, in the above S701, when the data collection device classifies the abnormal data using cluster analysis to determine the first data collection result, the method specifically includes:

[0136] S1: The data acquisition device selects historical abnormal data with the same abnormal data type from the historical abnormal database and establishes a cluster analysis library.

[0137] S2: The data acquisition device initializes K centroids in the cluster analysis library and assigns abnormal data to the nearest centroid to form K clusters.

[0138] S3: The data acquisition device recalculates the centroid of each cluster.

[0139] S4: The data acquisition device repeats S2 and S3 until the centroid no longer changes or the preset number of repetitions is completed to obtain at least one cluster.

[0140] S5: The data collection device determines the first data collection result according to the clustering result of the abnormal data.

[0141] In some embodiments of the present application, in the above S5, the method in which the data collection device determines the first data collection result according to the clustering result of the abnormal data specifically includes:

[0142] When abnormal data is classified into at least one cluster of the cluster analysis library, the historical risk type of the cluster to which the abnormal data belongs is used as the risk type of the abnormal data; when the abnormal data is not classified into at least one cluster of the cluster analysis library (that is, it forms a cluster alone), the risk valuation is determined based on the abnormal data type and the abnormal data, and the risk valuation is transmitted to the processing unit.

[0143] In some embodiments of the present application, the centroid calculation formula is as follows:

[0144]

[0145] Among them, Ck represents the centroid of the kth cluster, |Mk| represents the number of data in the kth cluster, and Ti represents the feature vector of the i-th data.

[0146] Specifically, the K-means clustering algorithm is used to match and classify the current abnormal data with the historical data. The historical data includes each historical abnormal data, historical abnormal data type, and historical risk type. By initializing the centroid, iterating the distribution, and adjusting the centroid, the classification of the data can be adjusted dynamically to ensure the accuracy of the clustering results. Each cluster represents a potential risk type. By comparing the clustering results of the current data with the historical data, the risk attributes of the current data can be inferred. In the case of forming a new cluster separately, the risk valuation is calculated based on the data characteristics to ensure that both data with historical precedents and data without precedents can be effectively processed.

[0147] It is understandable that cluster analysis can efficiently process a large amount of abnormal data, classify it according to historical data, quickly determine the current risk type, and improve processing efficiency and accuracy. When new abnormal data appears, it can be processed separately to ensure that the data can be scientifically evaluated regardless of whether there is a historical record, effectively reducing risk hazards.

[0148] In some embodiments of the present application, the risk valuation determined according to the abnormal data type and the abnormal data satisfies the following formula:

[0149]

[0150] Among them, R represents risk valuation, n represents the number of abnormal data types, Ej represents the data value of the j-th abnormal data, Sj represents the standard value of the j-th abnormal data, Wj represents the weight of the j-th abnormal data type, and αj represents the nonlinear weight index of the j-th abnormal data type.

[0151] Specifically, the difference between the specific value of abnormal data and its standard value is used to comprehensively assess the risk through weighting and nonlinear index. By calculating the ratio of the actual value to the standard value, the degree of deviation of the abnormal data can be quantified, and then its contribution to the overall risk can be determined. The weight allows adjustment according to the importance of different data types to ensure that the risk assessment reflects the actual risk situation. The introduction of nonlinear weight index further enhances the system's sensitivity to risk changes, making the assessment more delicate.

[0152] It is understandable that by comprehensively considering the abnormal data type, specific value and its standard value, a comprehensive risk valuation can be dynamically generated to ensure the accuracy and real-time nature of the assessment. It not only improves the ability to respond to abnormal situations, but also adapts to the risk characteristics of different types of data, enhances the safety and reliability of the oil and gas pipeline monitoring system, and effectively reduces potential safety risks.

[0153] In the above embodiment, by setting up a preprocessing unit connected to the extension line of the sensor group deployed on the pipeline, efficient and real-time data collection and processing are achieved. The data preprocessing cycle is flexibly determined according to the scale data of the sensor group, so as to adapt to the dynamic changes in the pipeline operation and ensure the timeliness and comprehensiveness of data collection. Secondly, by establishing a data group to be analyzed and eliminating redundant data, the data redundancy is significantly reduced and the accuracy of data analysis is improved. Then, the data matrix is ​​used to judge the abnormal data, which can not only quickly identify potential risks, but also compare with the historical abnormal database to ensure accurate handling of abnormal situations. The intelligence level and reliability of the oil and gas pipeline monitoring system are improved, the safety hazards are reduced, and the ability to respond to emergencies is enhanced.

[0154] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0155] The embodiment of the present application can divide the functional modules of the data acquisition device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. Optionally, the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0156] like Figure 8 , which is a schematic diagram of the structure of a data acquisition device provided in an embodiment of the present application. Figure 8 The data acquisition device shown includes: a communication unit 801 and a processing unit 802;

[0157] The communication unit 801 is used to collect scale data of the sensor group through the pre-processing unit;

[0158] The processing unit 802 is used to determine the data preprocessing cycle according to the scale data; the preprocessing unit is connected to the sensor group deployed on the pipeline through an extension line;

[0159] The communication unit 801 is also used to collect data from all sensors in the sensor group within the preprocessing period, establish a data group to be analyzed, determine whether there is redundant data in the data group to be analyzed according to the data type and data value of the collected sensor data, and remove the redundant data;

[0160] The processing unit 802 is also used to respectively establish a data matrix for the same type of data in the data group to be analyzed after removing redundant data according to the sensor position, perform abnormal data judgment on the data matrix, determine the abnormal data and determine the type of abnormal data;

[0161] The processing unit 802 is also used to compare the abnormal data and the abnormal data type as feature data with the historical abnormal data in the historical abnormal database, determine the data collection result according to the comparison result, and send the data collection result to the processing unit;

[0162] The step of determining the abnormal data and determining the type of the abnormal data includes:

[0163] Determine a center point (x0, y0) arbitrarily in the data matrix and determine the neighborhood radius;

[0164] Calculate the neighborhood average value based on the data values ​​of each coordinate point within the neighborhood radius;

[0165] When the difference between the data value of the center point and the average value of the neighborhood is greater than the data threshold, the data value of the center point is judged to be abnormal and is determined to be abnormal data;

[0166] According to the sensor type of the abnormal data, the abnormal data type is obtained;

[0167] The neighborhood average value is calculated by the following formula:

[0168]

[0169] Among them, P avg represents the average value of the neighborhood, e represents the neighborhood radius, (x0, y0) represents the coordinates of the center point, and P(x, y) represents the data value of the point (x, y) on the data matrix.

[0170] Optionally, the processing unit 802 is specifically configured to:

[0171] The scale data is compared with a plurality of preset scale data respectively, and a data preprocessing cycle is determined according to the comparison results.

[0172] Optionally, the plurality of preset scale data include first preset scale data and second preset scale data, and the first preset scale data is smaller than the second preset scale data; the processing unit 802 is specifically configured to:

[0173] When the scale data is less than or equal to the first preset scale data, the data preprocessing cycle is determined to be the first preset data processing cycle; when the scale data is greater than the first preset scale data and less than or equal to the second preset scale data, the data preprocessing cycle is determined to be the second preset data processing cycle; when the scale data is greater than the second preset scale data, the data preprocessing cycle is determined to be the third preset data processing cycle; the first preset data processing cycle is less than the second preset data processing cycle, and the second preset data processing cycle is less than the third preset data processing cycle.

[0174] Optionally, the processing unit 802 is specifically configured to:

[0175] The data of the same sensor in the data group to be analyzed are arranged in time series to form a data chain; when there are two or more data chains of the same type of data, and the data similarity between the two or more data chains is greater than a preset threshold, it is determined that there is redundant data; when it is determined that there is redundant data, the data mean within the data preprocessing cycle is obtained according to the data value in the data group to be analyzed, and each data of each data chain in the two or more data chains is compared with the data mean to obtain the data difference, the data chain with the largest data difference is retained, and the remaining data chains are eliminated.

[0176] Optionally, the processing unit 802 is specifically configured to:

[0177] When there is historical abnormal data of the same abnormal data type as the abnormal data in the historical abnormal database, cluster analysis is used to classify the abnormal data to determine the first data collection result; the first data collection result includes: risk type or risk valuation;

[0178] When there is no historical abnormal data of the same abnormal data type as the abnormal data in the historical abnormal database, a second data collection result is determined according to the abnormal data type and the abnormal data; the second data collection result includes: risk valuation, abnormal data and abnormal data type.

[0179] Optionally, the processing unit 802 is specifically configured to:

[0180] S1: Filter out historical abnormal data with the same abnormal data type in the historical abnormal database and establish a cluster analysis library;

[0181] S2: Initialize K centroids in the cluster analysis library and assign abnormal data to the nearest centroid to form K clusters;

[0182] S3: recalculate the centroid of each cluster;

[0183] S4: Repeat S2 and S3 until the centroid no longer changes or the preset number of repetitions is completed to obtain at least one cluster;

[0184] S5: Determine a first data collection result according to the clustering result of the abnormal data.

[0185] Optionally, the processing unit 802 is specifically configured to:

[0186] When the abnormal data is classified into at least one cluster of the cluster analysis library, the historical risk type of the cluster to which the abnormal data belongs is used as the risk type of the abnormal data; when the abnormal data is not classified into at least one cluster of the cluster analysis library, the risk valuation is determined according to the abnormal data type and the abnormal data.

[0187] Optionally, the centroid is calculated as follows:

[0188]

[0189] Among them, Ck represents the centroid of the kth cluster, |Mk| represents the number of data in the kth cluster, and Ti represents the feature vector of the i-th data.

[0190] Optionally, the processing unit 802 is specifically configured to:

[0191]

[0192] Among them, R represents risk valuation, n represents the number of abnormal data types, Ej represents the data value of the j-th abnormal data, Sj represents the standard value of the j-th abnormal data, Wj represents the weight of the j-th abnormal data type, and αj represents the nonlinear weight index of the j-th abnormal data type.

[0193] An embodiment of the present application also provides a computer-readable storage medium, which includes computer-executable instructions. When the computer-executable instructions are executed on a computer, the computer executes the data collection method provided in the above embodiment.

[0194] The embodiment of the present application also provides a computer program product, which can be directly loaded into a memory and contains software code. After the computer is loaded and executed, the computer program product can implement the data acquisition method provided in the above embodiment. Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention is described in detail with reference to the preferred embodiments, ordinary technicians in this field should understand that they can still modify or replace the technical solution of the present invention, and these modifications or equivalent replacements cannot make the modified technical solution deviate from the spirit and scope of the technical solution of the present invention.

[0195] The system provided in the above embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be decomposed or combined. For example, the modules in the above embodiment can be combined into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not regarded as improper limitations of the present invention.

[0196] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0197] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0198] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0199] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0200] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A data collection method based on an extension line, characterized in that: include: Collecting scale data of the sensor group through a preprocessing unit, and determining a data preprocessing period according to the scale data; The pre-processing unit is connected to the sensor group deployed on the pipeline via an extension line; Collecting data from all sensors in the sensor group during the preprocessing period to establish a data group to be analyzed, determining whether there is redundant data in the data group to be analyzed based on the data type and data value of the collected sensor data, and removing the redundant data; Establishing data matrices for the same type of data in the data group to be analyzed after removing redundant data according to the sensor positions, performing abnormal data judgment on the data matrix, determining the abnormal data and determining the type of the abnormal data; Compare the abnormal data and the abnormal data type as feature data with the historical abnormal data in the historical abnormal database, determine the data collection result according to the comparison result, and send the data collection result to the processing unit; The step of determining the abnormal data and determining the type of the abnormal data includes: Arbitrarily determine a center point (x0, y0) in the data matrix and determine the neighborhood radius; Calculate the neighborhood average value according to the data value of each coordinate point within the neighborhood radius; When the difference between the data value of the center point and the average value of the neighborhood is greater than the data threshold, it is determined that the data value of the center point is abnormal and is determined as the abnormal data; According to the sensor type of the abnormal data, obtaining the type of the abnormal data; The neighborhood average value is calculated by the following formula: Among them, P avg represents the average value of the neighborhood, e represents the neighborhood radius, (x0, y0) represents the coordinates of the center point, and P(x, y) represents the data value of the point (x, y) on the data matrix.

2. The data collection method based on the extension line according to claim 1 is characterized in that: The step of determining the data preprocessing period according to the scale data includes: The scale data is compared with a plurality of preset scale data respectively, and the data preprocessing period is determined according to the comparison result.

3. The data acquisition method based on the extension line according to claim 2 is characterized in that: The plurality of preset scale data include first preset scale data and second preset scale data, and the first preset scale data is smaller than the second preset scale data; The step of comparing the scale data with a plurality of preset scale data respectively and determining a data preprocessing period according to the comparison results includes: When the scale data is less than or equal to the first preset scale data, the data preprocessing cycle is determined to be the first preset data processing cycle; when the scale data is greater than the first preset scale data and less than or equal to the second preset scale data, the data preprocessing cycle is determined to be the second preset data processing cycle; when the scale data is greater than the second preset scale data, the data preprocessing cycle is determined to be the third preset data processing cycle; the first preset data processing cycle is less than the second preset data processing cycle, and the second preset data processing cycle is less than the third preset data processing cycle.

4. The data collection method based on the extension line according to claim 1, characterized in that: The step of judging whether there is redundant data in the data group to be analyzed according to the data type and the data value, and removing the redundant data, includes: The data of the same sensor in the data group to be analyzed are arranged in time series to form a data chain; when there are two or more data chains of the same type of data, and the data similarity between the two or more data chains is greater than a preset threshold, it is determined that the redundant data exists; when it is determined that the redundant data exists, the data mean value within the data preprocessing period is obtained according to the data value in the data group to be analyzed, and each data of each data chain in the two or more data chains is compared with the data mean value to obtain the data difference, retain the data chain where the maximum data difference is located, and eliminate the remaining data chains.

5. The data collection method based on extension line according to claim 1, characterized in that: The comparing the abnormal data and the abnormal data type as feature data with the historical abnormal data in the historical abnormal database, and determining the data collection result according to the comparison result, comprises: When there is historical abnormal data of the same abnormal data type as the abnormal data in the historical abnormal database, cluster analysis is used to classify the abnormal data to determine a first data collection result; the first data collection result includes: risk type or risk valuation; When there is no historical abnormal data of the same abnormal data type as the abnormal data in the historical abnormality database, a second data collection result is determined according to the abnormal data type and the abnormal data; the second data collection result includes: risk valuation, the abnormal data and the abnormal data type.

6. The data collection method based on the extension line according to claim 5, characterized in that: The adopting cluster analysis to classify the abnormal data to determine the first data collection result includes: S1: Filter out historical abnormal data with the same abnormal data type in the historical abnormal database, and establish a cluster analysis library; S2: Initialize K centroids in the cluster analysis library, assign the abnormal data to the nearest centroid, and form K clusters; S3: recalculate the centroid of each cluster; S4: Repeat S2 and S3 until the centroid no longer changes or the preset number of repetitions is completed to obtain at least one cluster; S5: Determine the first data collection result according to the clustering result of the abnormal data.

7. The data collection method based on the extension line according to claim 6, characterized in that: The determining the first data collection result according to the clustering result of the abnormal data includes: When the abnormal data is classified into at least one cluster of the cluster analysis library, the historical risk type of the cluster to which the abnormal data is classified is used as the risk type of the abnormal data; when the abnormal data is not classified into at least one cluster of the cluster analysis library, the risk valuation is determined according to the abnormal data type and the abnormal data.

8. The data collection method based on extension line according to claim 6, characterized in that: The centroid calculation formula is as follows: Wherein, Ck represents the centroid of the k-th cluster, |Mk| represents the amount of data in the k-th cluster, and Ti represents the feature vector of the i-th data.

9. The data collection method based on extension line according to claim 7, characterized in that: The determining the risk estimation value according to the abnormal data type and the abnormal data includes: Among them, R represents the risk valuation, n represents the number of the abnormal data types, Ej represents the data value of the j-th abnormal data, Sj represents the standard value of the j-th abnormal data, Wj represents the weight of the j-th abnormal data type, and αj represents the nonlinear weight index of the j-th abnormal data type.

10. A data acquisition device based on an extension line, characterized in that: include: A processor and a memory; wherein the memory is used to store one or more programs, and the one or more programs include computer-executable instructions. When the device is running, the processor executes the computer-executable instructions stored in the memory to enable the device to perform the method described in any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that: When the computer-executable instructions stored in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is capable of performing the method as claimed in any one of claims 1 to 9.

12. A computer program product, characterized in that The computer program product comprises: a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is caused to perform the method according to any one of claims 1 to 9.

Images