Data processing method and device, medium and electronic equipment

Abstract

The invention relates to a data processing method and device, a medium and electronic equipment. The method comprises the following steps: acquiring initial data of the equipment according to a preset acquisition period; according to a preset classification requirement, adding a classification label to the initial data, determining source data, and storing the source data to a database; and according to a target classification tag input by a user, screening the source data in the database, and determining target data corresponding to the target classification tag so as to improve the data processing efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular, to a data processing method, device, medium and electronic device. Background Art

[0002] As the number of equipment in various industries continues to increase and their intelligence levels continue to improve, a large number of equipment continue to generate massive amounts of data during operation. In the field of industrial production, such as various processing machine tools and automated production line equipment in manufacturing factories, power generation equipment and transmission equipment in the energy industry, these equipment are equipped with numerous sensors to collect various operating parameter data such as temperature, pressure, speed, current, voltage, etc., which brings huge pressure to data statistics and processing.

[0003] In related technologies, data is usually collected from the information system of some equipment first, and then some key data is recorded manually on a regular basis. This method is not only inefficient, but also prone to errors and cannot meet the requirements for data comprehensiveness and timeliness. At the same time, manual processing makes it difficult to conduct in-depth mining and analysis of data, and cannot fully tap the potential value of data. Summary of the invention

[0004] The purpose of the present disclosure is to provide a data processing method, device, medium and electronic device to improve data processing efficiency.

[0005] In order to achieve the above objectives, in a first aspect, the present disclosure provides a data processing method, the method comprising: Acquire the initial data of the device according to the preset collection cycle; Adding classification labels to the initial data according to preset classification requirements, determining source data, and saving the source data to a database; According to the target classification label input by the user, the source data in the database is screened to determine the target data corresponding to the target classification label.

[0006] Optionally, there are multiple preset classification requirements, and adding classification labels to the initial data according to the preset classification requirements to determine the source data includes: Input a plurality of the preset classification requirements into the data screening model to obtain the source data output by the data screening model, wherein the source data is obtained by the data screening model by screening the initial data in turn through each of the preset classification requirements and adding the classification label.

[0007] Optionally, the data screening model is trained in the following manner: Selecting a plurality of collected initial data and performing normalization processing to determine a plurality of sample initial data; Manually adding corresponding classification labels to each of the sample initial data to determine a plurality of verification source data; Input the plurality of sample initial data into the data screening model, sequentially pass through the plurality of nodes of the data screening model, add at least one classification label to each of the sample initial data, and output a plurality of sample source data, each of the nodes corresponding to one of the preset classification requirements; The parameters of the data screening model are updated according to the outputted multiple sample source data and the verification source data, and the training of the data screening model is completed until the preset conditions are met.

[0008] Optionally, adding classification labels to the initial data according to preset classification requirements to determine source data includes: The source data is determined by adding a classification label to the initial data according to at least one of data time, data type, data source device, data source system, keyword information, and identifier information.

[0009] Optionally, acquiring initial data of the device according to a preset collection period includes: Taking screenshots of the instrument of the device according to the preset collection period to determine the device data image; Recognize the text in the device data image and determine the image text data; The image text data is screened by using preset keywords and / or preset identifiers to determine the initial data.

[0010] Optionally, there are multiple device data images, and the identifying text in the device data image and determining the image text data includes: sequentially identifying the text in each of the device data images; In the event of an error in the recognition of any of the device data images, re-recognize the current device data image until the number of errors reaches a preset number, then mark the current device data image and perform recognition of the next device data image; When the current device data image is successfully identified, the identification of the next device data image is performed.

[0011] Optionally, the method further comprises: Get the data display format entered by the user; According to the data display format, the target data is displayed in at least one form of image, text and table.

[0012] In a second aspect, the present disclosure further provides a data processing device, the device comprising: An acquisition module is configured to acquire initial data of the device according to a preset acquisition period; A classification module, configured to add classification labels to the initial data according to preset classification requirements, determine source data, and save the source data to a database; The screening module is configured to screen the source data in the database according to the target classification label input by the user, and determine the target data corresponding to the target classification label.

[0013] In a third aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect of the present disclosure.

[0014] In a fourth aspect, the present disclosure further provides an electronic device, including: a memory having a computer program stored thereon; A processor is used to execute the computer program in the memory to implement the steps of the method described in the first aspect of the present disclosure.

[0015] Through the above technical solutions, the automatic collection, multi-dimensional classification storage and accurate screening of equipment data are realized. The comprehensiveness of data and management efficiency are improved, the database performance is optimized, the data can be accurately located to meet diverse needs, and the efficiency of equipment management and production operations is improved.

[0016] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings: Figure 1 It is a step flow chart of a data processing method proposed by an exemplary embodiment of the present disclosure.

[0018] Figure 2 It is a block diagram of a data processing device proposed by an exemplary embodiment of the present disclosure.

[0019] Figure 3 It is a block diagram of an electronic device according to an exemplary embodiment.

[0020] Figure 4 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0021] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0022] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0023] Figure 1 is a flowchart of a data processing method proposed by an exemplary embodiment of the present disclosure. Figure 1 As shown, the method can be applied to a data processing system, and the method comprises the following steps: In step S11, initial data of the device is acquired according to a preset acquisition cycle.

[0024] The initial data collected may be real-time data at the current moment, such as the vehicle speed or remaining power of the vehicle at the current moment, or it may be the accumulated total data for a period of time, for example, the power generation collected by the generator in a thermal power plant at the current moment is the total power generation for a period of time.

[0025] In one embodiment, different devices correspond to different preset collection cycles. For core equipment such as boilers, the operation process is relatively stable but involves many complex physical and chemical reactions. It is necessary to continuously monitor key parameters to ensure safe and efficient operation. For example, for parameters such as furnace temperature and pressure, since their changes are relatively slow and have a significant impact on the overall operation of the boiler, the collection cycle can be set to a shorter time interval, such as collecting once every 10 seconds. In this way, subtle changes in temperature and pressure can be captured in time so that a quick response can be made when abnormal trends occur to prevent potential safety accidents, such as boiler overpressure or overtemperature operation. For some auxiliary equipment, such as lighting system equipment in the factory area, its operating state is relatively fixed, and it is only in working state at a specific time (such as at night), and its operating parameters do not change frequently. For such equipment, the collection cycle can be set to be longer, for example, its working state (whether it is turned on, voltage and current and other basic parameters) is collected every 30 minutes. This can not only meet the monitoring needs of its basic operation, but also avoid wasting system resources and storage space due to too frequent data collection.

[0026] In one implementation, before collecting the initial data of the device, the device and system to be collected, the cycle time of the data collection cycle, and the start time and end time of multiple collection cycles may be set.

[0027] Taking the equipment data collection of thermal power plants as an example, first of all, all the equipment in the thermal power plant is classified in detail, which can be divided according to the function, importance, operating characteristics and other factors of the equipment. For example, the equipment in the thermal power plant is divided into several categories, such as main power generation equipment (boiler, steam turbine, generator), auxiliary equipment (pump, fan, cooling tower, etc.), electrical equipment (transformer, switchgear, etc.), and environmental monitoring equipment (temperature and humidity sensor, smoke sensor, etc.).

[0028] For each type of equipment, the initial range of the collection cycle is determined based on the equipment's operating characteristics, data importance and sensitivity, and system resource and storage limitations. For example, for major power generation equipment, the initial range of key parameter collection cycles is between 5 and 15 seconds; the collection cycle for auxiliary equipment is between 30 seconds and 5 minutes; the collection cycle for electrical equipment is between 10 seconds and 2 minutes; and the collection cycle for environmental monitoring equipment is between 10 minutes and 1 hour.

[0029] Then, in the actual operation process, the various parameters of each device are collected for a period of time (such as one week) during the trial operation, and the resource usage of the data processing system (such as CPU usage and network bandwidth occupancy) and the growth of data storage are monitored. According to the results of the trial operation, the collection cycle set initially is fine-tuned to achieve the best balance. For example, it is found that the data fluctuation of a temperature sensor of a certain boiler is very small under the 10-second collection cycle and the system resource occupancy is high. After analysis, the collection cycle is adjusted to 15 seconds, which not only ensures the effective monitoring of temperature changes, but also reduces the consumption of system resources.

[0030] In addition, for some equipment that runs on a fixed shift, such as a water pump, the start and end time of the collection cycle can match the equipment's operating shift. Assume that the water pump runs two shifts a day, from 8 a.m. to 4 p.m. and from 8 p.m. to 4 a.m. Then the start time of the data collection cycle for this equipment can be set to 8 a.m. and 8 p.m., and the end time can be set to 4 p.m. and 4 a.m. This allows effective data collection during equipment operation and stops collection during equipment downtime, avoiding the collection of invalid data and reducing the workload of data storage and processing.

[0031] For some intermittently running equipment, such as fire pumps, they will only start running during fire drills or fire alarms. For such equipment, the start time of the collection cycle can be set to when the equipment starts (the fire alarm system triggers the data collection start signal), and the end time can be set to a period of time after the equipment stops running (such as 10 minutes, so as to collect some subsequent parameter changes after the equipment stops, such as the pressure drop of the pipeline, etc.). In this way, key data of the equipment during operation can be accurately obtained, providing valuable basis for performance analysis and maintenance of the equipment.

[0032] By reasonably setting the collection cycles of different devices and the start and end times of multiple collection cycles, it is possible to ensure that while meeting the equipment monitoring needs, the data collection system resources are efficiently utilized, data storage is effectively managed, and strong data support is provided for the safe and stable operation and performance optimization of the equipment.

[0033] In one embodiment, the initial data may be at least one of an image, a document, and a table. For example, for some devices that cannot perform digital collection, a screenshot of the device data displayed by the device instrument may be taken, and the captured image or text in the image may be used as the initial data of the device. For devices that can perform digital collection, data collection is directly performed on the device to determine the collected text or table as the initial data of the device.

[0034] In step S12, classification labels are added to the initial data according to preset classification requirements, source data is determined, and the source data is saved in a database.

[0035] In one embodiment, the preset classification requirement includes at least one of data time, data type, data source device, data source system, keyword information, and identifier information. Each preset classification requirement corresponds to at least one classification tag for classifying the initial data.

[0036] In one embodiment, first, it is necessary to set classification requirements and classification labels according to device type, business requirements, data characteristics, and usage scenarios. For example, in an e-commerce business scenario, the preset classification requirements include product classification requirements, price classification requirements, and brand classification requirements, where the classification labels of product classification requirements include clothing, electronic products, and household items, the classification labels of price classification requirements include high price, medium price, and low price, and the classification labels of brand classification requirements include domestic brands and foreign brands.

[0037] Then, after collecting the initial data, according to the preset classification requirements and the classification labels corresponding to each preset classification requirement, analyze each piece of initial data one by one to determine which classification labels it meets the characteristics defined by, and then classify it into the corresponding category. Taking the commodity data in the e-commerce business as an example, if a commodity record shows that it is a smart phone, the price is 5,000 yuan, and the brand is brand A, then according to the preset classification labels pre-set in the above method, it can be classified into the "high price" range under the "electronic products" category, and belongs to the "brand A" category, and the corresponding classification labels are added to it respectively.

[0038] Finally, after classification and adding classification labels, all data sets under each classification label constitute the source data. For example, in the above e-commerce case, all product data classified as "electronic products" and meeting relevant classification labels such as "high price" and "Brand A" together constitute the source data for these classification dimensions. Source data is the basis for subsequent data analysis, query and application. After classification and sorting, it becomes more organized and convenient for targeted management and use.

[0039] In addition, you can choose a suitable database to classify and store the source data, such as relational databases MySQL, Oracle, or non-relational databases MongoDB. Generally speaking, in a relational database, different tables can be created to correspond to different categories, and the fields in the table store specific data content and related classification identification information. For example, for source data classified as "electronic products", you can create a table called "electronic_products", which contains fields such as "product_name", "price", and "brand", and then insert the corresponding product data into this table according to the classification rules. In a non-relational database, the source data is stored in the form of documents (such as JSON format documents in MongoDB), and each document contains the corresponding data content and key-value pair information related to the classification label, which is convenient for subsequent query and retrieval.

[0040] In one embodiment, the PPA (Robotic Process Automation) function of the data processing system can be used to add classification labels to the initial data and automatically store them in the database.

[0041] In step S13, the source data in the database is screened according to the target classification label input by the user to determine the target data corresponding to the target classification label.

[0042] In one embodiment, after the user inputs the target classification label, the data processing system executes corresponding screening logic in the database storing the source data according to these target classification labels to determine the target data corresponding to the target classification label. The database includes relational databases and non-relational databases. Different types of databases have different screening methods. Relational databases use SQL (Structured Query Language) statements for screening. Non-relational databases, such as MongoDB, can use query syntax based on JSON format.

[0043] In one embodiment, after receiving the target classification labels input by the user, the data processing system converts these target classification labels into screening condition statements corresponding to the database type to implement screening of source data in the database. After the database screening operation, all data records that meet the target classification labels input by the user are determined as target data.

[0044] After the target data is extracted from the database, it is returned to the user or the subsequent data processing flow in an appropriate form. The return form can be determined according to the specific application scenario. For example, when displaying device data in a Web application, the target data will be converted into JSON format and returned to the front-end page for display. In the data analysis tool, it is directly provided to the analyst in the form of a table or data frame for further data analysis operations.

[0045] Through the above technical solutions, the automatic collection, multi-dimensional classification storage and accurate screening of equipment data are realized. The comprehensiveness of data and management efficiency are improved, the database performance is optimized, the data can be accurately located to meet diverse needs, and the efficiency of equipment management and production operations is improved.

[0046] Optionally, there are multiple preset classification requirements, and classifying the initial data according to the preset classification requirements to determine the source data includes: Input a plurality of the preset classification requirements into the data screening model to obtain the source data output by the data screening model, wherein the source data is obtained by the data screening model by screening the initial data in turn through each of the preset classification requirements and adding the classification label.

[0047] It is worth noting that in actual data processing scenarios, the use of multiple preset classification requirements can classify the initial data more carefully and comprehensively from multiple dimensions. Each preset classification requirement represents a characteristic or attribute of a data. By comprehensively applying these classification requirements, the data can be divided into more precise categories, which facilitates the subsequent accurate screening and use of data for different needs. For example, in the context of thermal power plant equipment data, only the equipment type classification requirement can only roughly distinguish the data generated by different equipment, but after adding multiple classification requirements such as operating status, data source, and collection time, various specific data subsets can be more clearly located and managed.

[0048] In one embodiment, the data screening model must first be constructed, which can be implemented based on machine learning algorithms (such as classification algorithms such as decision trees and machine forests) or rule-based programming logic. The model has multiple pre-set classification labels, and the input of the model is the initial data. For example, taking rule-based programming logic as an example, in the data processing of thermal power plant equipment, the initial data is a record set including equipment number, collection time, various parameter values ​​and other information, and the preset classification requirements include "equipment type", "operating status", and "parameter type", among which the classification labels corresponding to "equipment type" include boilers and turbines, the classification labels corresponding to "operating status" include normal and fault, and the classification labels corresponding to "parameter type" include temperature, pressure and flow.

[0049] Then, the data screening model will filter the initial data in turn according to each preset classification requirement. For example, the first round of screening is performed based on the "equipment type", and all data from boiler equipment are selected to form an intermediate subset. Then, for this subset of boiler equipment data, the data of boiler equipment in the "normal operation" state is further screened out based on the "operating status", so that the data range is further narrowed. Then, according to the "parameter type", data such as "temperature parameters" are filtered out from this smaller data subset. This process is a progressive and gradually refined screening process. After each screening of a preset classification requirement, the data becomes more targeted.

[0050] Finally, while filtering, the data filtering model will add corresponding classification labels to the data that pass the corresponding filtering conditions. For example, for the temperature parameter data of the filtered boiler under normal operating conditions, label information such as "boiler", "normal operation", and "temperature parameters" will be added to these data records to indicate the relevant attributes of these data. These labeled data eventually constitute the source data, and data that meets specific needs can be easily retrieved and extracted from the source data based on these labels.

[0051] For example, the initial data collected from the thermal power plant equipment is shown in Table 1 below: Table 1 (initial data table)

[0052] The preset classification requirements include "equipment type (classification label: boiler-B, turbine-T, generator-G)", "operating status (classification label: normal-N, fault-F)", and "parameter type (classification label: temperature-Temp, speed-Speed, power-Power).

[0053] The data screening model is constructed according to the above preset classification requirements, and the initial data shown in Table 1 is input into the constructed data screening model, wherein each preset classification requirement corresponds to a node of the data screening model, and the initial data performs a classification judgment when passing through each node and then adds different classification labels, and finally obtains the source data described in Table 2 below: Table 2 (Source data table)

[0054] After filtering and adding labels according to each preset classification requirement, these data with multiple labels constitute the source data. If you want to find the temperature parameter data of the boiler under normal operating conditions in the future, you can quickly and accurately filter it out from the source data based on these labels to meet specific data usage needs.

[0055] In this way, by utilizing multiple preset classification requirements in combination with a data screening model, the initial data can be classified and managed more finely, thus improving the availability and pertinence of the data and facilitating the subsequent extraction of target data based on different business scenarios and analysis requirements.

[0056] Optionally, the data screening model is trained in the following manner: In the first step, multiple collected initial data are selected and normalized to determine multiple sample initial data.

[0057] In one embodiment, when selecting initial data as sample initial data, a variety of representative initial data can be selected to cover various situations and data features. For example, in the equipment data of a thermal power plant, it is necessary to select initial data of different equipment, different operating periods, different parameter value ranges, etc. to ensure the diversity of samples. For example, there are data collected when the boiler is running at high load, normal load, and low load, as well as initial data of the turbine at the startup, stable operation, and shutdown stages. Then the selected initial data is normalized so that the data screening model can learn and process these data more effectively.

[0058] In the second step, corresponding classification labels are manually added to each of the sample initial data to determine multiple verification source data.

[0059] The third step is to input the multiple sample initial data into the data screening model, and add at least one classification label to each of the sample initial data through the multiple nodes of the data screening model in turn, and output multiple sample source data, each of which corresponds to a preset classification requirement.

[0060] In one embodiment, the data screening model is a decision tree model, and the data screening model can be constructed as a structure including multiple nodes, each node is responsible for performing classification operations according to a preset classification requirement. For example, the first node is to judge the type of equipment based on information such as the equipment number, which corresponds to the preset classification requirement of "equipment type"; the data classified by this node then enters the next node, such as judging the operating status based on the operating parameter range of the equipment, corresponding to the preset classification requirement of "operating status", and so on. After the sample initial data is input into the model, the data will flow through each node in turn, and at each node, it will be judged according to the corresponding classification rules and the corresponding preset classification labels will be added, and finally multiple sample source data with labels will be obtained from the output end of the model.

[0061] The fourth step is to update the parameters of the data screening model according to the output of the multiple sample source data and the verification source data, until the preset conditions are met, thus completing the training of the data screening model.

[0062] In one embodiment, by comparing the sample source data output by the data screening model with the manually annotated verification source data, an evaluation index can be calculated to measure the accuracy of the data screening model, wherein the evaluation index includes at least one of the accuracy rate (the proportion of samples correctly classified by the model to the total number of samples), the recall rate (the proportion of actual positive examples correctly predicted by the model), and the F1 value (the harmonic mean of the accuracy rate and the recall rate). If the output result of the model is inconsistent with the verification source data, it means that there is a deviation in the classification of the model, and the model parameters can be updated by adjusting the node splitting rules, feature selection order, etc.

[0063] Among them, the preset condition can be that the training rounds of the data screening model reach a preset number, or the initial sample data is divided into a validation set and a training set, and the evaluation index of the validation set reaches a set threshold, such as an accuracy rate of more than 90%.

[0064] Optionally, classifying the initial data according to preset classification requirements to determine the source data includes: The source data is determined by adding a classification label to the initial data according to at least one of data time, data type, data source device, data source system, keyword information, and identifier information.

[0065] Among them, keyword information is to extract words or phrases with key meanings from the data content as labels. Usually, these keywords can reflect the core theme, important features or specific uses of the data. For example, extract subject words from document data, extract keywords related to fault types from equipment fault record data, etc. Identifier information is a pre-set code, number or symbol with a specific identification function, which is used to uniquely or specifically refer to a certain type of data, a certain device, a certain project, etc. For example, the barcode of a product, the unique serial number of a device, the number of a project, etc. can all be used as identifier information to add tags for accurate identification and distinction.

[0066] It is worth noting that data time can be information related to the time point, time period or time range of data collection. For example, the specific collection time in the format of year, month, day, hour, minute and second, or time period labels divided by morning, noon and evening, working days and non-working days, different seasons, etc. By adding classification labels by time, it is convenient to track the changing trend of equipment data over time, analyze the differences in the operating characteristics of equipment in different time periods, etc. The data type refers to the format or nature category of the data itself. Common ones include numerical data (such as specific values ​​of temperature and pressure), text data (such as equipment fault description text), image data (such as monitoring images of the appearance of the equipment), and audio data (such as sound records when the equipment is running).

[0067] Clarifying the data type helps to adopt appropriate data processing methods and storage strategies. Different types of data often have different focuses and technical means when analyzing and applying. The data source device is used to determine which specific physical device generated the initial data, which is very critical in scenarios with many different devices. For example, in a smart factory, there are various processing equipment, testing equipment, transmission equipment, etc. Clarifying which specific device the collected initial data comes from can quickly locate the operating status of the equipment and related problems. In a complex information environment, different systems or devices are responsible for data processing in different areas or types. Distinguishing the source system or device helps to sort out the overall data architecture and integrate data analysis across systems.

[0068] After adding classification labels according to the above dimensions, the source data is made to have multiple classification labels, which provides a favorable data basis for subsequent data screening and analysis.

[0069] In an optional implementation, when collecting the initial data of the device, if the device data can be directly read or collected, the initial data is directly obtained; if the device data cannot be directly read or collected, and the device data of the device can only be displayed on the instrument of the device, the above step S11 can be performed in the following manner: First, screenshots of instruments of the device are taken according to the preset collection period to determine device data images.

[0070] In one embodiment, when each acquisition cycle arrives, the instrument of the corresponding device is screenshotted using an automated screenshot tool or a software module integrated in the device monitoring system. These device data images can intuitively reflect the readings, status indications and other information of the instrument at a specific moment. For example, in the screenshot image of the boiler pressure gauge, you can clearly see the scale position pointed by the pressure gauge pointer and the pressure unit marked on the dial; the screenshot of the turbine tachometer will display the specific value of the speed and the relevant range identification, etc. The image format is common JPEG, PNG and other formats.

[0071] Then, the text in the device data image is recognized to determine the image text data.

[0072] In one implementation, in order to extract valuable information from the device data image, it is necessary to use OCR (Optical Character Recognition) technology. OCR technology pre-processes the image, such as grayscale, noise reduction, binarization, etc., to improve the accuracy of text recognition, and then uses algorithm steps such as character segmentation, feature extraction, and comparison with predefined character templates to identify the text content in the image, so as to realize the conversion of the text content in the image into a text format that can be edited and processed by a computer.

[0073] In another embodiment, the text recognition process may be affected by factors such as image quality (such as image blur, uneven lighting, etc.), text font and size, text and background contrast, etc., resulting in errors in the recognition results. Therefore, before recognizing image text, the device data image is pre-processed by image enhancement technology, or after the recognition is completed, the recognized image text data is post-processed by text processing technology, such as spelling check, grammar check, comparison and verification with known reasonable value ranges or formats, etc., to improve the accuracy of the image text data.

[0074] Finally, the image text data is screened by using preset keywords and / or preset identifiers to determine the initial data.

[0075] Among them, the preset keywords are words or phrases with key indicative meanings related to the operating status of the equipment, the names of important parameters, the types of faults, etc. For example, in thermal power plant equipment, the preset keywords are "overtemperature", "overload", "normal operation", "maintenance", etc. The preset keywords are used to quickly locate the data of the equipment in a specific operating state. The preset identifier is a code, number or symbol with a unique identification function, such as the equipment number, the instrument model, the system identifier to which the data belongs, etc., which facilitates the accurate distinction between data from different equipment and different sources. For example, each boiler has a unique equipment number "B001", "B002", etc. These identifiers can be used to filter out data related to specific equipment.

[0076] In one embodiment, after obtaining the image text data, the content including preset keywords or meeting the preset identifier requirements is searched in the image text data by string matching, and the data meeting the conditions is screened out. For example, in the large amount of text data after all the equipment data images are recognized by text, the data records including the keyword "overtemperature" are screened out, or all the image text data related to the boiler numbered B001 are screened out by the equipment number "B001" identifier. The data set obtained after such screening constitutes the source data, which is an important basis for subsequent in-depth analysis, report generation, equipment operation and maintenance decision-making, etc.

[0077] Through the above method, redundant data, erroneous data and non-target data can be effectively eliminated, thereby improving data processing efficiency and reducing the memory usage of the database by the source data.

[0078] Optionally, there are multiple device data images, and the identifying text in the device data image and determining the image text data includes: The text in each of the device data images is recognized in turn.

[0079] In the event of an error in the recognition of any of the device data images, the current device data image is re-recognized until the number of errors reaches a preset number, then the current device data image is marked and the recognition of the next device data image is performed.

[0080] When the current device data image is successfully identified, the identification of the next device data image is performed.

[0081] Among them, in the actual data collection and processing process, multiple equipment data images will be obtained. For example, screenshots of different instruments of multiple equipment in the factory, or screenshots of instrument status of the same equipment at different time points, will generate multiple equipment data images. It is necessary to apply OCR technology to these images one by one to extract the text information in them, so as to convert them into image text data that can be analyzed and processed later. This process is carried out in a certain order, such as in the order of image acquisition time, or in the order of equipment numbers from small to large, etc., and each equipment data image is input into the text recognition system for processing in turn to ensure that each image has a corresponding text recognition attempt and does not miss any potential data information in any image. However, the text recognition process may be affected by various factors and errors may occur. For example, the light environment of the equipment instrument is not good, resulting in dark or shadowed images, which affects the clarity of the text; the instrument itself has a special font, a small font size, or wear and stains, which makes it difficult to accurately identify the text content; or the image loses data or is damaged during transmission, resulting in a decrease in recognition accuracy. Therefore, an error handling mechanism needs to be set to deal with recognition errors.

[0082] For example, when an error occurs when recognizing a certain device data image, the system attempts to perform text recognition on the current device data image again, hoping to overcome recognition problems caused by accidental factors through multiple attempts and improve the recognition success rate. However, recognition cannot be repeated indefinitely, so a preset number of allowed errors is set in advance. If the number of repeated recognitions reaches the preset number and the text in the image cannot be accurately recognized, it means that the current image has serious quality problems or other difficult-to-solve interference factors. At this time, the device data image is marked, for example, as a "difficult to recognize image", and the relevant image number, acquisition time and other basic information are recorded. Then, the image is skipped and the recognition operation continues on the next device data image to ensure that the overall text recognition process can continue to advance and will not stagnate for a long time due to individual difficult-to-process images.

[0083] In one embodiment, after text recognition is performed on a device data image, the recognized text needs to be verified, such as by verifying whether the text recognition result is reasonable through grammar checking and judging the rationality of the data format, for example, judging whether the recognized pressure value is within a reasonable range. If the verification result is that expected, accurate and reasonable text information is extracted, then the same text recognition operation is performed on the next device data image in a predetermined order, and this cycle is repeated until the text recognition work of all device data images is completed, and finally the image text data corresponding to each image is obtained.

[0084] Through the above solution, when faced with various image quality problems and recognition difficulties, valuable image text data can be accurately extracted to ensure the smooth progress of the data processing of the entire device and provide complete basic data for subsequent further data screening and analysis operations.

[0085] Optionally, the data processing method provided by the present disclosure further includes: Get the data display format entered by the user.

[0086] According to the data display format, the target data is displayed in at least one form of image, text and table.

[0087] It is worth noting that in order to ensure the security and privacy of data, different systems usually use different file formats, which makes it impossible to schedule data for unified viewing or management between different systems. Therefore, in order to meet the personalized needs of different users for data viewing and analysis, this application also provides a data format conversion solution through the user page.

[0088] For example, the data processing system also includes a user page, which is provided with a special data display format selection menu, input box or drop-down list and other interactive components. The user enters the corresponding data display format requirements according to the actual usage scenario and the habit of viewing data. For example, if you want to intuitively see the trend chart of the equipment operation parameters changing over time, you will choose to display it in image format; if you are more concerned about the specific values ​​and the corresponding relationship between the parameters, you will tend to prefer a table format; and for brief descriptions of equipment status, etc., a text format is used.

[0089] Among them, text formats include txt, rtf, doc, docx; image formats include bmp, png, jpg, jpeg, svg, eps; table formats include xlsx, ods (OpenDocument Spreadsheet), csv.

[0090] The data display format input by the user can be a relatively clear and specific format type, such as directly selecting "Image (line chart)" to indicate that the data is desired to be displayed in the form of an image such as a line chart, or selecting "Table (rows and columns with headers and specific data)" to specify the presentation method of the table; it can also be a relatively broad description, such as entering "visual chart", and the system will further determine whether to use a bar chart, pie chart or other appropriate visual chart type to display the data based on the data characteristics and default rules.

[0091] By displaying the target data in the data display format input by the user, it can meet the diverse needs of different users for data viewing and analysis in different business scenarios, and improve the convenience and effectiveness of data use.

[0092] See also Figure 2 As shown, the present disclosure further provides a data processing device 500 , which includes an acquisition module 501 , a classification module 502 , and a screening module 503 .

[0093] The acquisition module 501 is configured to acquire initial data of the device according to a preset acquisition period.

[0094] The classification module 502 is configured to add classification labels to the initial data according to preset classification requirements, determine source data, and save the source data to a database.

[0095] The screening module 503 is configured to screen the source data in the database according to the target classification label input by the user, and determine the target data corresponding to the target classification label.

[0096] Optionally, there are multiple preset classification requirements, and the classification module 502 is configured to: Input a plurality of the preset classification requirements into the data screening model to obtain the source data output by the data screening model, wherein the source data is obtained by the data screening model by screening the initial data in turn through each of the preset classification requirements and adding the classification label.

[0097] Optionally, the classification module 502 is configured to: Selecting a plurality of collected initial data and performing normalization processing to determine a plurality of sample initial data; Manually adding corresponding classification labels to each of the sample initial data to determine a plurality of verification source data; Input the plurality of sample initial data into the data screening model, sequentially pass through the plurality of nodes of the data screening model, add at least one classification label to each of the sample initial data, and output a plurality of sample source data, each of the nodes corresponding to one of the preset classification requirements; The parameters of the data screening model are updated according to the outputted multiple sample source data and the verification source data, and the training of the data screening model is completed until the preset conditions are met.

[0098] Optionally, the classification module 502 is configured to: The source data is determined by adding a classification label to the initial data according to at least one of data time, data type, data source device, data source system, keyword information, and identifier information.

[0099] Optionally, the acquisition module 501 is configured to: Taking screenshots of instruments of the device according to the preset collection period to determine a device data image; Recognize the text in the device data image and determine the image text data; The image text data is screened by using preset keywords and / or preset identifiers to determine the initial data.

[0100] Optionally, there are multiple device data images, and the acquisition module 501 is configured to: sequentially identifying the text in each of the device data images; In the event of an error in the recognition of any of the device data images, re-recognize the current device data image until the number of errors reaches a preset number, then mark the current device data image and perform the recognition of the next device data image; When the current device data image is successfully identified, the identification of the next device data image is performed.

[0101] Optionally, the screening module 503 is configured to: Get the data display format entered by the user; According to the data display format, the target data is displayed in at least one form of image, text and table.

[0102] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0103] The present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the data processing method provided by the present disclosure is implemented.

[0104] The present disclosure also provides an electronic device, comprising: a memory having a computer program stored thereon; A processor is used to execute the computer program in the memory to implement the data processing method provided by the present disclosure.

[0105] Figure 3 FIG. 7 is a block diagram of an electronic device 700 according to an exemplary embodiment. Figure 3 As shown, the electronic device 700 may include: a processor 701 , a memory 702 . The electronic device 700 may also include one or more of a multimedia component 703 , an input / output (I / O) interface 704 , and a communication component 705 .

[0106] The processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above-mentioned data processing method. The memory 702 is used to store various types of data to support the operation of the electronic device 700. For example, these data may include instructions for any application or method used to operate on the electronic device 700, and application-related data, such as contact data, messages sent and received, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 702 or sent through the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules, and the other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 705 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0107] In an exemplary embodiment, the electronic device 700 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned data processing method.

[0108] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by the processor, the steps of the above-mentioned data processing method are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 702 including program instructions, and the above-mentioned program instructions can be executed by the processor 701 of the electronic device 700 to complete the above-mentioned data processing method.

[0109] Figure 4 1 is a block diagram of an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. Figure 4 The electronic device 1900 includes a processor 1922, which may be one or more, and a memory 1932 for storing a computer program executable by the processor 1922. The computer program stored in the memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the processor 1922 may be configured to execute the computer program to perform the above-mentioned data processing method.

[0110] In addition, the electronic device 1900 may further include a power supply component 1926 and a communication component 1950, wherein the power supply component 1926 may be configured to perform power management of the electronic device 1900, and the communication component 1950 may be configured to implement communication of the electronic device 1900, for example, wired or wireless communication. In addition, the electronic device 1900 may further include an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932.

[0111] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by the processor, the steps of the above-mentioned data processing method are implemented. For example, the non-transitory computer-readable storage medium can be the above-mentioned memory 1932 including program instructions, and the above-mentioned program instructions can be executed by the processor 1922 of the electronic device 1900 to complete the above-mentioned data processing method.

[0112] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program executable by a programmable device, and the computer program has a code portion for executing the above data processing method when executed by the programmable device.

[0113] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0114] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0115] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A data processing method, characterized in that: The method comprises: Acquire the initial data of the device according to the preset collection cycle; Adding classification labels to the initial data according to preset classification requirements, determining source data, and saving the source data to a database; According to the target classification label input by the user, the source data in the database is screened to determine the target data corresponding to the target classification label.

2. The method according to claim 1, characterized in that There are multiple preset classification requirements, and adding classification labels to the initial data according to the preset classification requirements to determine the source data includes: Input a plurality of the preset classification requirements into the data screening model to obtain the source data output by the data screening model, wherein the source data is obtained by the data screening model by screening the initial data in turn through each of the preset classification requirements and adding the classification label.

3. The method according to claim 2, characterized in that The data screening model is trained in the following way: Selecting a plurality of collected initial data and performing normalization processing to determine a plurality of sample initial data; Manually adding corresponding classification labels to each of the sample initial data to determine a plurality of verification source data; Input the plurality of sample initial data into the data screening model, sequentially pass through the plurality of nodes of the data screening model, add at least one classification label to each of the sample initial data, and output a plurality of sample source data, each of the nodes corresponding to one of the preset classification requirements; The parameters of the data screening model are updated according to the outputted multiple sample source data and the verification source data, and the training of the data screening model is completed until the preset conditions are met.

4. The method according to claim 1, characterized in that: The adding classification labels to the initial data according to the preset classification requirements to determine the source data includes: The source data is determined by adding a classification label to the initial data according to at least one of data time, data type, data source device, data source system, keyword information, and identifier information.

5. The method according to claim 1, characterized in that The obtaining of initial data of the device according to a preset collection period includes: Taking screenshots of instruments of the device according to the preset collection period to determine a device data image; Recognize the text in the device data image and determine the image text data; The image text data is screened by using preset keywords and / or preset identifiers to determine the initial data.

6. The method according to claim 5, characterized in that There are multiple device data images, and the step of identifying text in the device data image and determining image text data includes: sequentially identifying the text in each of the device data images; In the event of an error in the recognition of any of the device data images, re-recognize the current device data image until the number of errors reaches a preset number, then mark the current device data image and perform recognition of the next device data image; When the current device data image is successfully identified, the identification of the next device data image is performed.

7. The method according to claim 1, characterized in that The method further comprises: Get the data display format entered by the user; According to the data display format, the target data is displayed in at least one form of image, text and table.

8. A data processing device, characterized in that: The device comprises: An acquisition module is configured to acquire initial data of the device according to a preset acquisition period; A classification module, configured to add classification labels to the initial data according to preset classification requirements, determine source data, and save the source data to a database; The screening module is configured to screen the source data in the database according to the target classification label input by the user, and determine the target data corresponding to the target classification label.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 7.

Images