Image processing method, system, electronic device and storage medium

Abstract

The present application relates to an image processing method, system, electronic device and storage medium. By obtaining uploaded operation and maintenance pictures and performing image preprocessing on the operation and maintenance pictures, a preset trained data recognition model is used to extract the key content contained in the operation and maintenance pictures and perform data preprocessing on the key content, and a preset logical processing algorithm is used to logically compare the key content with the preset display material data to generate a comparison result, and abnormal data and matching failure information are preset processed, the comparison result is preset marked and the marked comparison result is fed back in a preset display mode. The present application can realize automated data extraction and accurate comparison, thereby improving operation and maintenance efficiency, reducing labor costs and reducing the impact of human factors, and ensuring data accuracy.

Description

Technical Field

[0001] The present application relates to the field of operation and maintenance image processing technology, and more specifically to an image processing method, an image processing system, an electronic device and a computer storage medium. Background Art

[0002] Operation and maintenance refers to the process of managing, monitoring, optimizing, and maintaining control equipment, systems, and their related resources. The core goal of operation and maintenance is to ensure high availability, security, and performance stability of the system to support the continuous operation of the business.

[0003] In the manufacturing process, production equipment usually runs continuously for a long time to meet the needs of efficient and stable production. However, the operating status of the equipment is often dynamic and is affected by many factors, including but not limited to equipment aging, load changes, ambient temperature and humidity, and operator habits. Fluctuations in these factors may cause equipment failure or performance degradation, thereby affecting production efficiency and product quality. Therefore, accurately obtaining and analyzing equipment operating parameters is an important basis for ensuring production stability and product quality.

[0004] Currently, equipment operation and maintenance typically relies on operators individually checking equipment operating data, various parameters, and related information, recording and comparing the data, and referencing equipment operation logs for analysis and judgment. However, due to the large amount of data, the wide variety of parameters, and the complex production environment, this comparison method is inefficient and susceptible to human factors, resulting in inaccurate data recording, misjudgments, and delayed comparisons. Summary of the Invention

[0005] The present application provides an image processing method, system, electronic device and storage medium, which can realize automated data extraction and accurate comparison, thereby improving operation and maintenance efficiency, reducing labor costs and ensuring data accuracy.

[0006] In a first aspect, the present application provides an image processing method, which includes: obtaining uploaded operation and maintenance pictures and performing image preprocessing on the operation and maintenance pictures; using a preset trained data recognition model to extract key content contained in the operation and maintenance pictures and performing data preprocessing on the key content; using a preset logical processing algorithm to logically compare the key content with preset display material data to generate a comparison result, and performing preset processing on abnormal data and matching failure information; pre-marking the comparison result and feeding back the marked comparison result in a preset display method.

[0007] In an optional scheme of the first aspect, when performing image preprocessing, the method includes: identifying the format of the uploaded operation and maintenance image and standardizing the format of the operation and maintenance image, and removing blank and damaged images to generate a standard operation and maintenance image; grayscale, binarization, denoising, size normalization and contrast enhancement processing are performed on the standard operation and maintenance image in sequence to generate a processed operation and maintenance image.

[0008] In an optional scheme of the first aspect, the preset training method of the data recognition model includes: generating a training data set based on historical annotations and corrected operation and maintenance pictures and dividing the training data set into a training set and a verification set of preset proportions; loading a pre-trained model of the PaddleOCR framework and performing model configuration, and then importing the training set and verification set for the pre-trained model; using the PP-LCNet convolutional neural network as the backbone network for feature extraction and using connection time series classification and NRTR methods to perform text recognition on the extracted features; calculating the loss of connection time series classification and NRTR methods; using the Adam optimizer for gradient update and using the Cosine learning rate adjustment strategy; calculating the accuracy; looping the training process until the preset number of iterations is reached or the model meets the preset requirements, and then generating a data recognition model.

[0009] In an optional scheme of the first aspect, when performing data preprocessing on the key content, the method includes: deleting redundant spaces in the key content through regular expressions or string replacement; deleting special characters in the key content; correcting noise characters in the key content through preset noise processing; standardizing the date format, number format and character encoding of the key content; correcting words in the key content through a spelling proofreading algorithm or a spell checking tool and using a preset dictionary or language model to replace unrecognizable words when unrecognizable words are detected; and converting all numerical values ​​contained in the text of the key content into a unified format.

[0010] In an optional scheme of the first aspect, after performing a logical comparison, the method includes: comparing the device image with the preset display material data to determine whether they match, if not, returning device unmatched information, if so, comparing the device parameters with the preset display material data to determine whether they comply with the configuration; if not, returning device configuration error information, if so, using a correction logic processing method to process all texts in the key content; determining whether all texts in the key content contain correction failures, if so, setting the correction status of the device to failure and marking the device in the failed correction status, if not, comparing the preset text of the key content with the preset display material data, and then outputting the comparison result and shielding or deleting the abnormal data and matching failure information.

[0011] In an optional scheme of the first aspect, the correction logic method includes: determining whether the key content contains preset keywords, and if so, extracting the text containing the preset keywords and all subsequent data related to the text, and converting the extracted data into a standard format; if not, recording the error and generating an unrecognized error prompt message, and then deleting the text that does not contain the preset keywords through a traversal method or a list parsing method; integrating and feeding back all processed texts, error records and error prompt messages in the key content.

[0012] In an optional scheme of the first aspect, when comparing the preset text of the key content with the preset display material data, the method includes: comparing the text of the range, calibration cycle, working mode and communication parameters contained in the key content with the preset display material data using a string matching method and outputting text that meets the standard range, calibration cycle, working mode and communication parameters; judging whether the text in the key content has a delay parameter, if so, extracting the delay parameter and filtering out the text containing the delay parameter; if not, comparing the text of the batch information contained in the key content with the preset display material data using a string matching method and automatically cleaning the text that does not meet the standard batch in the key content; comparing the alarm value contained in the key content with the preset display material data using a string matching method and outputting text that meets the standard alarm value; comparing the time, temperature and fan parameters contained in the key content with the preset display material data and outputting text that meets the standard time, temperature and fan parameters; comparing the curve coefficient and curve offset contained in the key content with the preset display material data and outputting text that meets the standard curve coefficient and curve offset.

[0013] In an optional scheme of the first aspect, when the comparison result is preset-marked and the marked comparison result is fed back in a preset display mode, the method includes: automatically identifying abnormal or unmatched information in the comparison result and processing the comparison result, abnormal or unmatched information in the form of a chart or list; feeding back parameters that meet the standards, parameters with deviations, abnormal or unmatched information in the form of a chart or list, and archiving and storing the uploaded operation and maintenance pictures, identified key content, comparison results and identification information.

[0014] In an optional scheme of the first aspect, when performing image preprocessing, the method further includes: calculating the quality score of the uploaded operation and maintenance images through the structural similarity index and the peak signal-to-noise ratio, and arranging the operation and maintenance images in a preset order according to the above quality scores; and denoising the operation and maintenance images using an adapted image denoising method according to the quality arrangement order, wherein the first denoising is performed on the operation and maintenance images whose quality scores are lower than a preset threshold or whose quality arrangement order is lower than a preset position, and the second denoising is performed on the operation and maintenance images whose quality scores are higher than the preset threshold or whose quality arrangement order is higher than the preset position.

[0015] In the second aspect, the present application also provides an image processing system using the above method, including: a data upload module, used to obtain the uploaded operation and maintenance data and perform a first data preprocessing on the operation and maintenance data; a data identification module, used to use a preset trained data identification model to extract the key content contained in the operation and maintenance data and perform a second data preprocessing on the key content; a data comparison module, used to perform a logical comparison between the key content and the preset display material data through a preset logical processing algorithm to generate a comparison result, and perform preset processing on abnormal data and matching failure information; a result display module, used to preset the comparison result and feedback the marked comparison result in a preset display method.

[0016] In a third aspect, the present application also provides an electronic device comprising: at least one processor; and at least one memory, wherein the at least one memory is coupled to the at least one processor and is used to store instructions to be executed by the at least one processor, wherein the instructions, when executed by the at least one processor, enable the electronic device to perform the image processing method.

[0017] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image processing method is implemented.

[0018] It should be understood that the foregoing general description and the following detailed description are merely illustrative and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate one or more embodiments of the present application and, together with the description, serve to explain the principles of the present application and to enable one of ordinary skill in the relevant art to make and use the present application.

[0020] Figure 1 is a flowchart of an exemplary image processing method according to some embodiments of the present application.

[0021] Figure 2 This is a flowchart of an exemplary model training method according to some embodiments of the present application.

[0022] Figure 3 This is a schematic diagram of an exemplary labeled image according to some embodiments of the present application.

[0023] Figure 4 This is a schematic diagram of an exemplary automatic image annotation according to some embodiments of the present application.

[0024] Figure 5 This is a schematic diagram of an exemplary manual correction of image annotation according to some embodiments of the present application.

[0025] Figure 6 This is a flowchart of an exemplary data preprocessing method according to some embodiments of the present application.

[0026] Figure 7 This is a flowchart of an exemplary comparison method using a preset logic processing algorithm according to some embodiments of the present application.

[0027] Figure 8 is a flowchart of an exemplary correction logic method according to some embodiments of the present application.

[0028] Figure 9 This is a module connection diagram of an exemplary image processing system according to some embodiments of the present application.

[0029] Figure 10 is a schematic structural diagram of an exemplary electronic device according to some embodiments of the present application. DETAILED DESCRIPTION

[0030] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to provide a thorough understanding of the embodiments of this application.

[0031] The image processing method and system of the present application are automated operation and maintenance methods and systems based on image recognition technology and text processing technology, aiming to achieve automated data extraction and accurate comparison, thereby improving operation and maintenance efficiency, reducing labor costs, and ensuring data accuracy.

[0032] Therefore, the image processing system of the present application is provided with at least the functions of image uploading and receiving, text recognition and extraction, data logic processing and comparison, result feedback and display, and data storage and management. Among them, the image processing system adopts Python as the main development language, and utilizes Python's advantages in data processing, image processing and machine learning to achieve efficient collaboration and data interaction among various modules within the system.

[0033] Therefore, users registered in the image processing system can upload a variety of operation and maintenance pictures including equipment parameters, wall materials, etc., and the system supports batch uploading and multiple picture formats (such as JPEG, PNG, GIF, BMP, TIFF, SVG, WebP, HEIF, PDF, RAW and other existing picture formats). The front end can use HTML5 File API is used for file selection, and the back-end can use frameworks such as Flask / Django to receive files to ensure user convenience; at the same time, special training and optimization are carried out for operation and maintenance images that match the user's industry and system application environment to improve the recognition accuracy of the OCR model in specific scenarios, and then the operation and maintenance images uploaded by the user are processed, and the text content in the uploaded images is automatically identified and extracted, covering key information such as equipment parameters, equipment numbers, production data, etc.; and through the set custom logic processing algorithm, the OCR recognized and processed data is compared with the preset wall display material data to determine whether each parameter meets the requirements and automatically identify abnormal or mismatched information to ensure the accuracy and real-time nature of the comparison results; and then the comparison results are intuitively displayed to the user in the form of charts or lists, clearly identifying which parameters meet the standards and which have deviations, so as to facilitate operation and maintenance personnel to make timely adjustments and decisions; and the image processing system of this application will archive and manage uploaded images, recognition data and comparison results, support subsequent queries, statistics and data analysis, and help user companies achieve long-term management and continuous optimization of operation and maintenance data.

[0034] Therefore, reference Figure 1 As shown, Figure 1 A flowchart illustrating an exemplary image processing method according to some embodiments of the present application is provided. The present application relates to an image processing method, and the method includes:

[0035] S1. Obtain uploaded operation and maintenance pictures and perform picture preprocessing on the operation and maintenance pictures.

[0036] Specifically, in this application, the operation and maintenance pictures uploaded by users will be archived and stored in a database specifically set up by the operation and maintenance personnel for subsequent processing and analysis.

[0037] In some examples of the present application, in S1, the stored operation and maintenance pictures are obtained and the operation and maintenance pictures are subjected to format standardization, abnormal picture removal, grayscale conversion, binarization, denoising, size normalization, and contrast enhancement. The specific methods include:

[0038] Identify the format of each uploaded operation and maintenance picture and convert the operation and maintenance pictures of different formats into a unified system standard format, wherein the conversion of the picture format can be achieved through image pixel matrix operations. The system standard format is set by the operation and maintenance personnel. For example, this application refers to the setting of the jpg format; at the same time, the identified blank pictures and damaged pictures are deleted, and the operation and maintenance pictures that exceed the recognition limit are compressed to produce standard operation and maintenance pictures for subsequent image processing. In one optional example of the present application, if it is identified that the picture is damaged or the clarity is low and cannot be recognized, the Stable Diffusion image generation algorithm based on the diffusion model or the ControlNet algorithm used to control the large diffusion model generation process is used to automatically complete the damaged and missing areas in the operation and maintenance picture to improve the completeness of the information; logical missing information can be generated based on historical recognition results and the equipment knowledge map set by the operation and maintenance personnel, rather than random completion.

[0039] After the operation and maintenance pictures are standardized, they are grayscaled to convert the color images into grayscale images. That is, each pixel of the operation and maintenance pictures is set to: 0.299R+0.587G+0.114B.

[0040] Furthermore, the grayscale operation and maintenance picture is binarized to convert the grayscale image into a black and white image, that is, the pixel value greater than the set threshold is set to 255, and the pixel value less than the set threshold is set to 0, so that the subsequent OCR (optical character recognition) can more easily recognize the text; wherein the set threshold shown can be set by the operation and maintenance personnel.

[0041] Furthermore, the binarized operation and maintenance pictures are subjected to denoising using a filtering algorithm. The filtering algorithms include but are not limited to Gaussian filtering algorithm, wavelet filtering algorithm, sliding average filtering algorithm, median filtering algorithm, bilateral filtering algorithm, high / low pass filtering algorithm, etc., which are specifically set by the operation and maintenance personnel. In this application, the Gaussian filtering algorithm is used as an example to reduce high-frequency noise: Where G(x,y) is the filtered pixel value, σ controls the blur level and is usually set to 1 to 3; x and y are pixel coordinates. The image is then morphologically processed, including erosion and dilation. Erosion removes small noise points to make the font in the image clearer, while dilation repairs damaged text and enhances edges.

[0042] Furthermore, the size of the operation and maintenance pictures that have undergone denoising and morphological processing is adjusted to a standard size, that is, the operation and maintenance pictures that exceed the standard size are reduced, and the operation and maintenance pictures that are below the standard size are enlarged, so that all operation and maintenance pictures are adjusted to be consistent with the standard size; wherein, the size normalization shown can be achieved by proportional scaling, padding scaling, etc.

[0043] S2. Use a preset trained data recognition model to extract key content contained in the operation and maintenance image and perform data preprocessing on the key content.

[0044] Specifically, refer to Figure 2 As shown, Figure 2 A flowchart illustrating an exemplary model training method according to some embodiments of the present application is provided. The preset training process of the data recognition model includes:

[0045] S20, generating a training data set based on the historical annotations and the corrected operation and maintenance images, and dividing the training data set into a training set and a validation set of preset proportions;

[0046] Among them, the training data set is divided into training set and validation set in a ratio of 8:2 or 9:1.

[0047] S21. Load the pre-trained model of the PaddleOCR framework and configure the model, and then import the training set and validation set for the pre-trained model.

[0048] Among them, the PaddleOCR framework of this application is an open source optical character recognition (OCR) tool library developed based on the PaddlePaddle deep learning framework, which aims to provide efficient and accurate text recognition technology. PaddleOCR supports multiple text recognition tasks, including text detection, text recognition and text direction detection, and is suitable for different types of text recognition needs such as scene text, documents, tables, books, etc.

[0049] S22. The PP-LCNet convolutional neural network is used as the backbone network for feature extraction and the connection temporal classification and NRTR methods are used to perform text recognition on the extracted features.

[0050] Among them, the PPLCNetV3 convolutional neural network is used as the backbone network (Backbone) to extract features, and its scaling ratio is 0.95; the extracted features are then passed to the connection temporal classification (CTC) and NRTR (No-RecurrenceSequence-to-Sequence Model For Scene Text Recognition, non-recursive sequence to sequence scene text recognition model) for text recognition; among them, NRTR uses the complete Transformer structure to encode and decode the input image.

[0051] S23. Calculate the loss of the connection time series classification and NRTR methods.

[0052] Among them, the CTC loss is calculated as:

[0053]

[0054] , where X is the input sequence, Y is the target text, A is the possible alignment path, and B -1 (Y) is the set of all possible pathways for transcribing Y;

[0055] NRTR loss is calculated as:

[0056]

[0057] , where N is the target sequence length in the NRTR loss calculation, that is, the number of characters to be recognized in the OCR task; i is the position of the currently processed character in the target sequence, y i is the target character, i.e. the i-th character in the real label;

[0058] P(y i |X) is the probability output by the Transformer decoder.

[0059] S24. Use Adam optimizer for gradient update and adopt Cosine learning rate adjustment strategy.

[0060] Among them, the parameter update formula of the Adam optimizer is:

[0061] m t =β1m t-1 +(1-β1)g t

[0062]

[0063] Among them, m t is the exponentially weighted average of the gradient, v t is the exponentially weighted average of the squared gradient, g tis the gradient of the loss calculated for the current batch with respect to the parameters, α is the learning rate, β1 and β2 are the momentum decay coefficients, ∈ is a smoothing term to prevent zero division errors, and θ t is the model parameter value after the current training step; parameter update mainly depends on m t and v t , where m t Control direction, v t Control step size.

[0064] In this way, the momentum method and RMSProp are combined to adaptively adjust the learning rate using the first-order and second-order moment estimates of the gradient.

[0065] During the optimization process, the Adam optimizer can not only speed up the convergence speed, but also reduce gradient oscillation and improve model stability.

[0066] Among them, the formula for using the Cosine learning rate adjustment strategy is:

[0067]

[0068] , where η t is the current learning rate used to dynamically adjust the step size during training, η max is the maximum learning rate, η min is the minimum learning rate, T cur is the current training progress, and T is the total number of training rounds. The cosine function is used to control the periodic decrease of the learning rate to improve the convergence stability of the model. The use of the Cosine learning rate can prevent the learning rate from being too large, resulting in jumping out of the optimal solution, or from being too small, resulting in training stagnation.

[0069] S25. Calculate the accuracy.

[0070] The calculation formula for accuracy is:

[0071]

[0072] Where I is the indicator function, is the model prediction, yi is the true label, and N is the number of samples for accuracy calculation.

[0073] S26, loop the training process of S22-S25 until the preset number of iterations is reached or the model meets the preset requirements, and then generate a data recognition model.

[0074] Among them, the preset number of loss iterations and preset requirements are set by the operation and maintenance personnel according to actual needs; the trained model is set as the data recognition model.

[0075] Among them, reference Figures 3 to 5 As shown, Figure 3A schematic diagram showing an exemplary labeled image in some embodiments of the present application is shown. Figure 4 A schematic diagram showing an exemplary automatic image annotation in some embodiments of the present application is shown. Figure 5 A schematic diagram illustrating an exemplary manual correction of image annotation in some embodiments of the present application is shown. Before performing preset training, operation and maintenance personnel need to annotate and correct historical operation and maintenance images. The specific method includes:

[0076] Open the annotation program of the pre-trained model of the PaddleOCR framework and use the automatic recognition function of the PaddleOCR framework to identify historical operation and maintenance pictures. Then the operation and maintenance personnel manually correct the annotation results based on the automatic recognition results.

[0077] In some examples of this application, reference Figure 6 As shown, the figure shows a flowchart of an exemplary data preprocessing method of some embodiments of the present application; after the data recognition model extracts the key content contained in the operation and maintenance image, the present application performs data preprocessing on the key content, and the specific processing method includes:

[0078] S200. Delete redundant spaces in the key content through regular expressions or string replacement.

[0079] When OCR extracts text, extra spaces may appear, especially at the beginning or end of a line or between different words. The extra spaces are deleted in step S200; for example, a regular expression is used to match multiple space characters and replace them with a single space.

[0080] S201. Delete special characters of the key content. For example, special characters include meaningless characters such as “@”, “#”, and “$”.

[0081] S202, correcting noise characters in the key content by using a preset noise process; illustratively, noise characters such as "0" and "o", "1" and "l", etc., are corrected by using a character mapping or dictionary;

[0082] S203: Standardize the date format, number format, and character encoding of the key content.

[0083] Among them, if the date format recognized by OCR is not uniform, such as "2020-02-20" and "20 / 02 / 2022", it needs to be uniformly converted to a standard format; if the number format recognized by OCR is not uniform, such as "3,000" is changed to "3000" and "3.5K" is changed to "3500", it needs to be uniformly converted to a standard format; if different character encodings appear in the OCR process, they need to be uniformly converted to a standard encoding.

[0084] S204: Correct the words of the key content using a spelling proofreading algorithm or a spelling check tool and replace unrecognizable words using a preset dictionary or language model when unrecognizable words are detected.

[0085] Among them, spelling proofreading algorithms include but are not limited to dictionary matching, edit distance, word or letter-level spelling correction algorithms, etc. Spell checking tools include but are not limited to PySpellChecker, PyEnchant, TextBlob, etc.; preset dictionaries are industry-related dictionaries (such as device parameters, common units, device models, etc.) built by operation and maintenance personnel, and preset language models include but are not limited to n-gram and Chinese Spelling Correct models.

[0086] S205: Convert all numerical values ​​contained in the text of the key content into a unified format.

[0087] Among them, numbers, dates, currencies, etc. in the text may be presented in different formats, and they need to be regularized and normalized to ensure the uniformity of the data.

[0088] S3. Perform a logical comparison between the key content and the preset display material data through a preset logical processing algorithm to generate a comparison result, and perform preset processing on abnormal data and matching failure information.

[0089] Specifically, refer to Figure 7 As shown, Figure 7 A flowchart of an exemplary method for performing comparison using a preset logic processing algorithm in some embodiments of the present application is shown; in S3, when performing logic comparison using the preset logic processing algorithm, the method includes:

[0090] S30: Compare the device image with the preset display data to determine whether they match.

[0091] If the device image does not match the corresponding device parameters in the preset display data, execute S300: return the device unmatched information; if the device image matches the corresponding device parameters in the preset display data, execute S31: compare the device parameters with the preset display data to determine whether they meet the configuration requirements, so as to ensure that each device meets the standards and avoid incorrect configuration.

[0092] If the device parameters do not meet the configuration standards, then execute S310: return device configuration error information; if the device parameters meet the standards, then execute S32: use a correction logic processing method to process all texts in the key content.

[0093] In some examples of this application, reference Figure 8 As shown, Figure 8A flowchart of an exemplary correction logic processing method according to some embodiments of the present application is shown; the correction logic processing method includes:

[0094] S320: Determine whether the key content contains preset keywords; wherein the preset keywords are set by operation and maintenance personnel according to actual application fields and user needs.

[0095] If the key content contains preset keywords, execute S3200: extract the text containing the preset keywords and all subsequent data related to the text, and convert the extracted data into a standard format. After the extraction is completed, the text matching the preset keywords will be shielded or deleted in the key content to avoid subsequent repeated extraction of data.

[0096] For example, all data in the key content are identified in sequence. For example, whenever a set keyword such as "water sample", "distilled water", "reagent" is identified in a certain text of the key content, the index position corresponding to "water sample", "distilled water", "reagent" is found, and then all the values ​​after it are extracted, and the extracted values ​​are standardized. For example, "once a week", "once every 6 days", "once a month", etc. are converted to "7 days", "6 days", etc.

[0097] "30 days" and other corresponding formats, refer to the standardized processing form in the above data preprocessing for details; then, the corresponding keywords such as "water sample", "distilled water", "reagent" and so on are steadily blocked or deleted in the key content to avoid subsequent repeated data extraction; then all processed text data are integrated into a list to ensure that each text has been processed and has complete feedback information

[0098] If the key content does not contain the preset keywords, execute S3201: record the error and generate an unrecognized error prompt message, and then delete the text that does not contain the preset keywords through a traversal method or a list parsing method.

[0099] S321: Integrate and feed back all processed texts, error records, and error prompt information in the key content.

[0100] S33: Determine whether all texts in the key content contain correction failures. If so, execute S330: set the correction status of the device to failure and mark the device in the failed correction status. If not, execute S34: compare the preset text of the key content with the preset display material data, and then output the comparison results and block or delete the abnormal data and matching failure information.

[0101] Among them, it is checked whether each line of text in the key content contains calibration failure. If it does, the calibration status of the device is set to failure and the devices that failed the calibration are marked to facilitate subsequent operations.

[0102] Among them, in S34, according to different application fields, different key content texts are set to be compared with preset display data. For example, this application takes a certain environmental monitoring field as an example, and specifically refers to:

[0103] S340. Compare the text of the range, calibration cycle, working mode and communication parameters contained in the key content with the preset display data using a string matching method and output the text of the range, calibration cycle, working mode and communication parameters that meets the standards, so that the text of the range, calibration cycle, working mode and communication parameters can be displayed in a matching position; and after outputting the text of the range, calibration cycle, working mode and communication parameters that meet the standards, delete or shield the text of the range, calibration cycle, working mode and communication parameters contained in the current key content to avoid repeated input or erroneous data and improve system stability.

[0104] S341. Determine whether the text in the key content has a delay parameter. If so, extract the delay parameter and filter out the text containing the delay parameter. If not, compare the text containing batch information in the current key content with the preset display material data using a string matching method and automatically clean the text in the key content that does not meet the standard batch.

[0105] Among them, different devices may store batch numbers before and after the batch keyword, so it is necessary to check the complete string and the partial string before the batch at the same time, and then display the batch text that meets the standards in the matching position; and after outputting the batch text that meets the standards, the corresponding batch text contained in the current key content will be deleted or blocked to avoid repeated input or erroneous data, thereby improving system stability.

[0106] S342. Compare the alarm value contained in the key content with the preset display data using a string matching method and output the text that meets the standard alarm value, so that the alarm value text that meets the standard can be displayed in a matching position; and after outputting the alarm value text that meets the standard, delete or block the alarm value text contained in the current key content to avoid repeated input or erroneous data, thereby improving system stability.

[0107] S343. Compare the time, temperature and fan parameters contained in the key content with the preset display data and output the text that meets the standard time, temperature and fan parameters, so that the time, temperature and fan parameter text that meets the standard can be displayed in a matching position; and after outputting the time, temperature and fan parameter text that meets the standard, delete or block the time, temperature and fan parameter text contained in the current key content to avoid repeated input or erroneous data and improve system stability.

[0108] S344. Compare the curve coefficient and curve offset contained in the key content with the preset display material data and output the text that meets the standard curve coefficient and curve offset, so that the curve coefficient and curve offset text that meets the standard can be displayed in a matching position; and after outputting the curve coefficient and curve offset that meet the standard, delete or shield the curve coefficient and curve offset text contained in the current key content to avoid repeated input or erroneous data and improve system stability.

[0109] S4. Preset marking is performed on the comparison result and the marked comparison result is fed back in a preset display mode.

[0110] Specifically, in S4, when the comparison result is marked with a preset label and the marked comparison result is fed back in a preset display mode, the method includes:

[0111] Automatically identify abnormal or unmatched information in the comparison results and process the comparison results, abnormal or unmatched information in a graphical or tabular form.

[0112] Parameters that meet the standards, parameters with deviations, abnormal or mismatched information will be fed back in the form of charts or lists. At the same time, the uploaded operation and maintenance pictures, identified key content, comparison results and identification information will be archived and stored.

[0113] In some embodiments of the present application, when performing image preprocessing, the method further includes:

[0114] S100: Calculate the quality scores of the uploaded operation and maintenance pictures by using the structural similarity index and the peak signal-to-noise ratio, and arrange the operation and maintenance pictures by quality in a preset order according to the quality scores.

[0115] The structural similarity index is used to measure the structural similarity between two images. In particular, in the denoising process, SSIM can reflect the denoising effect. The specific calculation formula of the structural similarity index is:

[0116]

[0117] In this embodiment: x and μy are the average brightness of image x and image y respectively, and are the variances of image x and image y, σ xy is the covariance between images x and y, and C1 and C2 are constants used to stabilize the calculation. For example, assume there are two images: the uploaded original image x and the noise-processed image y. By calculating the SSIM (Structural Similarity Index) value, we can determine whether the image is affected by noise. SSIM values ​​closer to 1 indicate better image quality, while smaller values ​​indicate more noise.

[0118] Among them, PSNR (peak signal-to-noise ratio) is an important indicator for measuring image quality. The specific calculation formula of PSNR is:

[0119]

[0120] Among them, MAX I is the maximum pixel value in the image. For example, for an 8-bit image, the maximum value is 255. MSE is the mean square error, which is used to measure the difference between images. The greater the difference, the larger the MSE and the smaller the PSNR.

[0121] S101: De-noising the operation and maintenance pictures using an adapted image denoising method according to the quality arrangement order, wherein a first denoising process is performed on the operation and maintenance pictures whose quality scores are lower than a preset threshold or whose quality arrangement order is lower than a preset position, and a second denoising process is performed on the operation and maintenance pictures whose quality scores are higher than a preset threshold or whose quality arrangement order is higher than a preset position.

[0122] The preset threshold and preset ranking are set by the operation and maintenance personnel based on actual needs. For example, for images with a quality score below the preset threshold or a quality ranking below the preset ranking, a stronger denoising method, such as Wavelet Transform or Median Filter, is used; while for images with a quality score above the preset threshold or a quality ranking above the preset ranking, a mild denoising method, such as Mean Filter, is selected to avoid over-processing.

[0123] By introducing metrics such as SSIM and PSNR to evaluate image quality, and combining them with noise detection mechanisms and adaptive data cleaning methods, the preprocessing process of operation and maintenance data can be optimized in different environments, ensuring data accuracy and consistency, thereby improving the overall performance of the automated operation and maintenance system.

[0124] In some embodiments of the present application, after performing data preprocessing, the method further includes:

[0125] After image preprocessing, Faster R-CNN convolutional neural network or YOLO (You Only Look Once) target detection algorithm is used for image recognition to identify the data information contained in the operation and maintenance images.

[0126] The identified data information is compared one by one with the key content extracted by the data recognition model. By comparing the numbers, formulas and text information of the two, it is determined whether they are consistent. If so, the correction is passed. If not, the correction fails and the corresponding content is marked for subsequent manual error correction to ensure that the final extracted data is consistent with the device configuration.

[0127] Among them, when performing the comparison, the Levenshtein distance text comparison algorithm can be used to calculate the edit distance of the two strings to determine whether the text matches. For example, assuming that the key content extracted by the data recognition model is "device model: AB123", and the data information recognized by the image is "AB123 device model", by calculating the Levenshtein distance, if the distance is 1, it means that the text is similar and the correction is judged to be passed; for example, assuming that the formula in the image is E=mc 2 , OCR extraction is E = m·c 2 , then perform symbol matching processing and determine whether it is a correct formula expression through formula recognition rules.

[0128] By comparing the information from image recognition with the content extracted by the data recognition model, we can effectively reduce correction failures caused by errors in the data recognition model, and combine the advantages of both to ensure accurate recognition and correction of complex images and text information.

[0129] Furthermore, in this application, when comparing device parameters, not only the basic parameters of the device are compared, but also multi-level information such as the device's operating history, maintenance records, usage environment, etc. can be compared. Through multi-level parameter comparison, the status of the device can be verified from multiple dimensions, making the comparison more comprehensive, thereby providing more accurate operation and maintenance feedback.

[0130] Furthermore, this application can also introduce a self-learning mechanism to continuously accumulate historical comparison data and optimize the logical comparison rules and parameters. This allows the system to adjust the comparison strategy based on historical data, enabling long-term stable operation. For example, during the device information comparison process, the comparison threshold can be automatically adjusted based on past comparison successes and failures, gradually improving matching accuracy.

[0131] Therefore, reference Figure 9 As shown, Figure 9A schematic diagram showing module connections of an exemplary image processing system in some embodiments of the present application is shown; in some embodiments of the present application, the present application also relates to an image processing system, including:

[0132] The data uploading module 100 is configured to obtain uploaded operation and maintenance data and perform a first data preprocessing on the operation and maintenance data.

[0133] The data identification module 101 is configured to extract key content contained in the operation and maintenance data using a preset trained data identification model and perform a second data preprocessing on the key content.

[0134] The data comparison module 102 is used to perform a logical comparison between the key content and the preset display material data through a preset logical processing algorithm to generate a comparison result, and perform a preset processing on abnormal data and matching failure information.

[0135] The result display module 103 is used to pre-mark the comparison result and feed back the marked comparison result in a pre-set display mode.

[0136] In some embodiments, reference Figure 10 As shown, Figure 10 The present invention is a schematic diagram of the structure of an electronic device used to implement embodiments of the present application. The electronic device includes a memory 201 and a processor 202. The memory 201 stores a computer program executable by the processor 202. When the processor 202 executes the computer program, the method described in the above embodiments is implemented. The number of the memory 201 and the processor 202 can be one or more.

[0137] The electronic device also includes:

[0138] The communication interface 203 is used to communicate with external devices and perform data exchange transmission.

[0139] If the memory 201, processor 202, and communication interface 203 are implemented independently, the memory 201, processor 202, and communication interface 203 can be connected to each other via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0140] Optionally, in a specific implementation, if the memory 201, the processor 202 and the communication interface 203 are integrated on a chip, the memory 201, the processor 202 and the communication interface 203 can communicate with each other through an internal interface.

[0141] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present application when the program is executed by the processor 202.

[0142] An embodiment of the present application also provides a chip, which includes a processor 202 for calling and executing instructions stored in the memory 201 from the memory 201, so that a communication device equipped with the chip executes the method provided in the embodiment of the present application.

[0143] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor 202 and a memory 201. The input interface, the output interface, the processor 202 and the memory 201 are connected through an internal connection path. The processor 202 is used to execute the code in the memory 201. When the code is executed, the processor 202 is used to execute the method provided in the embodiment of the application.

[0144] It should be understood that the processor 202 may be a central processing unit (CPU) 202, or may be another general-purpose processor 202, a digital signal processor (DSP) 202, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor 202 may be a microprocessor 202 or any conventional processor 202, etc. It is worth noting that the processor 202 may be a processor 202 that supports the Advanced RISC Machine (ARM) architecture.

[0145] Furthermore, the memory 201 may include a read-only memory 201 and a random access memory 201, and may also include a non-volatile random access memory 201. The memory 201 may be a volatile memory 201 or a non-volatile memory 201, or may include both volatile and non-volatile memory 201. The non-volatile memory 201 may include a read-only memory 201 (ROM), a programmable read-only memory 201 (PROM), an erasable programmable read-only memory 201 (EPROM), an electrically erasable programmable read-only memory 201 (EEPROM), or a flash memory. The volatile memory 201 may include a random access memory 201 (RAM), which serves as an external cache. By way of example and not limitation, many forms of RAM are available. For example, static random access memory 201 (Static RAM, SRAM), dynamic random access memory 201 (Dynamic Random Access Memory, DRAM), synchronous dynamic random access memory 201 (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory 201 (Double Data Rate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory 201 (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory 201 (Synchlink DRAM, SLDRAM) and direct memory bus random access memory 201 (DirectRambus RAM, DR RAM).

[0146] In the above embodiments, all or part of the embodiments may be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0147] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An image processing method, characterized in that: The method comprises: Obtain uploaded operation and maintenance pictures and perform image preprocessing on the operation and maintenance pictures; Using a pre-trained data recognition model to extract key content contained in the operation and maintenance image and perform data preprocessing on the key content; Performing a logical comparison between the key content and the preset display material data through a preset logic processing algorithm to generate a comparison result, and performing preset processing on abnormal data and matching failure information; Preset marking of the comparison results and feeding back the marked comparison results in a preset display mode; After performing the logical comparison, the method includes: Compare the device image with the preset display data to determine whether they match. If not, return a device unmatched message. If so, compare the device parameters with the preset display data to determine whether they meet the configuration requirements. If not, a device configuration error message is returned; if so, a correction logic processing method is used to process all texts in the key content; Determine whether all text in the key content contains calibration failure. If so, set the calibration status of the device to failed and mark the device in the failed calibration status. If not, compare the preset text of the key content with the preset display material data, and then output the comparison result and block or delete the abnormal data and matching failure information; When comparing the preset text of the key content with the preset display material data, the method includes: Compare the text of the key content including the range, calibration cycle, working mode and communication parameters with the preset display data using the string matching method and output the text that meets the standard range, calibration cycle, working mode and communication parameters; Determine whether the text in the key content has a delay parameter. If so, extract the delay parameter and filter out the text containing the delay parameter. If not, compare the text containing batch information in the key content with the preset display material data using a string matching method and automatically clean the text in the key content that does not meet the standard batch; Compare the alarm value contained in the key content with the preset display data using a string matching method and output the text that meets the standard alarm value; Compare the time, temperature and fan parameters included in the key content with the preset display data and output text that meets the standard time, temperature and fan parameters; Compare the curve coefficient and curve offset contained in the key content with the preset display material data and output text that meets the standard curve coefficient and curve offset.

2. The image processing method according to claim 1, wherein: When performing image preprocessing, the method includes: Identify the format of the uploaded operation and maintenance pictures, standardize the format of the operation and maintenance pictures, remove blank and damaged pictures, and generate standard operation and maintenance pictures; The standard operation and maintenance images are sequentially processed with grayscale, binarization, denoising, size normalization, and contrast enhancement to generate processed operation and maintenance images.

3. The image processing method according to claim 1 or 2, characterized in that: The preset training method of the data recognition model includes: Generate a training data set based on the historical annotations and the corrected operation and maintenance images, and split the training data set into a training set and a validation set with a preset ratio; Load the pre-trained model of the PaddleOCR framework and configure the model, and then import the training set and validation set for the pre-trained model; The PP-LCNet convolutional neural network is used as the backbone network for feature extraction and the connection temporal classification and NRTR methods are used to perform text recognition on the extracted features. Calculate the loss of connecting temporal classification and NRTR methods; Use Adam optimizer for gradient update and adopt Cosine learning rate adjustment strategy; Calculate accuracy; The training process is repeated until the preset number of iterations is reached or the model meets the preset requirements, and then a data recognition model is generated.

4. The image processing method according to claim 3, wherein: When performing data preprocessing on the key content, the method includes: Delete extra spaces in the key content through regular expressions or string replacement; Delete the special characters of the key content; Correcting the noise characters of the key content by using a preset noise processing; Standardizing the date format, number format, and character encoding of the key content; Correcting the words in the key content through a spelling proofreading algorithm or a spell checking tool and replacing unrecognizable words using a preset dictionary or language model when unrecognizable words are detected; Convert all numerical values ​​contained in the text of the key content into a unified format.

5. The image processing method according to claim 1, wherein: The correction logic method includes: Determine whether the key content contains preset keywords, If yes, extract the text containing the preset keyword and all subsequent data related to the text, and convert the extracted data into a standard format; If not, an error is recorded and an unrecognized error message is generated, and the text that does not contain the preset keywords is deleted through a traversal method or a list analysis method; All processed texts, error records and error prompt information in the key content are integrated and fed back.

6. The image processing method according to claim 1 or 5, characterized in that: When the comparison result is marked with a preset label and the marked comparison result is fed back in a preset display mode, the method includes: Automatically identify abnormal or mismatched information in the comparison results and process the comparison results, abnormal or mismatched information in a graphical or tabular form; Parameters that meet the standards, parameters with deviations, abnormal or mismatched information will be fed back in the form of charts or lists. At the same time, the uploaded operation and maintenance pictures, identified key content, comparison results and identification information will be archived and stored.

7. The image processing method according to claim 1, wherein: When performing image preprocessing, the method further includes: Calculate the quality scores of the uploaded operation and maintenance images using the structural similarity index and peak signal-to-noise ratio, and sort the operation and maintenance images by quality in a preset order based on the above quality scores; The operation and maintenance pictures are denoised using an adapted image denoising method according to the quality arrangement order, wherein a first denoising process is performed on the operation and maintenance pictures whose quality scores are lower than a preset threshold or whose quality arrangement order is lower than a preset position, and a second denoising process is performed on the operation and maintenance pictures whose quality scores are higher than the preset threshold or whose quality arrangement order is higher than a preset position.

8. An image processing system using the method according to any one of claims 1 to 7, characterized in that: include: A data uploading module, configured to obtain uploaded operation and maintenance data and perform a first data preprocessing on the operation and maintenance data; A data recognition module, configured to extract key content contained in the operation and maintenance data using a preset trained data recognition model and perform a second data preprocessing on the key content; A data comparison module is used to perform a logical comparison between the key content and the preset display material data using a preset logic processing algorithm to generate a comparison result, and to perform preset processing on abnormal data and matching failure information; The result display module is used to preset a mark on the comparison result and feed back the marked comparison result in a preset display mode.

9. An electronic device, characterized in that: include: at least one processor; At least one memory, the at least one memory being coupled to the at least one processor and being used to store instructions executed by the at least one processor, the instructions, when executed by the at least one processor, causing the electronic device to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which implements the method according to any one of claims 1 to 7 when executed by a processor.

Images