Image processing method and system, electronic equipment and storage medium

Abstract

The invention relates to an image processing method and system, electronic equipment and a storage medium. The method comprises the following steps: acquiring an uploaded operation and maintenance picture, carrying out picture preprocessing on the operation and maintenance picture, extracting key contents contained in the operation and maintenance picture by adopting a preset trained data identification model, and carrying out data preprocessing on the key contents; and performing logic comparison on the key content and preset display data through a preset logic processing algorithm to generate a comparison result, performing preset processing on the abnormal data and the matching failure information, performing preset identification on the comparison result, and feeding back the identified comparison result in a preset display mode. According to the method, automatic data extraction and accurate comparison can be realized, so that the operation and maintenance efficiency is improved, the labor cost is reduced, the influence of human factors is reduced, and the data accuracy is ensured.

Description

Technical Field

[0001] This application relates to the technical field of operation and maintenance image processing, and more particularly to an image processing method, an image processing system, an electronic device, and a computer storage medium. Background Art

[0002] Operation and maintenance refers to the process of managing, monitoring, optimizing, and maintaining control devices, systems, and their related resources. The core goal of operation and maintenance is to ensure the high availability, security, and performance stability of the system to support the continuous operation of the business.

[0003] During the manufacturing production process, production equipment usually runs continuously for a long time to meet the requirements of efficient and stable production. However, the operating state of the equipment is often dynamically changing and is affected by various factors, including but not limited to equipment aging, load changes, environmental temperature and humidity, and the operating habits of operators. The fluctuations of these factors may cause equipment failures or performance degradation, thus affecting production efficiency and product quality. Therefore, accurately obtaining and analyzing equipment operation parameters is an important basis for ensuring production stability and product quality.

[0004] Currently, the operation and maintenance of equipment usually rely on operation and maintenance personnel to check the operation data, various parameters, and relevant materials of the equipment one by one, record and compare the data, and analyze and judge with reference to the equipment operation logs. However, due to the large amount of data, the variety of parameters, and the complex production environment, this comparison method is inefficient and is easily affected by human factors, resulting in inaccurate data records, judgment errors, or comparison lags. Summary of the Invention

[0005] This application provides an image processing method, system, electronic device, and storage medium, which can achieve automated data extraction and accurate comparison, thereby improving operation and maintenance efficiency, reducing labor costs, and ensuring data accuracy.

[0006] In a first aspect, this application provides an image processing method, the method includes: obtaining the uploaded operation and maintenance image and performing image preprocessing on the operation and maintenance image; using a data recognition model trained in advance to extract the key content included in the operation and maintenance image and performing data preprocessing on the key content; performing logical comparison between the key content and preset display material data through a preset logic processing algorithm to generate a comparison result, and performing preset processing on abnormal data and matching failure information; performing preset identification on the comparison result and feeding back the identified comparison result in a preset display manner.

[0007] In an alternative solution of the first aspect, when performing image preprocessing, the method includes: identifying the format of the uploaded operation and maintenance image and standardizing the format of the operation and maintenance image, and removing blank and damaged images to generate a standard operation and maintenance image; sequentially performing grayscale conversion, binarization, denoising, size normalization, and contrast enhancement on the standard operation and maintenance image to generate a processed operation and maintenance image.

[0008] In an alternative solution of the first aspect, the preset training method of the data recognition model includes: generating a training data set based on historical annotations and the corrected operation and maintenance images and splitting the training data set into a training set and a validation set in a preset ratio; loading the pre-trained model of the PaddleOCR framework and performing model configuration, and then importing the training set and the validation set into the pre-trained model; using the PP-LCNet convolutional neural network as the backbone network for feature extraction and using the connectionist temporal classification and NRTR methods to perform text recognition on the extracted features; calculating the losses of the connectionist temporal classification and NRTR methods; using the Adam optimizer for gradient update and adopting the Cosine learning rate adjustment strategy; calculating the accuracy; repeating the training process until the preset number of iterations is reached or the model meets the preset requirements, and then generating the data recognition model.

[0009] In an alternative solution of the first aspect, when performing data preprocessing on the key content, the method includes: deleting the extra spaces in the key content through regular expressions or string replacement; deleting the special characters in the key content; correcting the noise characters in the key content through preset noise processing; standardizing the date format, number format, and character encoding of the key content; correcting the words in the key content through a spelling correction algorithm or a spelling check tool and using a preset dictionary or language model for substitution when an unrecognized word is detected; converting all the numerical values included in the text of the key content into a unified format.

[0010] In an alternative solution of the first aspect, after performing logical comparison, the method includes: comparing the device image with the preset display material data to determine whether they match. If not, return the device unmatched information. If so, compare the device parameters with the preset display material data to determine whether they meet the configuration; if not, return the device configuration error information. If so, use the correction logic processing method to process all the text in the key content; determine whether all the text in the key content contains a correction failure. If so, set the correction status of the device to failed and mark the device with the failed correction status. If not, perform a comparison between the preset text in the key content and the preset display material data, and then output the comparison result and perform a shielding or deletion process on the abnormal data and the matching failure information.

[0011] In an alternative embodiment of the first aspect, the correction logic method includes: determining whether the key content contains a preset keyword; if so, extracting the text containing the preset keyword and all subsequent relevant data, and converting the extracted data into a standard format; if not, recording an error and generating an unrecognized error prompt message, and then deleting the text that does not contain the preset keyword through a traversal method or a list parsing method; integrating and feedbacking all processed text, error records, and error prompt messages in the key content.

[0012] In an alternative embodiment of the first aspect, when comparing the preset text of the key content with the preset display data, the method includes: comparing the text of the range, calibration period, working mode, and communication parameters contained in the key content with the preset display data using a string matching method and outputting the text that meets the standard range, calibration period, working mode, and communication parameters; determining whether there is a delay parameter in the text of the key content, if so, extracting the delay parameter and filtering out the text containing the delay parameter, if not, comparing the text containing the batch information in the key content with the preset display data using a string matching method and automatically cleaning the text in the key content that does not meet the standard batch; comparing the alarm values contained in the key content with the preset display data using a string matching method and outputting the text that meets the standard alarm values; comparing the time, temperature, and fan parameters contained in the key content with the preset display data and outputting the text that meets the standard time, temperature, and fan parameters; comparing the curve coefficients and curve offsets contained in the key content with the preset display data and outputting the text that meets the standard curve coefficients and curve offsets.

[0013] In an alternative embodiment of the first aspect, when presetting the comparison results and feedbacking the marked comparison results in a preset display manner, the method includes: automatically marking the abnormal or unmatched information in the comparison results and processing the comparison results, abnormal or unmatched information in the form of a chart or a list; feedbacking the parameters that meet the standards, the parameters with deviations, and the abnormal or unmatched information in a chart or list display manner, and at the same time archiving and storing the uploaded operation and maintenance pictures, identified key content, comparison results, and marking information.

[0014] In an alternative solution of the first aspect, when performing image preprocessing, the method further includes: calculating a quality score of the uploaded operation and maintenance image through the structural similarity index and the peak signal-to-noise ratio, and arranging the operation and maintenance images in a quality order according to the above quality score; performing denoising processing on the operation and maintenance images by using an adapted image denoising method according to the quality arrangement order, wherein, performing a first denoising process on the operation and maintenance images with a quality score lower than a preset threshold or a quality arrangement order lower than a preset position, and performing a second denoising process on the operation and maintenance images with a quality score higher than the preset threshold or a quality arrangement order higher than the preset position.

[0015] In a second aspect, the present application further provides an image processing system using the above method, including: a data uploading module, configured to obtain the uploaded operation and maintenance data and perform first data preprocessing on the operation and maintenance data; a data recognition module, configured to extract key content included in the operation and maintenance data by using a data recognition model that has undergone preset training and perform second data preprocessing on the key content; a data comparison module, configured to generate a comparison result by logically comparing the key content with preset display material data through a preset logic processing algorithm, and perform preset processing on abnormal data and matching failure information; a result display module, configured to perform preset marking on the comparison result and feedback the marked comparison result in a preset display manner.

[0016] In a third aspect, the present application further provides an electronic device, including: at least one processor; at least one memory, the at least one memory being coupled to the at least one processor and configured to store instructions executed by the at least one processor, the instructions when executed by the at least one processor causing the electronic device to execute the image processing method described above.

[0017] In a fourth aspect, the present application further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the image processing method described above is implemented.

[0018] It should be understood that the above general description and the following detailed description are only exemplary and do not limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings incorporated herein and forming a part of the specification illustrate one or more embodiments of the present application and, together with the description, are used to explain the principles of the present application and to enable those of ordinary skill in the relevant art to make and use the present application.

[0020] Figure 1 is a flowchart of an exemplary image processing method according to some embodiments of the present application.

[0021] Figure 2 It is a flowchart of an exemplary model training method according to some embodiments of the present application.

[0022] Figure 3 It is a schematic diagram of an exemplary labeled picture according to some embodiments of the present application.

[0023] Figure 4 It is a schematic diagram of an exemplary automatic picture annotation according to some embodiments of the present application.

[0024] Figure 5 It is a schematic diagram of an exemplary manual correction of picture annotation according to some embodiments of the present application.

[0025] Figure 6 It is a flowchart of an exemplary data preprocessing method according to some embodiments of the present application.

[0026] Figure 7 It is a flowchart of an exemplary method for comparison using a preset logic processing algorithm according to some embodiments of the present application.

[0027] Figure 8 It is a flowchart of an exemplary correction logic method according to some embodiments of the present application.

[0028] Figure 9 It is a schematic diagram of module connections of an exemplary image processing system according to some embodiments of the present application.

[0029] Figure 10 It is a schematic diagram of the structure of an exemplary electronic device according to some embodiments of the present application. Detailed implementation manners

[0030] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present application.

[0031] The image processing method and system of the present application are automated operation and maintenance methods and systems based on image recognition technology and text processing technology, aiming to achieve automated data extraction and accurate comparison, thereby improving operation and maintenance efficiency, reducing labor costs, and ensuring data accuracy.

[0032] Thus, the image processing system of the present application is at least provided with functions of picture uploading and receiving, text recognition and extraction, data logic processing and comparison, result feedback and display, and data storage and management. Among them, the image processing system uses Python as the main development language, and utilizes the advantages of Python in the fields of data processing, image processing, and machine learning to achieve efficient cooperation and data interaction among various modules within the system.

[0033] Thus, users registered in the image processing system can upload various operation and maintenance pictures containing device parameters, wall-mounted materials, etc. The system supports batch uploading and multiple picture formats (such as existing picture formats like JPEG, PNG, GIF, BMP, TIFF, SVG, WebP, HEIF, PDF, RAW, etc.). Among them, HTML5 File API can be used for file selection at the front end, and frameworks such as Flask / Django can be used at the back end to receive files to ensure the convenience of users during use. At the same time, the operation and maintenance pictures matching the user's industry and system application environment are specifically trained and optimized to improve the recognition accuracy of the OCR model in specific scenarios, and then the operation and maintenance pictures uploaded by users are processed to automatically identify and extract the text content in the uploaded pictures, covering key information such as device parameters, device numbers, production data, etc. And through the set custom logic processing algorithm, the data after OCR recognition and processing is compared with the preset data of the wall-mounted display materials to determine whether each parameter meets the requirements and automatically mark the abnormal or mismatched information to ensure the accuracy and real-time of the comparison result. Furthermore, the comparison result is intuitively displayed to the user in the form of a chart or list, clearly indicating which parameters meet the standards and which have deviations, facilitating the operation and maintenance personnel to make adjustments and decisions in a timely manner. And the image processing system of the present application will archive and manage the uploaded pictures, recognition data, and comparison results, support subsequent query, statistics, and data analysis, and help the user enterprise achieve long-term management and continuous optimization of operation and maintenance data.

[0034] Thus, referring to Figure 1 shown, Figure 1 shows a flowchart of an exemplary image processing method according to some embodiments of the present application; an image processing method involved in the present application, the method includes:

[0035] S1. Obtain the uploaded operation and maintenance picture and perform picture preprocessing on the operation and maintenance picture.

[0036] Specifically, in the present application, the operation and maintenance pictures uploaded by users will be archived and stored in a database specifically set by the operation and maintenance personnel for subsequent processing and analysis.

[0037] In some examples of this application, in S1, the stored operation and maintenance pictures are obtained and processed for format standardization, abnormal picture removal, grayscale conversion, binarization, denoising, size normalization, contrast enhancement, etc. The specific methods include:

[0038] Identify the format of each uploaded operation and maintenance picture and uniformly convert operation and maintenance pictures in different formats into the system standard format. Among them, the conversion of the picture format can be achieved through image pixel matrix operations. The system standard format is set by the operation and maintenance personnel. Exemplarily, this application refers to setting the jpg format. At the same time, delete the identified blank pictures and damaged pictures, and compress the operation and maintenance pictures that exceed the recognition limit to produce standard operation and maintenance pictures for subsequent picture processing. Among them, in an optional example of this application, if it is recognized that the picture is damaged or the clarity is too low to be recognized, the image generation algorithm based on the diffusion model of Stable Diffusion or the algorithm of ControlNet used to control the generation process of the large diffusion model is used to automatically complete the damaged and missing areas in the operation and maintenance pictures to improve the information integrity; logical missing information can be generated according to historical recognition results and the device knowledge graph set by the operation and maintenance personnel, rather than randomly completing.

[0039] After the standardization processing of the operation and maintenance pictures, perform grayscale processing on the operation and maintenance pictures to convert the color image of the operation and maintenance pictures into a grayscale image, that is, set each pixel of the operation and maintenance pictures to: 0.299R + 0.587G + 0.114B.

[0040] Furthermore, perform binarization processing on the grayscale operation and maintenance pictures to convert the grayscale image into a black and white image, that is, set the pixel values greater than the set threshold to 255 and the pixel values less than the set threshold to 0 to make it easier to recognize text by subsequent OCR (Optical Character Recognition); among them, the set threshold can be set by the operation and maintenance personnel.

[0041] Furthermore, use a filtering algorithm to perform denoising processing on the binarized operation and maintenance pictures. The filtering algorithms include but are not limited to Gaussian filtering algorithm, wavelet filtering algorithm, moving average filtering algorithm, median filtering algorithm, bilateral filtering algorithm, high / low pass filtering algorithm, etc., which are specifically set by the operation and maintenance personnel. In this application, the Gaussian filtering algorithm is exemplarily used to reduce high-frequency noise: Among them, G(x, y) is the pixel value after filtering, σ is used to control the degree of blurring, usually set to 1-3; x, y are pixel coordinates; then perform morphological processing on the operation and maintenance pictures, including erosion operation and dilation operation. Remove small noise points through erosion to make the fonts in the pictures clearer; repair damaged text through dilation and enhance the edges.

[0042] Further, adjust the size of the operation and maintenance pictures that have undergone denoising and morphological processing to the standard size, that is, shrink the operation and maintenance pictures that exceed the standard size and enlarge the operation and maintenance pictures that are smaller than the standard size, so as to adjust all operation and maintenance pictures to be consistent with the standard size; among them, the size normalization can adopt proportional scaling, padding scaling, etc.

[0043] S2. Use a data recognition model trained in advance to extract the key content contained in the operation and maintenance pictures and perform data preprocessing on the key content.

[0044] Specifically, refer to Figure 2 as shown Figure 2 shows a flowchart of an exemplary model training method according to some embodiments of the present application; the preset training process of the data recognition model includes:

[0045] S20. Generate a training data set based on the historical annotations and the corrected operation and maintenance pictures and split the training data set into a training set and a validation set in a preset ratio;

[0046] Among them, the training data set is divided into a training set and a validation set in a ratio of 8:2 or 9:1.

[0047] S21. Load the pre-trained model of the PaddleOCR framework and perform model configuration, and then import the training set and the validation set into the pre-trained model.

[0048] Among them, the PaddleOCR framework of the present application is an open-source optical character recognition (OCR) tool library developed based on the PaddlePaddle deep learning framework, aiming to provide efficient and accurate text recognition technology. PaddleOCR supports a variety of text recognition tasks, including text detection, text recognition, and text direction detection, and is suitable for different types of text recognition requirements such as scene text, certificates, tables, and books.

[0049] S22. Use the PP-LCNet convolutional neural network as the backbone network for feature extraction and use the connectionist temporal classification and NRTR methods to perform text recognition on the extracted features.

[0050] Among them, the PPLCNetV3 convolutional neural network is used as the backbone to extract features, and its scaling ratio is 0.95; then the extracted features are passed to the connection temporal classification (CTC) and NRTR (No-Recurrence Sequence-to-Sequence Model For Scene Text Recognition) for text recognition; among them, NRTR uses the complete Transformer structure to encode and decode the input picture.

[0051] S23. Calculate the losses of the connection temporal classification and NRTR methods.

[0052] Among them, the CTC loss is calculated as:

[0053]

[0054] , where X is the input sequence, Y is the target text, A is the possible alignment path, and B -1 (Y) is the set of all paths that can transcribe Y;

[0055] The NRTR loss is calculated as:

[0056]

[0057] , where N is the length of the target sequence in the NRTR loss calculation, that is, the number of characters to be recognized in the OCR task; i is the position of the currently processed character in the target sequence, and y i is the target character, that is, the i-th character in the true label;

[0058] P(y i |X) is the probability output by the Transformer decoder.

[0059] S24. Use the Adam optimizer for gradient update and adopt the Cosine learning rate adjustment strategy.

[0060] Among them, the parameter update formula of the Adam optimizer is:

[0061] m t =β 1 m t-1 +(1-β 1 )g t

[0062]

[0063] Among them, m t is the exponential weighted average of the gradient, and v tis the exponentially weighted average of the squared gradient, g t is the gradient of the loss with respect to the parameters computed for the current batch, α is the learning rate, β 1 and β 2 are the momentum decay coefficients, ∈ is a smoothing term to prevent division by zero errors, θ t is the value of the model parameters after the current training step; the parameter update mainly depends on m t and v t where m t controls the direction and v t controls the step size.

[0064] Thus, combining the momentum method and RMSProp, the learning rate is adaptively adjusted using the first and second moment estimates of the gradient,

[0065] During the optimization process, the Adam optimizer can both accelerate the convergence speed and reduce gradient oscillations, improving the model stability.

[0066] Among them, the formula for adopting the Cosine learning rate adjustment strategy is:

[0067]

[0068] where η t is the current learning rate used to dynamically adjust the step size during training, η max is the maximum learning rate, η min is the minimum learning rate, T cur is the current training progress, T is the total number of training epochs; the periodic decrease of the learning rate is controlled by the cosine function to improve the model convergence stability; by adopting the Cosine learning rate, it can prevent the learning rate from being too large and causing it to jump out of the optimal solution, or too small and causing the training to stagnate.

[0069] S25. Calculate the accuracy.

[0070] Among them, the formula for the accuracy is:

[0071]

[0072] where I is the indicator function, is the model prediction, yi is the true label, and here N is the number of samples for accuracy calculation.

[0073] S26. Loop through the training process of S22 - S25 until the preset number of iterations is reached or the model meets the preset requirements, and then generate a data recognition model.

[0074] Among them, the preset number of iterations and preset requirements for the loss are set by the operation and maintenance personnel according to the actual needs; the trained model is set as the data recognition model.

[0075] Among them, refer to Figures 3 to 5 As shown, Figure 3 Figure 4 shows a schematic diagram of an exemplary labeled picture of some embodiments of the present application. Figure 4 Figure 5 shows a schematic diagram of an exemplary automatic picture annotation of some embodiments of the present application. Figure 5 Figure 6 shows a schematic diagram of an exemplary manual correction of picture annotation of some embodiments of the present application; before performing preset training, the operation and maintenance personnel need to label and correct historical operation and maintenance pictures. The specific method includes:

[0076] Open the annotation program of the pre-trained model of the PaddleOCR framework and use the automatic recognition function of the PaddleOCR framework to recognize historical operation and maintenance pictures, and then the operation and maintenance personnel manually correct the annotation results according to the automatic recognition results.

[0077] In some examples of the present application, refer to Figure 6 As shown, the figure shows a flowchart of an exemplary data preprocessing method of some embodiments of the present application; after the data recognition model extracts the key content included in the operation and maintenance pictures, the present application performs data preprocessing on the key content. The specific processing method includes:

[0078] S200. Delete the extra spaces in the key content through regular expressions or string replacement.

[0079] Among them, when OCR extracts text, extra spaces may appear, especially at the beginning or end of a line or between different words. The extra spaces are deleted through step S200; for example, multiple space characters are matched using regular expressions and replaced with a single space.

[0080] S201. Delete the special characters in the key content. Exemplarily, special characters such as "@", "#", "$" and other meaningless characters.

[0081] S202. Correct the noise characters in the key content through preset noise processing; Exemplarily, noise characters such as "0" and "o", "1" and "l", etc. are corrected through character mapping or dictionary.

[0082] S203. Standardize the date format, number format and character encoding of the key content.

[0083] Among them, if the date formats recognized by OCR are not unified, such as "2020-02-20" and "20 / 02 / 2022", they need to be uniformly converted to the standard format; if the number formats recognized by OCR are not unified, such as "3,000" modified to "3000", "3.5K" modified to "3500", they need to be uniformly converted to the standard format; if different character encodings appear during the OCR process, they need to be uniformly converted to the standard encoding.

[0084] S204. Correct the words of the key content through a spelling correction algorithm or a spelling check tool, and when an unrecognized word is detected, use a preset dictionary or language model for substitution.

[0085] Among them, the spelling correction algorithm includes but is not limited to dictionary matching, edit distance, word or letter-level spelling correction algorithms, etc., and the spelling check tool includes but is not limited to PySpellChecker, PyEnchant, TextBlob, etc.; the preset dictionary is constructed by the operation and maintenance personnel with industry-related dictionaries (such as device parameters, common units, device models, etc.), and the preset language model includes but is not limited to n-gram, Chinese Spelling Correct model, etc.

[0086] S205. Convert all numerical values included in the text of the key content into a unified format.

[0087] Among them, numbers, dates, currencies, etc. in the text may be presented in different formats, and regularization and normalization processing need to be performed on them to ensure data uniformity.

[0088] S3. Generate a comparison result by logically comparing the key content with the preset display material data through a preset logic processing algorithm, and perform preset processing on abnormal data and matching failure information.

[0089] Specifically, refer to Figure 7 as shown Figure 7 shows a flowchart of a comparison method using an exemplary preset logic processing algorithm in some embodiments of the present application; in S3, when performing logical comparison through the preset logic processing algorithm, the method includes:

[0090] S30. Compare the device picture with the preset display material data to determine whether there is a match.

[0091] If no corresponding device parameters are matched in the preset display material data according to the device picture, execute S300: return the device unmatched information; if corresponding device parameters are matched in the preset display material data according to the device picture, execute S31: compare the device parameters with the preset display material data to determine whether they meet the configuration to ensure that each device meets the standard and avoid incorrect configuration.

[0092] If the device parameters do not meet the configuration standard, execute S310: return the device configuration error information; if the device parameters meet the standard, execute S32: process all the text in the key content using a correction logic processing method.

[0093] In some examples of the present application, refer toFigure 8 As shown Figure 8 Figure 8 shows a flowchart of an exemplary correction logic processing method for some embodiments of the present application; the correction logic processing method includes:

[0094] S320. Determine whether the preset keyword is included in the key content; wherein, the preset keyword is set by the operation and maintenance personnel according to the actual application field and user requirements.

[0095] If the key content contains the preset keyword, then execute S3200: Extract the text containing the preset keyword and all subsequent relevant data, and convert the extracted data into a standard format. After the extraction is completed, mask or delete the text matching the preset keyword in the key content to avoid repeated data extraction later.

[0096] Exemplarily, sequentially identify all data in the key content. For example, whenever a set keyword such as "water sample", "distilled water", "reagent", etc. is found in a certain text of the key content, find the corresponding index positions of "water sample", "distilled water", "reagent", etc., and then extract all the values after them, and standardize the extracted values. For example, convert "once a week", "once every 6 days", "once a month", etc. into corresponding formats such as "7 days", "6 days"

[0097] "30 days", etc., specifically refer to the standardization processing form in the above data preprocessing; furthermore, mask or delete the steadily corresponding to the set keywords such as "water sample", "distilled water", "reagent", etc. in the key content to avoid repeated data extraction later; then integrate all processed text data into a list to ensure that each text has been processed and there is complete feedback information

[0098] If the key content does not contain the preset keyword, then execute S3201: Record the error and generate an unrecognized error prompt message, and then delete the text that does not contain the preset keyword through a traversal method or a list parsing method.

[0099] S321. Integrate and feedback all processed text, error records, and error prompt messages in the key content.

[0100] S33. Determine whether all texts in the key content contain unqualified calibration. If so, execute S330: Set the calibration status of the device to failed and mark the device with the failed calibration status. If not, execute S34: Compare the preset text in the key content with the preset display material data, and then output the comparison result and mask or delete the abnormal data and matching failure information.

[0101] Among them, check whether each line of text in the key content contains a calibration failure. If it does, set the calibration status of the device to failed and mark those devices with calibration failures for subsequent operations.

[0102] Among them, in S34, according to different application fields, set different key content texts to be compared with the preset display material data. Exemplarily, this application takes a certain environmental monitoring field as an example, specifically referring to:

[0103] S340. Compare the texts of the range, calibration period, working mode, and communication parameters included in the key content with the preset display material data using a string matching method and output the texts of the range, calibration period, working mode, and communication parameters that meet the standards, so as to display the texts of the range, calibration period, working mode, and communication parameters at the matching positions; and after outputting the texts of the range, calibration period, working mode, and communication parameters that meet the standards, delete or mask the texts of the range, calibration period, working mode, and communication parameters included in the current key content to avoid repeated input or incorrect data and improve system stability.

[0104] S341. Determine whether there is a delay parameter in the text of the key content. If so, extract the delay parameter and filter out the text containing the delay parameter. If not, compare the text of the key content containing batch information with the preset display material data using a string matching method and automatically clean the text of the key content that does not meet the standard batch.

[0105] Among them, different devices may store batch numbers before and after the batch keyword, so it is necessary to check both the complete string and the string before the batch, and then display the batch text that meets the standard at the matching position; and after outputting the batch text that meets the standard, delete or mask the corresponding batch text included in the current key content to avoid repeated input or incorrect data and improve system stability.

[0106] S342. Compare the alarm values included in the key content with the preset display material data using a string matching method and output the text of the alarm values that meet the standards, so as to display the text of the alarm values that meet the standards at the matching positions; and after outputting the text of the alarm values that meet the standards, delete or mask the text of the alarm values included in the current key content to avoid repeated input or incorrect data and improve system stability.

[0107] S343. Compare the time, temperature, and fan parameters included in the key content with the preset display data, and output the text of the time, temperature, and fan parameters that meet the standards, so as to display the text of the time, temperature, and fan parameters that meet the standards at the matching positions; and after outputting the text of the time, temperature, and fan parameters that meet the standards, delete or mask the text of the time, temperature, and fan parameters included in the current key content to avoid repeated input or incorrect data and improve the system stability.

[0108] S344. Compare the curve coefficient and curve offset included in the key content with the preset display data, and output the text of the curve coefficient and curve offset that meet the standards, so as to display the text of the curve coefficient and curve offset that meet the standards at the matching positions; and after outputting the curve coefficient and curve offset that meet the standards, delete or mask the text of the curve coefficient and curve offset included in the current key content to avoid repeated input or incorrect data and improve the system stability.

[0109] S4. Preset-identify the comparison result and feedback the identified comparison result in a preset display manner.

[0110] Specifically, in S4, when preset-identifying the comparison result and feedbacking the identified comparison result in a preset display manner, the method includes:

[0111] Automatically identify the abnormal or unmatched information in the comparison result and process the comparison result, abnormal or unmatched information in the form of a chart or list.

[0112] Feedback the parameters that meet the standards, the parameters with deviations, and the abnormal or unmatched information in the form of a chart or list display, and at the same time archive and store the uploaded operation and maintenance pictures, identified key content, comparison result, and identification information.

[0113] In some embodiments of the present application, when performing image preprocessing, the method further includes:

[0114] S100: Calculate the quality score of the uploaded operation and maintenance picture through the structural similarity index and peak signal-to-noise ratio, and arrange the operation and maintenance pictures in a preset order according to the above quality score.

[0115] Among them, the structural similarity index is used to measure the structural similarity of two images. Especially in the denoising process, SSIM can reflect the denoising effect. Specifically, the calculation formula of the structural similarity index is:

[0116]

[0117] Among them, in this embodiment: μ x and μy are the average brightness of image x and image y respectively, and are the variances of image x and image y respectively, σ xy is the covariance of image x and image y, C 1 and C 2 are constants used for stable calculation. Exemplarily, assume there are two images: the original uploaded image x and the image y after noise processing. By calculating the SSIM (Structural Similarity Index) value, it can be known whether the image is affected by noise. The closer the SSIM value is to 1, the better the image quality, and the smaller the value, the more noise there is.

[0118] Among them, PSNR (Peak Signal-to-Noise Ratio) is an important indicator for measuring image quality. The specific calculation formula of the PSNR is:

[0119]

[0120] Among them, MAX I is the maximum pixel value in the image. For example, for an 8-bit image, the maximum value is 255; MSE is the mean square error, which is used to measure the difference between images. The greater the difference, the greater the MSE and the smaller the PSNR.

[0121] S101: Denoise the operation and maintenance pictures using an adapted image denoising method according to the quality ranking order. Among them, perform the first denoising process on the operation and maintenance pictures with a quality score lower than the preset threshold or a quality ranking order lower than the preset position, and perform the second denoising process on the operation and maintenance pictures with a quality score higher than the preset threshold or a quality ranking order higher than the preset position.

[0122] Among them, the preset threshold and preset position are set by the operation and maintenance personnel according to actual needs. Exemplarily, for example: for images with a quality score lower than the preset threshold or a quality ranking order lower than the preset position, use a stronger denoising method, such as Wavelet Transform or median filtering; while for images with a quality score higher than the preset threshold or a quality ranking order higher than the preset position, choose mild denoising such as mean filtering to avoid overprocessing.

[0123] By introducing indicators such as SSIM and PSNR to evaluate image quality, and combining methods such as noise detection mechanisms and adaptive data cleaning, it is possible to optimize the preprocessing process of operation and maintenance data in different environments, ensure the accuracy and consistency of the data, and thus improve the overall performance of the automated operation and maintenance system.

[0124] In some embodiments of the present application, after data preprocessing, the method further includes:

[0125] For the operation and maintenance pictures after picture preprocessing, use the Faster R-CNN convolutional neural network or the YOLO (You Only Look Once) object detection algorithm for image recognition to identify the data information contained in the operation and maintenance pictures;

[0126] Compare the recognized data information with the key content extracted by the data recognition model one by one. By comparing the numbers, formulas, and text information of the two, judge whether they are consistent. If so, the correction passes; if not, the correction fails, and mark the corresponding content for subsequent manual correction of errors to ensure that the finally extracted data is consistent with the device configuration.

[0127] Among them, when making the comparison, the Levenshtein distance text comparison algorithm can be used to calculate the edit distance of two strings, so as to judge whether the texts match. Exemplarily, assume that the key content extracted by the data recognition model is "Device model: AB123", and the data information recognized by the image is "AB123 device model". By calculating the Levenshtein distance, if the distance is 1, it means the texts are similar and the correction is judged to pass; Exemplarily, assume that the formula in the image is E = mc 2 , and the OCR extracts it as E = m·c 2 , then perform symbol matching processing, and judge that this is the correct formula expression through the formula recognition rules.

[0128] By comparing the information recognized by the image with the content extracted by the data recognition model, it can effectively reduce the correction failure caused by the error of the data recognition model, and combine the advantages of the two to ensure the accurate recognition and correction of complex images and text information.

[0129] Furthermore, in this application, when comparing device parameters, not only compare the basic parameters of the device, but also compare multi-level information such as the operation history, maintenance records, and usage environment of the device. Through multi-level parameter comparison, the status of the device can be verified from multiple dimensions, making the comparison more comprehensive, so as to provide more accurate operation and maintenance feedback.

[0130] Furthermore, in this application, a self-learning mechanism can also be introduced to continuously accumulate comparison historical data and optimize the rules and parameters of logical comparison. Furthermore, the comparison strategy can be adjusted according to historical data, so that the system can run stably for a long time. Exemplarily, for example, during the device information comparison process, the comparison threshold can be automatically adjusted according to the success or failure cases of previous comparisons, gradually improving the matching accuracy.

[0131] Thus, as shown in Figure 9 shown, Figure 9The figure shows a schematic diagram of the module connection of an exemplary image processing system according to some embodiments of the present application; in some embodiments of the present application, the present application also relates to an image processing system, including:

[0132] A data upload module 100, configured to obtain the uploaded operation and maintenance data and perform first data preprocessing on the operation and maintenance data.

[0133] A data recognition module 101, configured to use a data recognition model that has undergone preset training to extract the key content included in the operation and maintenance data and perform second data preprocessing on the key content.

[0134] A data comparison module 102, configured to perform a logical comparison between the key content and preset display data through a preset logic processing algorithm to generate a comparison result, and perform preset processing on abnormal data and matching failure information.

[0135] A result display module 103, configured to perform preset marking on the comparison result and feedback the marked comparison result in a preset display manner.

[0136] In some embodiments, as shown in Figure 10 shown, Figure 10 is a schematic diagram of the structure of an electronic device for implementing the embodiments of the present application. The electronic device includes: a memory 201 and a processor 202, and a computer program that can run on the processor 202 is stored in the memory 201. When the processor 202 executes the computer program, the methods in the above embodiments are implemented. The number of the memory 201 and the processor 202 can be one or more.

[0137] The electronic device further includes:

[0138] A communication interface 203, configured to communicate with external devices and perform data interaction and transmission.

[0139] If the memory 201, the processor 202, and the communication interface 203 are implemented independently, the memory 201, the processor 202, and the communication interface 203 can be interconnected through a bus and complete communication with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity of representation, Figure 10 only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0140] Optionally, in a specific implementation, if the memory 201, the processor 202, and the communication interface 203 are integrated on a single chip, the memory 201, the processor 202, and the communication interface 203 can communicate with each other through an internal interface.

[0141] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which when executed by the processor 202 implements the method provided in the embodiment of the present application.

[0142] An embodiment of the present application further provides a chip, which includes a processor 202 for calling and running instructions stored in the memory 201 from the memory 201, so that a communication device equipped with the chip executes the method provided in the embodiment of the present application.

[0143] An embodiment of the present application further provides a chip, including: an input interface, an output interface, a processor 202, and a memory 201. The input interface, the output interface, the processor 202, and the memory 201 are connected through an internal connection path. The processor 202 is configured to execute the code in the memory 201, and when the code is executed, the processor 202 is configured to execute the method provided in the embodiment of the application.

[0144] It should be understood that the above-mentioned processor 202 may be a central processing unit 202 (CPU), or may also be other general-purpose processors 202, digital signal processors 202 (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor 202 may be a microprocessor 202 or any conventional processor 202, etc. It is worth noting that the processor 202 may be a processor 202 that supports the advanced reduced instruction set machines (ARM) architecture.

[0145] Further, the above-mentioned memory 201 may include a read-only memory 201 and a random access memory 201, and may also include a non-volatile random access memory 201. The memory 201 may be a volatile memory 201 or a non-volatile memory 201, or may include both volatile and non-volatile memories 201. Among them, the non-volatile memory 201 may include a read-only memory 201 (Read-Only Memory, ROM), a programmable read-only memory 201 (Programmable ROM, PROM), an erasable programmable read-only memory 201 (Erasable PROM, EPROM), an electrically erasable programmable read-only memory 201 (Electrically EPROM, EEPROM), or a flash memory. The volatile memory 201 may include a random access memory 201 (Random Access Memory, RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static random access memory 201 (Static RAM, SRAM), dynamic random access memory 201 (Dynamic Random Access Memory, DRAM), synchronous dynamic random access memory 201 (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory 201 (Double Data Rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory 201 (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory 201 (Synchlink DRAM, SLDRAM), and direct rambus random access memory 201 (Direct Rambus RAM, DR RAM).

[0146] In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0147] As described above, it is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claimed rights.

Claims

1. An image processing method, characterized in that: The method comprises: Obtain uploaded operation and maintenance pictures and perform picture preprocessing on the operation and maintenance pictures; Using a preset trained data recognition model to extract key content contained in the operation and maintenance picture and perform data preprocessing on the key content; The key content is logically compared with the preset display material data through a preset logic processing algorithm to generate a comparison result, and abnormal data and matching failure information are processed in a preset manner; The comparison result is marked with a preset label and the marked comparison result is fed back in a preset display mode.

2. The image processing method according to claim 1, characterized in that: When performing image preprocessing, the method includes: Identify the format of the uploaded operation and maintenance pictures and standardize the format of the operation and maintenance pictures, and remove blank and damaged pictures to generate standard operation and maintenance pictures; The standard operation and maintenance pictures are processed in sequence by grayscale conversion, binarization, denoising, size normalization and contrast enhancement to generate processed operation and maintenance pictures.

3. The image processing method according to claim 1 or 2, characterized in that: The preset training method of the data recognition model includes: Generate a training data set based on the historical annotations and the corrected operation and maintenance images, and divide the training data set into a training set and a validation set with a preset ratio; Load the pre-trained model of the PaddleOCR framework and configure the model, and then import the training set and validation set for the pre-trained model; The PP-LCNet convolutional neural network is used as the backbone network for feature extraction and the connection temporal classification and NRTR methods are used to perform text recognition on the extracted features; Calculate the loss of the connection time classification and NRTR methods; Use Adam optimizer for gradient update and adopt Cosine learning rate adjustment strategy; Calculate accuracy; The training process is repeated until the preset number of iterations is reached or the model meets the preset requirements, thereby generating a data recognition model.

4. The image processing method according to claim 3, characterized in that: When performing data preprocessing on the key content, the method includes: Delete the redundant spaces of the key content by regular expression or string replacement; Delete the special characters of the key content; Correcting the noise characters of the key content by means of a preset noise process; Standardizing the date format, number format and character encoding of the key content; Correcting the words of the key content by a spelling proofreading algorithm or a spell checking tool and replacing the words with a preset dictionary or language model when unrecognizable words are detected; All numerical values ​​contained in the text of the key content are converted into a unified format.

5. The image processing method according to claim 1 or 4, characterized in that: After performing the logical comparison, the method includes: Compare the device image with the preset display data to determine whether they match. If not, return the device unmatched information. If yes, compare the device parameters with the preset display data to determine whether they meet the configuration. If not, then return device configuration error information, if so, then use a correction logic processing method to process all texts in the key content; Determine whether all the texts in the key content contain correction failures. If so, set the correction status of the device to failure and mark the device in the failed correction status. If not, compare the preset text of the key content with the preset display material data, and then output the comparison results and block or delete the abnormal data and matching failure information.

6. The image processing method according to claim 5, characterized in that: The correction logic method comprises: Determine whether the key content contains preset keywords, If yes, extract the text containing the preset keyword and all subsequent data related to the text, and convert the extracted data into a standard format; If not, an error is recorded and an unrecognized error prompt message is generated, and the text that does not contain the preset keywords is deleted through a traversal method or a list parsing method; All processed texts, error records and error prompt information in the key content are integrated and fed back.

7. The image processing method according to claim 5, characterized in that: When comparing the preset text of the key content with the preset display material data, the method includes: The text of the key content including the measuring range, calibration cycle, working mode and communication parameters is compared with the preset display data by using the string matching method and the text that meets the standard measuring range, calibration cycle, working mode and communication parameters is output; Determine whether the text in the key content has a delay parameter. If so, extract the delay parameter and filter out the text containing the delay parameter. If not, compare the text containing batch information in the key content with the preset display material data using a string matching method and automatically clean the text in the key content that does not meet the standard batch; The alarm value contained in the key content is compared with the preset display data using a string matching method and a text that meets the standard alarm value is output; Compare the time, temperature and fan parameters included in the key content with the preset display data and output text that meets the standard time, temperature and fan parameters; Compare the curve coefficient and curve offset contained in the key content with the preset display material data and output text that meets the standard curve coefficient and curve offset.

8. The image processing method according to claim 1, 6 or 7, characterized in that: When the comparison result is marked with a preset mark and the marked comparison result is fed back in a preset display mode, the method includes: Automatically identify abnormal or unmatched information in the comparison results and process the comparison results, abnormal or unmatched information in a chart or list form; Parameters that meet the standards, parameters with deviations, abnormal or mismatched information will be fed back in the form of charts or lists. At the same time, the uploaded operation and maintenance pictures, identified key content, comparison results and identification information will be archived and stored.

9. The image processing method according to claim 1, characterized in that: When performing image preprocessing, the method further includes: Calculate the quality scores of the uploaded operation and maintenance pictures by using the structural similarity index and the peak signal-to-noise ratio, and arrange the operation and maintenance pictures by quality in a preset order according to the above quality scores; An adaptive image denoising method is used to perform denoising on the operation and maintenance pictures according to the quality arrangement order, wherein a first denoising process is performed on the operation and maintenance pictures whose quality scores are lower than a preset threshold or whose quality arrangement order is lower than a preset position, and a second denoising process is performed on the operation and maintenance pictures whose quality scores are higher than a preset threshold or whose quality arrangement order is higher than a preset position.

10. An image processing system using the method according to any one of claims 1 to 9, characterized in that: include: A data uploading module, used for acquiring uploaded operation and maintenance data and performing a first data preprocessing on the operation and maintenance data; A data identification module, used to extract key contents contained in the operation and maintenance data by using a preset trained data identification model and perform a second data preprocessing on the key contents; A data comparison module, used to perform a logical comparison between the key content and the preset display material data through a preset logic processing algorithm to generate a comparison result, and to perform preset processing on abnormal data and matching failure information; The result display module is used to preset a mark on the comparison result and feed back the marked comparison result in a preset display mode.

11. An electronic device, characterized in that: include: at least one processor; At least one memory, the at least one memory is coupled to the at least one processor and is used to store instructions executed by the at least one processor, and when the instructions are executed by the at least one processor, the electronic device performs the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images