Cross-platform instrument software data review and redrawing method based on image recognition

Abstract

The embodiment of the present application discloses a kind of cross-platform instrument software data review and redrawing method based on image recognition, belong to computer software automation control technical field, the method includes: the target software is opened target data, makes target software display target atlas;From the target atlas, the atlas area containing chromatographic peak type line chart is intercepted;According to matching accumulation peak configuration rule, the atlas area is analyzed, and target drag peak data is obtained;According to the target drag peak data, redraws chromatographic peak on target software interface.The embodiment of the present application can realize cross-platform general, chromatographic peak recognition rate is high, and redrawing precision is high.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer software automatic control, in particular to a cross-platform instrument software data review and redrawing method based on image recognition. Specifically, it is a method for realizing third-party software interface element positioning, automatic operation and data graph intelligent processing based on image recognition technology, which is especially suitable for automatic data review and redrawing of cross-platform instrument control software without API interface. BACKGROUND

[0002] In the field of chromatographic analysis, the chromatographic peak type line graphs generated by the instrument supporting software of manufacturers such as Agilent, Shimadzu China and Thermal Power need to be manually reviewed for peak shape, baseline calibration and redrawing of abnormal peaks. This process has the problems of low efficiency (single operation time ≥ 15 minutes), high subjective error rate (differences in judgment by different personnel > 30%) and the like. The existing technology has not effectively solved this problem, and the specific analysis is as follows:

[0003] Existing technology one: GUI automation tool based on static template matching (represented by SikuliX)

[0004] Core structure:

[0005] Image template library: store screenshots (template resolution 72 dpi) of target controls (such as buttons, icons);

[0006] Matching engine: use normalized cross-correlation (NCC) algorithm to calculate the similarity of screen area and template;

[0007] Operation executor: when the similarity > threshold value (default 0.8), drive the mouse to click the center point of the matching area.

[0008] The working principle is as follows:

[0009] Screen capture → template matching → similarity calculation → click execution.

[0010] Literature basis:

[0011]

[0012] The disadvantages of the existing technology one are as follows:

[0013] Chromatographic peak recognition completely fails:

[0014] Static template can only match fixed icons (such as "save" button), and cannot analyze the dynamic characteristics of chromatographic peaks (peak height and peak width change); see the Figure 1 -b, when the measured chromatographic peak (red) and the template peak (blue) are deformed by > 15%, the matching failure rate is as high as 92% (literature data).

[0015] Redrawing accuracy does not meet industrial standards:

[0016] Coordinate mapping error ±3 pixels (literature p.7, Section 4.2), resulting in redrawing line offset (see the above literature Figure 1 -c arrow); sub-pixel (<1 pixel) accuracy required for chromatographic analysis cannot be achieved.

[0017] Prior art two: instrument manufacturer closed macro tool (represented by Agilent Macro Commander)

[0018] Core structure:

[0019] Script editor: record user operation to generate VBScript command (such as OpenFile("DATA001.CH"));

[0020] API mapping layer: convert script command to instrument software internal interface call.

[0021] The working principle is as follows:

[0022] Execute script → call private API → operate software.

[0023] Literature basis:

[0024] Agilent Technologies, "ChemStation Macro Command Reference", 2018, p.12-15.

[0025] The shortcomings of prior art two are as follows:

[0026] Fundamental lack of redrawing function:

[0027] API only supports basic operations (open / save file), without chromatographic peak pattern access interface (literature p.14, Table 2);

[0028] Cross-platform compatibility is 0:

[0029] Limited to Agilent's own software, instruments of other manufacturers such as Hot-Plat and Shimadzu China cannot call their private API. Invention content

[0030] Therefore, the embodiment of the present application provides a cross-platform instrument software data review and redrawing method based on image recognition, which can realize cross-platform generalization, high chromatographic peak recognition rate and high redrawing accuracy.

[0031] A cross-platform instrument software data review and redrawing method based on image recognition, comprising:

[0032] Step S101: Operate the target software to open the target data and display the target map.

[0033] Step S102: Extract the spectral region containing the chromatographic peak shape from the target spectrum;

[0034] Step S103: Analyze the spectral region according to the matching peak configuration rules to obtain the target peak data;

[0035] Step S104: Based on the target peak data, redraw the chromatographic peaks on the target software interface.

[0036] The present invention has the following beneficial effects:

[0037] 1. High-precision control positioning: Through the CNOCR+OpenCV dual-level fault tolerance mechanism (confidence threshold trigger backup scheme), the control recognition failure problem in multi-language environments is completely solved, and the robustness of cross-platform operation is greatly improved.

[0038] 2. Broad-spectrum peak recognition capability: The three-mode peak decision engine (positive time window matching / interval peak scanning / template correlation coefficient comparison) fully covers complex industrial sample scenarios, significantly improving the ability to detect unconventional peak shapes.

[0039] 3. Industrial-grade redrawing accuracy assurance: The drawing-capture-verification closed-loop system ensures that every redrawing operation is quantifiable and traceable, eliminating the randomness of manual operation and meeting the compliance requirements of instrument analysis data.

[0040] 4. Full-process automation: The dual-loop architecture of materials and data enables unattended operation of the entire process from data import to report generation, significantly reducing the processing cycle of a single sample. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating the cross-platform instrument software data verification and redrawing method based on image recognition of the present invention.

[0043] Figure 2 for Figure 1 A schematic diagram of the actual operation of step S101;

[0044] Figure 3 for Figure 1 A schematic diagram of the actual operation of step S102;

[0045] Figure 4 For Figure 1 Actual operation schematic diagram of step S103 in the embodiment;

[0046] Figure 5 For Figure 1 Actual operation schematic diagram of step S104 in the embodiment;

[0047] Figure 6 Schematic diagram of chromatographic peak rule configuration in the embodiment Figure 1 ;

[0048] Figure 7 Schematic diagram of chromatographic peak rule configuration in the embodiment Figure 2 ;

[0049] Figure 8 Schematic diagram of chromatographic peak rule configuration in the embodiment Figure 3 ;

[0050] Figure 9 Schematic diagram of chromatographic peak rule configuration in the embodiment Figure 4 ;

[0051] Figure 10 Schematic diagram of chromatographic peak rule configuration in the embodiment Figure 5 ;

[0052] Figure 11 Schematic diagram of chromatographic peak rule configuration in the embodiment ;

[0053] Figure 12 Schematic diagram of chromatographic peak rule configuration in the embodiment ;

[0054] Figure 13 Schematic diagram of chromatographic peak rule configuration in the embodiment Specific embodiments

[0055] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings.

[0056] In the description of the present application, it should be understood that the terms "center", "transverse", "longitudinal", "front", "back", "left", "right", "up", "down", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only used to facilitate the description of the present application and simplify the description, and therefore cannot be understood as indicating or implying that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the scope of protection of the present application.

[0057] It should be noted that the embodiments described are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0058] To solve the problems in the prior art, such as: the recognition rate of traditional image matching technology on dynamic chromatographic peak shape < 10%, private API blocking of manufacturers leading to failure of cross-platform automation scheme, and drawing offset caused by mouse operation error > 3 pixels, the present application constructs a closed-loop control method based on image recognition to realize: (1) cross-platform general operation: establishing a relative coordinate system through feature marker recognition, supporting the adaptation of Agilent / Thermo / SHIMADZU VOC software; (2) multi-rule integral peak matching and target drag peak data extraction: based on atlas region data, performing positive integral peak, interval integral peak and template integral peak according to different species matching integral peak configuration rules; (3) intelligent correction of chromatographic peak: analyzing dynamic parameters such as peak height ratio and tailing factor to generate redrawing instructions; (4) sub-pixel precision control: using Gaussian interpolation algorithm to compress the operation error to < 0.5 pixels.

[0059] The embodiment of the present application provides a cross-platform instrument software (automation) data review and redrawing method based on image recognition, as shown in Figure 1 , which comprises the following steps:

[0060] Step S101: opening target data by operating target software, so that the target software displays a target atlas;

[0061] In the present application, the instrument is a chromatograph, and the target software is a software matched with the chromatograph. As shown in Figure 2 , when the target software opens the target data, it can be manually operated, mechanically operated by a physical mouse or simulated by a software mouse, etc. To realize automation (unmanned operation), the software mouse simulation mode is preferably adopted.

[0062] That is, as an optional embodiment, the step S101 comprises:

[0063] Step S1011: obtaining a target software interface after starting the target software;

[0064] In this step, the Windows GDI interface can be called to intercept the target software window (resolution ≥1920×1080) to output an RGB image stream (frame rate 1fps), so as to obtain the target software interface.

[0065] Step S1012: establishing a relative coordinate system through feature marker recognition in the target software interface;

[0066] Preferably, the step S1012 comprises:

[0067] Step S10121: Extracting interface feature markers in the target software interface according to a pre-constructed configuration file, wherein the configuration file defines the interface feature marker templates in each manufacturer's software and their default offsets;

[0068] This step is to extract interface feature markers. In actual implementation, in the instrument software interface, fixed and unchanged graphical elements such as “spectrum coordinate axis intersection point”, “spectrum frame”, “scale line starting point” and the like are identified. These elements are stable in position and consistent in style in different versions, and are referred to as interface feature markers.

[0069] Step S10122: Positioning the interface feature markers in the target software interface according to the configuration file;

[0070] This step is marker positioning. In actual implementation, image template matching (OpenCV cv2.matchTemplate function) can be used to accurately obtain the pixel positions of these elements in the screenshot, such as the lower left corner coordinates of the spectrum being .

[0071] Step S10123: Establishing a relative coordinate system using the interface feature markers.

[0072] This step is to establish a relative coordinate system. In actual implementation, a certain marker (such as the lower left corner intersection point of the spectrum) can be taken as the origin to construct the coordinate system of the local spectrum. Other coordinate values are calculated relative to the origin.

[0073] The establishment of the relative coordinate system can be completed through the above steps S10121-S10123. The above process can adapt to multiple platform software interfaces. By defining the marker templates and their default offsets in each manufacturer's software through a configuration file, positioning adaptation of the same drawing logic under multiple interfaces is achieved. For example, if the Agilent spectrum starting point is (300, 800) and the Shimadzu is (200, 750), accurate drawing in the target area can still be achieved through relative coordinate transformation without the need to retrain the template.

[0074] Step S1013: In the target software interface, using a text recognition engine to locate the “Open” button to obtain target coordinates;

[0075] In this step, the text recognition engine can be various OCR software capable of recognizing interface text, such as CNOCR.

[0076] Step S1014: Determining whether the confidence of the target coordinates is greater than a preset confidence threshold. If yes, simulate mouse clicks on the target coordinates to open the target data.

[0077] In this step, the confidence preset threshold can be flexibly set as needed, such as 80%, 85%, 90%, etc.

[0078] Preferably, the step S1014 comprises:

[0079] If the confidence of the target coordinate is less than or equal to the confidence preset threshold, the icon features (color + contour) of the "open" button in the target software interface are matched by using OpenCV to obtain the target coordinate, and the mouse is simulated to click the target coordinate to open the target data.

[0080] The above steps S1011-S1014 can solve the problem of invalid positioning of multi-language controls (Chinese recognition rate 98.5%), reduce the dependence on fixed icons, and eliminate manual navigation operations, thereby ensuring the data loading accuracy of 100%.

[0081] Step S102: cutting a chromatogram area containing a chromatographic peak type line graph from the target chromatogram;

[0082] This step can be specifically as shown in Figure 3 For cutting the chromatogram area, an artificial neural network model can be used for automatic recognition and cutting, or the cutting can be performed according to preset coordinates. The preset coordinates can be flexibly set according to the software interfaces of various manufacturers. For example, for the manufacturer Agilent, the preset coordinates can be x=300-900px, y=200-400px.

[0083] Step S103: analyzing the chromatogram area according to a matching peak integration configuration rule to obtain target peak dragging data;

[0084] This step can be specifically as shown in Figure 4 In the process of obtaining the target peak dragging data, the target peak type position is determined, and a marking range is generated. The peak integration can include the following three types of peak integrations:

[0085] Normal position peak integration: the peak type is searched within a fixed range according to the time axis without offset;

[0086] Interval peak integration: a time is artificially set, and the corresponding peak type is searched within the time interval;

[0087] Template peak integration: according to image recognition, the peak type of the cut picture is compared with the template picture, and then drawing is performed.

[0088] As an optional embodiment, in the step S103, the matching peak integration configuration rule comprises normal position peak integration, interval peak integration scanning, and / or template peak integration comparison, wherein:

[0089] The positive peak matching is locking the peak in a preset retention time window. In practice, the preset retention time window can be a set retention time before and after a certain time (such as 0.1 min, 0.3 min, 0.5 min, 1 min, etc.), which can be flexibly set according to requirements.

[0090] The interval peak scanning is scanning the local maximum value in the specified time interval by using the OpenCV algorithm, detecting the highest peak position, and performing dynamic noise suppression in the peak identification process to ensure the accuracy of the peak position identification.

[0091] The template peak comparison is using OpenCV to calculate the correlation coefficient of the current peak and the standard template. When the correlation coefficient is greater than the preset coefficient threshold, it is determined to be matched, so as to realize shape adaptive matching. In practice, the preset coefficient threshold can be flexibly set according to requirements, such as 0.6, 0.7, 0.8, 0.9, etc.

[0092] In this way, accurate positioning can be realized for different types of chromatographic peaks (fixed position peak, interval peak, and shape matching peak), and a marking range that can be used for redrawing operation is generated, and the peak positioning accuracy can reach more than 96%. The rule configuration can be referred to Figures 6-10 , wherein, Figure 6 corresponding to the template peak parameter setting on / off; Figure 7 corresponding to the template picture in the material folder; Figure 8 corresponding to the original peak situation, the peak is incorrect; Figure 9 corresponding to the peak situation after the template comparison peak, the peak is correct; Figure 10 corresponding to the peak rule configuration can be set separately according to different species.

[0093] As another optional embodiment, the step S103 comprises:

[0094] extracting the substance name corresponding to the atlas area by using a text recognition engine;

[0095] In practice, CNOCR can extract the substance name (such as "benzene").

[0096] As still another optional embodiment, the step S103 comprises:

[0097] analyzing the scale value of the atlas area by using OpenCV.

[0098] In practice, the retention time number can be identified.

[0099] Step S104: Redraw the chromatographic peak on the target software interface according to the target peak dragging data.

[0100] This step can be specifically as shown in Figure 5 .

[0101] As an optional embodiment, the step S104 comprises:

[0102] Step S1041: calculating a redraw path according to the target peak dragging data by using a Gaussian interpolation algorithm;

[0103] Step S1042: simulating mouse movement according to the redraw path to redraw the chromatographic peak;

[0104] Step S1043: recording the drawing path in real time by using OpenCV;

[0105] Step S1044: calculating the pixel coincidence degree of the redrawn peak and the target template, and determining that the redrawing is successful if the pixel coincidence degree is greater than a preset coincidence degree threshold.

[0106] In this step, the preset coincidence degree threshold can be flexibly set according to requirements, such as 80%, 85%, 90%, 95%, etc. The target template is specifically a local different template peak type picture folder established for different species. The software will compare the drawn peak with the peak type pictures in the picture folder, and if the coincidence degree reaches, the curve in the template peak type picture is used for positioning and redrawing.

[0107] Preferably, the step S1044 comprises:

[0108] If the pixel coincidence degree is less than or equal to the preset coincidence degree threshold, it is determined that the redrawing fails, the redraw path is regenerated, and the step S1042 is turned to.

[0109] In the step S1044, the regeneration of the redraw path preferably comprises:

[0110] Comparing the labeled peak with the peak type in the template peak type picture folder, if there is no peak type template in the template library that meets the preset coincidence degree threshold, no template comparison is performed, and the redraw path is calculated according to the peak type image recognition.

[0111] In this way, through the above steps S1041-S1044, the industrial level precision (error <2 pixels) can be ensured, the redrawing operation can be accurately traced, and the randomness of manual operation can be avoided.

[0112] Preferably, the step S1041 comprises:

[0113] Step S10411: extracting a peak dragging feature point sequence from the target peak dragging data;

[0114] In this step, the peak dragging feature point sequence is extracted. In specific implementation, the pixel point coordinate sequence {(x1, y1), (x2, y2)......(x n , y nThis sequence reflects the original contour morphology of the trailing peak.

[0115] Step S10412: Perform Gaussian filtering noise reduction on the peak feature point sequence;

[0116] This step involves Gaussian filtering for noise reduction. In practice, a one-dimensional Gaussian filter can be applied to the coordinate sequence to remove sharp abrupt changes and image noise during the image edge extraction process, thereby improving curve smoothness.

[0117] Step S10413: Construct a Gaussian interpolation function using the sequence of dragging peak feature points after Gaussian filtering and noise reduction;

[0118] This step involves constructing a Gaussian interpolation function. Specifically, a Gaussian kernel function can be used to fit the peak sequence to construct a continuous function f(x), whose expression is:

[0119] ;

[0120] in, , The original coordinates are... This is a bandwidth parameter that can be dynamically adjusted based on the peak length.

[0121] Step S10414: Perform equal-interval sampling on the Gaussian interpolation function to generate a discrete set of mouse drawing path points;

[0122] This step generates drawing points by discretizing the path. In practice, the continuous function f(x) can be sampled at equal intervals according to a preset step size Δx to generate a set of discrete mouse drawing path points {(x,f(x))}, so as to ensure fitting accuracy and smoothness of mouse movement.

[0123] Step S10415: Map the discrete mouse drawing path point set from the image coordinate system to the screen coordinates of the target software window to obtain the redrawing path.

[0124] This step involves coordinate normalization and window mapping. In practice, the above drawing points (i.e., the set of discrete mouse drawing path points) can be mapped from the image coordinate system to the screen coordinates of the target software window. In this way, the relative positioning method can be used to adapt to the software interfaces of different manufacturers, ensuring that the path remains consistent under cross-platform operation.

[0125] Thus, the calculation of the redrawing path can be completed through the above steps S10411-S10415, as follows: Figure 11 As shown, Figure 11 The process of transforming the peak data into the redraw path is clearly shown, where the red curve is the Gaussian interpolation fitting path and the blue dots are the generated discrete drawing points.

[0126] Further, to ensure the accuracy of the redrawn path, it is also necessary to analyze whether the actual peak type has a trailing peak. After analyzing the presence and degree of the trailing peak, the redrawn path is adjusted again and then drawn on the third-party software. That is, the step S10415 further includes:

[0127] Step S10416: Analyzing and optimizing the adjustment of the redrawn path.

[0128] In the implementation of this step, dynamic parameter analysis and optimization adjustment can be used, including the following steps:

[0129] Step A1: Calculate the peak height ratio;

[0130] In the implementation of this step, the target peak top coordinates (x p ,y p ) and the baseline height y_b are extracted from the image, and the peak height is calculated. Through normalization processing (for example, based on the full image height), the peak height ratio is obtained.

[0131] Step A2: Calculate the tailing factor;

[0132] In the implementation of this step, the half-peak width and the tail width are extracted from the peak image, and the tailing factor is calculated, which is used to identify non-ideal trailing peaks.

[0133] Step A3: Correct the redrawn path according to the peak height ratio and the tailing factor.

[0134] In the implementation of this step, according to the peak height ratio H_norm and the tailing factor T, the drawing speed, path density, and shape correction algorithm are determined. For example, the rules are as follows:

[0135] If T>1.5 (severe trailing), a "tailing correction sub-path" is automatically inserted in the redrawn path;

[0136] If H_norm<0.2 (small peak), the redrawn path is set to a low-resolution mode to reduce overfitting errors;

[0137] For obvious asymmetric peaks, the template library is called for shape matching and the redrawn path is generated according to the standard template.

[0138] In this way, the redrawn path after the optimization adjustment obtained through the above steps A1-A3 is a sequence of coordinate path points and path behavior parameters (drawing speed, curvature adjustment points, etc.), which is used for subsequent simulation mouse call drawing.

[0139] As another alternative embodiment, step S104 further comprises:

[0140] Step S1040: performing drag peak optimization on the target drag peak data.

[0141] Preferably, step S1040 comprises:

[0142] Secondly, checking the distance between the peak top and the baseline, and excluding noise peaks with an amplitude less than a preset amplitude threshold.

[0143] The above step S1040 can ensure a correct rate of 90% (only 30% for traditional methods), and can effectively exclude false drawings.

[0144] As still another alternative embodiment, the method further comprises:

[0145] Step S105: repeatedly performing steps S101 to S104 until the chromatographic peaks of all substances in the target data are redrawn.

[0146] This step can realize redrawing of the chromatographic peaks of all substances in the target data, and can realize automatic redrawing and support unattended batch processing.

[0147] The method of the present application will be described in detail below in combination with a specific embodiment.

[0148] As shown in Figure 12 the overall flow in this embodiment is generally divided into 17 steps, which are as follows:

[0149] 1. Start the program;

[0150] 2. Configure the basic parameter interface;

[0151] 3. Export the source data to a specified location;

[0152] 4. Import the target data to a specified location;

[0153] 5. Open the target data;

[0154] 6. Locate the target map;

[0155] 7. Analyze the target data (obtain substance name, retention time, map pixel, time axis information, etc.);

[0156] 8. Match the integral peak configuration rules (such as positive integral peak, interval integral peak, template integral peak, etc.);

[0157] 9. According to the corresponding rules, analyze the map data to obtain the target drag peak data in the map;

[0158] 10. Optimize the drag peak data to ensure the correctness of the drag peak position;

[0159] 11. Simulate drawing operation;

[0160] 12. Analyze drawing results, drawing success or failure;

[0161] 13. Save process record;

[0162] 14. Switch substances, perform next substance operation, restart cycle from step 6;

[0163] 15. Close target data;

[0164] 16. Export target report;

[0165] 17. Switch target data, perform next source data export to designated location, restart cycle from step 3 until data processing is complete.

[0166] According to the above-mentioned cycle, the automatic process is completed. The above-mentioned steps are specifically explained as follows:

[0167] Step 1-2: Start and configure

[0168] Technical implementation:

[0169] Load instrument-specific configuration file (Agilent / Shimadzu / Thermo, etc.);

[0170] Initialize CNOCR engine (lightweight model densenet_lite_136-gru);

[0171] Effect:

[0172] Provide parameter benchmark for subsequent operation, initialization time <1 second.

[0173] Step 3-5: Data preparation

[0174] Technical implementation:

[0175] Source data is exported to a fixed path (such as D: / source / );

[0176] Target data is imported from D: / source / ;

[0177] Data is imported into a fixed folder directory, and then the directory is opened from a third-party software to perform automatic process operation on the files.

[0178] CNOCR locates the "Open" button: confidence > 85%, click coordinates;

[0179] Effect:

[0180] Eliminate manual navigation operation, data loading accuracy 100%.

[0181] Steps 6-7: Spectral Analysis

[0182] Technical Implementation:

[0183] Extract the map region according to the preset coordinates (Agilent: x=300-900px, y=200-400px).

[0184] CNOCR extracts the name of the substance (e.g., "benzene").

[0185] OpenCV parsing of tick values: identifying retention time numbers;

[0186] Effects:

[0187] Substance recognition rate > 98% (thousands of tests), time < 3 seconds.

[0188] Step 8: The peak matching rules can be shown in Table 1:

[0189] Table 1

[0190]

[0191] Steps 9-10: Peak dragging optimization

[0192] Technical Implementation:

[0193] Within the identified peak range, analyze the tail extension characteristics of the peak and extract the trailing peak data (including parameters such as trailing length, trailing factor, and tail morphology).

[0194] The distance between the peak and the baseline is verified twice to exclude noise peaks with an amplitude of less than 5%.

[0195] Effects:

[0196] The accuracy rate reached 90% (compared to 30% for traditional methods).

[0197] Steps 11-12: Redraw and Verify

[0198] Technical Implementation:

[0199] Move the mouse along a preset path (CSV coordinate sequence);

[0200] OpenCV captures trajectories in real time;

[0201] Calculate the overlap between the redrawn peak and the template pixels:

[0202] = \frac{\text{overlapping pixels}}{\text{total pixels}} \times 100%>95\.

[0203] Effects:

[0204] Ensure industrial-grade precision (error < 2 pixels);

[0205] Steps 13-17: Cycle and output

[0206] Technical implementation:

[0207] Material cycle: CNOCR locates the "Next" button to switch materials;

[0208] Data cycle: Repeat steps 3-14 to process new data sources;

[0209] Report generation: Summary of success rate statistics (PDF format);

[0210] Effect:

[0211] Support unattended batch processing (continuous operation ≥ 72 hours).

[0212] The method of the present application is compared with the following comparative examples / alternatives to demonstrate the advantages of the present application.

[0213] I. Alternative 1 - Absolute coordinate macro control

[0214] Technical implementation:

[0215] Pre-recorded screen absolute coordinates of manual operation trajectory (e.g. click (120, 240) → drag to (300, 500)), execute fixed coordinate sequence.

[0216] Application scenario:

[0217] Only supports Agilent ChemStation v2.1 version under specific resolution (1920x1080).

[0218] Disadvantages and deficiencies:

[0219] 1. Zero fault tolerance:

[0220] Window position offset > 5 pixels, i.e. click the wrong button;

[0221] 100% failure due to vendor software version update (e.g. v2.1 → v2.2).

[0222] 2. Chromatographic peak redrawing failure:

[0223] Cannot recognize peak shape changes, still performs fixed path redrawing on tailing peaks.

[0224] 3. High maintenance cost:

[0225] Each new vendor requires re-recording of the entire set of coordinates.

[0226] The advantages of the main scheme of the present application are shown in Table 2:

[0227] Table 2

[0228]

[0229] II. Alternative 2: Static Template Matching

[0230] Technical Implementation:

[0231] Store the "ideal template" image of the chromatographic peak, compare the similarity of the current peak shape through the SSIM algorithm, and trigger the redrawing if the similarity is lower than the threshold.

[0232] Matching Logic:

[0233]

[0234] If the similarity is less than 0.8, it is determined as an abnormality.

[0235] Application Scenario:

[0236] It is suitable for standard sample detection with stable and unchanged peak shape, but cannot handle actual sample fluctuations.

[0237] Disadvantages and Inadequacies:

[0238] 1. Dynamic Failure:

[0239] Peak shape drift > 10% (common in environmental VOC samples) is a false judgment;

[0240] Drag peak factor changes cannot be detected.

[0241] 2. Instruction Rigidity:

[0242] Only "overall redrawing" can be performed, and it cannot be accurately corrected for local peak defects (such as tailing).

[0243] 3. Template Inflation:

[0244] Hundreds of templates need to be maintained for different peak types, with storage overhead > 1GB.

[0245] The advantages of the main scheme of the present application are shown in Table 3:

[0246] Table 3

[0247]

[0248] Dynamic template updating: that is, image comparison is added, different templates can be built for different peak types through image comparison, and the peak type template folder can be directly put into the template peak type picture;

[0249] Local operation + instruction compiler: The software will patrol the template folder under the picture comparison, in the drawing, will be according to the path of the picture in the drawing copy positioning redrawing, no need to rejudge the whole redrawing;

[0250] Convolution feature extraction: the basic way is to store the peak type logic judgment in the code, and draw in a fixed way. If there is a tail or abnormal peak type that is not in the logic, it needs to be modified again. After adding image comparison, it can be directly compared and drawn according to the comparison threshold. Storage only needs to put the picture.

[0251] Three, Beta (test version) version description:

[0252] Beta version 1: single OCR positioning engine (V1.0)

[0253] Technical scheme:

[0254] Only CNOCR recognition control text (no OpenCV backup scheme);

[0255] Accumulative peak only supports positive position accumulation peak rule (fixed time window matching);

[0256] Redrawing without verification link.

[0257] Advantages (feasibility): short development cycle; can handle standard products.

[0258] Disadvantages and limitations:

[0259] Chinese character recognition rate is only 75% (such as "peak area" misrecognized as "peak face").

[0260] Interval accumulation peak / template accumulation peak function is missing → complex sample missing rate > 40%;

[0261] Redrawing results cannot be verified → error rate > 25%.

[0262] Beta version 2: OpenCV redrawing verification prototype (V1.5)

[0263] Technical scheme:

[0264] Add OpenCV redrawing verification module;

[0265] Introduce interval accumulation peak rule (local maximum value scanning);

[0266] Keep single OCR positioning.

[0267] Advantages (feasibility):

[0268] Redrawing results can be quantitatively verified (pixel coincidence degree > 90% to determine success);

[0269] Interval peak stacking improves the detection rate of complex samples.

[0270] Disadvantages and limitations:

[0271] No backup plan when OCR fails → control positioning success rate is only 82%;

[0272] Template peak stacking is not implemented → irregular peak shape cannot be processed;

[0273] Manual material switching → cannot handle batch continuous processing.

[0274] The main scheme of the present application is the iterative breakthrough process of the Beta version as shown in Table 4:

[0275] Table 4

[0276]

[0277] Four, another embodiment of the method of the present application is a hardware coordinate mapping based mechanical arm control, and the technical scheme is described as follows:

[0278] No windows software is used, and the hardware scheme is implemented, specifically:

[0279] First, the control software needs to be separately deployed for the industrial camera, sensor and mechanical arm.

[0280] The control software needs to operate the industrial camera to capture the video interface of the industrial computer third-party software, and after image recognition, the laser displacement sensor is used to locate the icon, folder directory, coordinate, etc., and after positioning, the mechanical arm holds the mouse to perform automatic process. The structural composition is shown in Table 5:

[0281] Table 5

[0282]

[0283] Technical means:

[0284] 1. Camera capture software interface → 2. Laser sensor positioning screen absolute coordinate → 3. Mechanical arm holding physical mouse to execute click.

[0285] The working principle is as shown in Figure 13 .

[0286] The ability to achieve the purpose of the invention is shown in Table 6:

[0287] Table 6

[0288]

[0289] This embodiment has the following defects:

[0290] High cost: hardware cost > main scheme pure software;

[0291] No intelligent decision: only pre-recorded coordinates can be executed, and no abnormal chromatographic peak can be identified;

[0292] Destructive deployment: laser reflection markers need to be installed on the instrument screen.

[0293] In summary, the key technical points of the image recognition-based cross-platform instrument software data review and redrawing method of the application are as follows:

[0294] 1. Dynamic feature-instruction conversion mechanism

[0295] The core logic of converting chromatographic pattern features (peak height / peak width / tailing factor) into user operation instructions (mouse path / keyboard sequence) in real time.

[0296] 2. Multi-rule collaborative decision architecture

[0297] Supporting engine architecture for multiple chromatographic peak identification rules (time window matching / noise suppression scanning / shape comparison);

[0298] Each rule can be independently or combined to take effect.

[0299] 3. Operation-verification forced closed loop

[0300] Trajectory capture and pixel-level error quantification verification (coincidence degree > threshold) after executing operation instructions.

[0301] 4. Configurable cross-platform adaptation

[0302] Decoupling vendor software features (window coordinates / control features / operation logic) through configuration files.

[0303] Therefore, the application has the following beneficial effects:

[0304] (I) Direct technical effect

[0305] 1. High-precision control positioning:

[0306] Through the CNOCR+OpenCV double fault-tolerant mechanism (confidence threshold triggers backup scheme), the control recognition failure problem in a multi-language environment is completely solved, and the cross-platform operation robustness is significantly improved.

[0307] 2. Broad-spectrum peak integration identification capability:

[0308] Three-mode peak integration decision engine (positive time window matching / interval peak scanning / template correlation coefficient comparison) comprehensively covers industrial complex sample scenarios, and significantly improves the detection capability of unconventional peak shapes.

[0309] 3. Industrial-grade redrawing precision guarantee:

[0310] The drawing-capturing-verification closed-loop system ensures quantifiable traceability of each redrawing operation, eliminates the randomness of manual operations, and meets the data compliance requirements of instrumental analysis.

[0311] 4. Full-process automation:

[0312] The material / data dual-cycle architecture realizes full-link unattended operation from data import to report generation, greatly compressing the single-sample processing period.

[0313] (2) Industry-level chain reaction:

[0314] 1. Structural optimization of labor cost:

[0315] Full-process automation replaces manual review positions, freeing up high-skilled labor to focus on value-creating tasks and restructuring human resource allocation.

[0316] 2. Revolutionary improvement in detection throughput:

[0317] Batch continuous processing breaks through the bottleneck of traditional single-machine operation, significantly improving the utilization rate of instruments and equipment, and achieving a magnitude leap in unit capacity.

[0318] 3. Reconstruction of data credibility system:

[0319] The traceable redrawing verification mechanism meets the core requirements of the laboratory certification system and provides technical-level support for the judicial acceptance of detection reports.

[0320] 4. Collapse of cross-platform adaptation costs:

[0321] The configuration file-driven rapid adaptation solution compresses the support period for new instruments to less than 20% of traditional solutions, accelerating the expansion of the detection network.

[0322] (3) Deep social value:

[0323] 1. Environmental governance efficiency upgrade:

[0324] The improvement of VOC detection efficiency directly supports the accurate analysis of key regional pollution sources and helps achieve the air pollution prevention and control target ahead of schedule.

[0325] 2. Breaking the ecological barrier of scientific instruments:

[0326] Domestic instrument software accesses the automation system through standardized control interfaces, promoting the process of industrial self-control.

[0327] 3. Perfecting the chain of judicial evidence:

[0328] The zero-human intervention data production process meets the legal requirements of the Ecological Environment Monitoring Regulations for data authenticity, strengthening the effectiveness of environmental law enforcement evidence.

[0329] 4. International standard speech right promotion:

[0330] The automation operation chain with industrial grade precision provides a technical implementation example for international standards such as ISO 11095.

[0331] The above-described embodiments are merely preferred embodiments of the present application and are not intended to limit the scope of the present application. Various modifications and improvements to the technical solutions of the present application made by those of ordinary skill in the art without departing from the design spirit of the present application shall fall within the scope of protection of the present application.

Claims

1. An image recognition based cross-platform instrument software data review and redrawing method, characterized in that, The method comprises the following steps: Step S101: The target software opens the target data, so that the target software displays a target atlas; Step S102: A chromatographic peak type line chart containing atlas region is intercepted from the target atlas; Step S103: The atlas region is analyzed according to a matching integral peak configuration rule, and target peak dragging data is obtained; Step S104: The chromatographic peak is redrawn on the target software interface according to the target peak dragging data. In the step S103, the matching integral peak configuration rule comprises: positive integral peak matching, interval integral peak scanning and template integral peak comparison, wherein: The positive integral peak matching is to lock the peak value within a preset retention time window; The interval integral peak scanning is to scan the local maximum value by using OpenCV, and to detect the highest peak within a specified time window; The template integral peak comparison is to calculate the correlation coefficient of the current peak and the standard template by using OpenCV, and when the correlation coefficient is greater than a preset threshold value, it is determined that the matching is successful; The step S104 comprises: Step S1041: A redraw path is calculated according to the target peak dragging data by using a Gaussian interpolation algorithm; Step S1042: The chromatographic peak is redrawn by simulating the movement of a mouse according to the redraw path; Step S1043: The drawing path is recorded in real time by using OpenCV; Step S1044: The pixel coincidence degree of the redrawn peak and the target template is calculated, and if the pixel coincidence degree is greater than a preset threshold value, it is determined that the redrawing is successful.

2. The image recognition based cross-platform instrument software data review and redrawing method of claim 1, wherein, The step S101 comprises: Step S1011: A target software interface after the target software is started is obtained; Step S1012: A relative coordinate system is established by recognizing a feature marker in the target software interface; Step S1013: A target coordinate is obtained by positioning an "open" button in the target software interface by using a text recognition engine; Step S1014: It is determined whether the confidence degree of the target coordinate is greater than a preset threshold value, and if yes, the target coordinate is simulated to be clicked by a mouse to open the target data.

3. The image recognition based cross-platform instrument software data review and redrawing method of claim 2, wherein, The step S1012 comprises: Step S10121: Interface feature markers are extracted in the target software interface according to a pre-constructed configuration file, wherein the configuration file defines the interface feature marker templates and their default offsets in each manufacturer software; Step S10122: The interface feature markers are positioned in the target software interface according to the configuration file; Step S10123: The relative coordinate system is established by using the interface feature markers.

4. The image recognition based cross-platform instrument software data review and redrawing method of claim 2, wherein, The step S1014 comprises: If the confidence degree of the target coordinate is less than or equal to the preset threshold value, the target coordinate of an icon feature of the "open" button in the target software interface is obtained by using OpenCV, and the target coordinate is simulated to be clicked by a mouse to open the target data.

5. The image recognition based cross-platform instrument software data review and redrawing method of claim 1, wherein, The step S103 comprises: The material name corresponding to the atlas region is extracted by using a text recognition engine; And / or, the scale value of the atlas region is analyzed by using OpenCV.

6. The image recognition based cross-platform instrument software data review and redrawing method of claim 1, wherein, The step S1041 comprises: Step S10411: A peak dragging feature point sequence is extracted from the target peak dragging data; Step S10412: The peak dragging feature point sequence is subjected to Gaussian filtering and noise reduction processing. Step S10413: Construct a Gaussian interpolation function by using the denoised and de-blobbed feature point sequence; Step S10414: Perform equidistant sampling on the Gaussian interpolation function to generate a discrete mouse-drawing path point set; Step S10415: Map the discrete mouse-drawing path point set from the image coordinate system to the screen coordinate of the target software window to obtain the redrawn path.

7. The image recognition based cross-platform instrument software data review and redrawing method of claim 1, wherein, The step S1044 includes: If the pixel coincidence degree is less than or equal to the coincidence degree preset threshold, it is determined that the redrawing fails, the redrawn path is regenerated, and the step S1042 is turned to; The regeneration of the redrawn path includes: Comparing the labeled peak with the peak type in the template peak type image folder, if there is no peak type template in the template library that meets the coincidence degree preset threshold, the redrawn path is directly recognized and calculated according to the peak type image.

8. The image recognition based cross-platform instrument software data review and redrawing method of claim 1, wherein, The step S104 further includes: Step S1040: Perform peak-drag optimization on the target peak-drag data; The step S1040 includes: Secondly, check the distance between the peak top and the baseline, and exclude noise peaks with an amplitude less than the amplitude preset threshold.

9. The image recognition based cross-platform instrument software data review and redrawing method of claim 1, wherein, The method further includes: Step S105: Repeat the steps S101 to S104 until the chromatographic peak of all substances in the target data is redrawn.

Images