A method, device, medium and product for generating an Excel data report based on image recognition

Abstract

This application relates to the field of information technology and discloses a method, device, medium, and product for generating Excel data reports based on image recognition. The method includes: obtaining machine-coded text information based on an oscilloscope image and OCR technology; determining the image type of the oscilloscope image based on image feature identifiers in the machine-coded text information; extracting parameter names and corresponding values ​​from predefined data area coordinates according to the image type to form key-value pairs of structured data; and adding the structured data to a workbook according to autofill rules to obtain the target Excel data report.

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to a method, device, medium and product for generating Excel data reports based on image recognition. Background Technology

[0002] Currently, in numerous industries and fields such as electronic engineering and hardware development, scientific experiments, automotive electronics, quality control and production testing, medical, and logistics, a large amount of data from images needs to be collected and processed. For example, engineers in the electronic engineering and hardware development industry need to record the waveform parameters captured by the oscilloscope as electronic data when debugging circuit boards; this process is usually done manually. In recent years, even though some organizations have introduced OCR systems, manual intervention is still required for review and supplementary recording of documents with low system recognition confidence, abnormally complex formats, or unclear text.

[0003] The inventors discovered that the related technologies have at least the following technical problems: In recording oscilloscope image parameters, manual reading and input of parameters is very time-consuming. Long-term repetitive work inevitably leads to misreading and incorrect input. Furthermore, low efficiency and high error rate directly increase costs. Even with rigorous secondary verification, it is impossible to guarantee that all errors can be detected. Summary of the Invention

[0004] One objective of this application is to provide a method for generating Excel data reports based on image recognition, which at least solves the problems of low efficiency and high error rate caused by relying on manual input of oscilloscope image parameters.

[0005] To achieve the above objectives, some embodiments of this application provide the following aspects:

[0006] In a first aspect, some embodiments of this application provide a method for generating Excel data reports based on image recognition. The method includes: obtaining machine-coded text information based on an oscilloscope image and OCR technology; determining the image type of the oscilloscope image based on image feature identifiers in the machine-coded text information; extracting parameter names and corresponding values ​​from predefined data region coordinates according to the image type to form key-value pairs of structured data; and adding the structured data to a workbook according to autofill rules to obtain a target Excel data report.

[0007] Secondly, some embodiments of this application also provide an electronic device, the electronic device comprising: one or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the method described above.

[0008] Thirdly, some embodiments of this application also provide a computer-readable medium having computer program instructions stored thereon, which can be executed by a processor to implement the method described above.

[0009] Fourthly, some embodiments of this application also provide a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method described above.

[0010] Compared with related technologies, the solution provided in this application utilizes OCR technology to convert oscilloscope images into machine-coded text information, which can save time for manual image observation to a certain extent. Furthermore, since different oscilloscope image types require different parameters and corresponding values, image feature identifiers are obtained based on the machine-coded text information. The image type of the oscilloscope image can then be determined based on these identifiers. Parameters and corresponding values ​​are then extracted from the target data area according to the determined image type, resulting in structured data. This method of determining the image type through image feature identifiers and then identifying the target data area based on the image type to form structured data eliminates the need for manual identification of image types, parameter locations, and data, saving time and preventing misreading and incorrect input. Finally, according to autofill rules, the structured data is automatically added to the corresponding position in the workbook to obtain the target Excel data report. In this way, the process of oscilloscope image recognition and parameter entry eliminates the need for manual identification, filling, or verification, improving efficiency, reducing the rate of misreading and incorrect input caused by repetitive manual work, and saving costs. Attached Figure Description

[0011] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0012] Figure 1 An exemplary flowchart of a method for generating Excel data reports based on image recognition is provided for some embodiments;

[0013] Figure 2 Schematic diagrams of oscilloscope images provided for some embodiments;

[0014] Figure 3 Schematic diagrams of oscilloscope images provided for some embodiments;

[0015] Figure 4 This is a schematic diagram of the structure of an electronic device provided for some embodiments. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] The following terms are used in this document.

[0018] OCR, Optical Character Recognition;

[0019] RMS, Root Mean Square, is also often referred to as the "effective value".

[0020] First Embodiment

[0021] The first embodiment of this application relates to a method for generating Excel data reports based on image recognition. For example... Figure 1 As shown, the method may include the following steps:

[0022] Step S10: Obtain machine-encoded text information based on the oscilloscope image and OCR technology;

[0023] Step S20: Determine the image type of the oscilloscope image based on the image feature identifier in the machine-encoded text information;

[0024] Step S30: Based on the image type, extract parameter names and corresponding values ​​from the target data region to form key-value pair structured data;

[0025] Step S40: According to the autofill rules, add the structured data to the workbook to obtain the target Excel data report.

[0026] The following sections will provide a detailed explanation of each of the above steps.

[0027] Regarding step S10, specifically, the oscilloscope image can be a screenshot or a captured image of the oscilloscope screen. The process of generating the oscilloscope image can be as follows: the oscilloscope probe acquires the electrical signal at a certain test point in the circuit and converts the electrical signal into a stable waveform image on the screen. Machine-coded text information is data obtained from the oscilloscope image using OCR recognition technology, which is convenient for automatic parsing and processing by a computer program. Specifically, machine-coded text information can be obtained by recognizing text and numerical information in the oscilloscope image using OCR technology, and then converting the text and numerical information into strings and numbers that the computer can understand and process. In this embodiment, the machine-coded text information can include text and numerical information from the oscilloscope image.

[0028] The oscilloscope images provided in this application embodiment can be referenced. Figure 2 , Figure 3 .in, Figure 2 The waveforms and measurement data of two voltage signals are displayed, shown as yellow and blue waveforms respectively. The oscilloscope measured both voltage signals. Regarding parameter data, the oscilloscope measured both channels, focusing primarily on the "absolute voltage value (V)" and "time difference (Δt)". Figure 3 The waveform and measurement data of a current signal are displayed; in terms of parameter data, the oscilloscope mainly measures the "peak current (Max)", "minimum current (Min)" and "RMS value of current".

[0029] It is understandable that machine-encoded text information may include at least the above. Figure 2 , Figure 3 The parameters in the data include, for example, “absolute voltage (V)”, “time difference (Δt)”, “peak current (Max)”, “minimum current (Min)”, and “RMS current”.

[0030] Specifically, in step S20, the image feature identifier can be a key parameter in the machine-coded text information that characterizes the image type of the oscilloscope image, including text, numbers, and combinations of text and numbers. In this embodiment, the image type of the oscilloscope image is used to represent the category of the waveform in the oscilloscope image. In this step, the image feature identifier is identified in the machine-coded text information, and then the image type of the oscilloscope image is determined based on the identified image feature identifier.

[0031] Regarding step S30, specifically, based on the image type determined in the previous step, key parameters in the oscilloscope image of interest can be further obtained, and the region in the oscilloscope image containing the key parameters constitutes the target data region. Figure 3For example, the key parameter names in the target data area could be "Max" in the second row and "RMS" in the fourth row of the table in the lower left corner, with corresponding values ​​of "2.28" and "917.0m" respectively. Key-value pairs, as a general data structure, can store parameter names and corresponding values ​​in "key" and "value" respectively. Then, the extracted key-value pairs can be organized according to a certain logic to form structured data. By extracting the truly needed key parameters from the redundant information in the oscilloscope image, and repeatedly extracting key-value pairs to form structured data such as JSON data, the structured data can be directly accessed by databases or data analysis software.

[0032] Specifically, in step S40, the autofill rule is a predefined instruction that specifies how the data in the structured data is entered and where it is entered in the workbook. In this step, the structured data is added to the workbook through the autofill rule, resulting in a complete target Excel data report without manual intervention, thus automating data processing.

[0033] Understandably, existing oscilloscope image data processing methods rely on manual recognition and transcription. Although a few organizations have introduced OCR technology for simple recognition, followed by manual data selection and entry, this can improve work efficiency to some extent. However, daily work requires a large amount of oscilloscope data, which necessitates inputting dozens or even hundreds of data points into an Excel spreadsheet. This task is handled by dedicated personnel, and manual processing not only takes up a lot of time but also restricts the accuracy and timeliness of the data. The inventors discovered through research that recognition and entry is a repetitive task, and that automated recognition and entry can be achieved for specific types of oscilloscope images.

[0034] It is easy to see that, compared with related technologies, the solution provided in this application converts text and numerical information in oscilloscope images into machine-encoded text information using OCR recognition technology, enabling computers to understand and process it. Using image feature identifiers in the machine-encoded text information as the basis for determining the image type of the oscilloscope image, this automatic classification method helps handle massive amounts of oscilloscope screenshots, and "labeling" image types facilitates subsequent searching and management. The process of identifying target data areas from redundant oscilloscope images and then extracting key-value pairs to form structured data simplifies the process, helping to create standardized data that can be directly accessed. Finally, automated data entry rules are defined, and the structured raw data is transferred to the workbook automatically. The entire process requires no manual intervention, achieving end-to-end automated processing and eliminating the inefficiency and error-prone nature of manual data processing.

[0035] Second Embodiment

[0036] The second embodiment of this application relates to a method for generating Excel data reports based on image recognition. The second embodiment is an improvement upon the first embodiment, specifically in that it provides a method for determining the image type of an oscilloscope image based on machine-encoded text information.

[0037] Furthermore, in one embodiment, machine-encoded text information can be obtained based on the oscilloscope image and OCR technology, i.e., step S10 may include:

[0038] Step S101: Perform preprocessing operations on the oscilloscope image to obtain a preprocessed image; the preprocessing operations include at least one of grayscale conversion, binarization, noise removal, image rotation correction, and perspective transformation;

[0039] Step S102: Recognize the preprocessed image using OCR technology to obtain machine-encoded text information.

[0040] Specifically, in step S101, a preprocessed image is obtained by performing preprocessing operations on the oscilloscope image. These preprocessing operations include at least one of the following: grayscale conversion, binarization, noise removal, image rotation correction, and perspective transformation. The purpose of these preprocessing operations is to transform the original oscilloscope image into a standardized image to facilitate subsequent OCR recognition. In the preprocessing operations, grayscale conversion converts the color image to a grayscale image, changing each pixel's red, green, and blue values ​​to a single grayscale value to reduce processing complexity. Binarization, based on grayscale conversion, simplifies the image into foreground and background, eliminating grayscale transition bands to facilitate clear text outline separation by OCR. Noise removal primarily uses filtering algorithms to remove noise from the image. Image rotation correction detects and corrects the overall tilt angle of the image to ensure the text is level. Perspective transformation corrects distortion caused by incorrect shooting angles. After a series of preprocessing operations, the resulting preprocessed image is then used for OCR recognition, improving recognition accuracy.

[0041] Specifically, in step S102, after obtaining the preprocessed image, the OCR performs text and number information detection and recognition on the preprocessed image and outputs machine-encoded text information.

[0042] Optionally, in some embodiments, after obtaining the machine-encoded text information, the image type of the oscilloscope image can be further determined based on the image feature identifiers in the machine-encoded text information. That is, step S20 may include:

[0043] Step S201: Extract image feature identifiers from the machine-encoded text information;

[0044] Step S202: Compare the image feature identifier with a preset type feature library; the type feature library contains time interval feature templates for the first type of oscilloscope image and extreme value feature templates for the second type of oscilloscope image.

[0045] Step S203: Determine the category to which the image feature identifier belongs based on the comparison results.

[0046] Specifically, for step S201, key parameters are extracted from the machine-encoded text information as image feature identifiers, and a set of these image feature identifiers is output. As mentioned in the description of step S20, the image feature identifiers come from key parameters in the machine-encoded text information that characterize the image type of the oscilloscope image. For example, referring to... Figure 2 and Figure 3 Key parameters can be "VCC", "STB", "Max", "Min", "RMS", "5.0ms / div", "3.44mV", etc.

[0047] Specifically, in step S202, after obtaining the set of image feature identifiers, the image feature identifiers are compared with a preset type feature library. The preset type feature library is a predefined library containing different standard feature templates, specifically including time interval feature templates for first-type oscilloscope images and extreme value feature templates for second-type oscilloscope images.

[0048] The first type of oscilloscope image time interval feature template contains a unit-value correlation weight matrix. This template primarily focuses on time-related parameters. The unit-value correlation weight matrix has two predefined dimensions: the first dimension, "time unit," defines different time units, and the second dimension, "value," defines a value range for each time unit. Different time units and their corresponding values ​​have different weights in the matrix. For example, when using the first type of oscilloscope image time interval feature template, based on the extracted image feature identifier set, the cell where the unit and value range intersect in the matrix is ​​located to obtain the weight result. The weights of one or more parameters are then summed to obtain a total score. This weight matrix scoring method avoids rigid classification or grading judgments, and even for non-typical unit-value types, reasonable results can be given through weighting.

[0049] The second type of oscilloscope image's extreme value feature template contains a combined confidence parameter. The extreme value feature template primarily focuses on parameters related to voltage or current intensity, and the combined confidence parameter is a confidence parameter obtained by evaluating image feature identifiers using preset rules. For example, the preset rules can include multiple rules: rule A corresponds to a 30% increase in confidence for the presence of the "Max" parameter, and rule B corresponds to a 40% increase in confidence for the presence of the "RMS" parameter. If an image feature identifier triggers both rule A and rule B simultaneously, then its total confidence relative to the extreme value feature template is 30% + 40% = 70%. Obtaining the confidence parameter for each image feature identifier based on each rule is intuitive and easy to maintain; if a new image type needs to be added, only the preset rules need to be modified.

[0050] Specifically, for step S203, the image feature identifiers extracted in step S201 can be substituted into the time interval feature template of the first type of oscilloscope image and the extreme value feature template of the second type of oscilloscope image for calculation. Then, the two scores are compared. If the total weight score of the time interval feature is higher than the confidence parameter of the identifier combination in the extreme value feature template, it is determined to be the first type of oscilloscope image; otherwise, it is determined to be the second type of oscilloscope image.

[0051] By employing multi-dimensional matching of unit-value correlation weight matrices and identifier combination confidence parameters, the limitations of misjudgment based on single features are overcome, improving classification accuracy. Furthermore, the system can intelligently identify the technology type of oscilloscope images and provide accurate information for subsequent analysis steps. This not only reduces the need for manual intervention but also provides a reliable technical foundation for the automated processing of large-scale, diverse oscilloscope data.

[0052] Optionally, in some embodiments, the step of extracting image feature identifiers from the machine-encoded text information, i.e., step S201, may include:

[0053] Step S2011: For the time interval identifier of the first type of oscilloscope image, extract the associated combination containing the time unit field and the interval value field;

[0054] Step S2012: Extract the maximum value identifier and the valid value identifier for the feature identifiers of the second type of oscilloscope image.

[0055] Specifically, for step S2011, the time unit field covers the full spelling and abbreviation of "s (seconds)," "ms (milliseconds)," "μs (microseconds)," and "ns (nanoseconds)," and achieves dynamic recognition of multiple time units through a character matching algorithm. The interval value field is a floating-point value with a decimal point, and satisfies a preset position association rule with the time unit field, such as the value field being located within a fixed character spacing to the left or right of the time unit field.

[0056] Specifically, for step S2012, the maximum value identifier is a combination of the prefix qualifier "Max", "maximum value" and its variant form with the peak symbol "↑", and the effective value identifier is a combination of the prefix qualifier "RMS", "effective value" and its variant form with the root mean square symbol "~". The maximum value identifier and the effective value identifier are presented in a hierarchical or parallel structure in the text information.

[0057] By carefully considering the full spelling and abbreviation of time units, prefixes, variations, and symbols, the system can process images from different versions of oscilloscopes. By defining positional association rules and association structures, rather than simply identifying keywords, irrelevant text information can be excluded, reducing the probability of erroneous extraction. This series of designs enables accurate capture of image feature identifiers in complex and non-standard text environments, thus ensuring the accuracy and reliability of subsequent processing.

[0058] Optionally, in some embodiments, comparing the image feature identifier with a preset type feature library, i.e., step S202, may include:

[0059] Step S2021: Perform multi-dimensional matching calculations on the image feature identifiers and feature data in the type feature library;

[0060] Step S2022: Based on the multi-dimensional matching calculation results, determine the similarity score between the image feature identifier and each feature data;

[0061] Step S2023: When the similarity score reaches a preset threshold, the image type to which the corresponding feature data belongs is taken as a candidate type;

[0062] Step S2024: By prioritizing and analyzing the confidence of multiple candidate types, the final matching type of the image feature identifier is determined.

[0063] Specifically, for step S2021, regarding the time interval identifier, the matching dimensions include time unit format matching degree, numerical format compliance, and positional correlation between the unit and the numerical value. For example, for the time interval representation, assuming a feature identifier (20.00, ns) is extracted, regarding the time unit format matching degree, it is necessary to check the degree of matching between the identified unit "ns" and the standard unit "ns" in the feature library. Since "ns" and "ns" match perfectly, the matching degree is 100%. Regarding the numerical format compliance, it is necessary to check whether the numerical value 20.00 conforms to the floating-point format and whether its value is within a reasonable range. Since 20.00 is a standard floating-point number, the compliance is 100%. Regarding the positional correlation between the unit and the numerical value, it is necessary to evaluate whether the relative positions of the numerical value and the unit in the original text conform to the preset positional correlation rules, such as the numerical value on the left and the unit on the right, with a spacing of no more than 2 characters. Since "20.00ns" is correct, the correlation degree is 100%.

[0064] Specifically, for the maximum value and valid value identifiers, the matching dimensions include prefix qualifier variant similarity, symbol combination integrity, and hierarchical structure consistency between identifiers. For example, assuming two feature identifiers are extracted: "Max:1.024A" and "RMS:577.3mA", regarding prefix qualifier variant similarity, the identified prefix needs to be compared with the standard prefix in the feature library. "Max" has a 100% similarity with "Max"; "RMS" has a 100% similarity after normalization. Regarding symbol combination integrity, auxiliary symbol evidence other than the text prefix needs to be found to enhance the confirmation. No nearby conventional symbols were found for either of the two feature identifiers. Regarding hierarchical structure consistency between identifiers, the structural relationship between Max and RMS in the original text layout is evaluated to see if it meets expectations. If it does, the consistency score is higher.

[0065] Specifically, for step S2022, the matching results of each dimension in step S2021 are weighted and summed to obtain the overall similarity score of the image feature identifier relative to the feature data in the type feature library.

[0066] Specifically, in step S2023, if the similarity score of any image feature identifier exceeds a preset threshold, then the image type to which it belongs is included in the candidate pool. The preset threshold can be 70%-100%. For example, when both the time interval identifier score and the extreme value identifier combination score exceed the preset threshold, then both the first type of oscilloscope image and the second type of oscilloscope image are listed as candidate types.

[0067] Specifically, in step S2024, priority ranking is used to determine which type should be given priority when an image feature identifier matches multiple candidate types. For example, the priority ranking rule could be: if an image feature identifier is unique to a certain type of image, while another image feature identifier is common, then the class containing the unique feature is given priority. Confidence analysis quantitatively evaluates each candidate type. Confidence is a comprehensive score derived from the similarity score calculated in step S2022, the weight score in the time interval feature template, and the identifier combination confidence parameter score in the extreme value feature template. When making the final decision, there are two scenarios: confidence-driven and priority ranking-driven. Confidence-driven is used when there is a significant difference in confidence levels, eliminating the need for priority ranking; the higher confidence level can be selected directly. Priority ranking is used when confidence levels are very close, where, based on business priority rules, extreme value class measurements are preferred over time interval class measurements.

[0068] By employing multi-dimensional matching calculations, we can move beyond relying on single keywords and avoid misjudgments caused by errors in recognizing individual characters in OCR. Correct classification is still possible through positional association. Furthermore, each step—multi-dimensional matching calculations, similarity scoring, and confidence analysis—has clear rules, allowing us to trace the basis for classification decisions during debugging or application. Prioritization and confidence analysis enable more rational decision-making, improving the automation and intelligence of data processing.

[0069] Third Embodiment

[0070] The third embodiment of this application relates to a method for extracting parameter names and corresponding values ​​from a target data region based on the image type. The third embodiment is an improvement upon the first and / or second embodiments, specifically in that it provides a method for extracting parameter names and corresponding values.

[0071] Optionally, in some embodiments, step S30, which involves extracting parameter names and corresponding values ​​from the target data region based on the image type, may include:

[0072] Step S301: Dynamically load the corresponding data region coordinate configuration file according to the image type of the oscilloscope image; wherein, the time interval data region coordinate configuration file of the first type of oscilloscope image is associated with the pixel coordinate range of the time axis scale line and includes coordinate offset parameters of numerical field and unit field; the maximum value or effective value data region coordinate configuration file of the second type of oscilloscope image is associated with the pixel coordinate mapping relationship of the waveform peak point and distinguishes the coordinate boundary between the maximum value region and the effective value region.

[0073] Step S302: Locate the target data region based on the coordinate parameters and image pixel ratio in the configuration file;

[0074] Step S303: Extract parameter names and corresponding values ​​from the target data region.

[0075] Regarding step S301, specifically, for different image types, the layout of key parameters on the oscilloscope screen varies, and the purpose of the data area coordinate configuration file is to record the layout pattern of key parameters. In particular, for the first type of oscilloscope image, the time interval data area can be concentrated in the center of the screen; for the second type of oscilloscope image, the maximum value or effective value data area can be concentrated at the bottom of the screen.

[0076] For example, assuming the key parameters of the first type of oscilloscope image are located in the lower left of the center of the image, the configuration file is as follows:

[0077] {

[0078] "data_region": {

[0079] "top_left_x": 900,

[0080] "top_left_y": 200,

[0081] "width": 300,

[0082] "height": 400

[0083] },

[0084] "value_unit_relationship": {

[0085] "x_offset": -60,

[0086] "y_offset": 0

[0087] }

[0088] }

[0089] Here, "data_region" represents the region containing the key parameters, defining a bounding box (900, 200, 300, 400). "value_unit_relationship" defines the coordinate relationship between the numeric and unit fields. "x_offset": -60 indicates that after finding the coordinates of a numeric field, moving 60 pixels to the left will find its unit field. "y_offset": 0 indicates that the numeric and unit fields are roughly on the same horizontal line. Therefore, the workflow is roughly as follows: first, the key parameters are identified within the bounding box; then, based on the configuration file, the unit fields are identified in the vicinity of (x-60, y); finally, the numeric and unit fields are combined. This configuration file allows for the automatic extraction of the required time parameters from complex information.

[0090] For example, assuming the key parameters of the second type of oscilloscope image are located in the lower left corner of the image, the configuration file is as follows:

[0091] {

[0092] "data_region": {

[0093] "top_left_x": 950,

[0094] "top_left_y": 150,

[0095] "width": 300,

[0096] "height": 400

[0097] },

[0098] "sub_regions": {

[0099] "max_region": {

[0100] "top_left_x": 950,

[0101] "top_left_y": 200,

[0102] "width": 300,

[0103] "height": 50

[0104] },

[0105] "rms_region": {

[0106] "top_left_x": 950,

[0107] "top_left_y": 280,

[0108] "width": 300,

[0109] "height": 50

[0110] }

[0111] }

[0112] }

[0113] Here, "sub_regions" distinguishes the coordinate boundaries between the maximum value region and the effective value region, "max_region" distinguishes the maximum value region, and "rms_region" distinguishes the effective value region. The pixel coordinate mapping relationship of the waveform peak points is a rule of the oscilloscope image UI layout, pre-fixed in the configuration file. In this example, "top_left_y":280" is the embodiment of the pixel coordinate mapping relationship. This avoids directly identifying the waveform image, transforming the image analysis problem into a coordinate addressing problem, and using "sub_regions" to distinguish different parameters, avoiding data misinterpretation.

[0114] Specifically, in step S302, the coordinate parameters in the configuration file are scaled proportionally according to the current image size and resolution to obtain a specific bounding box on the image, i.e., the target data region. Taking the example in step S301 as an illustration, for the first type of oscilloscope image, the large box defined by "data_region" is used as the target data region, and each pair of numerical fields and unit fields is located and associated according to the "value_unit_relationship" rule; for the second type of oscilloscope image, both the "max_region" box and the "rms_region" box can be used as the target data region.

[0115] Specifically, in step S303, after the target data area has been determined, OCR recognition technology is used again to identify and extract the data, and then simple text segmentation rules are used to extract the parameter names and corresponding values.

[0116] By defining the target data region and then extracting it, the method avoids recognizing the entire image and only processes the key parts. The configuration file solves the correlation problem in the OCR recognition process, and the region division in the configuration file prevents issues such as cross-referencing and misreading. It can be seen that even though different types of oscilloscopes have different interface layouts, as long as a corresponding configuration file is created, it can adapt to and process the corresponding image type.

[0117] Furthermore, in one embodiment, locating the target data region based on the coordinate parameters and image pixel ratio in the configuration file, i.e., step S302, may include:

[0118] For the first type of oscilloscope image, a region localization algorithm based on time series features is used to accurately locate the time interval data region;

[0119] For the second type of oscilloscope image, a region localization algorithm based on extreme value features is used to locate the maximum value data region and the effective value data region respectively;

[0120] The coordinate parameters in the configuration file are converted into the target region in the actual image through coordinate mapping.

[0121] Specifically, since the coordinate parameters in the configuration file are defined based on a specific resolution, but the actual resolution of the processed images may be different, it is necessary to map the theoretical coordinates to the coordinates of the actual processed images, and then select an appropriate algorithm for precise positioning for different image types.

[0122] Before localization, the mapping from theoretical coordinates to actual coordinates is completed. First, the width W_ref and height H_ref of the reference image are obtained from the configuration file. Then, the actual width W_actual and actual height H_actual of the current image are read, the scaling ratio is calculated, and finally, the coordinate mapping is completed.

[0123] The calculation method for scaling ratio is as follows:

[0124] scale_x = W_actual / W_ref;

[0125] scale_y=H_actual / H_actual;

[0126] Here, scale_x represents the scaling factor for the image width, and scale_y represents the scaling factor for the image height.

[0127] The calculation method for coordinate transformation for each coordinate point (x_ref, y_ref) in the configuration file is as follows:

[0128] x_actual = x_ref × scale_x;

[0129] y_actual = y_ref × scale_y;

[0130] Where x_actual and y_actual represent the actual coordinates after transformation.

[0131] After completing the actual coordinate localization, different algorithms are employed for different image types to address potential errors in the actual images. Specifically, for the first type of oscilloscope image, the region localization algorithm based on time series features initially locates the approximate area contained in the actual coordinates, then searches for time series-related keywords within that area. Once a keyword is identified, the time interval data area is located based on the combination of numerical and unit fields in the configuration file. For the second type of oscilloscope image, the region localization algorithm based on extreme value features first locates the maximum and effective values ​​separately, finding the corresponding values ​​within their respective areas. Then, the coordinate boundaries between the maximum and effective value areas in the configuration file are verified to locate the maximum and effective value data areas.

[0132] By employing coordinate mapping, the problem of different coordinates caused by varying screen resolutions is solved. Through a classification approach, different algorithms are used for the first and second types of oscilloscope images to ensure that the numerical values ​​and units in the first type of oscilloscope image are fully recovered, and that key parameters in the second type of oscilloscope image are accurately located within their respective regions. This improves device adaptability and achieves high-precision target data area positioning.

[0133] Optionally, in some embodiments, the step of extracting parameter names and corresponding values ​​from the target data region, i.e., step S303, may include:

[0134] Step S3031: Perform image enhancement processing on the target data area to improve the clarity of the text area;

[0135] Step S3032: Extract parameter names and corresponding values ​​from the target data region using multimodal text recognition technology;

[0136] Step S3033: Perform semantic consistency verification and format standardization on the extracted parameter names and values;

[0137] Step S3034: When the recognition result is ambiguous or erroneous, intelligent correction is performed based on contextual association analysis;

[0138] Step S3035: Generate standardized key-value pairs from the corrected parameter names and values ​​according to a preset data structure.

[0139] Specifically, in step S3031, an edge sharpening algorithm is used to highlight the character edges of numerical values ​​and units in the time interval region, and a contrast enhancement algorithm is used to distinguish prefix identifiers from symbols in the extreme value parameter region.

[0140] Specifically, in step S3032, a floating-point recognition model is used for the time interval values, and a symbol recognition model (recognizing "↑" and "~") and a text recognition model are combined for the extreme value parameters.

[0141] Specifically, in step S3033, the time interval values ​​are uniformly converted to the format of "abbreviation of standard unit of value", such as converting "500 microseconds" to "500μs". The extreme value parameters establish a binding and verification relationship of "prefix identifier - symbol - value", such as "Max" must be bound to "↑" and the peak value.

[0142] Specifically, in step S3034, the ambiguity of the time interval is verified by the reasonableness of the unit conversion, such as correcting "1000ms" to "1s". The extreme value error is verified by the correlation between the waveform pixel coordinates and the value. If the value is too large, the waveform vertex pixel position is matched and re-identified.

[0143] Specifically, in step S3035, the parameter names and values ​​corrected by the above steps are used to generate standardized key-value pairs according to a preset data structure.

[0144] Image enhancement processing improves the recognition accuracy of OCR. Based on the recognition results, domain knowledge is introduced to enhance data reliability, thus transforming raw text into standardized structured data.

[0145] Fourth embodiment

[0146] The fourth embodiment of this application relates to an autofill method. The fourth embodiment is an improvement upon any one or more of the first, second, and third embodiments, specifically in that it provides a method for filling structured data into an Excel spreadsheet according to autofill rules.

[0147] The automatic filling method provided in this embodiment is described below. The method described below can be arbitrarily combined with the methods in the first, second and third embodiments above.

[0148] Optionally, in some embodiments, the step of adding structured data to the workbook according to autofill rules to obtain the target Excel data report, i.e., step S40, may include:

[0149] Step S401: Initialize the workbook, check and unmerge the target cells;

[0150] Step S402: Based on the product number, traverse the data rows in the specified number column of the workbook; if a matching number is found, return the corresponding row number as the target row; if no matching number is found, return the maximum row number of the workbook plus one to create a new row.

[0151] Step S403: According to the mapping rules, fill the time interval values ​​into the R result data column and fill the maximum or valid value values ​​into the N result data column to obtain the target Excel data report.

[0152] Specifically, the process of adding structured data to the workbook according to autofill rules to obtain the target Excel data report can be divided into four stages:

[0153] Phase 1: Initialize the workbook, check and unmerge the target cells.

[0154] The purpose of this stage is to prepare a clean workbook environment for subsequent data writing. Unmerging cells prevents data from being written to merged cells, which could cause program errors. An example script configuration is shown below:

[0155] def ensure_workbook(excel_path):

[0156] """Open or create a workbook"""

[0157] if os.path.exists(excel_path):

[0158] wb = load_workbook(excel_path)

[0159] ws = wb.active

[0160] else:

[0161] wb = Workbook()

[0162] ws = wb.active

[0163] ws.title = "Sheet1"

[0164] # Before writing the table header, unmerge any merged cells first.

[0165] for (r, c, val) in [(1, 3, "ID"), (1, 13, "R result"), (1, 17, "N result")]:

[0166] unmerge_if_needed(ws, r, c)

[0167] if ws.cell(row=r, column=c).value in (None, ""):

[0168] ws.cell(row=r, column=c, value=val)

[0169] return wb, ws

[0170] Phase 2: Based on the product number, traverse the data rows in the workbook with the specified number column.

[0171] The purpose of this stage is to determine whether to update or create a new record based on the current data. An example script configuration is as follows:

[0172] def find_or_create_row(ws, code):

[0173] """Search for the row in column C based on the number; if not found, return to the next row."""

[0174] max_row = ws.max_row

[0175] for r in range(2, max_row + 1):

[0176] val = ws.cell(row=r, column=3).value # Column C

[0177] cell_code = (str(val).strip().upper() if val is not Noneelse "")

[0178] if cell_code == code:

[0179] return r

[0180] return max_row + 1

[0181] Phase 3: Fill the data into the specified result column according to the mapping rules.

[0182] The purpose of this stage is to populate the correct columns with different data. Before data entry, the configuration file for the mapping rules is loaded. The rules define that if the data is of the time-unit type, it should be entered into column R; if the data is of the maximum value or valid value type, it should be entered into column N. Then, using the target row number determined in stage 2, combined with the mapping rules, the data is entered. An example of the script configuration is as follows:

[0183] def write_result(ws, row_idx, code, kind, value_str):

[0184] """

[0185] Write result:

[0186] - Column C number (fill in if empty)

[0187] - Write R values ​​in column M

[0188] - Write the N value in column Q

[0189] """

[0190] col_C, col_M, col_Q = 3, 13, 17

[0191] # Number column

[0192] unmerge_if_needed(ws, row_idx, col_C)

[0193] if not ws.cell(row=row_idx, column=col_C).value:

[0194] ws.cell(row=row_idx, column=col_C, value=code)

[0195] # Numerical Values

[0196] if value_str:

[0197] try:

[0198] val = float(value_str)

[0199] except ValueError:

[0200] val = None

[0201] else:

[0202] val = None

[0203] if kind == 'R':

[0204] unmerge_if_needed(ws, row_idx, col_M)

[0205] ws.cell(row=row_idx, column=col_M, value=val)

[0206] elif kind == 'N':

[0207] unmerge_if_needed(ws, row_idx, col_Q)

[0208] ws.cell(row=row_idx, column=col_Q, value=val)

[0209] else:

[0210] raise ValueError("kind must be 'R' or 'N'")

[0211] Through the above steps, a complete target Excel data table is obtained. The data is stored strictly according to the defined columns, without the interference of merged cells, and can update existing data and append new data, avoiding duplicate records. Furthermore, the product number allows tracing the product's historical records. In short, this process achieves full automation of data entry, eliminating the inefficiency and errors associated with manual operations.

[0212] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0213] Fifth embodiment

[0214] Some embodiments of this application also provide an electronic device. The electronic device can be various forms of digital computer, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, etc. The electronic device can also be various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices.

[0215] The electronic device includes: one or more processors; and a memory storing computer program instructions that, when executed, cause the processor to perform the steps of the methods provided in any one or more of the above embodiments. Figure 4An exemplary structural diagram of the electronic device is disclosed. The electronic device includes one or more processors 1101, a memory 1102, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations. The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0216] The electronic device may further include an input device 1103 and an output device 1104. The processor 1101, memory 1102, input device 1103 and output device 1104 may be connected by a bus or other means, as shown in the figure, which is connected by a bus.

[0217] Input device 1103 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 1104 may include a display device, auxiliary lighting device (e.g., LED), and haptic feedback device (e.g., vibration motor). The display device may include, but is not limited to, a liquid crystal display, a light-emitting diode display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0218] To provide interaction with the user, the electronic device can be a computer. The computer has: a display device (e.g., a cathode ray tube or LCD monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback); and input from the user can be received in any form (e.g., voice input or tactile input).

[0219] In this embodiment, a computer-readable medium stores a computer program / instructions that, when executed by a processor, implement the steps of the methods provided in any one or more of the above embodiments. This computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into that device. The aforementioned computer-readable medium carries one or more computer-readable instructions.

[0220] The memory 1102 can serve as a non-transitory computer-readable storage medium, used to store non-transitory software programs, non-transitory computer-executable programs, and modules. The processor 1101 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 1102, thereby implementing the program instructions / modules corresponding to the methods provided in any one or more of the embodiments described above in this application.

[0221] The memory 1102 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 1102 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1102 may optionally include memory remotely located relative to the processor 1101, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0222] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0223] Computer-readable media include permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory, static random access memory, dynamic random access memory, other types of random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory or other memory technologies, read-only optical discs, digital versatile optical discs or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0224] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including local area networks (LANs) or wide area networks (WANs), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0225] In the above embodiments, all or part of the implementation can be achieved through software, hardware, firmware, or any combination thereof. For example, it can be implemented using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of this application can be executed by a processor to implement the above steps or functions. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as RAM memory, magnetic or optical drives, floppy disks, and similar devices. In addition, some steps or functions of this application can be implemented in hardware, for example, as circuitry that cooperates with a processor to perform the various steps or functions.

[0226] The computer program product provided in this application includes one or more computer programs / instructions. When executed by a processor, these computer programs / instructions generate, in whole or in part, the processes or functions described in this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0227] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0228] The scope of this application is defined by the appended claims rather than the foregoing description, and is therefore intended to encompass all variations falling within the meaning and scope of equivalents of the claims. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device in software or hardware. Terms such as "first," "second," etc., are used only for distinguishing descriptions and do not indicate any particular order, nor should they be construed as indicating or implying relative importance.

[0229] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily made by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims, and the above embodiments should be regarded as exemplary and non-limiting.

Claims

1. A method for generating Excel data reports based on image recognition, characterized in that, The method includes: Based on the oscilloscope image and OCR technology, machine-encoded text information is obtained; Based on the image feature identifiers in the machine-encoded text information, the image type of the oscilloscope image is determined; Based on the image type, parameter names and corresponding values ​​are extracted from the target data region to form key-value pairs of structured data; According to the autofill rules, the structured data is added to the workbook to obtain the target Excel data report.

2. The method according to claim 1, characterized in that, The method of determining the image type of the oscilloscope image based on the image feature identifier in the machine-encoded text information includes: Extract image feature identifiers from the machine-encoded text information; The image feature identifier is compared with a preset type feature library; the type feature library contains time interval feature templates for first type oscilloscope images and extreme value feature templates for second type oscilloscope images; The category to which the image feature identifier belongs is determined based on the comparison results; The specific type of the oscilloscope image is determined based on the judgment result; wherein, the first type of oscilloscope image is identified by a time interval identifier, and the second type of oscilloscope image is identified by a maximum value identifier and a valid value identifier.

3. The method according to claim 2, characterized in that, The extraction of image feature identifiers from the machine-encoded text information includes: For the time interval markers of the first type of oscilloscope image, extract the associated combination containing the time unit field and the interval value field; For the feature identifiers of the second type of oscilloscope image, extract the maximum value identifier and the effective value identifier.

4. The method according to claim 2, characterized in that, The step of comparing the image feature identifier with a preset type feature library includes: Perform multi-dimensional matching calculations on the image feature identifiers and feature data in the type feature library; Based on the multi-dimensional matching calculation results, the similarity score between the image feature identifier and each feature data is determined; When the similarity score reaches a preset threshold, the image type to which the corresponding feature data belongs is taken as a candidate type; The final matching type of the image feature identifier is determined by prioritizing and analyzing the confidence of multiple candidate types.

5. The method according to claim 2, characterized in that, The step of extracting parameter names and corresponding values ​​from the target data region according to the image type includes: Based on the image type of the oscilloscope image, the corresponding data region coordinate configuration file is dynamically loaded; wherein, the time interval data region coordinate configuration file of the first type of oscilloscope image is associated with the pixel coordinate range of the time axis scale line, and includes coordinate offset parameters of numerical field and unit field; the maximum value or effective value data region coordinate configuration file of the second type of oscilloscope image is associated with the pixel coordinate mapping relationship of the waveform peak point, and distinguishes the coordinate boundary between the maximum value region and the effective value region. Based on the coordinate parameters and image pixel ratio in the configuration file, locate the target data region; Extract the parameter names and corresponding values ​​from the target data region.

6. The method according to claim 5, characterized in that, The step of extracting parameter names and corresponding values ​​from the target data region includes: Image enhancement processing is performed on the target data area to improve the clarity of the text area; Parameter names and corresponding values ​​are extracted from the target data region using multimodal text recognition technology; The extracted parameter names and values ​​are subjected to semantic consistency verification and format standardization. When the recognition results are ambiguous or erroneous, intelligent correction is performed based on contextual correlation analysis; The corrected parameter names and values ​​are used to generate standardized key-value pairs according to a preset data structure.

7. The method according to any one of claims 1 to 6, characterized in that, The process of adding structured data to the workbook according to autofill rules to obtain the target Excel data report includes: Initialize the workbook, check and unmerge the target cells; Based on the product number, iterate through the data rows in the specified number column of the workbook; if a matching number is found, return the corresponding row number as the target row; if no matching number is found, return the maximum row number of the workbook plus one to create a new row. According to the mapping rules, fill the time interval values ​​into the R result data column and fill the maximum or valid value values ​​into the N result data column to obtain the target Excel data report.

8. An electronic device, characterized in that, The electronic device includes: One or more processors; and A memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the method as described in any one of claims 1 to 7.

9. A computer-readable medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

Images