Method and system for extracting Mongolian power table data based on image recognition

Abstract

The invention relates to the field of image processing, optical character recognition and natural language processing, and discloses a Mongolian power table data extraction method and system based on image recognition, and the method comprises the following steps: carrying out the image preprocessing of an input power table image sample, and improving the image quality; extracting a table frame structure, optimizing table line segment positions, and segmenting table cells; identifying a power business field corresponding to each cell and establishing a mapping relationship between the cells and the power business fields; and based on the mapping relationship, generating structured spreadsheet data, and filling the structured spreadsheet data with the power business data. According to the method, by introducing table structure optimization, context associated character extraction and industry term correction technologies, the efficiency and accuracy of Mongolian table data processing are effectively improved, and technical support is provided for digital management of the power industry and intelligent application of minority languages.

Description

Technical Field

[0001] The present invention relates to the technical fields of image processing, optical character recognition and natural language processing, and in particular to a method and system for extracting Mongolian power table data based on image recognition. Background Art

[0002] In the power industry, a large amount of tabular data is used to record and manage user information, power consumption, and equipment parameters. These data are traditionally stored in paper or picture form. Especially in Mongolian-speaking areas, due to the particularity of the language and technical environment, the digital processing of table content is very difficult. At present, the automated processing technology for tabular data mainly relies on the general optical character recognition (OCR) method and traditional table structure parsing technology. However, the existing technology faces multiple difficulties and limitations when processing Mongolian power tabular data.

[0003] First, Mongolian characters have the characteristics of connected strokes and variable glyph structures, and their contextual relevance is strong. The existing OCR system has a low recognition accuracy rate for Mongolian characters, and is prone to misrecognition due to the similarity between characters. In addition, power tables usually contain a large number of industry-specific terms and terms, such as units and equipment names. The existing technology lacks a dedicated processing solution for industry terms, and inaccurate or inconsistent word meanings are prone to occur during the translation process, thus affecting the professionalism and practicality of the translation results.

[0004] Secondly, the layout of power tables is complex and diverse, and there may be problems such as line breakage, tilt, and noise interference. Traditional table structure analysis methods have high requirements for line quality and cannot accurately extract table frames when the structure is incomplete or there is a lot of noise. In addition, when the table has a non-standard layout or multiple nested forms, the existing methods have poor versatility and often need to rely on manual intervention or specific templates, which makes it difficult to meet the needs of efficient batch processing.

[0005] At the same time, when processing multilingual tabular data, existing technologies usually only focus on common languages, and have weak processing capabilities for minority languages ​​such as Mongolian. Especially in the translation between Mongolian and Chinese, due to the significant differences in grammar, vocabulary and expression habits between the two languages, the existing neural machine translation model does not work well for Mongolian-Chinese translation, and it is difficult to generate high-quality, context-consistent translation results. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention provides a method and system for extracting Mongolian power table data based on image recognition, which solves the problems of low processing efficiency and poor accuracy caused by character complexity, incomplete table structure and inaccurate terminology translation during the automated extraction and translation of Mongolian power table data.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: A method for extracting Mongolian power table data based on image recognition comprises the following steps:

[0008] Capturing the natural language expression of the input data, performing speech recognition on the natural language expression, and extracting power business data from the natural language expression;

[0009] Perform image preprocessing on the input power table image sample;

[0010] Extract the frame structure of the table, optimize the position of the table segments, and segment the table cells;

[0011] Identify the power service data field corresponding to each of the table cells, and establish a mapping relationship between the table cells and the power service data field;

[0012] Based on the mapping relationship, structured electronic form data is generated, and the electric power service data is filled into the structured electronic form data.

[0013] Preferably, the image preprocessing comprises the following steps:

[0014] A grayscale algorithm based on pixel weighted average is used to convert the color image into a grayscale image;

[0015] Use wavelet transform to remove image noise and suppress high-frequency noise in the wavelet high-frequency component;

[0016] Enhance the edges of table lines through the Canny edge detection algorithm;

[0017] Use Hough transform to detect the image tilt angle and perform perspective correction to make the table lines vertical or horizontal.

[0018] in,

[0019] The formula of the grayscale algorithm based on pixel weighted average is:

[0020] Gray = 0.299R + 0.587G + 0.114B, Gray is the gray value of the pixel, R, G and B are the weights of the red, green and blue channels of the pixel respectively

[0021] The formula of the Canny edge detection algorithm is:

[0022] G(x, y) is the value of the table line edge enhancement, x and y are the coordinate positions relative to the center of the Gaussian kernel, σ is the standard deviation, e is a natural constant, and π is the circumference of a circle.

[0023] Preferably, the steps of extracting the frame structure of the table, optimizing the positions of the table segments, and segmenting the table cells include:

[0024] Detecting edge straight line segments in the power table image sample based on Hough transform, and screening out straight line segments in horizontal and vertical directions;

[0025] Construct an energy function based on geometric optimization theory to optimize the position of table segments;

[0026] The energy function is optimized iteratively to minimize the energy. After the optimization is completed, the cell area of ​​the table is divided based on the intersection coordinates. The boundary coordinates of the cell are determined by the optimized table line segments.

[0027] in,

[0028] The energy function based on geometric optimization theory is:

[0029] E represents the Laplace energy, which is used to measure the smoothness of the dangerous section of the table. i To optimize the current position vector of the table segment, To optimize the expected position vector of the table segment, i is an integer greater than 0. i The closer the value is to When the value is , it means that the smoother the table segments are, the better the optimization effect is.

[0030] Preferably, the step of constructing an energy function based on geometric optimization theory to optimize the position of the table line segments comprises:

[0031] The target energy function of table segment optimization is constructed to minimize the gradient change and position deviation of the table segment. The target energy function is expressed as:

[0032]

[0033] Where E is the energy function value, Ω is the image area, L(x, y) represents the actual position of the table segment, x and y represent the image coordinates, respectively. represents the gradient of L(x, y), G(x, y) represents the geometric center of the table, α is the weight parameter of gradient consistency and position deviation, dxdy represents the integral operation of the entire image area, ||·|| 2 Represents the square of the Euclidean distance between vectors;

[0034] The energy function E is derived using a gradient descent algorithm to obtain the update direction of the table segment position. The formula of the gradient descent algorithm is:

[0035]

[0036] Among them, L (t+1) (x, y) represents the updated position of the table segment after the t+1th iteration, L (t) (x, y) represents the position of the line segment at the tth iteration, η is the optimization step size factor, Represents the gradient of the energy function with respect to the line segment position;

[0037] According to the formula of the gradient descent algorithm, the position of the table line segment is repeatedly updated until the convergence condition is met, and the convergence condition is:

[0038] |L (t+1) (x, y)-L (t) (x, y)|<δ

[0039] Among them, δ is the convergence threshold in the optimization process;

[0040] The optimized line segments are repaired, and the minimum spanning tree of the line segment endpoints is constructed based on the minimum spanning tree weight function to minimize the Euclidean distance of the completed line segment. The minimum spanning tree weight function is:

[0041]

[0042] Where W(T) represents the total weight of the spanning tree, T is the edge set of the spanning tree, (u, v)∈T represents the edge in the spanning tree, and d(u, v) represents the Euclidean distance between endpoints u and v;

[0043] After completing the broken lines, confirm the integrity of the table structure and generate the final boundaries of the cells based on the optimized table frame.

[0044] Preferably, the step of identifying the power service data field corresponding to each of the table cells and establishing a mapping relationship between the table cells and the power service data field comprises:

[0045] Extract the character area from the cell, use the image binarization method to process the cell image, use the 0tsu algorithm to calculate the global threshold, and convert the cell image into a binary image according to the global threshold; after binarization, further use the connected domain analysis method to mark and extract the possible character area;

[0046] The detected character area is segmented, the text line is analyzed using the vertical projection method, and the cumulative value of each column of pixels is calculated. The boundary of the character segmentation is determined according to the continuous zero value position of the cumulative value, and a single character area is segmented;

[0047] Perform feature extraction on the segmented single character area and use convolutional neural network to extract multi-scale spatial features of characters;

[0048] For Mongolian characters with contextual associations, a language model based on the self-attention mechanism is used to optimize the recognition results, and the Transformer model is used to process the character sequence;

[0049] Input the optimized character features into the fully connected layer for classification and output the character category label;

[0050] The language model is used to correct errors in the recognition results, and the power industry-specific dictionary is used to match and replace terms in character sequences.

[0051] Preferably, the step of generating structured electronic form data based on the mapping relationship and filling the electric power business data into the structured electronic form data comprises:

[0052] According to the extracted Mongolian character content, the Mongolian characters in the cell are reorganized into a continuous text sequence according to the character segmentation result and the cell coordinate order;

[0053] Use a pre-trained neural machine translation model to translate Mongolian text;

[0054] Match and optimize the generated Chinese translation texts with terms, and use the power industry-specific terminology dictionary to replace and verify the words in the translation results;

[0055] Rearrange the translated Chinese text according to the cell coordinates of the original table and output it in a spreadsheet format.

[0056] Preferably, the pre-trained neural machine translation model is based on a Transformer architecture, consisting of an encoder and a decoder.

[0057] Preferably, the data of the output electronic form includes the Mongolian original text and the translated Chinese content, which are stored in corresponding cells.

[0058] The present invention also provides a system for extracting Mongolian power table data based on image recognition, which is used to execute the above method. The system comprises:

[0059] An image processing module, used to capture the natural language expression of the input data, perform speech recognition on the natural language expression, extract the power business data from the natural language expression, and grayscale, denoise, edge enhance and perspective correct the input power table image sample;

[0060] Table detection module, used to extract the frame structure of the table, optimize the position of table segments, and segment table cells;

[0061] A character recognition module, used to identify the electric power business field corresponding to each of the table cells, and establish a mapping relationship between the table cells and the electric power business data field;

[0062] The translation module is used to translate the extracted Mongolian content into Chinese by the translation model;

[0063] The data output module is used to generate structured electronic spreadsheet data based on the mapping relationship, fill the electric power business data into the structured electronic spreadsheet data, and output the translated structured data in the form of an electronic spreadsheet.

[0064] The present invention also provides a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented.

[0065] The invention provides a method and a system for extracting Mongolian power table data based on image recognition.

[0066] It has the following beneficial effects:

[0067] 1. The present invention uses automated image recognition and translation technology to achieve rapid data extraction and translation of Mongolian tables in the power industry, replacing the traditional manual entry method. Through the automated process of image preprocessing and table frame extraction, the time for table data processing is effectively shortened, meeting the needs of large-scale power data digitization.

[0068] 2. The present invention combines optical character recognition technology and context optimization model, and can handle the problems of Mongolian characters such as connected strokes, morphological similarity and context dependency. Character features are extracted through convolutional neural networks, and the recognition results are optimized using the Transformer model based on the self-attention mechanism, which significantly improves the accuracy of Mongolian character extraction and meets the requirements of high-precision data processing in the power industry.

[0069] 3. The present invention introduces a terminology dictionary and semantic correction mechanism for the electric power industry in the process of translating Mongolian into Chinese, and dynamically matches and optimizes the translation results. The term matching rules can effectively identify and correct proper nouns in the electric power industry, avoid translation errors, and provide accurate translation results for professional field applications.

[0070] 4. The present invention can generate electronic spreadsheets or JSON data from the translated bilingual content in the structured form of the original form, support data export in different formats, and facilitate users to use it directly in subsequent operations. This multi-format structured output method enhances the readability and usability of data, laying the foundation for the information management of the power industry.

[0071] 5. The present invention detects and repairs the table frame by introducing geometric optimization theory, solves problems such as broken table lines, tilt and noise interference, and ensures the structured extraction capability of complex tables. In addition, the system can adapt to a variety of power table templates and complex scenarios, enhancing the robustness of the algorithm and the breadth of practical applications.

[0072] 6. By integrating technologies such as image processing, OCR, translation models and term matching, the present invention provides an efficient solution for data processing of minority languages ​​such as Mongolian, reduces the obstacles to intelligent management caused by language differences, and promotes the application and promotion of minority languages ​​in modern power management.

[0073] 7. The present invention introduces an adaptive structured data extraction method, which enables efficient migration from a table data extraction technology in a specific scenario to other scenarios. In this method, the algorithm can automatically learn the differences in table structures in different scenarios and adjust the data processing flow according to these differences. When applied to a new power scenario, there is no need to redesign the structured process from scratch, but to quickly adapt to the new data format and content through transfer learning, thereby reducing the development time and resource investment when applying to different scenarios.

[0074] 8. In the power industry of Inner Mongolia, due to its unique geographical and climatic conditions, the present invention adopts technologies such as adaptive image enhancement, rule-based field recognition and professional terminology matching. This solution effectively addresses the common problems of image quality differences, inconsistent table formats and special terminology in Inner Mongolia. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0076] Figure 2 It is a schematic diagram of the system structure of the present invention;

[0077] Figure 3 It is a schematic diagram of the computer device structure of the present invention.

[0078] Among them, 100, image processing module; 200, table detection module; 300, character recognition module; 400, translation module; 500, data output module; 40, computer equipment; 41, processor; 42, memory; 43, storage medium. DETAILED DESCRIPTION

[0079] The following will be combined with the drawings in the specification of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0080] Please see attached Figure 1 The embodiment of the present invention provides a method for extracting Mongolian power table data based on image recognition, aiming to automatically extract and translate Mongolian content in the power table and output it as structured electronic table data. The steps of the method are described in detail below.

[0081] S1. Capture the natural language expression of the input data, perform speech recognition on the natural language expression, extract the power business data from the natural language expression, and perform image preprocessing on the input power table image sample.

[0082] First of all, in the method of extracting Mongolian power table data based on image recognition, image preprocessing is the basic link of the entire technical process, which aims to improve the quality of input images, remove noise and interference, and provide high-quality input data for subsequent table structure extraction and Mongolian character recognition. Generally speaking, power table image samples may contain background noise, uneven light or tilt during shooting, which will directly affect the accuracy of subsequent steps.

[0083] In a possible implementation, the embodiment of the present invention gradually improves the quality of the input image through a series of image processing techniques, including image grayscale, denoising, edge enhancement and perspective correction, to provide reliable basic data for subsequent processing steps.

[0084] Specifically, in this embodiment, for the input color power table image sample, the image is first grayed to reduce the computational complexity. The gray formula is:

[0085] I(x,y)=0.299·R(x,y)+0.587·G(x,y)+0.114·B(x,y)

[0086] Among them, R(x, y), G(x, y), B(x, y) represent the red, green, and blue channel values ​​of the pixel in the image, respectively, and I(x, y) represents the grayscale value. As an option, the grayscale processing can slightly adjust the weighting coefficients of different color channels to adapt to the influence of lighting conditions in special scenes.

[0087] In some embodiments, in order to further remove the noise interference in the image, a denoising method based on wavelet transform is used. Specifically, the grayscale image is decomposed into low-frequency and high-frequency components. The low-frequency component contains the main information of the image, while the high-frequency component mainly reflects noise and detail information. The high-frequency components LH, HL, and HH are subjected to soft threshold processing. The specific processing formula is:

[0088]

[0089] Among them, x represents the pixel value of the high-frequency component, λ is the threshold, and sign(x) represents the sign function, which is used to retain the positive and negative sign information of the pixel value. Through soft threshold processing, high-frequency noise can be effectively removed while retaining the main features of the image.

[0090] As an option, the denoised image is further edge enhanced in this embodiment. Generally, edge enhancement can significantly improve the clarity of table lines. In the specific implementation, the Canny algorithm is used, which calculates the gradient of the image in the horizontal and vertical directions to identify areas with significant grayscale changes. The gradient calculation formula is:

[0091]

[0092] Among them, G x and G y are the gradients of the image in the x and y directions, respectively, and are calculated as follows:

[0093] G x =I*S x , G y =I*S y

[0094] Where I is the input image, S x and S y They represent Sobel filter kernels respectively, and the specific definition of the filter kernel is determined according to the requirements of edge detection. Through this method, the edge information of the table lines can be effectively extracted.

[0095] In some embodiments, in order to correct the perspective distortion caused by the shooting angle in the image, a perspective correction method based on Hough transform is further introduced. Specifically, the straight line edge in the image is first detected by Hough transform, and its parameter model is:

[0096] ρ=xcosθ+ysinθ

[0097] Among them, ρ represents the distance from the straight line to the origin of the image, and θ is the angle between the straight line and the horizontal axis. The possible (ρ, θ) parameter combinations are recorded by the accumulator to extract the main horizontal and vertical lines in the table.

[0098] After detecting the table line segments, the image rotation angle is calculated using the direction information of the line segments, and the image is corrected based on the affine transformation to restore the table lines to a horizontal or vertical state. As an option, the specific parameters of the affine transformation can be adjusted according to user needs to adapt to different table layout scenarios.

[0099] In some embodiments, in order to verify the preprocessing effect, this embodiment performs a visual inspection on the processed image after denoising, edge enhancement and perspective correction to ensure that the lines of the table are clear and the structure is complete.

[0100] Through the above steps, the present invention realizes high-quality preprocessing of input power table image samples, and provides reliable input data support for subsequent table structure extraction and Mongolian character recognition.

[0101] S2. Extract the frame structure of the table, optimize the position of the table segments, and divide the table cells.

[0102] Generally, the frame structure in the power table may be difficult to identify due to broken lines, irregular layout or angle distortion during shooting. In order to accurately extract table segments and segment cells, this embodiment detects straight lines through Hough transform, adjusts the position of line segments through geometric optimization, and combines the cell division algorithm to complete the structural processing of the table.

[0103] In this embodiment, firstly, based on the preprocessing result in the aforementioned step S1, edge detection is used to generate edge information in the image, and on this basis, long straight line segments are extracted. Specifically, based on the Hough transform parameter model in step S1, different (ρ, θ) parameter combinations are recorded through an accumulator mechanism, high-frequency parameters are screened out, and the main straight line segments in the image are determined.

[0104] In some embodiments, in order to remove background noise and irrelevant lines, the detected straight line segments are further screened. Generally, by constraining the directionality of the straight line, the line segments close to the horizontal and vertical directions are retained, and the screening conditions are:

[0105] |θ-k·90°|<∈

[0106] in, represents the direction multiple, and ∈ is the tolerance range. Through this screening step, the false detection of line segments can be significantly reduced and the accuracy of the framework structure can be improved.

[0107] In order to further optimize the position and continuity of the line segment, this embodiment constructs an energy function based on geometric optimization theory to adjust the position of the line segment. The energy function consists of the following two parts:

[0108] The gradient consistency term is used to measure the edge strength consistency of the line segment and is expressed as:

[0109]

[0110] in, Represents the gradient of a line segment.

[0111] The position deviation term is used to describe the deviation between the position of the line segment and the ideal geometric center, and is expressed as:

[0112] ||L(x,y)-G(x,y)|| 2

[0113] Among them, L(x, y) represents the position of the current line segment, and G(x, y) represents the ideal geometric center position.

[0114] Combining the above two parts, the overall form of the energy function is:

[0115]

[0116] Where E is the energy function value, Ω is the image area, L(x, y) represents the actual position of the table segment, x and y represent the image coordinates, respectively. represents the gradient of L(x, y), G(x, y) represents the geometric center of the table, α is the weight parameter of gradient consistency and position deviation, dxdy represents the integral operation of the entire image area, ||·|| 2 Represents the square of the Euclidean distance between vectors.

[0117] During the optimization process, the gradient descent algorithm is used to update the line segment position, and the update formula is:

[0118]

[0119] in:

[0120] L (t) (x, y) and L (t+1) (x, y) represent the position of the table line segment after the tth and t+1th iterations respectively;

[0121] η is the optimization step size factor, which is used to control the amplitude of each update;

[0122] Represents the gradient of the energy function with respect to the line segment position, and the calculation formula is:

[0123]

[0124] in:

[0125] is the Laplace operator of the line segment gradient, which is used to describe the gradient change and is expressed as:

[0126]

[0127] and are the second-order partial derivatives of the line segment position in the x and y directions, respectively.

[0128] In the optimization process, in order to ensure that the iteration can converge effectively, it is necessary to set the convergence condition. In general, the convergence condition is that the change in position between two consecutive iterations is less than the preset threshold δ, which can be expressed as:

[0129] |L (t+1) (x, y)-L (t) (x, y)|<δ

[0130] in:

[0131] |L (t+1) (x, y)-L (t) (x, y)| represents the change in the position of the line segment between two iterations, and its calculation formula is:

[0132]

[0133] Among them, x t+1 and t+1 represents the line segment coordinates of the t+1th iteration, x t and t are the coordinates of the tth iteration.

[0134] When the convergence condition is met, the optimization process is terminated and the optimized table line segment position is output. Through the above method, the position error of the table line segment can be effectively corrected to improve the accuracy and consistency of the line segment.

[0135] In some embodiments, in order to solve the problem of incomplete structure caused by broken or missing lines in the table, the present invention further introduces a minimum spanning tree algorithm to complete the line segments. Specifically, by constructing a line segment endpoint graph G = (V, E), where V is the set of line segment endpoints and E is the set of Euclidean distances between endpoints, the minimum spanning tree target is:

[0136]

[0137] in, Represents the Euclidean distance between the two endpoints. By completing the broken line segments, the integrity of the table frame can be ensured.

[0138] After the table line segments are optimized and completed, the cell area is divided based on the intersection positions of the line segments. Specifically, the cell boundary is determined by the intersection coordinates of the horizontal line segments and the vertical line segments. In some embodiments, to improve the segmentation efficiency, the intersection coordinates can be sorted and the row and column indexes of the cell can be established.

[0139] Through the above steps, this embodiment achieves complete extraction of the table frame and accurate division of the cell area, providing a clear structured data foundation for subsequent Mongolian character extraction and translation.

[0140] S3. Identify the electric power service data field corresponding to each of the table cells, and establish a mapping relationship between the table cells and the electric power service data field.

[0141] The purpose of extracting Mongolian characters in this step is to convert the Mongolian characters in the table cells into a parseable digital text form for subsequent translation and data storage. In general, the extraction of Mongolian characters faces complex technical challenges, such as character connection, glyph similarity, context dependency and other issues. The present invention achieves high-precision Mongolian character extraction by adopting a multi-step processing method of character region detection, character segmentation, feature extraction and context optimization.

[0142] In one possible implementation, the present invention utilizes optical character recognition (OCR) technology, combined with a deep learning method based on a convolutional neural network (CNN) and a natural language processing (NLP) model, to comprehensively improve the accuracy and robustness of Mongolian character extraction.

[0143] Specifically, in this embodiment, the segmented cell image is first binarized to detect the character area. Generally, the 0tsu algorithm is used to calculate the global threshold T of the image, and the grayscale image I (x, y) is converted into a binary image by the following formula:

[0144]

[0145] Among them, I'(x, y) represents the pixel value after binarization, 1 represents the foreground (character area), and 0 represents the background. After binarization, the possible character area is further marked and extracted through the connected domain analysis method.

[0146] In a possible implementation, in order to ensure segmentation accuracy, this embodiment uses a vertical projection method to analyze the separation information between characters in the extracted character area. Specifically, the cumulative value P(y) of each column of pixels is calculated:

[0147]

[0148] Where W is the image width, and I′(x, y) is the pixel value after binarization. In general, when P(y) is zero, it means that the column does not contain characters and can be used as a separator for character segmentation.

[0149] As an option, for the case where the character area has serious connected strokes, a segmentation method based on contour analysis is further adopted. In a possible implementation, the outer boundary of the character is obtained by a contour tracing algorithm, and the segmentation point of the character is determined based on geometric features (such as aspect ratio, area).

[0150] In this embodiment, in order to extract the multi-scale features of a single character, a convolutional neural network (CNN) is used to extract features from the character image. Specifically, the character image I is input. c After processing through multiple convolutional layers, the feature map F is generated. The convolution operation formula is:

[0151] F=σ(W*I c +b)

[0152] in:

[0153] W represents the convolution kernel;

[0154] b represents bias;

[0155] σ represents the activation function (usually ReLU).

[0156] In some embodiments, in order to further improve the accuracy of recognition, this embodiment combines a language model based on a self-attention mechanism (such as Transformer) to perform context optimization on the character sequence. Specifically, the segmented character sequence is embedded as:

[0157] Z i =Embed(C i )+PosEnc(i)

[0158] in:

[0159] Zi represents the embedded representation of the i-th character;

[0160] Embed(C i ) indicates character embedding;

[0161] PosEnc(i) represents position encoding, which is used to preserve the position information of the character sequence.

[0162] In one possible implementation, a multi-head attention mechanism is used to calculate the contextual association between characters, and the attention weight is calculated as follows:

[0163]

[0164] in:

[0165] Q, K, and V are query, key, and value matrices, respectively;

[0166] d k is the dimension of the key matrix.

[0167] Through the above optimization process, the recognition errors of connected characters can be effectively reduced and the overall accuracy of character extraction can be improved.

[0168] After the recognition is completed, this embodiment uses the Softmax layer to classify the extracted features, and the classification formula is:

[0169] p(c)=Softmax(W c ·F+b c )

[0170] in:

[0171] p(c) represents the probability that the character belongs to category c;

[0172] W c and b c are the weight and bias of the classification layer, respectively.

[0173] In some embodiments, in order to further improve the accuracy of industry applications, the present embodiment performs post-processing on the extraction results in combination with the terminology dictionary of the power industry. Specifically, by matching the recognized character sequence with the entries in the terminology dictionary, the erroneous characters are automatically corrected.

[0174] Generally, the tabular data of the power industry contains a large number of proper nouns and specific terms, and the recognition accuracy of these terms is crucial to the practical application of the tabular data. In a possible implementation, this embodiment introduces a context-dependent term matching rule, which dynamically selects appropriate term entries for correction by analyzing the context of the character sequence.

[0175] Specifically, when the recognized character sequence S = [C1, C2, ..., C n ] and an entry in the term dictionary Dk = [D k1 , D k2 , ..., D km ]When the content is partially matched, the context consistency correction mechanism is used to calculate the matching score using the following formula:

[0176]

[0177] in:

[0178] δ(C i , D ki )=1 means C i =D ki , otherwise 0;

[0179] m is the total length of the term entry.

[0180] When the matching score is higher than the preset threshold, the current character sequence is automatically replaced with the entry D in the term dictionary. k , ensuring the accuracy of terminology.

[0181] In some embodiments, in order to address the problem of insufficient coverage of the term dictionary, this embodiment adds a self-learning term expansion function. Specifically, by analyzing a large amount of historical power table data, new term entries are automatically generated and added to the term dictionary.

[0182] In a possible implementation, the Mongolian-Chinese bilingual translation results are used to perform cluster analysis on character sequences that appear frequently and have significant similarities with existing terminology dictionary entries. The cluster center is represented as:

[0183]

[0184] in:

[0185] N k Indicates the number of character sequences of type k;

[0186] S i Represents the i-th character sequence belonging to this class.

[0187] The cluster center μ k Perform similarity matching with the existing term dictionary. If the matching degree is lower than the threshold, μ k Automatically added to the dictionary as a new term entry.

[0188] In a possible implementation, in order to further improve the robustness of term matching and correction, this embodiment introduces a semantic correction module based on a neural network. Specifically, a pre-trained bilingual embedding model (such as MUSE or FastText) is used to semantically embed the character sequence and the term dictionary entry. k Mapped to the same semantic space:

[0189]

[0190] in:

[0191] f(·) represents the semantic embedding function;

[0192] v S and Represent the semantic vectors of character sequences and term entries respectively.

[0193] Calculate the semantic similarity score:

[0194]

[0195] When the similarity score exceeds the threshold, the dictionary entry with the closest semantics is preferentially selected as the correction result.

[0196] As an extension, this embodiment also adds dynamic adjustment and multi-layer verification mechanisms. In general, the reliability of terminology correction is further improved by the following steps:

[0197] 1. First, perform term matching based on the initial recognition result of the character sequence and output the first candidate correction result;

[0198] 2. Secondly, the corrected results are input into the template matching system of the power industry data to verify whether the corrected terms conform to the table logic, such as voltage units, date formats, etc.;

[0199] 3. Finally, the verification results of terminology correction are recorded, and the dictionary weights are dynamically adjusted according to actual needs, giving higher priority to frequently used terms.

[0200] Through the above steps, this embodiment achieves accurate extraction from cell images to Mongolian characters, and can provide high-quality input data for subsequent translation and table generation.

[0201] S4. Generate structured electronic spreadsheet data based on the mapping relationship, and fill the electric power business data into the structured electronic spreadsheet data.

[0202] Generally speaking, Mongolian text has the characteristics of unique grammatical structure, complex character ligatures, strong context relevance, etc., which makes it difficult to directly carry out high-quality translation. In addition, the translation result also needs to be consistent with the original table structure, so as to ensure the data readability and logical consistency in the electronic form. Therefore, the present invention adopts a translation model based on neural machine translation (NMT), combined with term matching and multi-format output technology, to complete high-quality Mongolian-Chinese translation and table generation.

[0203] Specifically, in this embodiment, the Mongolian characters in the extracted cells are first reorganized into a sequence form. Generally, the characters in the cells are read in order according to their relative positions. The extracted character sequence T is represented as:

[0204] T=[C1,C2,...,C n ]

[0205] Among them, C i represents the i-th character, and n represents the total number of characters in the cell. As an option, for table contents with multiple lines or special alignment, the characters can be sorted by the priority of the center point coordinates to ensure the logic of sequence generation.

[0206] In a possible implementation, this embodiment uses a neural machine translation model based on the Transformer architecture to translate Mongolian sequences. Specifically, the translation model includes an encoder and a decoder. The encoder is responsible for mapping the Mongolian text sequence T into a context vector Z = [Z1, Z2, ..., Z n ], the decoder gradually generates the Chinese translation text H = [H1, H2, ..., H m ].

[0207] The encoding process calculates the relationship between characters through the self-attention mechanism and generates a context vector.

[0208] The decoding process uses an autoregressive strategy to gradually generate the translated text, and each character generated depends on the previously generated results. The specific decoding formula is:

[0209] H j =Decoder(H j-1 , Z) = softmax(W · [H j-1 , Z]+b)

[0210] in:

[0211] Hj represents the jth Chinese character generated;

[0212] H, -1 indicates the previously generated Chinese characters;

[0213] W and b are the weight and bias of the decoder respectively.

[0214] In a possible implementation, in order to improve the professionalism of the translation, this embodiment combines the terminology dictionary of the power industry to match and correct the translation results. Generally, when the vocabulary in the translation text is inconsistent with the entries in the terminology dictionary, they are replaced according to the following rules:

[0215]

[0216] in:

[0217] H j Represents the vocabulary in the translated text;

[0218] D stands for term dictionary;

[0219] S(H j ) indicates that the dictionary contains j The matching term.

[0220] In order to ensure the logic of the translation results, this embodiment further adopts a verification mechanism based on template matching. For example, for the content of the numeric field, unit field, etc., check whether its format meets the expectations. If the verification fails, the cell content will be marked as abnormal for manual adjustment by the user.

[0221] After the translation is completed, this embodiment matches the Mongolian original text with the Chinese translation result one by one, and generates bilingual table data according to the original structured form of the table. Generally, the following two output formats are supported:

[0222] Spreadsheet format (Excel): Display the bilingual content in separate cells, and users can view the translation results directly in the table.

[0223] JSON format: bilingual content is stored in the form of key-value pairs, where the key is the coordinates of the cell and the value is an object containing Mongolian and Chinese content.

[0224] In some embodiments, in order to improve the compatibility of the system, the present invention supports user-defined output formats. For example, the user can choose to export only the Chinese translation results, or choose to store them in the form of a CSV file.

[0225] Through the above steps, this embodiment realizes the conversion from Mongolian character sequence to Chinese translation text and generates structured electronic spreadsheet data, which provides convenience for practical application.

[0226] In general, the present invention improves the quality of input images through image preprocessing, uses table structure extraction algorithms to detect and optimize table segments, segments cell areas, and extracts Mongolian character content in combination with optical character recognition technology. Furthermore, a neural machine translation model is used to translate Mongolian content into Chinese, and the translation results are corrected in combination with a dictionary of electric power industry terms, and finally bilingual electronic spreadsheet data corresponding to the original table structure is generated. The present invention effectively improves the processing efficiency and accuracy of Mongolian electric power table data, and provides technical support for the intelligent management of the electric power industry.

[0227] As an implementation of the present invention, in order to generalize the algorithm for extracting data from Mongolian power tables to other different scenarios, the present invention also provides a structured data extraction framework. The framework is modularly designed and can be flexibly adjusted according to the needs of specific scenarios, thereby adapting to different industry needs.

[0228] Specifically, the present invention constructs a universal data structured framework to decompose the core technologies of power table data extraction (such as image recognition, text recognition, table structure analysis, etc.) into multiple reusable modules. This modular design enables customized optimization according to the characteristics of input data in different scenarios.

[0229] For example, in the image preprocessing module, according to the characteristics of different table images (such as the thickness of table lines, the type of noise, the format of the table, etc.), image enhancement, denoising and table line recognition can be automatically performed through adaptive algorithms. As an option, the image preprocessing module can automatically learn and adjust image features in different scenarios by introducing a pre-trained model based on a convolutional neural network, thereby improving the preprocessing accuracy.

[0230] Specifically, in different application scenarios, the image recognition module can adaptively adjust the recognition algorithm according to the style, content and complexity of different tables. For example, in the process of extracting power table data, the row and column structure of the table may be relatively fixed, while in tables in other industries (such as medical, financial, etc.), the format and field layout of the table may be more complex. Therefore, in this case, the table structure parsing module dynamically adjusts the parsing algorithm of the table structure to automatically identify tables of different formats and extract the corresponding data fields.

[0231] In some embodiments, the present invention further enhances the flexibility and applicability of the framework through transfer learning. Specifically, a preliminary model is obtained by pre-training on Mongolian power table data, and then the model is fine-tuned in new application scenarios using transfer learning technology. The fine-tuning process usually uses a small amount of annotated data to avoid retraining the model from scratch, thereby greatly improving the efficiency of the algorithm in other scenarios.

[0232] In the process of transfer learning, it is usually necessary to select a suitable migration strategy, such as using domain-based adaptation technology, so that the model can automatically adapt to the new data distribution when migrating from the power scenario to other scenarios such as medical or financial. For example, in the power scenario, the table usually contains standardized data fields such as voltage and current, while in the medical scenario, it may contain unstructured information such as medical record number, diagnosis, and drug name. In order to cope with these differences, the present invention uses domain adaptation technology to enable the model to recognize and extract data fields with similar structures in new scenarios.

[0233] In addition, the framework of the present invention also supports multi-language scene adaptation. As an option, the present invention introduces a multi-language support module based on Mongolian. By designing a universal OCR recognition model, the model can not only recognize Mongolian, but also extract data from tables in other minority languages ​​or common languages ​​(such as Chinese, English, etc.). This function provides huge application potential for the promotion of the present invention.

[0234] For example, in the medical field, patients' medical records often contain records in multiple languages, especially in multi-ethnic areas, where the mixed use of Mongolian, Chinese and other languages ​​may cause the accuracy of traditional OCR systems to decrease. Therefore, by introducing multilingual OCR technology, it is possible to ensure accurate data extraction in such a mixed language environment.

[0235] In another possible implementation, the present invention further improves the accuracy of data extraction by introducing automatic labeling technology for table fields. Specifically, in power tables, the types of fields (such as voltage, current, load, etc.) usually have fixed rules, while in tables of other industries, the field types may be more diverse. In order to cope with this situation, the present invention adopts automatic label generation technology based on deep learning to automatically identify the types of different fields and label them to ensure the accuracy of data structuring.

[0236] In general, the technical framework provided by the present invention enables the algorithm to be effectively promoted from the Mongolian power meter scenario to other industries and scenarios through modular design, transfer learning, domain adaptation, and multi-language support, and has strong versatility and adaptability.

[0237] As an implementation of the present invention, in order to optimize the power scenario in Inner Mongolia, the present invention also proposes a data extraction optimization scheme based on regional characteristics and the unique needs of the power industry. The optimization scheme combines the uniqueness of Inner Mongolia in terms of geography, climate, economy, etc., and designs a special preprocessing algorithm and data analysis method to improve the accuracy and efficiency of table data extraction.

[0238] Specifically, in Inner Mongolia, the collection of electricity meter data is often affected by extreme climate and natural conditions. For example, low temperature and dusty weather may lead to poor shooting quality of the meter, affecting subsequent image recognition. Therefore, the present invention introduces an adaptive image enhancement technology to deal with image noise and low contrast problems caused by weather conditions. In some embodiments, the image enhancement module uses deep learning technology to automatically identify the type of noise in the image and perform targeted denoising, thereby improving the image quality of the electricity meter.

[0239] In addition, the power tables in Inner Mongolia usually contain large-scale digital data, especially in power distribution and power consumption monitoring. The format of these data is usually fixed, but there may be certain differences in the table structure. For example, the date field, equipment number field, etc. may have format inconsistencies due to various reasons (such as equipment aging or human error). In response to this problem, the field recognition method based on rule learning in the present invention can automatically identify the field type and perform unified processing during the data parsing process to ensure that the extracted data meets the unified standard.

[0240] As an option, in the Inner Mongolia power industry, there are many complex tables involving power grids, which contain not only standardized power data, but also power grid status monitoring information, which is usually highly time-sensitive. In order to improve the timeliness and accuracy of data extraction, the present invention further optimizes the field parsing algorithm of the table and adopts a multi-level structured parsing method. Specifically, the method improves the accuracy and efficiency of table data extraction by establishing an association matrix between different data fields. In a possible implementation, for the frequently changing time-sensitive fields (such as date, timestamp, load value, etc.) in the power table, the present invention adopts a field recognition technology based on dynamic programming to ensure that the changes in these fields can be dynamically tracked.

[0241] In some embodiments, in order to deal with the special symbols and terms in the power table in Inner Mongolia, the present invention combines the professional terminology dictionary of the power industry and establishes a special term matching module to ensure that specific terms in the power table (such as voltage unit, power factor, etc.) can be accurately identified and extracted. This module constructs a professional terminology library and uses natural language processing (NLP) technology to match terms, thereby effectively avoiding errors that may occur in conventional OCR recognition.

[0242] Generally speaking, nationwide, the tabular data of the power industry has certain standardization, and the table fields and formats rarely change. However, due to the special geographical environment and power supply conditions, the power tables in Inner Mongolia often have special requirements and standards that are different from those in other parts of the country. For example, the power tables in Inner Mongolia may contain some special records for renewable energy such as wind power and solar power. The format and data processing method of these special records are different from those of traditional power tables, so the table data extraction algorithm needs to be adjusted accordingly.

[0243] As an option, for this special data, the present invention further introduces an automatic annotation technology based on knowledge graph, which automatically annotates and explains the special fields in the power table in Inner Mongolia by constructing a graph based on power industry knowledge. This knowledge graph can not only dynamically adapt to the data types unique to Inner Mongolia, but also automatically update with the development of the power industry, thereby ensuring high accuracy and high adaptability of data extraction.

[0244] In other embodiments, the present invention also takes into account the distribution of power companies in Inner Mongolia. In this region, there may be certain differences in the management mode of power companies and the content of the table. For example, some power companies may add custom fields in the table to record the status information of specific equipment. In response to this problem, the present invention designs a flexible table structure recognition method that can automatically identify the custom fields of power companies and perform structured processing on these fields according to actual needs.

[0245] Specifically, the present invention combines unsupervised learning algorithms and domain knowledge reasoning to intelligently infer the meaning and data type of custom fields based on the contextual information of the table, thereby ensuring the accuracy of power table data extraction. In order to further improve the effect of this method, the present invention also uses clustering analysis technology to group and match the data in the table, thereby enhancing the adaptability of the algorithm in diversified power data scenarios.

[0246] In summary, the present invention effectively solves the problem of table data extraction in the Inner Mongolia power scenario through meticulous domain adaptation and innovative algorithm optimization, and can be dynamically adjusted and optimized according to the specific needs of the region, thereby improving the accuracy and efficiency of data extraction.

[0247] The system for extracting Mongolian power table data based on image recognition described below and the method for extracting Mongolian power table data based on image recognition described above can be referred to each other.

[0248] Please see attached Figure 2 The present invention also provides a system for extracting Mongolian power table data based on image recognition, comprising:

[0249] The image processing module 100 is used to capture the natural language expression of the input data, perform speech recognition on the natural language expression, extract the power business data from the natural language expression, and grayscale, denoise, edge enhance and perspective correct the input power table image sample;

[0250] Table detection module 200, used to extract the frame structure of the table, optimize the position of the table line segments, and segment the table cells;

[0251] The character recognition module 300 is used to identify the electric power business field corresponding to each of the table cells, and establish a mapping relationship between the table cell and the electric power business data field;

[0252] A translation module 400 is used for translating the extracted Mongolian content into Chinese by a translation model;

[0253] The data output module 500 is used to generate structured electronic spreadsheet data based on the mapping relationship, fill the electric power business data into the structured electronic spreadsheet data, and output the translated structured data in the form of an electronic spreadsheet.

[0254] The system of this embodiment can be used to execute the above method embodiments, and its principles and technical effects are similar, which will not be repeated here.

[0255] Please refer to the attached Figure 3 The present invention further provides a computer device 40, comprising: a processor 41 and a memory 42, wherein the memory 42 stores a computer program executable by the processor, and when the computer program is executed by the processor, the above method is executed.

[0256] The present invention further provides a storage medium 43 on which a computer program is stored. When the computer program is run by the processor 41, the above method is executed.

[0257] Among them, the storage medium 43 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable red-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, disk or optical disk.

[0258] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for extracting Mongolian power table data based on image recognition, characterized in that: The following steps are involved: Capturing the natural language expression of the input data, performing speech recognition on the natural language expression, and extracting power business data from the natural language expression; Perform image preprocessing on the input power table image sample; Extract the frame structure of the table, optimize the position of the table segments, and segment the table cells; Identify the power service data field corresponding to each of the table cells, and establish a mapping relationship between the table cells and the power service data field; Based on the mapping relationship, structured electronic form data is generated, and the electric power service data is filled into the structured electronic form data.

2. The method for extracting Mongolian power table data based on image recognition according to claim 1 is characterized in that: The image preprocessing comprises the following steps: A grayscale algorithm based on pixel weighted average is used to convert the color image into a grayscale image; Use wavelet transform to remove image noise and suppress high-frequency noise in the wavelet high-frequency component; Enhance the edges of table lines using the Canny edge detection algorithm; Use Hough transform to detect the image tilt angle and perform perspective correction to make the table lines vertical or horizontal. in, The formula of the grayscale algorithm based on pixel weighted average is: Gray = 0.299R + 0.587G + 0.114B, Gray is the gray value of the pixel, R, G and B are the weights of the red, green and blue channels of the pixel respectively The formula of the Canny edge detection algorithm is: G(x, y) is the value of the table line edge enhancement, x and y are the coordinate positions relative to the center of the Gaussian kernel, σ is the standard deviation, e is a natural constant, and π is the circumference of a circle.

3. The method for extracting Mongolian power table data based on image recognition according to claim 1 is characterized in that: The steps of extracting the frame structure of the table, optimizing the positions of the table segments, and segmenting the table cells include: Detecting edge straight line segments in the power table image sample based on Hough transform, and screening out straight line segments in horizontal and vertical directions; Construct an energy function based on geometric optimization theory to optimize the position of table segments; The energy function is optimized iteratively to minimize the energy. After the optimization is completed, the cell area of ​​the table is divided based on the intersection coordinates. The boundary coordinates of the cell are determined by the optimized table segments. in, The energy function based on geometric optimization theory is: E represents the Laplace energy, which is used to measure the smoothness of the dangerous section of the table. i To optimize the current position vector of the table segment, To optimize the expected position vector of the table segment, i is an integer greater than 0. i The closer the value is to When the value is , it means that the smoother the table segments are, the better the optimization effect is.

4. The method for extracting Mongolian power table data based on image recognition according to claim 3 is characterized in that: The step of constructing an energy function based on geometric optimization theory and optimizing the position of the table line segments comprises: The target energy function of table segment optimization is constructed to minimize the gradient change and position deviation of the table segment. The target energy function is expressed as: Where E is the energy function value, Ω is the image area, L(x, y) represents the actual position of the table segment, x and y represent the image coordinates, respectively. represents the gradient of L(x, y), G(x, y) represents the geometric center of the table, α is the weight parameter of gradient consistency and position deviation, dxdy represents the integral operation of the entire image area, ||·|| 2 Represents the square of the Euclidean distance between vectors; The energy function E is derived using a gradient descent algorithm to obtain the update direction of the table segment position. The formula of the gradient descent algorithm is: Among them, L (t+1) (x, y) represents the updated position of the table segment after the t+1th iteration, L (t) (x, y) represents the position of the line segment at the tth iteration, η is the optimization step size factor, Represents the gradient of the energy function with respect to the line segment position; According to the formula of the gradient descent algorithm, the position of the table line segment is repeatedly updated until the convergence condition is met, and the convergence condition is: |L (t+1) (x,y)-L (t) (x,y)|<δ Among them, δ is the convergence threshold in the optimization process; The optimized line segments are repaired, and the minimum spanning tree of the line segment endpoints is constructed based on the minimum spanning tree weight function to minimize the Euclidean distance of the completed line segment. The minimum spanning tree weight function is: Where W(T) represents the total weight of the spanning tree, T is the edge set of the spanning tree, (u,v)∈T represents the edge in the spanning tree, and d(u,v) represents the Euclidean distance between endpoints u and v; After completing the broken lines, confirm the integrity of the table structure and generate the final boundaries of the cells based on the optimized table frame.

5. The method for extracting Mongolian power table data based on image recognition according to claim 1, characterized in that: The step of identifying the electric power service data field corresponding to each of the table cells and establishing a mapping relationship between the table cells and the electric power service data field comprises: Extract the character area from the cell, use the image binarization method to process the cell image, use the Otsu algorithm to calculate the global threshold, and convert the cell image into a binary image according to the global threshold; after binarization, further use the connected domain analysis method to mark and extract the possible character area; The detected character area is segmented, the text line is analyzed using the vertical projection method, and the cumulative value of each column of pixels is calculated. The boundary of the character segmentation is determined according to the continuous zero value position of the cumulative value, and a single character area is segmented; Perform feature extraction on the segmented single character area and use convolutional neural network to extract multi-scale spatial features of characters; For Mongolian characters with contextual associations, a language model based on the self-attention mechanism is used to optimize the recognition results, and the Transformer model is used to process the character sequence; Input the optimized character features into the fully connected layer for classification and output the character category label; The language model is used to correct the errors in the recognition results, and the power industry-specific dictionary is used to match and replace terms in the character sequences.

6. The method for extracting Mongolian power table data based on image recognition according to claim 1, characterized in that: The step of generating structured electronic form data based on the mapping relationship and filling the electric power business data into the structured electronic form data comprises: According to the extracted Mongolian character content, the Mongolian characters in the cell are reorganized into a continuous text sequence according to the character segmentation result and the cell coordinate order; Use a pre-trained neural machine translation model to translate Mongolian text; Match and optimize the generated Chinese translation texts with terms, and use the power industry-specific terminology dictionary to replace and verify the words in the translation results; Rearrange the translated Chinese text according to the cell coordinates of the original table and output it in a spreadsheet format.

7. The method for extracting Mongolian power table data based on image recognition according to claim 6 is characterized in that: The pre-trained neural machine translation model is based on the Transformer architecture and consists of an encoder and a decoder.

8. The method for extracting Mongolian power table data based on image recognition according to claim 6, characterized in that: The data of the output electronic form includes the Mongolian original text and the translated Chinese content, which are stored in the form of cells.

9. A system for extracting Mongolian power table data based on image recognition, used to execute the method according to any one of claims 1 to 8, characterized in that: The system comprises: An image processing module, used to capture the natural language expression of the input data, perform speech recognition on the natural language expression, extract the power business data from the natural language expression, and grayscale, denoise, edge enhance and perspective correct the input power table image sample; Table detection module, used to extract the frame structure of the table, optimize the position of table segments, and segment table cells; A character recognition module, used to identify the electric power business field corresponding to each of the table cells, and establish a mapping relationship between the table cells and the electric power business data field; The translation module is used to translate the extracted Mongolian content into Chinese by the translation model; The data output module is used to generate structured electronic spreadsheet data based on the mapping relationship, fill the electric power business data into the structured electronic spreadsheet data, and output the translated structured data in the form of an electronic spreadsheet.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images