Data processing method and device, equipment and storage medium

Abstract

The present disclosure provides a data processing method and device, equipment and storage medium, and relates to the technical field of computer. The method comprises the following steps: obtaining a to-be-processed image format page of a to-be-processed portable document format (PDF) file, wherein the to-be-processed image format page is obtained by converting a to-be-processed page format in the to-be-processed PDF file into an image; performing chart detection on the to-be-processed image format page by using a chart detection model to obtain information of a target chart region of the to-be-processed image format page; classifying the target chart region by using a chart classification model to obtain a chart category label of the target chart region, wherein the chart category label comprises a data chart category, a non-data chart category and a table category; and obtaining related content of the target chart region of the to-be-processed image format page of the to-be-processed PDF file according to the chart category label of the target chart region. The method improves the accuracy of extracting chart information in the PDF file.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a data processing method, apparatus, device, and readable storage medium. Background Technology

[0002] Portable Document Format (PDF) is a file format used for exchanging files in a way that is independent of applications, operating systems, and hardware. Research reports used for specialized research are often in PDF format, and the figures and tables in these reports typically contain important information needed by researchers. Current techniques for extracting figures and tables from PDF files based on image or table-based graphical features lack specificity and result in low accuracy of the obtained information.

[0003] As mentioned above, improving the accuracy of extracting chart information from PDF files has become an urgent problem to be solved.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this disclosure is to provide a data processing method, apparatus, device, and readable storage medium that can improve the accuracy of extracting chart information from PDF files to a certain extent.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] According to one aspect of this disclosure, a data processing method is provided, comprising: acquiring a page of a portable document format PDF file to be processed, wherein the page of a portable document format PDF file to be processed is obtained by converting the page of a portable document format PDF file to be processed into an image; performing chart detection on the page of a portable document format PDF file to be processed using a chart detection model to obtain information about a target chart region of the page of a portable document format PDF file; classifying the target chart region using a chart classification model to obtain a chart category label for the target chart region, wherein the chart category label includes data chart, non-data chart, and table; and obtaining relevant content of the target chart region of the page of a portable document format PDF file to be processed based on the chart category label of the target chart region.

[0008] According to one embodiment of this disclosure, obtaining the relevant content of the target chart area of ​​the image format page to be processed in the PDF file to be processed based on the chart category label of the target chart area includes: when the chart category label of the target chart area is a data chart, performing text detection on the target chart area using a text detection model to obtain the target text area in the target chart area; performing character recognition on the target text area using an optical character recognition method to obtain an optical character recognition result; and correcting the optical character recognition result based on the basic chart information obtained by performing low-level parsing of the PDF file to be processed to obtain the text information in the target text area.

[0009] According to one embodiment of this disclosure, the optical character recognition result is corrected based on the basic chart information obtained by performing low-level parsing on the PDF file to be processed, including: obtaining the similarity between a first string in the optical character recognition result and a second string in the basic chart information; if the similarity between the first string and the second string is greater than a preset similarity threshold, the first string in the optical character recognition result is modified to the second string.

[0010] According to one embodiment of this disclosure, the information of the target chart area includes the size information of the target chart area; the method further includes: determining whether the target chart area is of normal size based on the size information of the target chart area; classifying the target chart area using a chart classification model to obtain a chart category label for the target chart area, including: classifying the target chart area using a chart classification model to obtain a chart category label for the target chart area when the size of the target chart area is determined to be normal.

[0011] According to an embodiment of this disclosure, the information of the target chart area includes the location information of the target chart area; the method further includes: obtaining a target text line within a preset distance above and / or below the target chart area in the image format page to be processed based on the location information of the target chart area; obtaining the probability that the target text line is the title of the target chart in the target chart area using a chart title model; and obtaining the title of the target chart as the target text line when the probability that the target text line is the title of the target chart is greater than a preset probability threshold.

[0012] According to one embodiment of this disclosure, a chart detection model is used to perform chart detection on the image format page to be processed to obtain information about the target chart region of the image format page to be processed. This includes: using the chart detection model to perform chart detection on the image format page to be processed to obtain information about a predicted chart region, wherein the information about the predicted chart region includes position information, size information, and page number information of at least one predicted chart region; if, based on the position information of the predicted chart regions, it is determined that two predicted chart regions overlap, then, based on the position information and size information of the two predicted chart regions, it is determined that the two predicted chart regions will be merged into one target chart region, or that the two predicted chart regions will be adjusted to obtain two target chart regions that do not overlap; and / or if, based on the position information and page number information of the predicted chart regions, it is determined that a cross-page table exists, then the two predicted chart regions corresponding to the cross-page table will be merged to obtain the target chart region corresponding to the cross-page table.

[0013] According to an embodiment of this disclosure, a chart detection model is used to perform chart detection on the image format page to be processed, and information on the target chart region of the image format page to be processed is obtained, including:

[0014] The chart detection model is used to detect charts on the image format page to be processed, and information about predicted chart regions is obtained. The information about predicted chart regions includes the position information, size information, and page number information of at least one predicted chart region. Based on the page number information of the predicted chart regions, the position information and size information of the parsed chart regions in the image format page to be processed are obtained from the basic chart information obtained by low-level parsing of the PDF file to be processed. Based on the position information and size information of the predicted chart regions and the position information and size information of the parsed chart regions, if it is determined that there is a parsed chart region corresponding to the predicted chart region, the predicted chart region is taken as the target chart region, and the information of the target chart region is obtained. And / or based on the position information and size information of the parsed chart regions, if it is determined whether the predicted chart region contains multiple charts, the predicted chart region is split to obtain the information of the target chart region. And / or based on the position information and size information of the parsed chart regions, if it is determined that two adjacent predicted chart regions belong to the same chart, the two adjacent predicted chart regions are merged to obtain the information of the target chart region.

[0015] According to one embodiment of this disclosure, the information of the target chart region includes the position information and size information of the target chart region; performing chart detection on the image format page to be processed using a chart detection model to obtain the information of the target chart region of the image format page to be processed includes: performing chart detection on the image format page to be processed using the chart detection model to obtain the information of the predicted chart region, the information of the predicted chart region including the position information, size information and page number information of the predicted chart region; obtaining the position information and size information of the parsed chart region corresponding to the predicted chart region in the image format page to be processed from the basic chart information obtained by performing low-level parsing on the PDF file to be processed based on the page number information of the predicted chart region; adjusting the position and size of the predicted chart region based on the position information and size information of the parsed chart region to obtain the position information and size information of the target chart region.

[0016] According to one embodiment of this disclosure, obtaining the relevant content of the target chart area of ​​the image format page of the PDF file to be processed based on the chart category label of the target chart area includes: when the chart category label of the target chart area is a table, obtaining the text content corresponding to the target chart area in the image format page to be processed from the basic chart information obtained by performing low-level parsing on the PDF file to be processed.

[0017] According to one embodiment of this disclosure, obtaining the relevant content of the target chart area of ​​the image format page of the PDF file to be processed based on the chart category label of the target chart area includes: when the chart category label of the target chart area is a non-data chart, obtaining the target non-data chart of the image format from the target chart area of ​​the image format page to be processed.

[0018] According to one embodiment of this disclosure, the PDF file to be processed includes a first research report; obtaining relevant content of the target chart area of ​​the image format page to be processed in the PDF file includes: obtaining relevant content of the target chart area of ​​the first research report, wherein the relevant content of the target chart area of ​​the first research report includes at least one of text information in the target chart area of ​​the first research report and the title of the target chart; the method further includes: obtaining an input search term; matching the search term with the relevant content of the target chart area of ​​the first research report; and, if the search term successfully matches the relevant content of the target chart area of ​​the first research report, outputting a first search result in response to the operation of inputting the search term, wherein the first search result indicates that the first research report has been retrieved.

[0019] According to an embodiment of this disclosure, the PDF file to be processed further includes a second research report; the method further includes: obtaining the text content of the second research report; matching the search term with the text content of the second research report; and outputting a first search result when the search term successfully matches the relevant content of the target chart area of ​​the first research report, including: outputting the first search result when the search term successfully matches the relevant content of the target chart area of ​​the first research report and the search term successfully matches the text content of the second research report, wherein the first search result is further used to indicate that the second research report was retrieved and that the first research report has a higher priority than the second research report.

[0020] According to another aspect of this disclosure, a data processing apparatus is provided, comprising: a to-be-processed image acquisition module, configured to acquire a to-be-processed image format page of a portable document format PDF file, wherein the to-be-processed image format page is obtained by converting the to-be-processed page format in the PDF file into an image; a chart region information acquisition module, configured to perform chart detection on the to-be-processed image format page using a chart detection model to obtain information about a target chart region of the to-be-processed image format page; a chart region classification module, configured to classify the target chart region using a chart classification model to obtain a chart category label for the target chart region, wherein the chart category label includes data chart, non-data chart, and table; and a chart content acquisition module, configured to obtain relevant content of the target chart region of the to-be-processed image format page of the PDF file based on the chart category label of the target chart region.

[0021] According to an embodiment of this disclosure, the chart content acquisition module is further configured to: when the chart category label of the target chart area is a data chart, perform text detection on the target chart area using a text detection model to obtain a target text area in the target chart area; perform character recognition on the target text area using an optical character recognition method to obtain an optical character recognition result; and correct the optical character recognition result based on the basic chart information obtained by performing low-level parsing on the PDF file to be processed to obtain the text information in the target text area.

[0022] According to an embodiment of this disclosure, the chart content acquisition module is further configured to: obtain the similarity between a first string in the optical character recognition result and a second string in the chart basic information; and modify the first string in the optical character recognition result to the second string if the similarity between the first string and the second string is greater than a preset similarity threshold.

[0023] According to one embodiment of this disclosure, the information of the target chart area includes the size information of the target chart area; the chart area classification module is further configured to determine whether the target chart area is of normal size based on the size information of the target chart area; if the size of the target chart area is determined to be normal, the target chart area is classified using a chart classification model to obtain the chart category label of the target chart area.

[0024] According to an embodiment of this disclosure, the information of the target chart area includes the location information of the target chart area; the chart content acquisition module is further configured to obtain a target text line within a preset distance above and / or below the target chart area in the image format page to be processed, based on the location information of the target chart area; obtain the probability that the target text line is the title of the target chart in the target chart area using a chart title model; and obtain the title of the target chart as the target text line when the probability that the target text line is the title of the target chart is greater than a preset probability threshold.

[0025] According to an embodiment of this disclosure, the chart region information acquisition module is further configured to perform chart detection on the image format page to be processed using the chart detection model to obtain information of the predicted chart region. The information of the predicted chart region includes the position information, size information, and page number information of at least one predicted chart region. If it is determined that the positions of two predicted chart regions overlap based on the position information of the predicted chart regions, the module determines, based on the position information and size information of the two predicted chart regions, to merge the two predicted chart regions into one target chart region, or to adjust the two predicted chart regions to obtain two target chart regions that do not overlap. And / or if it is determined based on the position information and page number information of the predicted chart regions that there is a cross-page table, the module merges the two predicted chart regions corresponding to the cross-page table to obtain the target chart region corresponding to the cross-page table.

[0026] According to an embodiment of this disclosure, the chart region information acquisition module is further configured to: perform chart detection on the image format page to be processed using the chart detection model to obtain information of a predicted chart region, wherein the information of the predicted chart region includes position information, size information, and page number information of at least one predicted chart region; based on the page number information of the predicted chart region, obtain the position information and size information of the parsed chart region in the image format page to be processed from the basic chart information obtained by performing low-level parsing on the PDF file to be processed; based on the position information and size information of the predicted chart region and the position information and size information of the parsed chart region, if it is determined that there is a parsed chart region corresponding to the predicted chart region, take the predicted chart region as the target chart region and obtain the information of the target chart region; and / or based on the position information and size information of the parsed chart region, if it is determined that the predicted chart region contains multiple charts, split the predicted chart region to obtain the information of the target chart region; and / or based on the position information and size information of the parsed chart region, if it is determined that two adjacent predicted chart regions belong to the same chart, merge the two adjacent predicted chart regions to obtain the information of the target chart region.

[0027] According to an embodiment of this disclosure, the information of the target chart region includes the position information and size information of the target chart region; the chart region information acquisition module is further configured to use the chart detection model to perform chart detection on the image format page to be processed, and obtain the information of the predicted chart region, the information of the predicted chart region including the position information, size information and page number information of the predicted chart region; based on the page number information of the predicted chart region, the position information and size information of the parsed chart region corresponding to the predicted chart region in the image format page to be processed are obtained from the basic chart information obtained by performing low-level parsing on the PDF file to be processed; based on the position information and size information of the parsed chart region, the position and size of the predicted chart region are adjusted to obtain the position information and size information of the target chart region.

[0028] According to one embodiment of this disclosure, the chart content acquisition module is further configured to, when the chart category label of the target chart area is a table, obtain the text content corresponding to the target chart area in the image format page to be processed from the basic chart information obtained by performing low-level parsing on the PDF file to be processed.

[0029] According to one embodiment of this disclosure, the chart content acquisition module is further configured to acquire a target non-data chart in image format from the target chart area of ​​the image format page when the chart category label of the target chart area is a non-data chart.

[0030] According to an embodiment of this disclosure, the PDF file to be processed includes a first research report; the chart content acquisition module is further configured to: acquire relevant content of a target chart area of ​​the first research report, wherein the relevant content of the target chart area of ​​the first research report includes at least one of text information in the target chart area of ​​the first research report and the title of the target chart; the device further includes: a search term acquisition module, configured to acquire input search terms; a search term matching module, configured to match the search terms with the relevant content of the target chart area of ​​the first research report; and a search result output module, configured to output a first search result when the search terms are successfully matched with the relevant content of the target chart area of ​​the first research report, in response to the operation of inputting the search terms, wherein the first search result indicates that the first research report has been retrieved.

[0031] According to an embodiment of this disclosure, the PDF file to be processed further includes a second research report; the chart content acquisition module is further configured to: acquire the text content of the second research report; the search term matching module is further configured to match the search term with the text content of the second research report; the search result output module is further configured to output the first search result when the search term is successfully matched with the relevant content of the target chart area of ​​the first research report and the search term is successfully matched with the text content of the second research report, wherein the first search result is further configured to indicate that the second research report was retrieved and that the first research report has a higher priority than the second research report.

[0032] According to another aspect of this disclosure, an apparatus is provided, comprising: a memory, a processor, and executable instructions stored in the memory and executable in the processor, wherein the processor, when executing the executable instructions, implements any of the methods described above.

[0033] According to another aspect of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, which, when executed by a processor, implement any of the methods described above.

[0034] The data processing method provided in the embodiments of this disclosure acquires a page in image format obtained by converting the page format of a PDF file to be processed into an image. It then uses a chart detection model to perform chart detection on the image format page to obtain information about the target chart region. A chart classification model is used to classify the target chart region, obtaining chart category labels for the target chart region, including data charts, non-data charts, and tables. Based on the chart category labels of the target chart region, the relevant content of the target chart region of the image format page in the PDF file to be processed is obtained. By classifying the target chart region into data charts, non-data charts, and tables after detection, the relevant content of the chart region can be obtained more effectively according to the chart category labels, thereby improving the accuracy of extracting chart information from PDF files.

[0035] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this disclosure. Attached Figure Description

[0036] The above and other objects, features and advantages of this disclosure will become more apparent from a detailed description of exemplary embodiments thereof with reference to the accompanying drawings.

[0037] Figure 1 A schematic diagram of a system structure according to an embodiment of this disclosure is shown.

[0038] Figure 2 A flowchart of a data processing method according to an embodiment of this disclosure is shown.

[0039] Figure 3 It shows Figure 2 The step S204 shown is a schematic diagram of the processing procedure in one embodiment.

[0040] Figure 4 It shows Figure 2 The step S204 shown is a schematic diagram of the processing procedure in another embodiment.

[0041] Figure 5 It shows Figure 2 The step S204 shown is a schematic diagram of the processing procedure in another embodiment.

[0042] Figure 6 According to Figures 2 to 5 This diagram illustrates a document layout parsing process.

[0043] Figure 7 It shows Figure 2 The step S206 shown is a schematic diagram of the processing procedure in one embodiment.

[0044] Figure 8It is based on Figure 2 The diagram illustrates a method for obtaining chart titles.

[0045] Figure 9 It shows Figure 2 The step S208 shown is a schematic diagram of the processing procedure in one embodiment.

[0046] Figure 10 It shows Figure 9 The step S906 shown is a schematic diagram of the processing procedure in one embodiment.

[0047] Figure 11 According to Figure 9 and Figure 10 This diagram illustrates a process for acquiring data graph content.

[0048] Figure 12 according to Figures 2 to 11 A flowchart for parsing charts in a research report PDF document is shown.

[0049] Figure 13 The image format of a PDF document page to be processed is shown in an application scenario.

[0050] Figure 14 It is based on Figure 13 A schematic diagram of a chart detection result is shown.

[0051] Figure 15 It is based on Figure 14 An example diagram showing the merging of chart areas is provided.

[0052] Figure 16 It is based on Figure 15 An example diagram showing how to adjust a chart area is provided.

[0053] Figure 17 It is based on Figure 16 An example diagram of abnormal region removal is shown.

[0054] Figure 18 It is based on Figure 17 An example graph showing the results of a chart classification is provided.

[0055] Figure 19 It is based on Figure 18 An example diagram of a chart context is shown.

[0056] Figure 20 It is based on Figure 19 An example diagram showing chart title matching is provided.

[0057] Figure 21 It is based on Figure 18 and Figure 20An example diagram showing chart title matching is provided.

[0058] Figure 22 It is based on Figures 2 to 11 The diagram shows an example architecture of an intelligent investment research platform.

[0059] Figure 23 This is a flowchart illustrating a research report search method according to an exemplary embodiment.

[0060] Figure 24 This is a flowchart illustrating another research report search method according to an exemplary embodiment.

[0061] Figure 25 A block diagram of a data processing apparatus according to an embodiment of the present disclosure is shown.

[0062] Figure 26 This is a block diagram illustrating another data processing apparatus according to an exemplary embodiment.

[0063] Figure 27 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0064] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0065] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, apparatuses, steps, etc., can be employed. In other instances, well-known structures, methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0066] Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this disclosure, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. The symbol " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0067] In this disclosure, unless otherwise expressly specified and limited, the term "connection" and similar terms should be interpreted broadly, for example, it can refer to an electrical connection or the ability to communicate with each other; it can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this disclosure according to the specific circumstances.

[0068] In specialized research fields, researchers often need to find, read, and analyze research reports relevant to that field. For example, in investment research, researchers need to read numerous company purchase or publicly available research reports. Content analysis of research reports is fundamental to many platform functionalities, and chart extraction is a crucial supplement to the analysis results, in addition to regular text extraction. For instance, in structured analysis functions, presenting only key points extracted from text can easily lead to a summary-like presentation, lacking supporting evidence based on data visualization and analysis. Readers are prone to missing information while quickly browsing large amounts of text, whereas charts highlight core content, making them less likely to be overlooked and providing a good summary of the report's indicators and arguments. Furthermore, searches and recommendations based solely on text content are prone to including low-relevance content. A common scenario is that a research report mentions a keyword sporadically, and it's difficult to determine the relevance between the report's content and the keyword based solely on text. However, if the report's charts contain relevant content, the probability of a high relevance between the report and that keyword is much higher.

[0069] Research reports used for specialized research are typically in PDF format. The limitations of existing technologies for locating and extracting individual images or tables from PDF documents are mainly reflected in the following aspects: 1) While layout analysis technologies can locate images and tables, they rely on graphical features rather than deep learning-based object detection, resulting in weak generalization ability; 2) Chart parsing technologies usually focus on processing specific types of targets, i.e., only processing charts or only processing tables. The target to be processed is already categorized at input, without addressing the issue of chart differentiation; 3) Due to a lack of specific application scenarios, technical solutions are generally designed only up to chart extraction, without further processing of table and image content, such as analyzing chart types or included metrics. Overall, these technologies focus on solving technical problems, lacking connection to business scenarios and failing to provide solutions for extracting charts from PDF documents that meet actual business needs. Therefore, the accuracy of chart information extracted from PDF documents in real-world business scenarios is low.

[0070] Therefore, this disclosure provides a data processing method that uses a deep learning model to analyze the layout of a document, identifies areas containing charts, and then uses a classification model to determine whether the chart area is a table, a data graph, or a non-data graph. This achieves unified processing of charts and allows for more targeted acquisition of relevant content from chart areas based on chart category labels, thereby improving the accuracy of extracting chart information from PDF files.

[0071] Figure 1 An exemplary system architecture 10 is shown that can be applied to the data processing methods or data processing apparatus of this disclosure.

[0072] like Figure 1 As shown, system architecture 10 may include terminal device 102, network 104, and server 106. Terminal device 102 may be various electronic devices with a display screen and supporting input and output, including but not limited to smartphones, tablets, laptops, desktop computers, wearable devices, virtual reality devices, smart home devices, in-vehicle terminals, etc. Network 104 is used as a medium to provide a communication link between terminal device 102 and server 106. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc. Server 106 may be a server or server cluster providing various services, such as a web server, a backend processing server, a database server, etc.

[0073] Users can use terminal device 102 to interact with server 106 via network 104 to receive or send data. For example, a user can download a PDF file to be processed from server 106 to terminal device 102 via network 104, and then use PDF processing software on terminal device 102 to convert it into a multi-page image format. Another example is that a user can perform operations on terminal device 102, using the chart detection model on server 106 to perform chart detection on the image format page to be processed via network 104. Yet another example is that a user can perform operations on terminal device 102, sending the relevant content of the obtained target chart area to server 106 for storage via network 104.

[0074] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0075] Figure 2 This is a flowchart illustrating a data processing method according to an exemplary embodiment. For example... Figure 2 The method shown can be applied, for example, to the server side of the above system, or to the terminal devices of the above system.

[0076] refer to Figure 2 The method 20 provided in this embodiment may include the following steps.

[0077] In step S202, the image format page to be processed of the PDF file is obtained by converting the format of the page to be processed in the PDF file into an image.

[0078] In some embodiments, the PDF file to be processed is an entire PDF document that may contain unknown charts, and the PDF document typically contains rich text information, as well as images and / or tables that may contain an unknown number, type, size and location.

[0079] In some embodiments, for example, based on the form of the chart, these charts can be divided into several categories: (1) data charts, including various line charts, pie charts, bar charts, or other images representing quantity or statistical characteristics of quantity; (2) non-data charts, including various concept illustrations, process descriptions, photos, etc.; (3) tables, including tables mainly displaying numbers, tables mainly displaying text, and tables that combine numbers and text. Some of these tables have clear borders, some do not have borders, or have partial borders. Images in the PDF file to be processed can be embedded as whole images (the text in the image is also represented as an image), or they can be a mixture of images and related text in the image (such as legends, coordinate axis descriptions, quantity and size); while tables can be embedded as text or numbers.

[0080] In some embodiments, each page of the PDF file to be processed can be converted into an image format first, and one or more pages can be selected as the image format pages to be processed, serving as input for the next step of the chart detection model. For example, each page of the PDF can be converted into an RGB (red, green, blue) three-channel image format, serving as the basis for a chart detection and localization algorithm based on a deep learning model.

[0081] In step S204, a graph detection model is used to perform graph detection on the image format page to be processed, thereby obtaining information about the target graph region of the image format page to be processed.

[0082] In some embodiments, the chart detection model can be, for example, the Faster-RCNN algorithm model. This model detects the position and size of possible charts / tables on each page of a document, outputting the position as the coordinates of the top-left corner of the chart / table region plus its width and height values. The model parameters required by the algorithm are obtained through training on a labeled document image training set. The process of chart detection using the Faster-RCNN algorithm model can include the following steps: The input three-channel image is first processed by a Convolutional Neural Network (CNN) to transform it into a feature map; then, based on the feature map, a Region Proposal Network (RPN) is used to generate a candidate region set; next, combining the feature map and candidate region set information, feature maps of the candidate regions are obtained through Region of Interest (ROI) pooling; finally, a classification layer determines whether a candidate region is a chart, and a regression layer finely adjusts the position and size of the candidate regions determined to be charts.

[0083] In other embodiments, Fast-RCNN algorithm models can also be used as graph detection models. The number of network layers in Fast-RCNN, Faster-RCNN and other algorithm models can be optimized according to actual experimental results. This disclosure does not limit the specific network structure.

[0084] In some embodiments, after the chart detection model obtains the predicted chart area, the predicted chart area can be processed by merging, splitting, or adjusting its position. Specific implementation methods can be found in [reference needed]. Figures 3 to 6 .

[0085] In step S206, the target chart area is classified using a chart classification model to obtain the chart category label of the target chart area. The chart category label includes data chart, non-data chart, and table.

[0086] In some embodiments, for example, abnormally sized portions of all target chart areas can be filtered out first. Then, in the remaining target chart areas of normal size, a chart classification model is used to classify their contents. Chart categories include data charts, non-data charts, and tables. The predicted labels for classification may also include non-chart categories. For specific implementation methods of size filtering, please refer to [reference needed]. Figure 7 .

[0087] In some embodiments, the chart classification model can be a classification model based on a deep learning algorithm. The model input is the image information of the chart region. The algorithm outputs the probability that the content of the region is a data chart, a non-data chart, or a table based on features such as the distribution of lines, color fill, and color outline within the image. The category with the highest probability value is taken as the classification result. For example, the GoogLeNet model based on a convolutional neural network algorithm can be used. The training set consists of images collected from the internet, images converted from charts in research reports, and labeled samples collected from the internet.

[0088] In step S208, the relevant content of the target chart area of ​​the image format page to be processed in the PDF file is obtained according to the chart category label of the target chart area.

[0089] In some embodiments, for example, the title text of the target chart area can be extracted using a chart title model. Specific implementation details can be found in [reference needed]. Figure 8 .

[0090] In some embodiments, when the target chart area is labeled as a data chart, the focus can be on extracting the text content within it, and the extraction result can be used as the output. For example, specific implementation methods can be found in [reference needed]. Figures 9 to 11 .

[0091] In some embodiments, when the chart category label of the target chart area is a table, the text content corresponding to the target chart area in the image format page to be processed can be obtained from the basic chart information obtained by parsing the PDF file to be processed, and stored in the database as table content for use in related subsequent applications.

[0092] In some embodiments, if the chart category label of the target chart area is a non-data chart, the target non-data chart in image format is obtained from the target chart area of ​​the image format page to be processed. For non-data charts, the correct image position and size can be selected, and the image can be cropped and directly stored in the database.

[0093] According to the data processing method provided in this disclosure, a page in image format is obtained by converting the page format of a PDF file to be processed into an image. A chart detection model is used to detect charts in the image format page to obtain information about the target chart region. A chart classification model is then used to classify the target chart region, obtaining chart category labels for the target chart region, including data charts, non-data charts, and tables. Based on the chart category labels of the target chart region, the relevant content of the target chart region in the image format page of the PDF file to be processed is obtained. By classifying the target chart region into data charts, non-data charts, and tables after detection, the relevant content of the chart region can be obtained more effectively according to the chart category labels, thereby improving the accuracy of extracting chart information from PDF files.

[0094] The images and tables in the research report PDF documents obtained using the data processing method provided in this embodiment serve as an important supplement to the refined text content, allowing researchers to quickly understand the core indicators and arguments used in the report. Combined with the extraction of the report's chapter structure and core viewpoints, this empowers the analysis, presentation, and structured compilation of research reports. The charts and graphs extracted from the reports can also be used for searching and recommendation, helping researchers find reports with relevant indicators, enriching search dimensions, and improving search efficiency. Furthermore, the charts and graphs are a primary source of material for the research report generation function, helping researchers write and enrich their reports.

[0095] Figure 3 It shows Figure 2 The diagram shown illustrates step S204 in one embodiment. Figure 3 As shown in the present embodiment, step S204 may further include the following steps.

[0096] Step S302: Use the chart detection model to perform chart detection on the format page of the image to be processed, and obtain information of the predicted chart area. The information of the predicted chart area includes the position information, size information and page number information of at least one predicted chart area.

[0097] In some embodiments, the specific implementation of using the chart detection model to perform chart detection on the format page of the image to be processed can be referred to step S204.

[0098] Step S3042: If it is determined that the positions of two predicted chart areas overlap based on the position information of the predicted chart areas, then based on the position information and size information of the two predicted chart areas, it is determined to merge the two predicted chart areas into one target chart area, or to adjust the two predicted chart areas to obtain two target chart areas that do not overlap.

[0099] In some embodiments, for example, if two chart candidate regions partially overlap, and the percentage of their overlap is greater than 'a'% of the smaller (or larger, or the sum of the two predicted chart regions) area, the two chart candidate regions are merged into one. The parameter 'a' can be obtained by training on a labeled document image training set.

[0100] In some embodiments, for example, if two candidate chart regions partially overlap, and the percentage of their overlap is less than (or equal to) a% of the smaller (or larger, or the sum of the two predicted chart regions) area, the positions and sizes of the two predicted chart regions are adjusted so that they do not overlap. Parameter a is obtained by training on a set of labeled document images.

[0101] Step S3044: If it is determined that there is a cross-page table based on the location information and page number information of the predicted chart area, the two predicted chart areas corresponding to the cross-page table are merged to obtain the target chart area corresponding to the cross-page table.

[0102] In some embodiments, for example, based on the position, size, and page number of adjacent predicted chart areas, combined with basic chart information and possible other preset rule algorithms (e.g., related content exists, and / or the previous predicted chart area is at the bottom of the previous page, the next predicted chart area is at the top of the next page, etc.), it is determined whether a cross-page table exists. If it exists, the cross-page table is marked with its position, size, and page number, and then merged for use when extracting table content later.

[0103] Figure 4 It shows Figure 2 The illustrated step S204 is a process diagram in another embodiment. (See diagram below.) Figure 4 As shown in the present embodiment, step S204 may further include the following steps.

[0104] Step S402: Use the chart detection model to perform chart detection on the format page of the image to be processed, and obtain information of the predicted chart area. The information of the predicted chart area includes the position information, size information and page number information of at least one predicted chart area.

[0105] In some embodiments, the specific implementation of using the chart detection model to perform chart detection on the format page of the image to be processed can be referred to step S204.

[0106] Step S404: Based on the page number information of the predicted chart area, obtain the position and size information of the parsed chart area in the image format page to be processed from the basic chart information obtained from the underlying parsing of the PDF file to be processed.

[0107] In some embodiments, after inputting the PDF file to be processed, the basic information of the charts in the PDF file can be extracted based on the underlying data parsing method of the PDF document. This information may include various information of the Chinese text and charts, such as coordinates, rotation, size, content, etc., which can be used as auxiliary information for subsequent chart positioning and content extraction.

[0108] Step S4062: Based on the location and size information of the predicted chart area and the location and size information of the parsed chart area, if it is determined that there is a parsed chart area corresponding to the predicted chart area, the predicted chart area is taken as the target chart area, and the information of the target chart area is obtained.

[0109] In some embodiments, prediction chart areas that are not present in the basic chart information can be deleted based on the basic chart information.

[0110] Step S4064: Based on the location and size information of the parsed chart area, and in the case of determining whether the predicted chart area contains multiple charts, the predicted chart area is split to obtain the information of the target chart area.

[0111] In some embodiments, based on the position and size information in the basic chart information, combined with a preset rule algorithm (e.g., two non-overlapping chart positions in the basic chart information are within a predicted chart area, and the two chart positions in the basic chart information are not connected), it can be determined whether a predicted chart area contains only one chart. If the determination is negative, the predicted chart area is split to ensure that a predicted chart area contains only one chart.

[0112] Step S4066: Based on the location and size information of the parsed chart area, if it is determined that two adjacent predicted chart areas belong to the same chart, the two adjacent predicted chart areas are merged to obtain the information of the target chart area.

[0113] In some embodiments, based on the position and size information in the basic information of the chart, combined with a preset rule algorithm (e.g., the positions of two non-overlapping predicted chart areas are within the chart position area in the basic information of the chart, and the positions of the two predicted chart areas are adjacent), it can be determined whether two adjacent predicted chart areas point to the same chart. If the determination is yes, the adjacent predicted chart areas are merged.

[0114] like Figure 3 and Figure 4The method shown can be implemented by a chart merging and splitting model. The input of the chart merging and splitting model is the predicted chart area position, size, page number, etc., output by the chart detection model, as well as the basic chart information obtained from the underlying parsing. The output is the target chart area position, size, and other related information. Any one of steps S3042 and S3044 can be performed individually or in any order. Any one of steps S4062, S4064, and S4066 can be performed individually or any two or three of them in any order. This disclosure does not impose any restrictions.

[0115] Figure 5 It shows Figure 2 The step S204 shown is a schematic diagram of the processing procedure in another embodiment. (See diagram below.) Figure 5 As shown in the present embodiment, step S204 may further include the following step: adjusting the positioning of the prediction chart area.

[0116] Step S502: Use the chart detection model to perform chart detection on the format page of the image to be processed, and obtain information about the predicted chart area. The information about the predicted chart area includes the position information, size information and page number information of the predicted chart area.

[0117] In some embodiments, the specific implementation of using the chart detection model to perform chart detection on the format page of the image to be processed can be referred to step S204.

[0118] Step S504: Based on the page number information of the predicted chart area, obtain the position and size information of the parsed chart area corresponding to the predicted chart area in the image format page to be processed from the basic chart information obtained by low-level parsing of the PDF file to be processed.

[0119] Step S506: Based on the position and size information of the parsed chart area, adjust the position and size of the predicted chart area to obtain the position and size information of the target chart area.

[0120] In some embodiments, the position and size of the predicted chart area can be checked and adjusted based on the chart's basic information to ensure that the predicted chart area includes the range of the corresponding chart in the chart's basic information.

[0121] In some embodiments, redundant blank borders that are not present in the chart's basic information in the prediction chart area can be removed.

[0122] Figure 6 According to Figures 2 to 5 This diagram illustrates a document layout parsing process. Figure 6As shown, firstly, each page of the PDF file (PDF document) 6002 to be processed is converted into image format (S602) as input for the next step of the chart detection algorithm. Then, in the chart detection step (S604), for each page's image format, a chart detection algorithm based on a deep learning model is used to detect possible chart targets, including data charts, non-data charts, and tables, as detailed in step S204. Next, referring to the basic chart information output from the underlying parsing, it is decided whether to split a detected predicted chart region into multiple chart regions, or to merge multiple detected predicted chart regions into one chart region (S606), as detailed in step S204. Figure 3 and Figure 4 Based on the basic chart information output from the underlying parsing, the position and size of the predicted chart area are adjusted according to preset rules to obtain the final target chart area output. See details... Figure 5 The output after adjusting the positioning of the predicted chart area is the target chart area output after the document layout is parsed.

[0123] According to the document layout parsing method provided in this disclosure, the area where the figure or table is located in the document is obtained, so that each final determined target chart area contains only one figure or table, and each figure or table is contained in only one final determined target chart area.

[0124] Figure 7 It shows Figure 2 The step S206 shown is a schematic diagram of the processing procedure in one embodiment. (See attached diagram.) Figure 7 As shown in the present embodiment, step S206 may further include the following step: filtering out target chart areas with abnormal sizes.

[0125] Step S702: Determine whether the target chart area is the correct size based on its size information.

[0126] Step S704: If the target chart area is determined to be of normal size, the target chart area is classified using a chart classification model to obtain the chart category label of the target chart area.

[0127] In some embodiments, for example, the size anomaly of the target chart area can be defined as a target chart area with a width less than b% of the page width (e.g., b% = 10%, b% = 15%, or b% = 20%, etc.), and the parameter b can be obtained by training on a labeled document image training set. These charts can be image content that is not closely related to the content of the document itself, such as the logo of the research report author (e.g., a securities firm).

[0128] Figure 8 It is based on Figure 2The diagram illustrates a method for obtaining chart titles. Figure 8 As shown in this embodiment, the data processing method 20 may further include the following steps for obtaining the text title of the target chart area.

[0129] Step S802: Based on the location information of the target chart area, obtain the target text line within a preset distance above and / or below the target chart area in the image format page to be processed.

[0130] In some embodiments, the text lines that are closest to the target chart area above and below it can be extracted based on the location of the target chart area.

[0131] Step S804: Use the chart title model to obtain the probability that the target text line is the title of the target chart in the target chart area.

[0132] In some embodiments, for example, the chart title model can employ the FastText algorithm. FastText is a text classification model based on word vector algorithms, combining word vector models and classification models. The model input consists of possible candidate chart titles, and the output is the probability that the text is a chart title. If the probability is higher than a preset probability threshold, it is determined to be the title of the target chart. The training set used to train the model is formed by summarizing chart titles from research reports, chart-related corpora collected online, and other non-title text from research reports after annotation.

[0133] Step S806: When the probability that the target text line is the title of the target chart is greater than a preset probability threshold, the title of the target chart is obtained as the target text line.

[0134] In some embodiments, if there are multiple possible alternative titles, the probability of these text lines being figure / table titles is determined using a figure / table title model, and the result with the highest probability is taken as the title of the corresponding figure / table.

[0135] In some embodiments, such as when multiple charts / tables are adjacent to each other, the text content above and below the middle chart is likely to be the chart title. In this case, the title position of the middle chart should be determined by using a rule model based on the identification results of the title positions of the top and bottom charts / tables.

[0136] According to the method for obtaining text titles of target chart areas provided in this disclosure, the method analyzes the text immediately above and below the chart area and uses a probability model to determine which text corresponds to the chart / table titles. In a PDF document, chart titles may be above or below the chart, or two or more charts may be placed side-by-side in the same row. In this case, the corresponding chart titles may overlap. The method in this disclosure matches the chart titles with the corresponding titles in the PDF document, improving the accuracy of text title extraction.

[0137] Figure 9 It shows Figure 2 The step S208 shown is a schematic diagram of the processing procedure in one embodiment. (See attached diagram.) Figure 9 As shown in this embodiment, step S208 may further include the following step: extracting the text content in the target chart area of ​​the data chart type.

[0138] Step S902: Use a text detection model to perform text detection on the target chart area of ​​the data graph type to obtain the target text area in the target chart area.

[0139] In some embodiments, for example, a text detection model CTPN (Connectionist Text Proposal Network) can be used to detect text that may exist in the data graph and determine the location of the text. This text may include coordinate axis descriptions, legends, and other relevant text descriptions in the graph.

[0140] Step S904: Perform character recognition on the target text region using the Optical Character Recognition (OCR) method to obtain the optical character recognition result.

[0141] In some embodiments, OCR technology can be used to perform text recognition on the image of the target text region in the data image and output the text information in the image of the target text region.

[0142] Step S906: Based on the basic information of the charts obtained from the low-level parsing of the PDF file to be processed, the optical character recognition results are corrected to obtain the text information in the target text area.

[0143] In some embodiments, the optical character recognition result can be compared with the basic information of the charts parsed from the underlying PDF data to correct the OCR text recognition result. Specific implementation methods can be found in [reference needed]. Figure 10 .

[0144] Figure 10 It shows Figure 9 The step S906 shown is a schematic diagram of the processing procedure in one embodiment. (See attached diagram.) Figure 10 As shown in the present embodiment, step S906 may further include the following step: extracting the text content in the target chart area of ​​the data chart type.

[0145] Step S1002: Obtain the similarity between the first string in the optical character recognition result and the second string in the basic information of the chart.

[0146] In some embodiments, for example, a text similarity model can be used to determine whether the basic information of a chart contains OCR text recognition content. The text similarity model determines whether the basic information of a chart contains OCR text recognition content by calculating the similarity between a given OCR text recognition string and each string in the corresponding basic information of the chart. The text similarity model can use a traditional literal matching model, comparing the percentage of character overlap between the given OCR text recognition string and the string in the basic information of the chart (number of overlapping characters / number of characters in the compared string in the basic information of the chart), and this percentage is used as the text similarity score output by the model.

[0147] Step S1004: If the similarity between the first string and the second string is greater than a preset similarity threshold, the first string in the optical character recognition result is modified to the second string.

[0148] In some embodiments, when the similarity between the first string and the second string is greater than a preset similarity threshold c%, it is considered that the basic information of the chart contains OCR text recognition content. The parameter c can be obtained by training on a labeled training set.

[0149] In some embodiments, if the similarity between the first string and the second string is not greater than a preset similarity threshold c%, it is considered that the basic information of the chart does not contain OCR text recognition content, and there is no need to correct the OCR recognition result.

[0150] In other embodiments, if the basic information of the chart does not contain a second string, that is, it does not contain text content, it means that the data chart is an entire image embedded, the text in the chart is not embedded separately, and there is no need to correct the OCR recognition result.

[0151] In other embodiments, if the basic information of the chart contains OCR text recognition content, the relevant content in the OCR text recognition result is corrected based on the basic information of the image.

[0152] Figure 11 According to Figure 9 and Figure 10 This diagram illustrates a data graph content retrieval process. Figure 11 As shown, firstly, the text detection model CTPN is used to detect text in the data graph (S1102) to obtain the target text region in the target graph area, as detailed in step S902; then, text recognition is performed on the target text region based on OCR technology (S1104), as detailed in step S904; finally, the text recognition result is corrected based on the basic information of the graph (S1106), as detailed in steps S906 and... Figure 10 .

[0153] According to the method provided in the embodiments of this disclosure, for target chart areas of data chart type, while storing the data chart in the database in image format, the text information in the data chart can be extracted using the text detection model of the data chart and then OCR recognition can be performed. The data chart can be corrected according to the basic information of the chart for support of applications such as search recommendation.

[0154] The following section uses a real-world business scenario to illustrate... Figures 2 to 11 The methods described herein are as follows. Figure 12 according to Figures 2 to 11 This document illustrates a workflow for parsing charts and graphs in a research report PDF document. The workflow steps for this business scenario are as follows:

[0155] 1. At the beginning of each calendar day, the system for implementing the methods provided in the embodiments of this disclosure (hereinafter referred to as the "system") acquires newly added research reports for that day, which may include newly purchased research reports and publicly available research reports on the market, and stores them in the database along with metadata.

[0156] 2. At the designated time each day, the research report parsing function is executed: newly added research reports are retrieved from the database (S1202), and based on the underlying data parsing method of PDF files, various information of text and charts in the research report PDF document to be processed is extracted, including coordinates, rotation, size, content, etc. The relevant extraction results are used as "basic chart information" to provide auxiliary information for subsequent chart positioning and content extraction.

[0157] 3. The document layout parsing module in the system begins processing each newly added research report to obtain the segmented chart areas (S1204). Document layout parsing can include the following four processing steps:

[0158] (1) First, convert each page of the PDF document into an image format as input for the next step of the chart detection algorithm. Figure 13 The image format of a PDF document page to be processed is shown in an application scenario.

[0159] (2) In the chart detection step, for each page's image format, a chart detection model based on a deep learning algorithm is used to detect chart areas including data charts, non-data charts, and tables (see specific implementation details). Figure 2 ). Figure 14 It is based on Figure 13 A schematic diagram of a chart detection result is shown. For example... Figure 14 As shown, Figure 13 The "CITIC Securities" logo in the middle Figure 3 "", Figure 4 " and "Table 2" were detected and highlighted with boxes, and among them " Figure 4 The vertical axis data and the rest were detected as two chart areas.

[0160] (3) Refer to the basic chart information extracted in step 2 of the process to determine whether it is necessary to split a detected target into multiple chart regions or merge multiple detected targets into one chart region (see the specific implementation method). Figure 3 and Figure 4 ). Figure 15 It is based on Figure 14 An example diagram showing the merging of chart areas is shown. For example... Figure 15 As shown, Figure 14 "in Figure 4 The vertical axis data and the rest are combined into one chart area.

[0161] (4) Based on the basic chart information extracted in step 2 of the process, make necessary adjustments to the position and size of the chart area to obtain the final chart area output (see the specific implementation method). Figure 5 ). Figure 16 It is based on Figure 15 An example diagram showing how to adjust a chart area is provided. Figure 16 As shown, Figure 15 "in Figure 3 Cut off the excess blank parts on the left and right sides of the character.

[0162] 4. Classify the defined chart areas (S1206) to obtain chart category labels, which will serve as the basis for subsequent processing steps. The specific classification steps include:

[0163] (1) Remove abnormally sized areas (see detailed implementation method) Figure 7 ). Figure 17 It is based on Figure 16 An example diagram of abnormal region removal is shown. For example... Figure 17 As shown, Figure 16 The frame around the "CITIC Securities" logo has been removed, and it is no longer considered a target chart area.

[0164] (2) Classification is performed using a chart classification model. For specific implementation details, please refer to [the relevant documentation]. Figure 2 . Figure 18 It is based on Figure 17 An example chart showing the results of a classification process is provided. Figure 18 As shown, Figure 17 "in Figure 3 "", Figure 4 "Table 2" and "Table 2" are classified as "Non-data Chart", "Data Chart" and "Table", respectively.

[0165] 5. Based on the chart's location information, extract all immediately above and below text content, and use a chart title classification model to determine whether it is a chart title. Combining the determination result with the chart's location, match the corresponding chart title to the chart (S1208) (For specific implementation details, refer to...). Figure 8 ).

[0166] Figure 19 It is based on Figure 18 An example diagram of a chart context is shown. For example... Figure 19 As shown, first obtain Figure 18 "in Figure 3 "", Figure 4 The text immediately above and below “Table 2” is... Figure 18 "in Figure 3 The text immediately above " is " Figure 3 The company is located in the core area of ​​China's energy and chemical industry golden triangle, with the following text immediately following: "Source: Company Announcement"; Figure 18 "in Figure 3 The text immediately above " is " Figure 4 The text "Comparison of Average Prices of Thermal Coal in Various Regions of China (2020)" is immediately followed by the text "Source: Wind, CITIC Securities Research Department". Figure 18 The text immediately above "Table 2" is "Table 2: Comparison of the Company's Coal Purchase Price with Comparable Companies (RMB / ton)", and the text immediately below it is "Source: Company Prospectus, CITIC Securities Research Department".

[0167] Figure 20 It is based on Figure 19 An example diagram showing chart title matching is provided. Figure 20 As shown, a chart title classification model is then used to determine whether a text is a chart title, and this is matched with the corresponding chart area to obtain... Figure 18 "in Figure 3 "", Figure 4 The titles of "Table 2" and "Table 2" are respectively " Figure 3 The company is located in the core area of ​​China's energy and chemical industry golden triangle. Figure 4 Table 1: Comparison of Average Prices of Thermal Coal in Various Regions of China (2020) and Table 2: Comparison of Company's Coal Purchase Prices with Comparable Companies (RMB / ton).

[0168] 6. For different types of charts, extract different chart content (S1208) to provide data support for subsequent tasks such as searching. For non-data charts, directly store the original image content and the title matched in step 5 of the process into the database. For data charts, in addition to the image content and title content, use a data chart text extraction model to extract the text information contained therein, and store it in the database along with the text content of the data chart (see the specific implementation method). Figures 9 to 11 Similarly, for tables, based on the underlying data parsing results of the PDF document, all text content within the table area is extracted and stored in the database as table content, supplementing the image information for subsequent applications. Figure 21 It is based on Figure 18 and Figure 20 An example diagram showing chart title matching is provided. Figure 21 As shown, Figure 18 Non-data graphs in Figure 3 The title text was extracted, and... Figure 18 Data chart in Figure 4 "The title text and text content in the image were extracted. The title text of table "Table 2" and the text content of the cells in the table were extracted."

[0169] Figure 22 It is based on Figures 2 to 11 The diagram illustrates an example architecture of an intelligent investment research platform. Figure 22 As shown, the data layer 2202 can obtain research reports, survey reports, and internal research reports purchased by the company or publicly available in the market. The model layer 2204 can provide rich investment research data and intelligent model algorithms, such as chart parsing models 22042 including chart classification models and OCR models, and text parsing models 22044 including layout analysis models and text classification models. The intelligent investment research platform 22062 built in the application layer 2206 provides functions such as search recommendation, research report simplification, and research report generation. All related functions use both text and chart data, providing comprehensive data support and enriching the user experience. For example, users can search for research reports whose main text contains a certain keyword, or they can search for research reports whose chart titles / chart content contain that keyword; when reading research reports, users can view the text summaries of each section, as well as the charts contained in the corresponding sections. It is evident that the application of charts is of paramount importance in the various functions of the investment research platform.

[0170] After logging into the intelligent investment research platform 22062, researchers can enter topics or indicators of interest in the search box. The search engine will then retrieve results from its database based on both text content and chart titles / content, presenting a combined result. Researchers can choose to perform either a text-only search or a chart-only search through the search settings. The advantage of chart search is that it ensures highly relevant research reports are found, avoiding the situation where many results in pure text search only mention a specific term without elaborating on related content.

[0171] When a researcher opens a research report, the system displays the report's main text along with its chapter structure, key indicators, text summary, and included charts and graphs. This allows researchers to quickly grasp the main points of each chapter and gain an intuitive understanding of the key data arguments through the charts and graphs. If a researcher finds a section of interest, they can quickly jump to that section, improving the efficiency of reading the research report.

[0172] The intelligent investment research platform provided in this disclosure can help researchers find, read, and analyze research reports, improve the efficiency of writing research reports, accumulate investment research knowledge, and ultimately help improve the overall investment research efficiency and level of the company.

[0173] Figure 23 This is a flowchart illustrating a research report search method according to an exemplary embodiment. Figure 23 In this context, the first research report can correspond to the PDF file to be processed in the above embodiments. For example... Figure 23 As shown, the method 230 provided in this embodiment may include the following steps.

[0174] Step S2302: Obtain the input search terms.

[0175] In some embodiments, refer to Figure 22 Users can enter search terms on the search interface of the intelligent investment research platform 22062. Search terms can be keywords, topics of interest, or indicators, etc.

[0176] Step S2304: Match the search terms with the relevant content in the target chart area of ​​the first research report.

[0177] In some embodiments, refer to Figure 2 , Figures 8-10 It is possible to obtain the relevant content of the target chart area of ​​the first research report. The relevant content of the target chart area of ​​the first research report may include at least one of the text information in the target chart area of ​​the first research report and the title of the target chart.

[0178] Step S2306: If the search term successfully matches the relevant content of the target chart area of ​​the first research report, output the first search result in response to the operation of inputting the search term. The first search result is used to indicate that the first research report has been retrieved.

[0179] In some embodiments, the intelligent investment research platform 22062 can display the first successfully matched research report on the search results display interface, and can also display text summaries of each section of the first research report. (See also...) Figure 21 Based on the extracted chart content, users can also view the charts contained in the corresponding sections of the first research report.

[0180] Figure 24 This is a flowchart illustrating another research report search method according to an exemplary embodiment. Figure 24 In this context, both the first and second research reports can correspond to the PDF files to be processed in the above embodiments. For example... Figure 24 As shown, the method 240 provided in this embodiment may include the following steps.

[0181] Step S2402: Obtain the input search terms.

[0182] Step S2404: Match the search terms with the relevant content in the target chart area of ​​the first research report.

[0183] In some embodiments, the specific implementation of steps S2402 and S2404 can be referred to steps S2302 and S2304.

[0184] Step S2406: Match the search terms with the text content of the second research report.

[0185] In some embodiments, the text content of the second research report can be obtained, for example, by using OCR technology to recognize the text of the second research report.

[0186] In step S2408, if the search term successfully matches the relevant content of the target chart area of ​​the first research report and the search term successfully matches the text content of the second research report, the first search result is output. The first search result is also used to indicate that the second research report was retrieved and that the first research report has a higher priority than the second research report.

[0187] In some embodiments, the search engine in the intelligent investment research platform 22062 can search the database according to text content and chart title / chart content, and present the results obtained by the two methods in a comprehensive manner. For example, in the first search result, the first research report matching the chart content is placed before the second research report matching the text content, so as to indicate that the first research report has a higher priority than the second research report.

[0188] In some embodiments, researchers can choose to perform only text search or only chart search through the search settings in the intelligent investment research platform 22062, if they have a preference.

[0189] According to the research report search method provided in this disclosure, matching search terms with chart content can comprehensively present the search results in the database based on text content and chart titles / content. This ensures that the searched research reports are highly relevant and avoids the situation in pure text searches where a large number of results only mention a specific word without elaborating on related content, thus improving the accuracy of the search results. Figure 25 This is a block diagram illustrating a data processing apparatus according to an exemplary embodiment. Figure 25 The device shown can be applied, for example, to the server side of the above system, or to the terminal device of the above system.

[0190] refer to Figure 25The apparatus 250 provided in this embodiment may include a to-be-processed image acquisition module 2502, a chart region information acquisition module 2504, a chart region classification module 2506, and a chart content acquisition module 2508.

[0191] The image acquisition module 2502 can be used to acquire the image format page of the PDF file to be processed, which is obtained by converting the format of the page to be processed in the PDF file to be processed into an image.

[0192] The chart region information acquisition module 2504 can be used to perform chart detection on the image format page to be processed using a chart detection model, and obtain information about the target chart region of the image format page to be processed.

[0193] The chart region information acquisition module 2504 can also be used to perform chart detection on the format page of the image to be processed using a chart detection model to obtain information about the predicted chart regions. The information about the predicted chart regions includes the position information, size information, and page number information of at least one predicted chart region. If it is determined that the positions of two predicted chart regions overlap based on the position information of the predicted chart regions, it is determined, based on the position information and size information of the two predicted chart regions, to merge the two predicted chart regions into one target chart region, or to adjust the two predicted chart regions to obtain two target chart regions that do not overlap. And / or if it is determined that there is a cross-page table based on the position information and page number information of the predicted chart regions, the two predicted chart regions corresponding to the cross-page table are merged to obtain the target chart region corresponding to the cross-page table.

[0194] The chart region information acquisition module 2504 can also be used to perform chart detection on the image format page to be processed using a chart detection model to obtain information about predicted chart regions. The information about predicted chart regions includes the position information, size information, and page number information of at least one predicted chart region. Based on the page number information of the predicted chart regions, the position information and size information of the parsed chart regions in the image format page to be processed are obtained from the basic chart information obtained from the low-level parsing of the PDF file to be processed. Based on the position information and size information of the predicted chart regions, and the position information and size information of the parsed chart regions, if it is determined that there is a parsed chart region corresponding to the predicted chart region, the predicted chart region is taken as the target chart region, and the information of the target chart region is obtained. And / or based on the position information and size information of the parsed chart regions, if it is determined whether the predicted chart region contains multiple charts, the predicted chart region is split to obtain the information of the target chart region. And / or based on the position information and size information of the parsed chart regions, if it is determined that two adjacent predicted chart regions belong to the same chart, the two adjacent predicted chart regions are merged to obtain the information of the target chart region.

[0195] The information for the target chart area includes its location and size.

[0196] The chart region information acquisition module 2504 can also be used to perform chart detection on the page of the image format to be processed using a chart detection model, and obtain information about the predicted chart region. The information about the predicted chart region includes the position information, size information and page number information of the predicted chart region. Based on the page number information of the predicted chart region, the position information and size information of the parsed chart region corresponding to the predicted chart region in the page of the image format to be processed are obtained from the basic chart information obtained by low-level parsing of the PDF file to be processed. Based on the position information and size information of the parsed chart region, the position and size of the predicted chart region are adjusted to obtain the position information and size information of the target chart region.

[0197] The chart area classification module 2506 can be used to classify the target chart area using a chart classification model and obtain the chart category label of the target chart area. The chart category label includes data chart, non-data chart and table.

[0198] The information for the target chart area includes the size of the target chart area.

[0199] The chart area classification module 2506 can also be used to determine whether the target chart area is of normal size based on the size information of the target chart area; if the target chart area is determined to be of normal size, the chart classification model is used to classify the target chart area and obtain the chart category label of the target chart area.

[0200] The chart content acquisition module 2508 can be used to obtain the relevant content of the target chart area of ​​the image format page of the PDF file to be processed, based on the chart category label of the target chart area.

[0201] The chart content acquisition module 2508 can also be used to obtain target text lines within a preset distance above and / or below the target chart area in the image format page to be processed, based on the location information of the target chart area; obtain the probability that the target text line is the title of the target chart in the target chart area using the chart title model; and obtain the title of the target chart as the target text line when the probability that the target text line is the title of the target chart is greater than a preset probability threshold.

[0202] The chart content acquisition module 2508 can also be used to: when the chart category label of the target chart area is data chart, use a text detection model to perform text detection on the target chart area to obtain the target text area in the target chart area; perform character recognition on the target text area using an optical character recognition method to obtain the optical character recognition result; and correct the optical character recognition result based on the basic chart information obtained by low-level parsing of the PDF file to be processed to obtain the text information in the target text area.

[0203] The chart content acquisition module 2508 can also be used to: obtain the similarity between the first string in the optical character recognition result and the second string in the chart basic information; and modify the first string in the optical character recognition result to the second string if the similarity between the first string and the second string is greater than a preset similarity threshold.

[0204] The information for the target chart area includes the location information of the target chart area.

[0205] The chart content acquisition module 2508 can also be used to obtain the text content corresponding to the target chart area from the basic chart information obtained by low-level parsing of the PDF file to be processed when the chart category label of the target chart area is table.

[0206] The chart content acquisition module 2508 can also be used to obtain a target non-data chart in image format from the target chart area of ​​the image format page when the chart category label of the target chart area is a non-data chart.

[0207] Figure 26 This is a block diagram illustrating another data processing apparatus according to an exemplary embodiment. Figure 26 The device shown can be applied, for example, to the server side of the aforementioned system, or to the terminal devices of the aforementioned system. Figure 26 The device shown can, for example, be integrated into... Figure 25 The device shown can also be set up separately and exchange information when needed.

[0208] refer to Figure 26 The apparatus 260 provided in this embodiment may include a search term acquisition module 2602, a search term matching module 2604, and a search result output module 2606.

[0209] The search term acquisition module 2602 can be used to acquire the input search terms.

[0210] The term matching module 2604 can be used to match search terms with relevant content in the target chart area of ​​the first research report.

[0211] The term matching module 2604 can also be used to match terminology with the text content of the second research report.

[0212] The search result output module 2606 can be used to output the first search result when the search term is successfully matched with the relevant content of the target chart area of ​​the first research report, in response to the operation of inputting the search term. The first search result is used to indicate that the first research report has been retrieved.

[0213] The search result output module 2606 can also be used to output the first search result when the search term successfully matches the relevant content of the target chart area of ​​the first research report and the search term successfully matches the text content of the second research report. The first search result is also used to indicate that the second research report was retrieved and that the first research report has a higher priority than the second research report.

[0214] The specific implementation of each module in the device provided in this embodiment can be referred to the content of the above method, and will not be repeated here.

[0215] Figure 27 A schematic diagram of the structure of an electronic device according to an embodiment of this disclosure is shown. It should be noted that... Figure 27 The devices shown are merely examples of computer systems and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0216] like Figure 27 As shown, device 2700 includes a central processing unit (CPU) 2701, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 2702 or a program loaded from storage section 2708 into random access memory (RAM) 2703. RAM 2703 also stores various programs and data required for the operation of device 2700. CPU 2701, ROM 2702, and RAM 2703 are interconnected via bus 2704. Input / output (I / O) interface 2705 is also connected to bus 2704.

[0217] The following components are connected to I / O interface 2705: an input section 2706 including a keyboard, mouse, etc.; an output section 2707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 2708 including a hard disk, etc.; and a communication section 2709 including a network interface card such as a LAN card, modem, etc. The communication section 2709 performs communication processing via a network such as the Internet. A drive 2710 is also connected to I / O interface 2705 as needed. Removable media 2711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 2710 as needed so that computer programs read from them can be installed into storage section 2708 as needed.

[0218] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 2709, and / or installed from removable medium 2711. When the computer program is executed by central processing unit (CPU) 2701, it performs the functions defined above in the system of this disclosure.

[0219] It should be noted that the computer-readable medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0220] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0221] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor may be described as including a module for acquiring an image to be processed, a module for acquiring chart area information, a module for classifying chart areas, and a module for acquiring chart content. The names of these modules do not necessarily limit the module itself; for example, the module for acquiring an image to be processed may also be described as "a module for converting a PDF file to be processed into an image format to be processed."

[0222] In another aspect, this disclosure also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to include:

[0223] The process involves: acquiring the image format page of the PDF file to be processed (the page in the PDF file is converted into an image); performing chart detection on the image format page using a chart detection model to obtain information about the target chart area; classifying the target chart area using a chart classification model to obtain chart category labels, which include data charts, non-data charts, and tables; and finally, obtaining the relevant content of the target chart area of ​​the image format page in the PDF file based on the chart category labels.

[0224] Exemplary embodiments of this disclosure have been specifically shown and described above. It should be understood that this disclosure is not limited to the detailed structures, arrangements, or implementations described herein; rather, this disclosure is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.

Claims

1. A data processing method, characterized in that, include: After inputting a portable document format PDF file to be processed, the basic information of the charts in the PDF file to be processed is extracted based on the underlying data parsing method of the PDF document. The basic information of the charts includes text and the coordinates and content of the charts. Obtain the image format page to be processed from the PDF file to be processed, wherein the image format page to be processed is obtained by converting the format of the page to be processed in the PDF file to be processed into an image; The image format page to be processed is subjected to chart detection using a chart detection model to obtain information about the target chart area of ​​the image format page to be processed. The target chart area is classified using a chart classification model to obtain chart category labels for the target chart area. The chart category labels include data chart, non-data chart, and table. The chart classification model outputs the probability that the content of the target chart area is a data chart, non-data chart, or table based on the line distribution, color fill distribution, and color outline distribution features inside the image. The category with the highest probability value is taken as the classification result. Based on the chart category labels of the target chart area, obtain the relevant content of the target chart area of ​​the image format page of the PDF file to be processed; Based on the chart category labels of the target chart area, obtain the relevant content of the target chart area of ​​the image format page of the PDF file to be processed, including: If the chart category label of the target chart area is a data chart, a text detection model is used to perform text detection on the target chart area to obtain the target text area in the target chart area. The target text region is subjected to character recognition using an optical character recognition method to obtain optical character recognition results. Based on the basic information of the charts obtained by performing low-level parsing on the PDF file to be processed, the optical character recognition results are corrected to obtain the text information in the target text region; If the chart category label of the target chart area is a table, the text content corresponding to the target chart area in the image format page to be processed is obtained from the basic chart information obtained by low-level parsing of the PDF file to be processed.

2. The method according to claim 1, characterized in that, Based on the basic information of the charts obtained through low-level parsing of the PDF file to be processed, the optical character recognition results are corrected, including: Obtain the similarity between the first string in the optical character recognition result and the second string in the basic information of the chart; If the similarity between the first string and the second string is greater than a preset similarity threshold, the first string in the optical character recognition result is modified to the second string.

3. The method according to claim 1, characterized in that, The information of the target chart area includes the size information of the target chart area; The method further includes: Determine whether the target chart area is the correct size based on its size information. The target chart area is classified using a chart classification model to obtain chart category labels for the target chart area, including: If the target chart area is determined to be of normal size, the target chart area is classified using a chart classification model to obtain the chart category label of the target chart area.

4. The method according to claim 1, characterized in that, The information of the target chart area includes the location information of the target chart area; The method further includes: Based on the location information of the target chart area, obtain the target text line within a preset distance above and / or below the target chart area in the image format page to be processed; The probability that the target text line is the title of the target chart in the target chart area is obtained using a chart title model; When the probability that the target text line is the title of the target chart is greater than a preset probability threshold, the title of the target chart is obtained as the target text line.

5. The method according to claim 1, characterized in that, The image format page to be processed is subjected to chart detection using a chart detection model to obtain information about the target chart region of the image format page to be processed, including: The chart detection model is used to perform chart detection on the image format page to be processed to obtain information of the predicted chart region. The information of the predicted chart region includes the location information, size information and page number information of at least one predicted chart region. If, based on the location information of two predicted chart regions, it is determined that their positions overlap, then, based on the location and size information of the two predicted chart regions, it is determined to either merge the two predicted chart regions into a single target chart region, or adjust the two predicted chart regions to obtain two target chart regions that do not overlap; and / or If a cross-page table is determined to exist based on the location information and page number information of the predicted chart area, the two predicted chart areas corresponding to the cross-page table are merged to obtain the target chart area corresponding to the cross-page table.

6. The method according to claim 1, characterized in that, The image format page to be processed is subjected to chart detection using a chart detection model to obtain information about the target chart region of the image format page to be processed, including: The chart detection model is used to perform chart detection on the image format page to be processed to obtain information of the predicted chart region. The information of the predicted chart region includes the location information, size information and page number information of at least one predicted chart region. Based on the page number information of the predicted chart area, the position and size information of the parsed chart area in the image format page to be processed are obtained from the basic chart information obtained by performing low-level parsing on the PDF file to be processed. Based on the location and size information of the predicted chart region and the location and size information of the parsed chart region, if it is determined that a parsed chart region corresponding to the predicted chart region exists, the predicted chart region is taken as the target chart region, and the information of the target chart region is obtained; and / or Based on the location and size information of the parsed chart area, and considering whether the predicted chart area contains multiple charts, the predicted chart area is split to obtain information about the target chart area; and / or Based on the location and size information of the parsed chart area, if it is determined that two adjacent predicted chart areas belong to the same chart, the two adjacent predicted chart areas are merged to obtain the information of the target chart area.

7. The method according to claim 1, characterized in that, The information of the target chart area includes the location information and size information of the target chart area; The image format page to be processed is subjected to chart detection using a chart detection model to obtain information about the target chart region of the image format page to be processed, including: The chart detection model is used to perform chart detection on the image format page to be processed to obtain information about the predicted chart region. The information about the predicted chart region includes the location information, size information and page number information of the predicted chart region. Based on the page number information of the predicted chart area, the position and size information of the parsed chart area corresponding to the predicted chart area in the image format page to be processed are obtained from the basic chart information obtained by performing low-level parsing on the PDF file to be processed. Based on the position and size information of the parsed chart area, the position and size of the predicted chart area are adjusted to obtain the position and size information of the target chart area.

8. The method according to claim 1, characterized in that, Based on the chart category labels of the target chart area, obtain the relevant content of the target chart area of ​​the image format page of the PDF file to be processed, including: If the chart category label in the target chart area is a non-data chart, obtain the target non-data chart in image format from the target chart area of ​​the image format page to be processed.

9. The method according to claim 1, characterized in that, The PDF file to be processed includes the first research report; Obtain the relevant content of the target chart area of ​​the image format page of the PDF file to be processed, including: Obtain the relevant content of the target chart area of ​​the first research report, wherein the relevant content of the target chart area of ​​the first research report includes at least one of the text information in the target chart area of ​​the first research report and the title of the target chart; The method further includes: Get the input search terms; Match the search terms with the relevant content in the target chart area of ​​the first research report; If the search term successfully matches the relevant content of the target chart area of ​​the first research report, a first search result is output in response to the operation of inputting the search term. The first search result indicates that the first research report has been retrieved.

10. The method according to claim 9, characterized in that, The PDF file to be processed also includes a second research report; The method further includes: Obtain the text content of the second research report; Match the search terms with the text content of the second research report; If the search term successfully matches the relevant content of the target chart area in the first research report, the first search result is output, including: If the search term successfully matches the relevant content of the target chart area of ​​the first research report, and the search term successfully matches the text content of the second research report, the first search result is output. The first search result is also used to indicate that the second research report was retrieved and that the first research report has a higher priority than the second research report.

11. A data processing apparatus, characterized in that, include: The image to be processed module is used to acquire the image format page of the PDF file to be processed, which is a portable document format PDF file. The image format page is obtained by converting the format of the page to be processed in the PDF file to be processed into an image. The chart region information acquisition module is used to perform chart detection on the image format page to be processed using a chart detection model, and obtain information about the target chart region of the image format page to be processed. The chart region classification module is used to classify the target chart region using a chart classification model to obtain the chart category label of the target chart region. The chart category label includes data chart, non-data chart, and table. The chart classification model outputs the probability that the content of the target chart region is a data chart, non-data chart, or table based on the line distribution, color fill distribution, and color outline distribution features inside the image. The category with the highest probability value is taken as the classification result. The chart content acquisition module is used to obtain the relevant content of the target chart area of ​​the image format page of the PDF file to be processed based on the chart category label of the target chart area; The chart content acquisition module is further configured to, when the chart category label of the target chart area is a data chart, use a text detection model to perform text detection on the target chart area to obtain the target text area in the target chart area; perform character recognition on the target text area using an optical character recognition method to obtain an optical character recognition result; and correct the optical character recognition result based on the basic chart information obtained by low-level parsing of the PDF file to be processed to obtain the text information in the target text area. The basic chart information is extracted based on the low-level data parsing of the PDF document after inputting the PDF file to be processed, and includes the coordinates and content of the text and the chart. The chart content acquisition module is further configured to, when the chart category label of the target chart area is a table, obtain the text content corresponding to the target chart area in the image format page to be processed from the basic chart information obtained by low-level parsing of the PDF file to be processed.

12. An apparatus comprising: A memory, a processor, and executable instructions stored in the memory and executable in the processor, characterized in that the processor, when executing the executable instructions, implements the method as described in any one of claims 1-10.

13. A computer-readable storage medium having computer-executable instructions stored thereon, characterized in that, When the executable instructions are executed by the processor, they implement the method as described in any one of claims 1-10.

Images