Image-text reader processing method, model training method, device and system

Abstract

The invention relates to the technical field of data processing, and discloses an image-text reader processing method, a model training method, a model training device and a model training system.The method comprises the steps that a target file is subjected to standardized analysis based on a preset multi-format analysis model in AI NAS equipment, standardized content is obtained, text analysis operation is carried out on the standardized text content, and the standardized content is obtained; and carrying out image processing operation on the standardized image content to obtain target image content, and displaying the target text content and / or the target image content based on a pre-constructed cross-device response type interface. And based on the AI NAS device, carrying out device synchronization updating on the reading data of the target user for the target text content and / or the target image content. It can be seen that by implementing the method and the system, the analysis and rendering tasks can be migrated to the AI NAS equipment, multi-format analysis is supported, the analysis efficiency can be improved, intelligent text analysis and image processing can be achieved, reading content is optimized, multi-equipment interface display and cross-equipment synchronization are supported, the user reading fluency is improved, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular, to a processing method, a model training method, an apparatus, and a system for a graphic and text reader. Background Art

[0002] With the rapid development of computer technology, the traditional paper reading method can gradually no longer meet people's reading needs, and the efficient and convenient electronic reading method is gradually favored by people.

[0003] Currently, taking the mainstream e-books and comic readers on the market as an example, they support the parsing of specific formats by installing dedicated software, mainly relying on the local parsing ability of the terminal device. Therefore, the existing readers have problems of insufficient multi-format support and low parsing efficiency. Therefore, it is particularly important to propose a technical solution that can support multi-format parsing and improve parsing efficiency. Summary of the Invention

[0004] The present invention provides a processing method, a model training method, an apparatus, and a system for a graphic and text reader, which can support multi-format parsing and improve parsing efficiency.

[0005] To solve the above technical problems, in a first aspect of the present invention, a processing method for a graphic and text reader is disclosed. The method is applied to an AI NAS device, and the method includes:

[0006] Performing standardized parsing on a target file based on a preset multi-format parsing model in the AI NAS device to obtain standardized content, where the standardized content includes standardized text content and / or standardized image content;

[0007] When the standardized content includes the standardized text content, performing text analysis operations on the standardized text content to obtain target text content, where the text analysis operations include at least one of chapter division operations, graphic and text separation operations, and language translation operations;

[0008] When the standardized content includes the standardized image content, performing image processing operations on the standardized image content to obtain target image content, where the image processing operations include at least one of image enhancement operations, image denoising operations, and image intelligent stitching operations;

[0009] Displaying the target text content and / or the target image content based on a pre-constructed cross-device responsive interface, and synchronously updating the reading data of the target user for the target text content and / or the target image content on the AI NAS device.

[0010] As an alternative implementation, in the first aspect of the present invention, the standardized parsing of the target file based on the multi-format parsing model preset in the AI NAS device to obtain standardized content includes:

[0011] Decompose the standardized parsing task corresponding to the target file based on the multi-format parsing model preset in the AI NAS device to obtain multiple subtasks corresponding to the target file;

[0012] Dynamically evaluate the computing requirements of each subtask based on a preset task scheduling algorithm, and determine the parallel processing mechanism of the target file according to the computing requirements of each subtask;

[0013] According to the parallel processing mechanism, perform parallel processing on each subtask through the multi-format parsing model to obtain the task processing result corresponding to each subtask;

[0014] Merge the results of each task processing result to obtain the standardized content.

[0015] As an alternative implementation, in the first aspect of the present invention, when the standardized content includes the standardized text content, the text analysis operation on the standardized text content to obtain the target text content includes:

[0016] When the standardized content includes the standardized text content, combine the preset chapter features to identify the chapter information of the standardized text content, and perform chapter division operation on the standardized text content based on the chapter information to obtain the target text content, where the chapter information includes the chapter name and the hierarchical relationship; and / or,

[0017] Extract the image features and text features in the standardized text content, and perform joint encoding on the image features and the text features to obtain the encoding result;

[0018] Based on the preset feature weights and the encoding result, perform graphic and text separation operation on the standardized text content to obtain the target text content; and / or,

[0019] Identify the non-set language text content in the standardized text content, and perform language translation operation on the non-set language text content based on the user requirements to obtain the target text content, where the user requirements include the translation language type requirement and / or the translation range requirement.

[0020] As an alternative implementation, in the first aspect of the present invention, when the standardized content includes the standardized image content, the image processing operation on the standardized image content to obtain the target image content includes:

[0021] When the standardized content includes the standardized image content, perform an image enhancement operation on the standardized image content based on a super-resolution model to obtain target image content; and / or,

[0022] Extract noise information in the standardized image content, and perform an image denoising operation on the standardized image content for the noise information through a denoising autoencoder and an edge-preserving filtering technique to obtain target image content, where the noise information includes stain information and / or watermark information; and / or,

[0023] Extract feature point information in the standardized image content, and predict a target stitching area in the standardized image content according to the feature point information;

[0024] Perform an intelligent image stitching operation on the segmented images in the standardized image content according to the feature point information and the target stitching area to obtain target image content.

[0025] As an optional implementation manner, in the first aspect of the present invention, the device synchronization update of the reading data of the target user for the target text content and / or the target image content based on the AI NAS device includes:

[0026] Obtain the reading data of the target user for the target text content and / or the target image content on the target device, where the reading data includes reading progress data and reading setting data, and the reading setting data includes reading mode setting data and / or display content setting data;

[0027] Store the reading data in the AI NAS device, and synchronously update the reading data to each reading device corresponding to the target user based on the AI NAS device.

[0028] As an optional implementation manner, in the first aspect of the present invention, the method further includes:

[0029] Construct a deep learning model based on a preset structure in the AI NAS device;

[0030] Determine multiple file formats to be trained, and generate data structure tags for each of the file formats, where the data structure tags include chapter information tags, text content tags, and image resource tags;

[0031] Obtain the file to be trained, and based on the file to be trained and the data structure tags of each file format, perform model training on the deep learning model to obtain the multi-format parsing model. The file to be trained includes sub-files with multi-format complexity, and the sub-files with multi-format complexity include text sub-files in multiple languages and image sub-files with multiple resolutions.

[0032] As an optional implementation manner, in the first aspect of the present invention, the performing model training on the deep learning model based on the file to be trained and the data structure tags of each file format to obtain the multi-format parsing model includes:

[0033] Analyze the structural features of the file to be trained, and perform clustering analysis on the file to be trained according to the structural features and the data structure tags of each file format to obtain an analysis result;

[0034] Perform model training on the deep learning model according to the analysis result to obtain a multi-format parsing model;

[0035] Obtain the new format file and the parsing result of the new format file fed back by the user during the process of using the multi-format parsing model, and perform model incremental learning on the multi-format parsing model according to the new format file and the parsing result of the new format file to update the multi-format parsing model.

[0036] The second aspect of the present invention discloses a method for model construction and training of a multi-format parsing model. The method is applied to an AI NAS device, and the method includes:

[0037] Build a deep learning model in the AI NAS device based on a preset framework, and the preset framework includes a deep learning framework and a distributed computing framework;

[0038] Determine multiple file formats to be trained, and generate data structure tags for each file format. The data structure tags include chapter information tags, text content tags, and image resource tags;

[0039] Obtain the file to be trained, and based on the file to be trained and the data structure tags of each file format, perform model training on the deep learning model to obtain a multi-format parsing model. The file to be trained includes sub-files with multi-format complexity, and the sub-files with multi-format complexity include text sub-files in multiple languages and image sub-files with multiple resolutions.

[0040] As an optional implementation manner, in the second aspect of the present invention, the performing model training on the deep learning model based on the file to be trained and the data structure tags of each file format to obtain a multi-format parsing model includes:

[0041] Analyze the structural features of the file to be trained, and perform clustering analysis on the file to be trained according to the structural features and the data structure tags of each file format to obtain an analysis result;

[0042] Perform model training on the deep learning model according to the analysis result to obtain a multi-format parsing model;

[0043] Obtain the new format file and the parsing result of the new format file fed back by the user during the use of the multi-format parsing model, and perform model incremental learning on the multi-format parsing model according to the new format file and the parsing result of the new format file to update the multi-format parsing model.

[0044] As an optional implementation manner, in the second aspect of the present invention, the analyzing the structural features of the file to be trained, and performing clustering analysis on the file to be trained according to the structural features and the data structure tags of each file format to obtain an analysis result includes:

[0045] Perform file preprocessing on the file to be trained to obtain file content, where the file content includes text structure content and / or picture sequence content;

[0046] Perform tagging processing on the file content according to the data structure tags of each file format to obtain the processed tagged file content, and perform structural analysis on the tagged file content to obtain the structural features of the tagged file content;

[0047] Perform clustering analysis on the file to be trained according to the tagged file content and the structural features to obtain an analysis result.

[0048] The third aspect of the present invention discloses a processing device for a graphic and text reader, where the device is applied to an AI NAS device, and the device includes:

[0049] A parsing module, configured to perform standardized parsing on a target file based on a multi-format parsing model preset in the AI NAS device to obtain standardized content, where the standardized content includes standardized text content and / or standardized image content;

[0050] A text analysis module, configured to perform text analysis operations on the standardized text content when the standardized content includes the standardized text content to obtain target text content, where the text analysis operations include at least one of chapter division operations, graphic and text separation operations, and language translation operations;

[0051] An image processing module, configured to perform image processing operations on the standardized image content to obtain target image content when the standardized content includes the standardized image content, where the image processing operations include at least one of image enhancement operation, image denoising operation, and image intelligent stitching operation;

[0052] A display module, configured to display the target text content and / or the target image content based on a pre-constructed cross-device responsive interface;

[0053] An update module, configured to perform device synchronization and update on the reading data of the target user for the target text content and / or the target image content based on the AI NAS device.

[0054] As an optional implementation manner, in the third aspect of the present invention, the manner in which the parsing module performs standardized parsing on the target file based on a multi-format parsing model preset in the AI NAS device to obtain standardized content specifically includes:

[0055] Performing task decomposition on the standardized parsing task corresponding to the target file based on the multi-format parsing model preset in the AI NAS device to obtain multiple subtasks corresponding to the target file;

[0056] Dynamically evaluating the computing requirements of each subtask based on a preset task scheduling algorithm, and determining the parallel processing mechanism of the target file according to the computing requirements of each subtask;

[0057] According to the parallel processing mechanism, performing parallel processing on each subtask through the multi-format parsing model to obtain the task processing result corresponding to each subtask;

[0058] Merging the results of each task processing result to obtain standardized content.

[0059] As an optional implementation manner, in the third aspect of the present invention, the manner in which the text analysis module performs text analysis operations on the standardized text content to obtain target text content when the standardized content includes the standardized text content specifically includes:

[0060] When the standardized content includes the standardized text content, identifying the chapter information of the standardized text content in combination with preset chapter features, and performing chapter division operations on the standardized text content based on the chapter information to obtain target text content, where the chapter information includes chapter names and hierarchical relationships; and / or,

[0061] Extracting the image features and text features in the standardized text content, and performing joint encoding on the image features and the text features to obtain an encoding result;

[0062] Based on the preset feature weights and the encoding result, perform text-image separation operation on the standardized text content to obtain the target text content; and / or,

[0063] Identify the non-set language text content in the standardized text content, and perform language translation operation on the non-set language text content based on user requirements to obtain the target text content, where the user requirements include translation language type requirements and / or translation range requirements.

[0064] As an alternative implementation manner, in the third aspect of the present invention, when the standardized content includes the standardized image content, the manner in which the image processing module performs image processing operation on the standardized image content to obtain the target image content specifically includes:

[0065] When the standardized content includes the standardized image content, perform image enhancement operation on the standardized image content based on a super-resolution model to obtain the target image content; and / or,

[0066] Extract the noise information in the standardized image content, and perform image denoising operation on the standardized image content for the noise information through a denoising autoencoder and edge-preserving filtering technology to obtain the target image content, where the noise information includes stain information and / or watermark information; and / or,

[0067] Extract the feature point information in the standardized image content, and predict the target stitching area in the standardized image content according to the feature point information;

[0068] According to the feature point information and the target stitching area, perform intelligent image stitching operation on the segmented images in the standardized image content to obtain the target image content.

[0069] As an alternative implementation manner, in the third aspect of the present invention, the manner in which the update module synchronously updates the device based on the AI NAS device for the reading data of the target user for the target text content and / or the target image content specifically includes:

[0070] Obtain the reading data of the target user for the target text content and / or the target image content on the target device, where the reading data includes reading progress data and reading setting data, and the reading setting data includes reading mode setting data and / or display content setting data;

[0071] Store the reading data in the AI NAS device, and synchronously update the reading data to each reading device corresponding to the target user based on the AI NAS device.

[0072] As an alternative implementation, in the third aspect of the present invention, the device further includes:

[0073] A construction module, configured to construct a deep learning model based on a preset structure in the AI NAS device;

[0074] A determination module, configured to determine multiple file formats to be trained and generate data structure tags for each of the file formats, where the data structure tags include chapter information tags, text content tags, and image resource tags;

[0075] An acquisition module, configured to acquire a file to be trained and perform model training on the deep learning model based on the file to be trained and the data structure tags of each of the file formats, to obtain the multi-format parsing model, where the file to be trained includes sub-files with multi-format complexity, and the sub-files with multi-format complexity include text sub-files in multiple languages and image sub-files with multiple resolutions.

[0076] As an alternative implementation, in the third aspect of the present invention, the manner in which the acquisition module performs model training on the deep learning model based on the file to be trained and the data structure tags of each of the file formats to obtain the multi-format parsing model specifically includes:

[0077] Analyze the structural features of the file to be trained, and perform clustering analysis on the file to be trained according to the structural features and the data structure tags of each of the file formats to obtain an analysis result;

[0078] Perform model training on the deep learning model according to the analysis result to obtain the multi-format parsing model;

[0079] Acquire a new format file and the parsing result of the new format file fed back by a user during the use of the multi-format parsing model, and perform model incremental learning on the multi-format parsing model according to the new format file and the parsing result of the new format file to update the multi-format parsing model.

[0080] The fourth aspect of the present invention discloses a processing system for a graphic and text reader, where the system at least includes an electronic device and an AI NAS device, and the electronic device is communicatively connected to the AI NAS device;

[0081] The electronic device is configured with an application program that can access the AI NAS device, and reads the target text content and / or target image content processed by the AI NAS device based on the processing method of the graphic and text reader disclosed in the first aspect of the present invention through the application program.

[0082] The fifth aspect of the present invention discloses a computer storage medium storing computer instructions, which are used to execute the processing method of the graphic and text reader disclosed in the first aspect of the present invention when called.

[0083] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0084] In the embodiments of the present invention, a multi-format parsing model preset in the AI NAS device can be used to perform standardized parsing on the target file to obtain standardized content, that is, the parsing and rendering tasks can be migrated to the AI NAS device, supporting multi-format parsing, and the parsing efficiency can be improved. Text analysis operations are performed on the standardized text content to obtain the target text content, and image processing operations are performed on the standardized image content to obtain the target image content. Intelligent text analysis and image processing can be realized, the reading content can be optimized, the target text content and / or target image content are displayed based on the pre-constructed cross-device responsive interface, and the reading data of the target user for the target text content and / or target image content is synchronously updated based on the AI NAS device, supporting multi-device interface display and cross-device synchronization, improving the user reading fluency and enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0086] Figure 1 is a schematic flowchart of a processing method of a graphic and text reader disclosed in an embodiment of the present invention;

[0087] Figure 2 is a schematic flowchart of another processing method of a graphic and text reader disclosed in an embodiment of the present invention;

[0088] Figure 3 is a schematic flowchart of a training method of a multi-format parsing model disclosed in an embodiment of the present invention;

[0089] Figure 4 is a schematic structural diagram of a processing device of a graphic and text reader disclosed in an embodiment of the present invention;

[0090] Figure 5 is a schematic structural diagram of another processing device of a graphic and text reader disclosed in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0091] To enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0092] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or terminal comprising a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or terminals.

[0093] Referring to "embodiments" herein means that the specific features, structures or characteristics described in connection with the embodiments can be included in at least one embodiment of the present invention. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0094] The present invention discloses a processing method, a model training method, a device and a system for a graphic and text reader, which can migrate parsing and rendering tasks to an AI NAS device, support multi-format parsing, improve parsing efficiency, realize intelligent text analysis and image processing, optimize reading content, support multi-device interface display and cross-device synchronization, improve the user reading fluency, and improve the user experience. The following will be described in detail respectively.

[0095] Embodiment 1

[0096] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a processing method for a graphic and text reader disclosed in an embodiment of the present invention. Among them, Figure 1 the described processing method for the graphic and text reader can be applied to a processing device for the graphic and text reader, and the processing device for the graphic and text reader can be applied to an AI NAS device. Among them, the processing device for the graphic and text reader can include an intelligent server or an intelligent platform for standardizing the parsing of e-books and / or electronic images. The intelligent server includes an AI NAS end server or a cloud server, and the embodiments of the present invention do not make limitations. As Figure 1As shown, the processing method of the graphic and text reader may include the following operations:

[0097] 101. Perform standardized parsing on the target file based on a multi-format parsing model preset in the AI NAS device to obtain standardized content.

[0098] In an embodiment of the present invention, optionally, an AI reader system may be pre-installed in the AI NAS device. The AI reader system may include a multi-format parsing model and may be used to parse and render files, that is, the parsing and rendering tasks for electronic files may be migrated to the AI reader system in the AI NAS device to improve the parsing and rendering efficiency by using its powerful computing power and intelligent optimization functions. The present invention does not make any limitations.

[0099] In an embodiment of the present invention, optionally, the multi-format parsing model may be used to identify and parse various formats of e-books and electronic images (such as comics), including but not limited to PDF, EPUB, CBZ, and CBR, etc. The multi-format parsing model may parse the target file based on a unified data structure and standardize the content of the parsed file to obtain standardized content. The target file may include electronic text files and / or electronic image files, and the standardized content may include standardized text content and / or standardized image content. The present invention does not make any limitations.

[0100] 102. When the standardized content includes standardized text content, perform a text analysis operation on the standardized text content to obtain target text content.

[0101] In an embodiment of the present invention, optionally, when the standardized content includes standardized text content, that is, when parsing an e-book, a text analysis operation may be performed on the standardized text content to obtain target text content. The text analysis operation includes at least one of a chapter division operation, a text-image separation operation, and a language translation operation. Among them, the chapter division operation may include extracting chapter names and hierarchical relationships from the layout format of the e-book. The text-image separation operation may include separating the text and image content in the document. The language translation operation may perform paragraph-level translation or full-text translation on the non-native language document content in the document. The present invention does not make any limitations.

[0102] 103. When the standardized content includes standardized image content, perform an image processing operation on the standardized image content to obtain target image content.

[0103] In an embodiment of the present invention, optionally, when the standardized content includes standardized image content, that is, when performing parsing operations on electronic images, image processing operations may be performed on the standardized image content to obtain target image content. The image processing operations include at least one of image enhancement operations, image denoising operations, and image intelligent stitching operations. Among them, the image enhancement operation may include using super-resolution technology to improve the clarity of comic images, adjusting image contrast and brightness, and optimizing the display effect. The image denoising operation may include using a deep learning denoising model to remove noise and watermarks in the image. The image intelligent stitching operation may include intelligently and automatically stitching segmented comic pages, which is not limited in the present invention.

[0104] It should be noted that the execution order of step 102 and step 103 has no sequence relationship, that is, step 102 and step 103 can be executed simultaneously, or they can be executed in sequence. It should be noted that when step 102 and step 103 are executed simultaneously, the processing efficiency of the target file can be improved, and thus the parsing efficiency of the target file can be improved.

[0105] 104. Display the target text content and / or target image content based on a pre-constructed cross-device responsive interface, and synchronously update the reading data of the target user for the target text content and / or target image content based on the AI NAS device.

[0106] In an embodiment of the present invention, optionally, the target text content and / or target image content may be displayed based on a pre-constructed cross-device responsive interface. The pre-constructed cross-device responsive interface supports multi-device access and multiple reading modes, including night mode, audiobook mode, etc., and supports touch screen operations, keyboard shortcuts, and voice control. Specifically, a responsive interface can be developed using web technologies (such as HTML5 and JavaScript), which is compatible with devices such as PCs, tablets, and mobile phones, adapts to touch devices, and provides interactive operations such as swiping to turn pages and two-finger zooming. The reading data of the target user for the target text content and / or target image content can be synchronously updated based on the AI NAS device. Specifically, the AI NAS device can be connected to a cloud storage service to automatically synchronize reading data such as the user's reading progress, bookmarks, and notes, which is not limited in the present invention.

[0107] In an embodiment of the present invention, optionally, during the process of displaying the target text content and / or target image content based on a pre-constructed cross-device responsive interface, an incremental parsing technology is implemented to load content only when the user scrolls to the corresponding chapter, reducing resource consumption. The parsed files are stored through a caching technology to improve subsequent access speed, which is not limited in the present invention.

[0108] It can be seen that implementing Figure 1The processing method of the described graphic and text reader can perform standardized parsing on the target file based on a multi-format parsing model preset in the AI NAS device to obtain standardized content, perform text analysis operations on the standardized text content to obtain target text content, perform image processing operations on the standardized image content to obtain target image content, display the target text content and / or target image content based on a pre-constructed cross-device responsive interface, and perform device synchronization updates on the reading data of the target user for the target text content and / or target image content based on the AI NAS device. It can migrate the parsing and rendering tasks to the AI NAS device, support multi-format parsing, improve the parsing efficiency, support cross-device synchronization, and improve the user experience.

[0109] In an optional embodiment, when the standardized content includes standardized text content, performing text analysis operations on the standardized text content to obtain target text content may include the following operations:

[0110] When the standardized content includes standardized text content, combine preset chapter features to identify the chapter information of the standardized text content, and perform chapter division operations on the standardized text content based on the chapter information to obtain target text content. The chapter information includes chapter names and hierarchical relationships; and / or,

[0111] Extract the image features and text features in the standardized text content, and perform joint encoding on the image features and text features to obtain an encoding result;

[0112] Based on the preset feature weights and the encoding result, perform graphic and text separation operations on the standardized text content to obtain target text content; and / or,

[0113] Identify the non-set language text content in the standardized text content, and perform language translation operations on the non-set language text content based on user requirements to obtain target text content. The user requirements include translation language type requirements and / or translation range requirements.

[0114] In this optional embodiment, optionally, when the standardized content includes standardized text content, the chapter information of the standardized text content can be identified by combining preset chapter features. The preset chapter features can include the font styles of a large number of chapter titles, keywords of chapter titles, etc. The chapter information can include chapter names and hierarchical relationships. The standardized text content can be divided into chapters based on the chapter information to obtain target text content, that is, a table of contents tree of the standardized text content can be generated according to the chapter information. Optionally, a rule model (such as an analysis model based on the HTML / XML structure) and a deep learning model can also be combined to improve the compatibility with different document structures. This embodiment is not limited.

[0115] In this optional embodiment, optionally, image features and text features in the standardized text content can be extracted. Specifically, through NLP combined with computer vision technology, the text and image content in the document can be separated, and the image features and text features can be jointly encoded to obtain an encoding result. Based on the preset feature weights and the encoding result, an operation of separating text and images is performed on the standardized text content to obtain the target text content. The preset feature weights can be set by the user or automatically determined according to the document information. The feature weights for each format of the file can be the same or different. Optionally, image annotations for the image can also be automatically generated according to user requirements or system settings to facilitate the user's review. This embodiment does not make any limitations.

[0116] In this optional embodiment, optionally, non-set language text content in the standardized text content can be recognized. Among them, the user can preset the mother tongue language (such as Chinese or English), and the text and voice in the AI reader will be displayed or broadcast according to the mother tongue language set by the user. The non-set language text content can include text content of language types other than the mother tongue language set by the user. For example, when the mother tongue language set by the user is Chinese, the non-set language text content can include text content in languages other than Chinese. A language translation operation can be performed on the non-set language text content based on user requirements to obtain the target text content. User requirements include translation language type requirements and / or translation range requirements. The user can change the language type of the translated text through the translation language type requirements, and can change the range of the translated text through the translation range requirements. The AI reader supports paragraph-level translation and full-text translation. This embodiment does not make any limitations.

[0117] In this optional embodiment, optionally, the TF-IDF algorithm and a deep learning model (such as TextRank) can be used to generate keywords and content summaries for each chapter, and the user can customize the keyword weights according to actual needs. This embodiment does not make any limitations.

[0118] It can be seen that implementing this optional embodiment can, when the standardized content includes standardized text content, combine the preset chapter features to recognize the chapter information of the standardized text content, and based on the chapter information, perform a chapter division operation on the standardized text content, extract the image features and text features in the standardized text content, jointly encode the image features and text features to obtain an encoding result, based on the preset feature weights and the encoding result, perform an operation of separating text and images on the standardized text content, recognize the non-set language text content in the standardized text content, and perform a language translation operation on the non-set language text content based on user requirements to obtain the target text content, which can improve the efficiency of distinguishing text and image content, optimize the presentation method, enhance the readability of non-mother tongue content, and improve the user experience.

[0119] In another optional embodiment, when the standardized content includes standardized image content, the image processing operations performed on the standardized image content to obtain the target image content may include the following operations:

[0120] When the standardized content includes standardized image content, perform an image enhancement operation on the standardized image content based on a super-resolution model to obtain the target image content; and / or,

[0121] Extract the noise information in the standardized image content, and perform an image denoising operation on the standardized image content for the noise information through a denoising autoencoder and edge-preserving filtering technology to obtain the target image content, where the noise information includes stain information and / or watermark information; and / or,

[0122] Extract the feature point information in the standardized image content, and predict the target stitching area in the standardized image content according to the feature point information;

[0123] Perform an intelligent image stitching operation on the segmented image in the standardized image content according to the feature point information and the target stitching area to obtain the target image content.

[0124] In this optional embodiment, optionally, when the standardized content includes standardized image content, an image enhancement operation may be performed on the standardized image content based on a super-resolution model to obtain the target image content. Among them, the super-resolution model may include a super-resolution generative adversarial network (SRGAN), which can be used to improve image clarity and enhance detail performance. Moreover, during the process of performing the image enhancement operation on the standardized image content, a context attention mechanism may be introduced to reduce the probability of key details being lost in the enhanced image. Further, parameters such as image contrast and brightness can be adjusted to optimize the display effect, which is not limited in this embodiment.

[0125] In this optional embodiment, optionally, the noise information in the standardized image content can be extracted. The noise information includes stain information and / or watermark information. For example, the stains in a scanned document. An image denoising operation on the standardized image content for the noise information can be performed through a denoising autoencoder and edge-preserving filtering technology to obtain the target image content. Specifically, an image denoising operation on the standardized image content for the noise information can be performed based on a denoising autoencoder (Denoising Autoencoder) and a ResNet architecture. Moreover, during the denoising process, an edge-preserving filtering technology is introduced to ensure that the image clarity is not affected and support manual denoising by the user and / or automatic denoising by the AI reader according to the user's needs, which is not limited in this embodiment.

[0126] In this optional embodiment, optionally, feature point information in the standardized image content is extracted, and the target stitching region in the standardized image content is predicted based on the feature point information. The feature point information may include the boundary feature points of the patterns or scenes in the image. The continuity parameter and mutation parameter of the feature points can be determined based on the feature point information, and then the target stitching region in the standardized image content is predicted. The target stitching region may represent the segmentation region in the image caused by parsing or pagination. The segmented images in the standardized image content can be subjected to intelligent image stitching operations according to the feature point information and the target stitching region to obtain the target image content, and both vertical stitching and horizontal stitching modes are supported, which can be automatically selected according to the image layout format and are not limited in this embodiment.

[0127] It can be seen that implementing this optional embodiment can, when the standardized content includes standardized image content, perform image enhancement operations on the standardized image content based on the super-resolution model, extract the noise information in the standardized image content, and perform image denoising operations on the standardized image content for the noise information through the denoising autoencoder and edge-preserving filtering techniques, extract the feature point information in the standardized image content, predict the target stitching region in the standardized image content based on the feature point information, and perform intelligent image stitching operations on the segmented images in the standardized image content according to the feature point information and the target stitching region to obtain the target image content. It can dynamically adjust the enhancement effect according to the image content, and the intelligent image stitching technology improves the comic reading experience, realizes intelligent and precise image processing, and significantly improves the user experience.

[0128] In another optional embodiment, the device synchronization update of the reading data of the target user for the target text content and / or target image content based on the AI NAS device may include the following operations:

[0129] Obtain the reading data of the target user for the target text content and / or target image content on the target device. The reading data includes reading progress data and reading setting data, and the reading setting data includes reading mode setting data and / or display content setting data;

[0130] Store the reading data in the AI NAS device, and synchronize and update the reading data to each reading device corresponding to the target user based on the AI NAS device.

[0131] In this optional embodiment, optionally, the reading data of the target user for the target text content and / or target image content on the target device may include reading progress data and reading settings data. The reading settings data includes reading mode settings data and / or display content settings data. The reading mode settings data may include a night mode, an audiobook mode, etc. Among them, the night mode can automatically adjust the background color and text brightness to reduce eye fatigue. The audiobook mode can use text-to-speech (TTS) technology to convert the e-book content into voice for playback, and supports custom adjustment of the speech rate and tone. The display content settings data may include data such as the display size, color, font, and line spacing of the e-book text content, which are not limited in this embodiment.

[0132] In this optional embodiment, optionally, the reading data can be stored in the AI NAS device, and the user can log in to their account on multiple devices. When the user reads using the AI reader on the target device, based on the real-time data synchronization of WebSocket, the reading data can be stored in the AI NAS device, and then based on the AI NAS device, the reading data can be synchronously updated to each reading device corresponding to the target user to ensure that the reading content is consistent when the user switches between multiple devices. During the process of data transmission and synchronization, the data is encrypted to ensure the security of user data, which is not limited in this embodiment.

[0133] It can be seen that implementing this optional embodiment can obtain the reading data of the target user for the target text content and / or target image content on the target device, store the reading data in the AI NAS device, and based on the AI NAS device, synchronously update the reading data to each reading device corresponding to the target user. It can enable the user's reading progress and settings to be seamlessly synchronized between different devices, provide a consistent reading experience, and can reduce the parsing waiting time when the user reads on different devices, reduce the number of file parses, and improve the user's reading fluency.

[0134] Embodiment II

[0135] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of a processing method for a graphic and text reader disclosed in an embodiment of the present invention. Among them, Figure 2 the described processing method for the graphic and text reader can be applied to a processing device for the graphic and text reader. The processing device for the graphic and text reader can be applied to an AI NAS device. Among them, the processing device for the graphic and text reader may include an intelligent server or an intelligent platform for standardizing the parsing of e-books and / or electronic images. The intelligent server includes an AI NAS terminal server or a cloud server, which is not limited in an embodiment of the present invention. As Figure 2 shown, the processing method for the graphic and text reader may include the following operations:

[0136] 201. Decompose the standardized parsing task corresponding to the target file based on a preset multi-format parsing model in the AI NAS device to obtain multiple subtasks corresponding to the target file.

[0137] In an embodiment of the present invention, optionally, the preset multi-format parsing model in the AI NAS device may be a deep learning model based on the Transformer architecture and adopting a distributed computing framework, which can decompose the standardized parsing task of the target file to obtain multiple subtasks corresponding to the target file. For example, when the target file is an e-book file, the subtasks may include extracting text content, parsing chapter structures, and extracting embedded pictures or tables, etc.; when the target file is an electronic image file, the subtasks may include decompressing the picture sequence, image preprocessing, and text extraction in the image, etc. The present invention is not limited thereto.

[0138] 202. Dynamically evaluate the computing requirements of each subtask based on a preset task scheduling algorithm, and determine the parallel processing mechanism of the target file according to the computing requirements of each subtask.

[0139] In an embodiment of the present invention, optionally, the preset task scheduling algorithm may include the Round-Robin algorithm or the dynamic priority scheduling algorithm, which can dynamically evaluate the computing requirements of each subtask based on the preset task scheduling algorithm. Specifically, the file size and task complexity corresponding to each subtask can be determined, and then the computing requirements of each subtask can be evaluated. Then, according to the computing requirements of each subtask, the parallel processing mechanism of the target file is determined. The present invention is not limited thereto.

[0140] In an embodiment of the present invention, optionally, the parallel processing mechanism of the target file may include a multi-thread parallel processing mechanism, a distributed parallel processing mechanism, or a GPU acceleration processing mechanism. Among them, the multi-thread parallel processing mechanism may include using a multi-core CPU and multi-thread technology on a single AI NAS device to concurrently process different files or different subtasks of the same file through concurrent threads. The distributed parallel processing mechanism may include combining the distributed architecture of the AI NAS device to distribute the file parsing task to multiple AI NAS nodes or cloud servers for parallel processing. The GPU acceleration processing mechanism may include using a GPU accelerator to further improve the processing speed for computationally intensive tasks (such as running deep learning models or image processing). The present invention is not limited thereto.

[0141] 203. According to the parallel processing mechanism, perform parallel processing on each subtask through the multi-format parsing model to obtain the task processing result corresponding to each subtask.

[0142] In an embodiment of the present invention, optionally, according to the parallel processing mechanism, each subtask can be processed in parallel through a multi-format parsing model to obtain a task processing result corresponding to each subtask. During the processing, asynchronous I / O technology can be used to perform I / O operations (such as file reading and writing) in parallel during the parsing process to avoid performance degradation caused by I / O bottlenecks. The present invention does not make any limitations in this regard.

[0143] 204. Merge the results of each task processing to obtain the standardized content.

[0144] In an embodiment of the present invention, optionally, after each subtask is completed, the results of the subtasks can be merged through a unified task manager to generate a standardized output data structure.

[0145] 205. When the standardized content includes standardized text content, perform text analysis operations on the standardized text content to obtain the target text content.

[0146] 206. When the standardized content includes standardized image content, perform image processing operations on the standardized image content to obtain the target image content.

[0147] 207. Display the target text content and / or the target image content based on a pre-constructed cross-device responsive interface, and synchronously update the reading data of the target user for the target text content and / or the target image content on the AI NAS device.

[0148] In an embodiment of the present invention, for other descriptions of steps 205 - 207, please refer to the detailed descriptions of steps 102 - 104 in Embodiment 1 of the present invention, and the embodiments of the present invention will not be elaborated herein.

[0149] It can be seen that in the implementation Figure 2The processing method of the described graphic and text reader can decompose the standardized parsing task corresponding to the target file based on the multi-format parsing model preset in the AI NAS device to obtain multiple subtasks corresponding to the target file, dynamically evaluate the computing requirements of each subtask based on the preset task scheduling algorithm, and determine the parallel processing mechanism of the target file according to the computing requirements of each subtask. According to the parallel processing mechanism, each subtask is processed in parallel through the multi-format parsing model to obtain the task processing result corresponding to each subtask, and the results are merged to obtain the standardized content. It can reduce the parsing time and improve the resource utilization efficiency through the parallel processing mechanism during the file parsing process, reduce the occurrence of performance degradation caused by single-task overload or resource waste, parse multiple format files at the same time, and share the training knowledge of the model at the same time, improving the efficiency and accuracy of multi-format parsing. Perform text analysis operations on the standardized text content to obtain the target text content, perform image processing operations on the standardized image content to obtain the target image content, display the target text content and / or target image content based on the pre-constructed cross-device responsive interface, and synchronously update the reading data of the target user for the target text content and / or target image content based on the AI NAS device. It can migrate the parsing and rendering tasks to the AI NAS device, support multi-format parsing, improve the parsing efficiency, support cross-device synchronization, and improve the user experience. < / /

[0150] In an optional embodiment, the processing method of the graphic and text reader may further include the following operations:

[0151] Build a deep learning model based on a preset structure in the AI NAS device;

[0152] Determine multiple file formats to be trained, and generate data structure tags for each file format. The data structure tags include chapter information tags, text content tags, and image resource tags;

[0153] Obtain the files to be trained, and train the deep learning model based on the files to be trained and the data structure tags of each file format to obtain a multi-format parsing model. The files to be trained include sub-files with multi-format complexity, and the sub-files with multi-format complexity include text sub-files in multiple languages and image sub-files with multiple resolutions.

[0154] In this optional embodiment, optionally, the deep learning model with a preset structure may include a deep learning model based on the Transformer architecture and adopting a distributed computing framework. The various file formats to be trained may include PDF, EPUB, CBZ, CBR, etc. The data structure tags of each file format may include chapter information tags, text content tags, and image resource tags, and the data structure tags of each file format have a unified data structure, which is not limited in this embodiment.

[0155] In this optional embodiment, optionally, the file to be trained may include sub-files with multiple format complexities. The sub-files with multiple format complexities may include text sub-files in multiple languages and image sub-files with multiple resolutions. Preprocessing the file to be trained, including but not limited to decompression, extracting XML structures, and picture sequences, etc., may perform model training on the deep learning model based on the file to be trained and the data structure tags of each file format to obtain a multi-format parsing model, which is not limited in this embodiment.

[0156] It can be seen that implementing this optional embodiment can build a deep learning model based on a preset structure in the AI NAS device, determine various file formats to be trained, generate data structure tags for each file format, obtain the file to be trained, and perform model training on the deep learning model based on the file to be trained and the data structure tags of each file format to obtain a multi-format parsing model, which can perform multi-format parsing, solve the parsing limitations of traditional single formats, and improve parsing performance and efficiency.

[0157] In another optional embodiment, performing model training on the deep learning model based on the file to be trained and the data structure tags of each file format to obtain a multi-format parsing model may include the following operations:

[0158] Analyze the structural features of the file to be trained, and perform clustering analysis on the file to be trained according to the structural features and the data structure tags of each file format to obtain an analysis result;

[0159] Perform model training on the deep learning model according to the analysis result to obtain a multi-format parsing model;

[0160] Obtain the new format file and the parsing result of the new format file fed back by the user during the process of using the multi-format parsing model, and perform model incremental learning on the multi-format parsing model according to the new format file and the parsing result of the new format file to update the multi-format parsing model.

[0161] In this optional embodiment, optionally, analyze the structural features of the file to be trained, and perform clustering analysis on the file to be trained according to the structural features and the data structure tags of each file format to obtain an analysis result. Specifically, a feature-based self-supervised learning method can be used to perform clustering analysis on the structural features of the input file, so as to predict the parsing rules for unseen formats. Then, model training is performed on the deep learning model according to the analysis result to obtain a multi-format parsing model. Moreover, new format files and the parsing results of the new format files fed back by the user during the use of the multi-format parsing model can be obtained, and model incremental learning is performed on the multi-format parsing model according to the new format files and the parsing results of the new format files. Through the user feedback mechanism, the training set of the model is dynamically updated, and the parsing results of the new format files are used for model incremental learning to continuously optimize the model parsing performance and update the multi-format parsing model. This embodiment is not limited.

[0162] It can be seen that implementing this optional embodiment can analyze the structural features of the file to be trained, perform clustering analysis on the file to be trained according to the structural features and the data structure tags of each file format to obtain an analysis result, perform model training on the deep learning model according to the analysis result to obtain a multi-format parsing model, obtain new format files and the parsing results of the new format files fed back by the user during the use of the multi-format parsing model, and perform model incremental learning on the multi-format parsing model according to the new format files and the parsing results of the new format files to update the multi-format parsing model. It can improve the adaptability of the model to unseen format files based on adversarial training and self-supervised learning, and improve the long-term scalability of the parsing model through incremental learning, further improving the flexibility and parsing efficiency of the model.

[0163] Embodiment III

[0164] Please refer to Figure 3 , Figure 3 which is a schematic flowchart of a method for training a multi-format parsing model disclosed in an embodiment of the present invention. Among them, Figure 3 the described method for training a multi-format parsing model can be applied to the processing device of a graphic and text reader, and the processing device of the graphic and text reader can be applied to an AI NAS device. Among them, the processing device of the graphic and text reader can include an intelligent server or an intelligent platform for standardizing the parsing of e-books and / or electronic images. The intelligent server includes an AI NAS end server or a cloud server. This embodiment of the present invention is not limited. As Figure 3 shown, the method for training a multi-format parsing model can include the following operations:

[0165] 301. Build a deep learning model based on a preset framework in the AI NAS device.

[0166] In an embodiment of the present invention, optionally, the preset framework may include a deep learning framework and a distributed computing framework. The deep learning framework may include a Transformer architecture, and the distributed computing framework may be a framework based on a multi-task scheduling algorithm, such as the Round-Robin algorithm or the dynamic priority scheduling algorithm. That is, the deep learning model can implement multi-file format parsing through multi-task learning, and support Multi-Input Streams, process inputs of different formats simultaneously in the same model. Each file format corresponds to an input encoder, and the outputs of the encoders are uniformly parsed through a shared decoder, allowing the model to share weights between different formats while retaining the optimization ability for specific formats. The present invention does not make any limitations in this regard.

[0167] 302. Determine multiple file formats to be trained, and generate data structure tags for each file format.

[0168] In an embodiment of the present invention, optionally, the multiple file formats to be trained may include PDF, EPUB, CBZ, CBR, etc. The data structure tags for each file format may include chapter information tags, text content tags, and image resource tags, and the data structure tags for each file format have a unified data structure. The present invention does not make any limitations in this regard.

[0169] 303. Obtain the files to be trained, and based on the files to be trained and the data structure tags for each file format, perform model training on the deep learning model to obtain a multi-format parsing model.

[0170] In an embodiment of the present invention, optionally, the files to be trained may include sub-files with multi-format complexity. The sub-files with multi-format complexity may include text sub-files in multiple languages and image sub-files with multiple resolutions. Preprocessing the files to be trained, including but not limited to decompression, extracting XML structures, and picture sequences, etc., and based on the files to be trained and the data structure tags for each file format, perform model training on the deep learning model to obtain a multi-format parsing model. The present invention does not make any limitations in this regard.

[0171] It can be seen that implementing Figure 3 The described training method of the multi-format parsing model can build a deep learning model based on a preset structure in an AI NAS device, determine multiple file formats to be trained, generate data structure tags for each file format, obtain the files to be trained, and based on the files to be trained and the data structure tags for each file format, perform model training on the deep learning model to obtain a multi-format parsing model, which can perform multi-format parsing, solve the limitations of traditional single-format parsing, and improve parsing performance and efficiency.

[0172] In an alternative embodiment, model training of a deep learning model based on a file to be trained and data structure tags of each file format to obtain a multi-format parsing model may include the following operations:

[0173] Analyze the structural features of the file to be trained, and perform clustering analysis on the file to be trained according to the structural features and the data structure tags of each file format to obtain an analysis result;

[0174] Perform model training on the deep learning model according to the analysis result to obtain a multi-format parsing model;

[0175] Obtain a new format file and the parsing result of the new format file feedback by the user during the use of the multi-format parsing model, and perform model incremental learning on the multi-format parsing model according to the new format file and the parsing result of the new format file to update the multi-format parsing model.

[0176] In this alternative embodiment, optionally, a feature-based self-supervised learning method may be used to perform clustering analysis on the structural features of the input file, so as to predict the parsing rules of unseen formats. Perform model training on the deep learning model according to the analysis result to obtain a multi-format parsing model, and obtain a new format file and the parsing result of the new format file feedback by the user during the use of the multi-format parsing model. According to the new format file and the parsing result of the new format file, perform model incremental learning on the multi-format parsing model. Through the user feedback mechanism, dynamically update the training set of the model, and use the parsing result of the new format file for model incremental learning to continuously optimize the model parsing performance to update the multi-format parsing model. This embodiment is not limited.

[0177] It can be seen that implementing this alternative embodiment can analyze the structural features of the file to be trained, perform clustering analysis on the file to be trained according to the structural features and the data structure tags of each file format to obtain an analysis result, perform model training on the deep learning model according to the analysis result to obtain a multi-format parsing model, obtain a new format file and the parsing result of the new format file feedback by the user during the use of the multi-format parsing model, and perform model incremental learning on the multi-format parsing model according to the new format file and the parsing result of the new format file to update the multi-format parsing model. It can improve the adaptability of the model to unseen format files based on adversarial training and self-supervised learning, and improve the long-term scalability of the parsing model through incremental learning, further improving the flexibility and parsing efficiency of the model.

[0178] In another alternative embodiment, analyzing the structural features of the file to be trained and performing clustering analysis on the file to be trained according to the structural features and the data structure tags of each file format to obtain an analysis result may include the following operations:

[0179] Perform file preprocessing on the file to be trained to obtain the file content, which includes text structure content and / or picture sequence content;

[0180] Perform tagging processing on the file content according to the data structure tags of each file format to obtain the processed tagged file content, and perform structural analysis on the tagged file content to obtain the structural features of the tagged file content;

[0181] Perform clustering analysis on the file to be trained according to the tagged file content and the structural features to obtain the analysis result.

[0182] In this optional embodiment, optionally, file preprocessing can be performed on the file to be trained to obtain the file content. Specifically, the file to be trained can be decompressed to extract the file content in the file to be trained. The file content can include text structure content and / or picture structure content. The text structure content includes XML structure content. Tagging processing can be performed on the file content according to the data structure tags of each file format to obtain the processed tagged file content, that is, assign unified structured data tags to the file content according to the data structure tags of each file format. Perform structural analysis on the tagged file content to obtain the structural features of the tagged file content. Perform clustering analysis on the file to be trained according to the tagged file content and the structural features to obtain the analysis result. This embodiment is not limited.

[0183] It can be seen that implementing this optional embodiment can perform file preprocessing on the file to be trained to obtain the file content, perform tagging processing on the file content according to the data structure tags of each file format to obtain the processed tagged file content, and perform structural analysis on the tagged file content to obtain the structural features of the tagged file content. Perform clustering analysis on the file to be trained according to the tagged file content and the structural features to obtain the analysis result, which can improve the model training accuracy of the multi-format parsing model, and further improve the parsing efficiency and performance of the multi-format parsing model.

[0184] Embodiment 4

[0185] Please refer to Figure 4 , Figure 4 which is a schematic structural diagram of a processing device of a graphic and text reader disclosed in an embodiment of the present invention. Among them, Figure 4 the described processing device of the graphic and text reader can be applied to an AI NAS device. Among them, the processing device of the graphic and text reader can include an intelligent server or an intelligent platform for performing standardized parsing on e-books and / or electronic images. The intelligent server includes an AI NAS end server or a cloud server. This embodiment of the present invention is not limited. As Figure 4 shown, the processing device of the graphic and text reader can include:

[0186] The parsing module 401 is configured to perform standardized parsing on a target file based on a multi-format parsing model preset in the AI NAS device to obtain standardized content, where the standardized content includes standardized text content and / or standardized image content;

[0187] The text analysis module 402 is configured to, when the standardized content includes standardized text content, perform text analysis operations on the standardized text content to obtain target text content, and the text analysis operations include at least one of chapter division operations, graphic-text separation operations, and language translation operations;

[0188] The image processing module 403 is configured to, when the standardized content includes standardized image content, perform image processing operations on the standardized image content to obtain target image content, and the image processing operations include at least one of image enhancement operations, image denoising operations, and image intelligent stitching operations;

[0189] The display module 404 is configured to display the target text content and / or the target image content based on a pre-constructed cross-device responsive interface;

[0190] The update module 405 is configured to perform device synchronization and update on the reading data of the target user for the target text content and / or the target image content based on the AI NAS device.

[0191] It can be seen that implementing Figure 4 The processing device of the described graphic and text reader can perform standardized parsing on a target file based on a multi-format parsing model preset in the AI NAS device to obtain standardized content, perform text analysis operations on the standardized text content to obtain target text content, perform image processing operations on the standardized image content to obtain target image content, display the target text content and / or the target image content based on a pre-constructed cross-device responsive interface, and perform device synchronization and update on the reading data of the target user for the target text content and / or the target image content based on the AI NAS device. It can migrate the parsing and rendering tasks to the AI NAS device, support multi-format parsing, improve the parsing efficiency, support cross-device synchronization, and improve the user experience.

[0192] In an optional embodiment, as Figure 5 shown, the specific manner in which the parsing module 401 performs standardized parsing on the target file based on the multi-format parsing model preset in the AI NAS device includes:

[0193] Perform task decomposition on the standardized parsing task corresponding to the target file based on the multi-format parsing model preset in the AI NAS device to obtain multiple subtasks corresponding to the target file;

[0194] Dynamically evaluate the computing requirements of each subtask based on a preset task scheduling algorithm, and determine the parallel processing mechanism of the target file according to the computing requirements of each subtask;

[0195] According to the parallel processing mechanism, parallelly process each subtask through a multi-format parsing model to obtain the task processing result corresponding to each subtask;

[0196] Merge the results of each task processing to obtain the standardized content.

[0197] It can be seen that implementing Figure 5 The processing device of the described graphic reader can decompose the standardized parsing task corresponding to the target file based on the preset multi-format parsing model in the AI NAS device to obtain multiple subtasks corresponding to the target file, dynamically evaluate the computing requirements of each subtask based on the preset task scheduling algorithm, and determine the parallel processing mechanism of the target file according to the computing requirements of each subtask. According to the parallel processing mechanism, parallelly process each subtask through the multi-format parsing model to obtain the task processing result corresponding to each subtask, merge the results of each task processing to obtain the standardized content, and can reduce the parsing time and improve the resource utilization efficiency through the parallel processing mechanism during the file parsing process, reduce the occurrence of performance degradation caused by single-task overload or resource waste, parse multiple format files at the same time, and share the training knowledge of the model at the same time, improving the efficiency and accuracy of multi-format parsing. Perform text analysis operations on the standardized text content to obtain the target text content, perform image processing operations on the standardized image content to obtain the target image content, display the target text content and / or the target image content based on the pre-constructed cross-device responsive interface, and perform device synchronization updates on the reading data of the target user for the target text content and / or the target image content based on the AI NAS device. It can migrate the parsing and rendering tasks to the AI NAS device, support multi-format parsing, improve the parsing efficiency, support cross-device synchronization, and improve the user experience.

[0198] In another optional embodiment, as Figure 5 shown, when the standardized content includes standardized text content, the specific manner in which the text analysis module 402 performs text analysis operations on the standardized text content to obtain the target text content includes:

[0199] When the standardized content includes standardized text content, combine the preset chapter features to identify the chapter information of the standardized text content, and perform chapter division operations on the standardized text content based on the chapter information to obtain the target text content. The chapter information includes chapter names and hierarchical relationships; and / or,

[0200] Extract the image features and text features from the standardized text content, and jointly encode the image features and text features to obtain an encoding result;

[0201] Based on the preset feature weights and the encoding result, perform an operation of separating images and text from the standardized text content to obtain the target text content; and / or,

[0202] Identify the non-set language text content in the standardized text content, and perform a language translation operation on the non-set language text content based on the user's needs to obtain the target text content, where the user's needs include translation language type requirements and / or translation scope requirements.

[0203] It can be seen that when implementing Figure 5 The processing device of the described graphic and text reader can, when the standardized content includes the standardized text content, combine the preset chapter features to identify the chapter information of the standardized text content, and perform a chapter division operation on the standardized text content based on the chapter information, extract the image features and text features from the standardized text content, and jointly encode the image features and text features to obtain an encoding result, perform an operation of separating images and text from the standardized text content based on the preset feature weights and the encoding result, identify the non-set language text content in the standardized text content, and perform a language translation operation on the non-set language text content based on the user's needs to obtain the target text content, which can improve the efficiency of distinguishing graphic and text content, optimize the presentation method, enhance the readability of non-native content, and improve the user experience.

[0204] In another optional embodiment, as Figure 5 shown, when the standardized content includes the standardized image content, the specific ways for the image processing module 403 to perform image processing operations on the standardized image content to obtain the target image content include:

[0205] When the standardized content includes the standardized image content, perform an image enhancement operation on the standardized image content based on a super-resolution model to obtain the target image content; and / or,

[0206] Extract the noise information from the standardized image content, and perform an image denoising operation on the standardized image content for the noise information through a denoising autoencoder and an edge-preserving filtering technique to obtain the target image content, where the noise information includes stain information and / or watermark information; and / or,

[0207] Extract the feature point information from the standardized image content, and predict the target splicing area in the standardized image content according to the feature point information;

[0208] According to the feature point information and the target splicing area, perform an intelligent image splicing operation on the segmented images in the standardized image content to obtain the target image content.

[0209] It can be seen that implementing Figure 5 The processing device of the graphic reader described can, when the standardized content includes standardized image content, perform an image enhancement operation on the standardized image content based on a super-resolution model, extract noise information from the standardized image content, and perform an image denoising operation on the standardized image content for the noise information through a denoising autoencoder and edge-preserving filtering technology, extract feature point information from the standardized image content, predict the target stitching area in the standardized image content based on the feature point information, and perform an intelligent image stitching operation on the segmented images in the standardized image content according to the feature point information and the target stitching area to obtain the target image content. It can dynamically adjust the enhancement effect according to the image content, and the intelligent image stitching technology improves the comic reading experience, realizes intelligent and precise image processing, and significantly improves the user experience.

[0210] In another alternative embodiment, as Figure 5 shown, the specific manner in which the update module 405 synchronously updates the reading data of the target user for the target text content and / or target image content based on the AI NAS device includes:

[0211] Obtain the reading data of the target user for the target text content and / or target image content on the target device. The reading data includes reading progress data and reading setting data, and the reading setting data includes reading mode setting data and / or display content setting data;

[0212] Store the reading data in the AI NAS device, and synchronously update the reading data to each reading device corresponding to the target user based on the AI NAS device.

[0213] It can be seen that implementing Figure 5 The processing device of the graphic reader described can obtain the reading data of the target user for the target text content and / or target image content on the target device, store the reading data in the AI NAS device, and synchronously update the reading data to each reading device corresponding to the target user based on the AI NAS device. The reading progress and settings of the user can be seamlessly synchronized between different devices, providing a consistent reading experience, and can reduce the parsing waiting time when the user reads on different devices, reduce the number of file parses, and improve the reading fluency of the user.

[0214] In another alternative embodiment, as Figure 5 shown, the processing device of the graphic reader may further include:

[0215] A construction module 406, configured to construct a deep learning model in the AI NAS device based on a preset structure;

[0216] A determination module 407 is configured to determine multiple file formats to be trained and generate data structure tags for each file format. The data structure tags include chapter information tags, text content tags, and image resource tags.

[0217] An acquisition module 408 is configured to acquire files to be trained and perform model training on a deep learning model based on the files to be trained and the data structure tags of each file format, so as to obtain a multi-format parsing model. The files to be trained include sub-files with multi-format complexity, and the sub-files with multi-format complexity include text sub-files in multiple languages and image sub-files with multiple resolutions.

[0218] It can be seen that implementing Figure 5 The processing device of the described graphic and text reader can build a deep learning model based on a preset structure in an AI NAS device, determine multiple file formats to be trained, generate data structure tags for each file format, acquire files to be trained, and perform model training on the deep learning model based on the files to be trained and the data structure tags of each file format, so as to obtain a multi-format parsing model, which can perform multi-format parsing, solve the limitations of traditional single-format parsing, and improve parsing performance and efficiency.

[0219] In yet another alternative embodiment, as Figure 5 shown, the specific manner in which the acquisition module 408 performs model training on the deep learning model based on the files to be trained and the data structure tags of each file format to obtain a multi-format parsing model includes:

[0220] Analyze the structural features of the files to be trained, and perform clustering analysis on the files to be trained according to the structural features and the data structure tags of each file format to obtain an analysis result;

[0221] Perform model training on the deep learning model according to the analysis result to obtain a multi-format parsing model;

[0222] Acquire new format files and parsing results of the new format files fed back by the user during the process of using the multi-format parsing model, and perform model incremental learning on the multi-format parsing model according to the new format files and the parsing results of the new format files to update the multi-format parsing model.

[0223] It can be seen that implementing Figure 5The processing device of the described graphic and text reader can analyze the structural features of the file to be trained, and perform clustering analysis on the file to be trained according to the structural features and the data structure tags of each file format to obtain an analysis result. Then, based on the analysis result, model training is carried out on the deep learning model to obtain a multi-format parsing model. The new format files and the parsing results of the new format files fed back by the user during the use of the multi-format parsing model are obtained, and based on the new format files and the parsing results of the new format files, model incremental learning is carried out on the multi-format parsing model to update the multi-format parsing model. It can improve the adaptability of the model to unseen format files based on adversarial training and self-supervised learning, and improve the long-term scalability of the parsing model through incremental learning, further improving the flexibility and parsing efficiency of the model.

[0224] Embodiment Five

[0225] An embodiment of the present invention discloses a processing system of a graphic and text reader, characterized in that the system at least includes an electronic device and an AI NAS device, and the electronic device is communicatively connected to the AI NAS device;

[0226] The electronic device is configured with an application program that can access the AI NAS device, and reads the target text content and / or target image content processed by the AI NAS device based on the processing method of the graphic and text reader described in Embodiment One or Embodiment Two of the present invention through the application program.

[0227] Embodiment Six

[0228] An embodiment of the present invention discloses a computer storage medium, which stores computer instructions that, when called, are used to execute the steps in the processing method of the graphic and text reader described in Embodiment One or Embodiment Two of the present invention.

[0229] Embodiment Seven

[0230] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute the steps in the processing method of the graphic and text reader described in Embodiment One or Embodiment Two.

[0231] The device embodiments described above are only illustrative. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0232] Through the specific descriptions of the above embodiments, those skilled in the art can clearly understand that each implementation can be realized by means of software plus a necessary general hardware platform, and of course, it can also be realized by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, and the storage medium includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disk memories, tape memories, or any other medium that can be used to carry or store data and is computer-readable.

[0233] Finally, it should be noted that: What is disclosed in the processing method, model training method, device and system of a graphic reader disclosed in the embodiments of the present invention is only the preferred embodiments of the present invention, and is only used to illustrate the technical solutions of the present invention, rather than limiting it; Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A processing method for a graphic and text reader, the method being applied to an AI NAS device, characterized in that, The method includes: Based on a multi-format parsing model preset in the AI NAS device, performing standardized parsing on a target file to obtain standardized content, where the standardized content includes standardized text content and / or standardized image content; When the standardized content includes the standardized text content, performing a text analysis operation on the standardized text content to obtain target text content, where the text analysis operation includes at least one of chapter division operation, text-image separation operation, and language translation operation; When the standardized content includes the standardized image content, performing an image processing operation on the standardized image content to obtain target image content, where the image processing operation includes at least one of image enhancement operation, image denoising operation, and image intelligent stitching operation; Based on a pre-constructed cross-device responsive interface, displaying the target text content and / or the target image content, and based on the AI NAS device, synchronously updating the reading data of the target user for the target text content and / or the target image content on the device.

2. The processing method of the graphic and text reader according to claim 1, wherein The performing standardized parsing on a target file based on a multi-format parsing model preset in the AI NAS device to obtain standardized content includes: Based on the multi-format parsing model preset in the AI NAS device, decomposing the standardized parsing task corresponding to the target file into multiple subtasks corresponding to the target file; Based on a preset task scheduling algorithm, dynamically evaluating the computing requirements of each subtask, and according to the computing requirements of each subtask, determining the parallel processing mechanism of the target file; According to the parallel processing mechanism, performing parallel processing on each subtask through the multi-format parsing model to obtain the task processing result corresponding to each subtask; Performing result merging on each task processing result to obtain standardized content.

3. The processing method of the graphic and text reader according to claim 1 or 2, characterized in that, The performing a text analysis operation on the standardized text content to obtain target text content when the standardized content includes the standardized text content includes: When the standardized content includes the standardized text content, identifying the chapter information of the standardized text content in combination with preset chapter features, and based on the chapter information, performing a chapter division operation on the standardized text content to obtain target text content, where the chapter information includes chapter names and hierarchical relationships; and / or, Extracting the image features and text features in the standardized text content, and jointly encoding the image features and the text features to obtain an encoding result; Based on preset feature weights and the encoding result, performing a text-image separation operation on the standardized text content to obtain target text content; and / or, Identifying the non-set language text content in the standardized text content, and based on user requirements, performing a language translation operation on the non-set language text content to obtain target text content, where the user requirements include translation language type requirements and / or translation range requirements.

4. The processing method of the graphic and text reader according to claim 1 or 2, characterized in that, The performing an image processing operation on the standardized image content to obtain target image content when the standardized content includes the standardized image content includes: When the standardized content includes the standardized image content, perform an image enhancement operation on the standardized image content based on a super-resolution model to obtain target image content; and / or, Extract noise information from the standardized image content, and perform an image denoising operation on the standardized image content for the noise information through a denoising autoencoder and an edge-preserving filtering technique to obtain target image content, where the noise information includes stain information and / or watermark information; and / or, Extract feature point information from the standardized image content, and predict a target stitching area in the standardized image content according to the feature point information; Perform an intelligent image stitching operation on the segmented image in the standardized image content according to the feature point information and the target stitching area to obtain target image content.

5. The processing method of the graphic and text reader according to claim 1 or 2, characterized in that, The device synchronization update of the reading data of the target user for the target text content and / or the target image content based on the AINAS device includes: Obtain the reading data of the target user for the target text content and / or the target image content on the target device, where the reading data includes reading progress data and reading setting data, and the reading setting data includes reading mode setting data and / or display content setting data; Store the reading data in the AI NAS device, and synchronously update the reading data to each reading device corresponding to the target user based on the AI NAS device.

6. The processing method of the graphic and text reader according to claim 2, characterized in that, The method further includes: Construct a deep learning model based on a preset structure in the AI NAS device; Determine multiple file formats to be trained, and generate data structure tags for each of the file formats, where the data structure tags include chapter information tags, text content tags, and image resource tags; Obtain a file to be trained, and perform model training on the deep learning model based on the file to be trained and the data structure tags for each of the file formats to obtain the multi-format parsing model, where the file to be trained includes sub-files with multi-format complexity, and the sub-files with multi-format complexity include text sub-files in multiple languages and image sub-files with multiple resolutions.

7. The processing method of the graphic and text reader according to claim 6, characterized in that The performing model training on the deep learning model based on the file to be trained and the data structure tags for each of the file formats to obtain the multi-format parsing model includes: Analyze the structural features of the file to be trained, and perform clustering analysis on the file to be trained according to the structural features and the data structure tags for each of the file formats to obtain an analysis result; Perform model training on the deep learning model according to the analysis result to obtain a multi-format parsing model; Obtain a new format file and the parsing result of the new format file fed back by the user during the use of the multi-format parsing model, and perform model incremental learning on the multi-format parsing model according to the new format file and the parsing result of the new format file to update the multi-format parsing model.

8. A training method for a multi-format parsing model, characterized in that, The method is applied to an AI NAS device, and the method includes: Build a deep learning model based on a preset framework in the AI NAS device, where the preset framework includes a deep learning framework and a distributed computing framework; Determine various file formats to be trained, and generate data structure tags for each of the file formats. The data structure tags include chapter information tags, text content tags, and image resource tags; Obtain a file to be trained, and based on the file to be trained and the data structure tags of each file format, perform model training on the deep learning model to obtain a multi-format parsing model. The file to be trained includes sub-files with multi-format complexity, and the sub-files with multi-format complexity include text sub-files in multiple languages and image sub-files with multiple resolutions.

9. The training method of the multi-format parsing model according to claim 8, characterized in that The performing model training on the deep learning model based on the file to be trained and the data structure tags of each file format to obtain a multi-format parsing model includes: Analyze the structural features of the file to be trained, and based on the structural features and the data structure tags of each file format, perform clustering analysis on the file to be trained to obtain an analysis result; Perform model training on the deep learning model according to the analysis result to obtain a multi-format parsing model; Obtain a new format file and the parsing result of the new format file fed back by the user during the use of the multi-format parsing model, and based on the new format file and the parsing result of the new format file, perform model incremental learning on the multi-format parsing model to update the multi-format parsing model.

10. The training method of the multi-format parsing model according to claim 9, characterized in that, The analyzing the structural features of the file to be trained, and based on the structural features and the data structure tags of each file format, performing clustering analysis on the file to be trained to obtain an analysis result includes: Perform file preprocessing on the file to be trained to obtain file content, where the file content includes text structure content and / or picture sequence content; Perform tagging processing on the file content according to the data structure tags of each file format to obtain the processed tagged file content, and perform structural analysis on the tagged file content to obtain the structural features of the tagged file content; Perform clustering analysis on the file to be trained according to the tagged file content and the structural features to obtain an analysis result.

11. A processing device for a graphic and text reader, characterized in that, The device is applied to an AI NAS device, and the device includes: A parsing module for performing standardized parsing on a target file based on a preset multi-format parsing model in the AI NAS device to obtain standardized content, where the standardized content includes standardized text content and / or standardized image content; A text analysis module for, when the standardized content includes the standardized text content, performing text analysis operations on the standardized text content to obtain target text content. The text analysis operations include at least one of chapter division operations, text-picture separation operations, and language translation operations; An image processing module, configured to perform image processing operations on the standardized image content to obtain target image content when the standardized content includes the standardized image content, where the image processing operations include at least one of image enhancement operation, image denoising operation, and image intelligent stitching operation; A display module, configured to display the target text content and / or the target image content based on a pre-constructed cross-device responsive interface; An update module, configured to perform device synchronization and update on the reading data of the target user for the target text content and / or the target image content based on the AINAS device.

12. A processing system for a graphic and text reader, characterized in that, The system includes at least one electronic device and an AINAS device, and the electronic device is communicatively connected to the AI NAS device; The electronic device is configured with an application program that can access the AINAS device, and reads the target text content and / or the target image content processed by the AINAS device based on the processing method of the graphic and text reader according to any one of claims 1-7 through the application program.

13. A computer storage medium, characterized in that, The computer storage medium stores computer instructions, which are used to execute the processing method of the graphic and text reader according to any one of claims 1-7 when the computer instructions are called.

Images