A file automatic recognition and intelligent conversion method and device based on multi-modal AI

Abstract

The application discloses a kind of based on multimodal AI's file automatic identification and intelligent conversion method and device.Therein, method includes: to upload file carries out multilevel format check, multilevel format check includes by parsing file binary stream characteristic code and carries out first layer identification, by fusing file metadata and content fragment semantic feature and carries out second layer cross verification, and to the file of identification result suspect, call big model and carry out third layer semantic analysis, output format check result;According to format check result and file complexity, select general processing path or special processing path to extract file content, obtain structured content data;To structured content data carries out depth semantic analysis, identifies the core theme of content, entity relationship and logic structure, and constructs knowledge graph based on analysis result, knowledge graph is mapped as the structured data of target format, obtains converted data;Converted data is output as pre-set format file or write into database.

Description

Technical Field

[0001] This invention relates to the fields of computer file processing and artificial intelligence technology, and more specifically, to a method and apparatus for automatic file recognition and intelligent conversion based on multimodal AI. Background Technology

[0002] With the widespread use of internet applications, file upload functionality has become a core feature of various websites and systems. However, the following prominent problems still exist in existing technologies: First, different website platforms have varying levels of support for uploaded file formats. Many websites are designed to support only a few common formats (such as JPG, PNG, PDF, etc.). When users upload non-standard or rare format files, the system often directly reports an error or refuses to process them, seriously affecting user experience and the normal operation of the website.

[0003] Secondly, traditional file format identification mainly relies on file extensions, which is relatively insecure. Malicious users can modify file extensions to disguise script files and executable files as compliant formats for uploading, posing serious security risks to websites.

[0004] Furthermore, existing file parsing solutions require independent development of parsing logic for different formats. As the number of supported formats increases, the code maintenance cost rises exponentially, and it is difficult to quickly adapt to newly emerging file formats.

[0005] Finally, existing solutions can only handle single-modal data and are unable to cope with complex files that contain multiple elements such as images, tables, and text. Furthermore, they cannot effectively preserve the semantic structure and logical relationships of the content during format conversion. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a method and apparatus for automatic document recognition and intelligent conversion based on multimodal AI.

[0007] According to one aspect of the present invention, a method for automatic document recognition and intelligent conversion based on multimodal AI is provided, comprising: Uploaded files undergo multi-level format verification, which includes first-level identification by parsing the binary stream feature code of the file, second-level cross-validation by fusing file metadata and semantic features of content fragments, and third-level semantic analysis by calling a large model for files with questionable identification results, and outputting the format verification results. Based on the format validation results and file complexity, a general processing path or a dedicated processing path is selected to extract the file content and obtain structured content data. Deep semantic parsing is performed on structured content data to identify the core themes, entity relationships and logical structure of the content, and a knowledge graph is constructed based on the parsing results. The knowledge graph is then mapped into structured data in the target format to obtain the transformed data. The converted data can be output as a preset format file or written to a database.

[0008] According to another aspect of the present invention, a document automatic recognition and intelligent conversion device based on multimodal AI is provided, comprising: The verification module is used to perform multi-level format verification on uploaded files. The multi-level format verification includes the first level of identification by parsing the binary stream feature code of the file, the second level of cross-verification by fusing file metadata and semantic features of content fragments, and the third level of semantic analysis by calling a large model for files with questionable identification results, and outputting the format verification results. The extraction module is used to extract file content by selecting a general processing path or a special processing path based on the format validation results and file complexity, so as to obtain structured content data. The parsing module is used to perform deep semantic parsing on structured content data, identify the core themes, entity relationships and logical structures of the content, and build a knowledge graph based on the parsing results. The knowledge graph is then mapped into structured data in the target format to obtain the transformed data. The output module is used to output the converted data as a preset format file or write it to a database.

[0009] According to another aspect of the present invention, a computer-readable storage medium is provided, the storage medium storing a computer program for performing the methods described in any of the above aspects of the present invention.

[0010] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the method described in any of the preceding aspects of the present invention.

[0011] Therefore, the present invention has the following advantages compared with the prior art: Improved format recognition accuracy: Through a three-layer verification mechanism, it accurately identifies complex files such as those with disguised formats and those without file extensions, effectively preventing malicious file uploads.

[0012] Enhanced system scalability: The universal multimodal large model uniformly handles cross-format files, eliminating the need to develop independent parsing logic for each format, significantly reducing development and maintenance costs.

[0013] Enhanced multimodal content understanding capabilities: Through dedicated models for images, audio, and video, unstructured content is transformed into structured data, enabling unified processing of cross-modal information.

[0014] Improved semantic fidelity: Through semantic parsing, knowledge graphs, and intelligent error correction, the conversion results are ensured to retain the semantic logic and structural relationships of the original content.

[0015] Data security and compliance: Localized deployment + federated learning + intelligent de-identification comprehensively protect the data privacy of sensitive files.

[0016] Continuous self-optimization: The closed-loop self-learning system enables the model to adapt to new file formats and processing requirements, and the system capabilities continue to evolve. Attached Figure Description

[0017] Exemplary embodiments of the present invention can be more fully understood by referring to the following figures: Figure 1 This is a flowchart illustrating an exemplary embodiment of the present invention for a method of automatic document recognition and intelligent conversion based on multimodal AI. Figure 2 This is a schematic diagram of the structure of a file automatic recognition and intelligent conversion device based on multimodal AI provided in an exemplary embodiment of the present invention; Figure 3 This is the structure of an electronic device provided in an exemplary embodiment of the present invention. Detailed Implementation

[0018] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. It is obvious that the described embodiments are merely some embodiments of the present invention, and not all embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0019] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0020] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of the present invention are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0021] It should also be understood that in the embodiments of the present invention, "multiple" can refer to two or more, and "at least one" can refer to one, two or more.

[0022] It should also be understood that any component, data or structure mentioned in the embodiments of the present invention can generally be understood as one or more unless explicitly defined or given contrary instructions in the context.

[0023] Furthermore, the term "and / or" in this invention is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this invention generally indicates that the preceding and following related objects have an "or" relationship.

[0024] It should also be understood that the description of the various embodiments in this invention emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0025] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0026] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0027] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0028] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0029] The embodiments of this invention can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Well-known examples of terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0030] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0031] Exemplary methods Figure 1 This is a flowchart illustrating an exemplary embodiment of the present invention regarding an automatic document recognition and intelligent conversion method based on multimodal AI. This embodiment can be applied to electronic devices, such as… Figure 1 As shown, the document automatic recognition and intelligent conversion method 100 based on multimodal AI includes the following steps: Step 101: Perform multi-level format verification on the uploaded file. The multi-level format verification includes first-level identification by parsing the binary stream feature code of the file, second-level cross-validation by fusing file metadata and semantic features of content fragments, and third-level semantic analysis by calling a large model for files with questionable identification results, and output the format verification results. Step 102: Based on the format verification results and file complexity, select a general processing path or a special processing path to extract the file content and obtain structured content data. Step 103: Perform deep semantic parsing on the structured content data to identify the core themes, entity relationships and logical structure of the content, and construct a knowledge graph based on the parsing results. Map the knowledge graph into structured data in the target format to obtain the transformed data. Step 104: Output the converted data as a preset format file or write it to the database.

[0032] Specifically, this invention provides a method for automatic document recognition and intelligent conversion based on multimodal AI, comprising the following steps: Step 1: Intelligent verification of multimodal file formats (three-layer verification) Layer 1: Binary signature parsing. This involves using a lightweight NLP model to parse the signature of the file's binary stream (rather than relying solely on the file extension) to identify files with disguised formats, such as malicious scripts disguised as PDFs.

[0033] The second layer: Metadata cross-validation. It integrates semantic features of file metadata (such as file header, creation time, and size) and a small number of content fragments, and cross-validates the true format of the file through a lightweight classification model (such as FastText).

[0034] The third layer: Large model doubt judgment. For files with doubtful identification results (such as file extensions that do not match content features), the large model (such as GPT-4o / minis) is automatically called to analyze the semantics of the file fragments, and finally outputs "format confidence + correction suggestions".

[0035] Step 2: Adaptive Content Extraction AI Engine This step builds a layered AI extraction engine that combines general and specialized technologies, replacing hard-coded processing with a single technology stack. S2.1 General Layer Processing: Based on multimodal large models (such as LLaVA, Qwen-VL), it uniformly processes cross-format files, automatically identifies elements such as text, images, tables, and formulas in the files, and eliminates the need to develop separate parsing logic for each format.

[0036] S2.2 Dedicated Layer Processing: For highly complex scenarios (such as blurry scanned documents, multilingual audio, and low frame rate video), a dedicated fine-tuning model is automatically invoked.

[0037] Image processing: Pixel-level content segmentation and text extraction are achieved based on SAM (Segment Anything Model) + OCR large model (such as PP-OCRv4), supporting handwritten and distorted text recognition.

[0038] Audio processing: Real-time transcription of multiple languages ​​is achieved based on large speech models (such as Whisper v3), while emotions and keywords in the speech are extracted through sentiment analysis models.

[0039] Video processing: Extract keyframes and action sequences from videos based on spatiotemporal attention models (such as CSN) and automatically generate video summary text.

[0040] S2.3 Adaptive Scheduling: Based on the complexity and accuracy requirements of the file, the AI ​​decision model automatically selects either "fast processing with a general model" or "precise processing with a dedicated model" to balance efficiency and accuracy.

[0041] Step 3: Semantic-driven intelligent content transformation S3.1 Deep Semantic Analysis: Based on large language models (such as ChatGLM, Llama 3), deep semantic analysis is performed on the extracted content to identify the core theme, entity relationships, and logical structure of the content (such as the chapter hierarchy of a document and the row and column relationships of a table).

[0042] S3.2 Knowledge Graph Construction: Automatically constructs lightweight knowledge graphs, transforming unstructured content (such as image descriptions and audio transcriptions) into structured entity-relation pairs.

[0043] S3.3 Format Adaptation Mapping: Based on the target output format (such as JSON, database table, web page HTML), the format adaptation AI model is invoked to automatically map the knowledge graph into structured data in the target format, while preserving the semantic logic of the content (such as the row and column correspondence of tables and the contextual association of documents).

[0044] S3.4 Intelligent Error Correction: By comparing and learning the model to verify the semantic consistency between the converted data and the original content, it automatically corrects logical errors in the format conversion (such as missing JSON fields or misaligned table rows and columns).

[0045] Step 4: Multi-format output and storage The processed content is converted into a format supported by the website, including but not limited to TXT, CSV, JSON, Word, etc., or directly written into a relational or non-relational database to facilitate subsequent querying and analysis.

[0046] Step 5: Adapting the self-learning format to the AI ​​model S5.1 Data Layer: Collects user feedback (such as conversion errors, format recognition deviations) and processing logs, and automatically labels them as training samples.

[0047] S5.2 Model Layer: Through incremental fine-tuning technology, the parameters of the format recognition, content extraction, and conversion models are updated regularly to adapt to newly emerging file formats.

[0048] S5.3 Strategy Layer: Based on a reinforcement learning model, the processing strategy is automatically adjusted according to the file characteristics of different industry scenarios (such as finance, healthcare, and education).

[0049] Step Six: Lightweight AI Processing for Privacy Protection S6.1 Local Deployment: For uploaded sensitive files, lightweight AI models deployed locally are used for processing first, and the data is not transmitted to the cloud.

[0050] S6.2 Federated Learning Update: For model parameters that require collaborative optimization, federated learning technology is used to update the model across nodes without disclosing the original data, ensuring model performance while complying with data privacy regulations.

[0051] S6.3 Intelligent Desensitization: Automatically identifies sensitive information in files and automatically desensitizes it during content conversion using an AI desensitization model.

[0052] Therefore, the present invention has the following advantages compared with the prior art: Improved format recognition accuracy: Through a three-layer verification mechanism, it accurately identifies complex files such as those with disguised formats and those without file extensions, effectively preventing malicious file uploads.

[0053] Enhanced system scalability: The universal multimodal large model uniformly handles cross-format files, eliminating the need to develop independent parsing logic for each format, significantly reducing development and maintenance costs.

[0054] Enhanced multimodal content understanding capabilities: Through dedicated models for images, audio, and video, unstructured content is transformed into structured data, enabling unified processing of cross-modal information.

[0055] Improved semantic fidelity: Through semantic parsing, knowledge graphs, and intelligent error correction, the conversion results are ensured to retain the semantic logic and structural relationships of the original content.

[0056] Data security and compliance: Localized deployment + federated learning + intelligent de-identification comprehensively protect the data privacy of sensitive files.

[0057] Continuous self-optimization: The closed-loop self-learning system enables the model to adapt to new file formats and processing requirements, and the system capabilities continue to evolve.

[0058] Exemplary device Figure 2 This is a schematic diagram of the structure of a multimodal AI-based automatic document recognition and intelligent conversion device provided in an exemplary embodiment of the present invention. Figure 2 As shown, the device 200 includes: The verification module 210 is used to perform multi-level format verification on uploaded files. The multi-level format verification includes the first level of identification by parsing the binary stream feature code of the file, the second level of cross-verification by fusing file metadata and semantic features of content fragments, and the third level of semantic analysis by calling a large model for files with questionable identification results, and outputting the format verification results. The extraction module 220 is used to extract the file content by selecting a general processing path or a special processing path based on the format verification results and file complexity, so as to obtain structured content data. The parsing module 230 is used to perform deep semantic parsing on structured content data, identify the core themes, entity relationships and logical structures of the content, and construct a knowledge graph based on the parsing results. The knowledge graph is then mapped into structured data in the target format to obtain the converted data. The output module 240 is used to output the converted data as a preset format file or write it to a database.

[0059] Exemplary electronic devices Figure 3 This is the structure of an electronic device provided in an exemplary embodiment of the present invention. For example... Figure 3 As shown, the electronic device 30 includes one or more processors 31 and memory 32.

[0060] The processor 31 may be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0061] The memory 32 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 31 may execute the program instructions to implement the methods of the software programs of the various embodiments of the present invention described above, and / or other desired functions. In one example, the electronic device may also include an input device 33 and an output device 34, these components being interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0062] In addition, the input device 33 may also include, for example, a keyboard, a mouse, etc.

[0063] The output device 34 can output various information to the outside. The output device 34 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0064] Of course, for the sake of simplicity, Figure 3 Only some of the components of the electronic device relevant to the present invention are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.

[0065] Exemplary computer program products and computer-readable storage media In addition to the methods and apparatus described above, embodiments of the present invention may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of the present invention described in the "Exemplary Methods" section above.

[0066] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of the present invention. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0067] Furthermore, embodiments of the present invention may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the methods for information mining of historical change records according to various embodiments of the present invention as described in the "Exemplary Methods" section above.

[0068] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0069] The basic principles of the present invention have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in the present invention are merely examples and not limitations, and should not be considered as essential features of each embodiment of the present invention. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the present invention to the necessity of employing the aforementioned specific details.

[0070] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0071] The block diagrams of devices, systems, devices, and systems involved in this invention are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, systems, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0072] The methods and systems of the present invention may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of the present invention are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, the present invention may also be implemented as a program recorded on a recording medium, the program comprising machine-readable instructions for implementing the methods according to the present invention. Thus, the present invention also covers recording media storing programs for performing the methods according to the present invention.

[0073] It should also be noted that in the systems, apparatus, and methods of the present invention, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered equivalents of the present invention. The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the invention. Therefore, the invention is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0074] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the invention to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for automatic document recognition and intelligent conversion based on multimodal AI, characterized in that, include: Uploaded files undergo multi-level format verification, which includes a first-level identification by parsing the binary stream feature code of the file, a second-level cross-validation by fusing file metadata and semantic features of content fragments, and a third-level semantic analysis by calling a large model for files with questionable identification results, and outputting the format verification results. Based on the format verification results and file complexity, a general processing path or a special processing path is selected to extract the file content and obtain structured content data. Deep semantic parsing is performed on the structured content data to identify the core themes, entity relationships, and logical structure of the content. A knowledge graph is constructed based on the parsing results, and the knowledge graph is mapped into structured data in the target format to obtain the transformed data. The converted data is output as a preset format file or written to a database.

2. The method according to claim 1, characterized in that, The first layer of identification specifically includes: parsing the feature codes of the file binary stream using a lightweight natural language processing model to identify files with disguised formats; the second layer of cross-validation specifically includes: fusing file header, creation time, size metadata and content fragment semantic features, and cross-validating the true format of the file using a lightweight classification model; the third layer of semantic analysis specifically includes: for files whose extensions do not match their content features, calling a large model to analyze the semantics of file fragments, and outputting format confidence and correction suggestions.

3. The method according to claim 1, characterized in that, The general processing path is: to uniformly process cross-format files based on a multimodal large model, and automatically identify text, images, tables and formula elements in the files; the dedicated processing path is: to call a special fine-tuning model for precise processing in high-complexity scenarios.

4. The method according to claim 3, characterized in that, The dedicated processing path includes: for image files, pixel-level content segmentation and text extraction based on a visual segmentation model and an optical character recognition model; for audio files, multilingual transcription based on a speech model and extraction of emotions and keywords from the speech through a sentiment analysis model; and for video files, extraction of video keyframes and action sequences based on a spatiotemporal attention model and generation of video summary text.

5. The method according to claim 1, characterized in that, Also includes: The AI ​​decision-making model automatically selects a general processing path or a special processing path based on the complexity and accuracy requirements of the file, balancing processing efficiency and accuracy.

6. The method according to claim 1, characterized in that, Deep semantic parsing is performed on the structured content data to identify the core themes, entity relationships, and logical structure of the content. A knowledge graph is constructed based on the parsing results, and this knowledge graph is mapped to structured data in the target format to obtain the transformed data, including: Based on a large language model, the extracted content is subjected to deep semantic analysis to identify the core theme, entity relationships, and logical structure of document chapter hierarchy or table row and column association, and the speech analysis results are obtained. A lightweight knowledge graph is constructed based on the semantic parsing results, and unstructured content is transformed into structured entity-relation pairs based on the lightweight knowledge graph. Based on the target output format, the format adaptation AI model is invoked to map the lightweight knowledge graph into structured data in the target format, while preserving the semantic logic of the content. By comparing and learning models to verify the semantic consistency between the converted structured data and the original content, logical errors in the format conversion are automatically corrected.

7. The method according to claim 1, characterized in that, The preset format includes at least one of TXT, CSV, JSON, and Word formats; the database is a relational database or a non-relational database.

8. The method according to claim 1, characterized in that, Also includes: For sensitive files, lightweight AI models deployed locally are used for processing first, and the data is not transmitted to the cloud; model parameters are updated across nodes without leaking the original data through federated learning technology; sensitive information in the files is automatically identified and automatically desensitized during content conversion using an AI desensitization model.

9. The method according to claim 1, characterized in that, Also includes: Collect user feedback and processing logs and automatically label them as training samples. Use incremental fine-tuning technology to regularly update the parameters of the format recognition model, content extraction model and conversion model to adapt to newly emerging file formats. Based on reinforcement learning model, automatically adjust the processing strategy according to the file characteristics of different industry scenarios.

10. A document automatic recognition and intelligent conversion device based on multimodal AI, characterized in that, include: The verification module is used to perform multi-level format verification on uploaded files. The multi-level format verification includes first-level identification by parsing the binary stream feature code of the file, second-level cross-verification by fusing file metadata and semantic features of content fragments, and third-level semantic analysis by calling a large model for files with questionable identification results, and outputting the format verification results. The extraction module is used to extract the file content by selecting a general processing path or a special processing path based on the format verification result and the file complexity, so as to obtain structured content data. The parsing module is used to perform deep semantic parsing on the structured content data, identify the core theme, entity relationships and logical structure of the content, and construct a knowledge graph based on the parsing results. The knowledge graph is then mapped into structured data in the target format to obtain the converted data. The output module is used to output the converted data as a preset format file or write it to a database.

Images