Visual textualization two-way operation manual compiling system based on BERT natural language processing

Abstract

The invention relates to a visual textualization two-way operation manual compiling system based on BERT natural language processing, and the system comprises a text analysis module which is constructed based on BERT, carries out the text analysis of an input operation manual text, and extracts operation steps and key elements in the operation manual text, and the key elements at least comprise actions and objects; the visualization generation module is used for establishing a mapping relation between each key element and a visualization element based on the extracted key elements, and organizing the visualization element based on the operation steps as a sorting basis to obtain a video; the bidirectional conversion module is used for generating a second text according to the video and comparing the second text with the text of the operation manual, and if the similarity is lower than a pre-configured threshold value, the text analysis module re-extracts key elements in the text of the operation manual; and the interaction optimization module is used for receiving a feedback text of the user and extracting a prompt word based on the feedback text input by the user to update the text analysis module and the visualization generation module. Compared with the prior art, the method has the advantages that the continuity of the video generated based on the operation manual can be improved, and the like.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing and visualization, and in particular to a visualized textual bidirectional operation manual compilation system based on BERT natural language processing. Background Art

[0002] Currently, the banking industry has begun to apply more new platforms, new tools, and new technologies. The promotion of these new platforms, new tools, and new technologies is usually carried out by the head office's technology stack and distributed to each branch for implementation. This faces various difficulties in the promotion of innovative technologies, such as the branches' knowledge blind spots about innovative technologies, operating manuals that need to be compiled and distributed for promotion, and so on.

[0003] However, on the one hand, the operation manual is not intuitive. In order to lower the training threshold, in addition to the operation manual, a corresponding animation demonstration video will also be compiled. In this way, it is necessary to compile the operation manual and produce the animation demonstration video at the same time, and both need to be produced independently, which is inefficient.

[0004] On the other hand, some persons in charge may prefer to record demonstration videos and are not accustomed to making operation manuals. Therefore, requiring them to make additional operation manuals also increases the burden.

[0005] Although some existing technologies provide text visualization technology, for example, Chinese patent CN115249062A discloses a text generation video network model, method and device based on knowledge distillation and visual recurrent unit. First, a method for obtaining knowledge from a text generation image model through knowledge distillation to enhance the effect of a text generation video model is disclosed. The text generation image model is used as a teacher network, and the knowledge of mapping text to visual content is distilled from it. This knowledge is used as an abstract constraint shared by each frame in the generated video, thereby improving the semantic consistency of the text image generated for each frame.

[0006] However, such existing technologies are unable to achieve two-way conversion between text and video. Secondly, they are unable to deal with texts that are different from general texts or texts in other fields. Operation manuals often emphasize operation and execution, and existing video generation often ignores this point, resulting in the lack of coherence in the generated animation demonstration videos, and thus unable to provide effective guidance. Summary of the invention

[0007] The purpose of the present invention is to provide a visual text-based bidirectional operation manual compilation system based on BERT natural language processing.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] A visual textual bidirectional operation manual compilation system based on BERT natural language processing, including:

[0010] The text analysis module is built based on BERT and performs text analysis on the input operation manual text to extract the operation steps and key elements in the operation manual text, wherein the key elements include at least actions and objects;

[0011] A visualization generation module, based on the extracted key elements, establishes a mapping relationship between each key element and a visualization element, and organizes the visualization elements to obtain a video based on the operation steps as a sorting basis;

[0012] A bidirectional conversion module extracts the visual elements in the video and the time sequence of the positions of the visual elements, generates a second text based on the time sequence of the visual elements and the positions of the visual elements and the mapping relationship between the key elements and the visual elements, and compares the second text with the operation manual text. If the similarity is lower than a pre-configured threshold, the text analysis module re-extracts the key elements in the operation manual text.

[0013] The interactive optimization module receives the user's feedback text and extracts prompt words based on the feedback text input by the user to update the text analysis module and the visualization generation module.

[0014] The key elements also include judgment, loop and attributive.

[0015] The process of extracting key elements by the text analysis module includes:

[0016] Get the text data of the operation manual;

[0017] Performing cleaning and preprocessing, wherein the preprocessing includes word segmentation and removal of stop words;

[0018] Use the BERT network to extract features from the cleaned and preprocessed text data to obtain the semantic vector representation of each operation step;

[0019] Extract the key elements involved in each operation step.

[0020] The step of obtaining text data of the operation manual includes:

[0021] Read the operation manual file and determine the type of the operation manual based on the file suffix;

[0022] If the file type of the operation manual is a text document, the text in the file is extracted as the operation manual text;

[0023] If the file type of the operation manual is a pdf file, the objects and the positions of the objects in the file are identified, the text of each object is obtained, and the operation manual text is obtained by combining the objects according to the positions of the objects.

[0024] The obtaining of the text of each object includes:

[0025] Step A1: randomly select 2 pages, and determine whether each page contains a text object. If neither of the selected 2 pages contains a text object, execute step A4; otherwise, execute step A2;

[0026] Step A2: extracting text from the text object, and determining whether the text in any of the two selected pages exceeds the first set number of characters. If yes, execute step A3; otherwise, execute step A4.

[0027] Step A3: extracting text from all text objects in the file;

[0028] Step A4: extract the text from all text objects in the file, and extract the text from all image objects by OCR.

[0029] In the time series of the positions of the visualization elements, each element is a triple data group, wherein the triple data group includes: the horizontal coordinate of the anchor point, the vertical coordinate of the anchor point, and a timestamp.

[0030] The visualization elements include graphic elements and image elements.

[0031] The graphic elements are vector graphics, including at least: straight lines, circles, and arrows.

[0032] The image element is a bitmap or a vector image.

[0033] The prompt words include element names, actions and multiple attribute words.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. By organizing the key elements extracted based on BERT into steps and including actions, it is possible to provide timing and action attributes for the visualization elements, making the generated video more coherent, thereby greatly improving the flexibility and interactivity of use.

[0036] 2. Different text acquisition methods are adopted according to the file type of the operation manual, thereby improving the accuracy and speed of text acquisition.

[0037] 3. The method of randomly extracting two pages is used to identify whether the file is a picture, so that when identifying text files, it can avoid extracting invalid information from pictures and improve the efficiency and accuracy of text acquisition. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a block diagram of the composition of the present invention;

[0039] Figure 2This is a schematic diagram of the principle of the BERT network in the text analysis module;

[0040] Figure 3 A deployment diagram. DETAILED DESCRIPTION

[0041] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0042] A visual textual bidirectional operation manual compilation system based on BERT natural language processing, such as Figure 1 As shown, including:

[0043] The text analysis module is built based on BERT and performs text analysis on the input operation manual text to extract the operation steps and key elements in the operation manual text, where the key elements include at least actions and objects;

[0044] A visualization generation module, based on the extracted key elements, establishes a mapping relationship between each key element and a visualization element, and organizes the visualization elements to obtain a video based on the operation steps as a sorting basis;

[0045] A bidirectional conversion module extracts the visual elements in the video and the time sequence of the positions of the visual elements, generates a second text based on the time sequence of the visual elements and the positions of the visual elements and the mapping relationship between the key elements and the visual elements, and compares the second text with the operation manual text. If the similarity is lower than a pre-configured threshold, the text analysis module re-extracts the key elements in the operation manual text.

[0046] The interactive optimization module receives the user's feedback text and extracts prompt words based on the feedback text input by the user to update the text analysis module and the visualization generation module.

[0047] By organizing the key elements extracted based on BERT into steps and including actions, it is possible to provide timing and action attributes for visualization elements, making the generated video more coherent, thereby greatly improving the flexibility and interactivity of use.

[0048] In this embodiment, the key elements also include judgment, loop and attributive. Of course, in other embodiments, more types of key elements may also be included.

[0049] In some embodiments, the process of extracting key elements by the text analysis module includes:

[0050] Get the text data of the operation manual;

[0051] Perform cleaning and preprocessing, where preprocessing includes word segmentation and removal of stop words;

[0052] like Figure 2 As shown in the figure, the BERT network is used to extract features from the cleaned and preprocessed text data to obtain the semantic vector representation of each operation step;

[0053] Extract the key elements involved in each operation step.

[0054] In some embodiments, obtaining text data of the operation manual includes:

[0055] Read the operation manual file and determine the type of the operation manual based on the file suffix;

[0056] If the file type of the operation manual is a text document, the text in the file is extracted as the text of the operation manual;

[0057] If the file type of the operation manual is a pdf file, the objects and the positions of the objects in the file are identified, the text of each object is obtained, and the operation manual text is obtained by combining the objects according to the positions of the objects.

[0058] In particular, in this embodiment, obtaining the text of each object includes:

[0059] Step A1: randomly select 2 pages, and determine whether each page contains a text object. If neither of the selected 2 pages contains a text object, execute step A4; otherwise, execute step A2;

[0060] Step A2: extracting text from the text object, and determining whether the text in any of the two selected pages exceeds the first set number of characters. If yes, execute step A3; otherwise, execute step A4.

[0061] Step A3: extracting text from all text objects in the file;

[0062] Step A4: extract the text from all text objects in the file, and extract the text from all image objects by OCR.

[0063] The method of randomly extracting two pages is used to identify whether the file is a picture, so that when identifying text files, it can avoid extracting invalid information from the picture and improve the efficiency and accuracy of text acquisition.

[0064] In this embodiment, in the time series of the position of the visualization element, each element is a triple data group, wherein the triple data group includes: the horizontal coordinate of the anchor point, the vertical coordinate of the anchor point, and a timestamp.

[0065] In addition, during the actual operation process, the operation manual needs to be divided into sections, and a separate video needs to be generated for each section.

[0066] Generally, visualization elements include graphic elements and image elements. Graphic elements are vector graphics, including at least straight lines, circles, and arrows, and image elements are bitmaps or vector graphics.

[0067] The prompt words include element names, actions, and multiple attribute words.

[0068] The two-way seamless conversion between text and visualization of the operation manual content has been achieved, breaking through the one-way limitations of existing technologies. When promoting new platforms, new tools, and new technologies, the superior technology stack can be more efficient and intuitive for every subordinate technology staff to learn and watch, abandoning the previous inefficient model of branch representative training and the poor quality of new technologies with low popularity, greatly improving efficiency and comprehensiveness. At the same time, the technology also innovatively combines various websites or programs that have been promoted within the bank. BERT can automatically match the operation steps according to the current usage scenario; at the same time, branch technology staff can freely choose text descriptions or graphic demonstrations to learn and operate according to their own preferences, greatly improving the flexibility and interactivity of use.

[0069] Give full play to BERT's advantages in natural language understanding and achieve a deep integration of text content and visualization effects. Ensure the accuracy and completeness of the conversion results, avoid information loss, and improve the use value of the operation manual. With adaptive learning capabilities, it can continuously optimize the system's performance and interactive design based on user habits and feedback. Through continuous learning, the practicality and ease of use of the operation manual are continuously improved.

[0070] The system can be widely used in the compilation of operation manuals for various complex equipment or systems, greatly improving compilation efficiency and reducing labor costs. It can also be used in the fields of education and training, technical exchanges, etc. carried out by the head office to enhance the effect of knowledge transfer. As an intelligent operation manual compilation system, its core technical solution has strong innovation and practical value, and is expected to be widely used in related fields.

[0071] In general, the BERT-based visual textual bidirectional operation manual compilation system proposed in this application makes full use of advanced natural language processing technology and integrates it into the promotion of our bank in the field of innovative technology, realizing the seamless conversion of the operation manual content between text and visualization, and providing a more intelligent and interactive operation experience for the linkage work between superiors and subordinates. The system has significant advantages in improving writing efficiency and enhancing user experience, and has broad market prospects in multiple application scenarios.

[0072] In this application, the interactive optimization module and the text analysis module work together to form a complete two-way compilation tool. Users can choose to use visual or textual compilation methods according to their needs to obtain the required operation instructions. The results of the two compilation methods can be converted into each other, which is convenient for users to switch.

[0073] Finally, if Figure 3 As shown, during the system integration and testing process, each module is integrated into a complete visual text-based bidirectional operation manual compilation system to conduct functional testing, performance testing, user experience testing, etc., to ensure the stability and reliability of the system, and to continuously optimize and iterate to improve the compilation quality and user experience.

[0074] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

Claims

1. A visual textual bidirectional operation manual compilation system based on BERT natural language processing, characterized in that: include: The text analysis module is built based on BERT and performs text analysis on the input operation manual text to extract the operation steps and key elements in the operation manual text, wherein the key elements include at least actions and objects; A visualization generation module, based on the extracted key elements, establishes a mapping relationship between each key element and a visualization element, and organizes the visualization elements to obtain a video based on the operation steps as a sorting basis; A bidirectional conversion module extracts the visual elements in the video and the time sequence of the positions of the visual elements, generates a second text based on the time sequence of the visual elements and the positions of the visual elements and the mapping relationship between the key elements and the visual elements, and compares the second text with the operation manual text. If the similarity is lower than a pre-configured threshold, the text analysis module re-extracts the key elements in the operation manual text. The interactive optimization module receives the user's feedback text and extracts prompt words based on the feedback text input by the user to update the text analysis module and the visualization generation module.

2. According to claim 1, a visual textual bidirectional operation manual compilation system based on BERT natural language processing is characterized in that: The key elements also include judgment, loop and attributive.

3. According to claim 1, a visual textual bidirectional operation manual compilation system based on BERT natural language processing is characterized in that: The process of extracting key elements by the text analysis module includes: Get the text data of the operation manual; Performing cleaning and preprocessing, wherein the preprocessing includes word segmentation and removal of stop words; Use the BERT network to extract features from the cleaned and preprocessed text data to obtain the semantic vector representation of each operation step; Extract the key elements involved in each operation step.

4. According to claim 3, a visual textual bidirectional operation manual compilation system based on BERT natural language processing is characterized in that: The step of obtaining text data of the operation manual includes: Read the operation manual file and determine the type of the operation manual based on the file suffix; If the file type of the operation manual is a text document, the text in the file is extracted as the operation manual text; If the file type of the operation manual is a pdf file, the objects and the positions of the objects in the file are identified, the text of each object is obtained, and the operation manual text is obtained by combining the objects according to the positions of the objects.

5. According to claim 4, a visual textual bidirectional operation manual compilation system based on BERT natural language processing is characterized in that: The obtaining of the text of each object includes: Step A1: randomly select 2 pages, and determine whether each page contains a text object. If neither of the selected 2 pages contains a text object, execute step A4; otherwise, execute step A2; Step A2: extracting text from the text object, and determining whether the text in any of the two selected pages exceeds the first set number of characters. If yes, execute step A3; otherwise, execute step A4. Step A3: extracting text from all text objects in the file; Step A4: extract the text from all text objects in the file, and extract the text from all image objects by OCR.

6. According to claim 1, a visual textual bidirectional operation manual compilation system based on BERT natural language processing is characterized in that: In the time series of the positions of the visualization elements, each element is a triple data group, wherein the triple data group includes: the horizontal coordinate of the anchor point, the vertical coordinate of the anchor point, and a timestamp.

7. According to claim 1, a visual textual bidirectional operation manual compilation system based on BERT natural language processing is characterized in that: The visualization elements include graphic elements and image elements.

8. According to claim 7, a visual textual bidirectional operation manual compilation system based on BERT natural language processing is characterized in that: The graphic elements are vector graphics, including at least: straight lines, circles, and arrows.

9. According to claim 1, a visual textual bidirectional operation manual compilation system based on BERT natural language processing is characterized in that: The image element is a bitmap or a vector image.

10. A visual textual bidirectional operation manual compilation system based on BERT natural language processing according to claim 1, characterized in that: The prompt words include element names, actions and multiple attribute words.

Images