Conference summary automatic generation system and method based on natural speech processing technology

Through natural speech processing technology, deep learning is used to extract feature vectors of conference recordings and image data and generate conference summary, solving the problem of time-consuming manual collation and improving the efficiency and accuracy of conference records.

CN119047435AInactive Publication Date: 2024-11-29KUNMING QIANDI CULTURAL COMMUNICATION CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411324089.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-23
Publication Date
2024-11-29
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the prior art, the generation of conference summary needs to be manually sorted, time-consuming and labor-intensive, affecting efficiency.

Method used

Using a system based on natural speech processing technology, by obtaining the text information and conference image data of conference recording transcripts, deep learning technology is used to extract feature vectors and perform correlation analysis to generate conference abstracts.

Benefits of technology

It saves time for manual summary, improves the accuracy and work efficiency of meeting records, and the generated summary is more detailed and accurate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119047435A_ABST
    Figure CN119047435A_ABST
Patent Text Reader

Abstract

The invention relates to the field of conference summary generation, and particularly discloses a conference summary automatic generation system and method based on a natural speech processing technology, and the method comprises the steps: firstly obtaining text information transcribed by a conference record and conference image data containing conference information, and then employing a deep learning technology to generate a conference summary; and feature extraction and correlation analysis are performed on the two conferences, and finally the conference abstract is generated through the generator, so that the time and the demand of manual summarization are saved, and the recording accuracy and the working efficiency are further improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of conference summary generation, and more specifically, to a system and method for automatically generating conference summaries based on natural language processing technology. Background Art

[0002] A meeting is a systematic discussion activity led by organizers and leaders, which aims to achieve specific goals through collective discussion and decision-making. It is usually held at a predetermined time and place, and the meeting is conducted according to a predetermined procedure to ensure the effectiveness of the discussion and the achievement of goals. The content of the meeting recorded during the meeting will be made into a meeting record, and a short text description containing key information of the meeting will be generated based on the meeting record, namely the meeting summary.

[0003] In the modern office environment, meetings are a core communication and decision-making mechanism. In order to efficiently record and manage meeting content, meeting record systems and meeting voice transcription systems have been widely used. However, for meeting summaries, special manpower is required to organize them after the meeting is completed. This process is not only time-consuming and labor-intensive, but also seriously affects the efficiency of generating meeting summaries.

[0004] Therefore, a system and method for automatically generating conference summaries based on natural language processing technology is desired. Summary of the invention

[0005] In order to solve the above technical problems, this application is proposed. The embodiment of this application provides a system and method for automatically generating conference summaries based on natural language processing technology, which first obtains text information transcribed from conference recordings and conference image data containing conference information, then uses deep learning technology to perform feature extraction and correlation analysis on the two, and finally generates a conference summary through a generator, thereby saving the time and demand for manual summarization, thereby improving the accuracy of records and work efficiency.

[0006] According to one aspect of the present application, a system for automatically generating conference summaries based on natural language processing technology is provided, which includes: A conference information data acquisition module, used to acquire text information transcribed from conference recordings and conference image data containing conference information; A conference information data extraction module, used to extract the conference record text global semantic feature vector and the conference information tracking global feature vector from the text information transcribed from the conference recording and the conference image data containing the conference information; The conference summary generation module is used to generate a conference summary based on the conference record text global semantic feature vector and the conference information tracking global feature vector.

[0007] According to another aspect of the present application, a method for automatically generating a conference summary based on natural language processing technology is provided, which includes: Obtaining text information transcribed from the conference recording and conference image data containing conference information; Extracting a conference record text global semantic feature vector and a conference information tracking global feature vector from the text information transcribed from the conference recording and the conference image data containing the conference information; A conference summary is generated based on the conference record text global semantic feature vector and the conference information tracking global feature vector.

[0008] Compared with the prior art, the present application provides a system and method for automatically generating conference summaries based on natural speech processing technology, which first obtains text information transcribed from conference recordings and conference image data containing conference information, and then uses deep learning technology to perform feature extraction and correlation analysis on the two, and finally generates a conference summary through a generator, thereby saving the time and need for manual summarization, thereby improving the accuracy of records and work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1 Schematic diagram of a system for automatically generating conference summaries based on natural language processing technology according to an embodiment of the present application.

[0011] Figure 2 It is a block diagram of a conference information data extraction module in a conference summary automatic generation system based on natural language processing technology according to an embodiment of the present application.

[0012] Figure 3 It is a block diagram of a conference image information feature extraction unit in a conference summary automatic generation system based on natural language processing technology according to an embodiment of the present application.

[0013] Figure 4 It is a block diagram of a conference summary generation module in a conference summary automatic generation system based on natural language processing technology according to an embodiment of the present application.

[0014] Figure 5 The present invention is a flowchart of a method for automatically generating conference summaries based on natural language processing technology according to an embodiment of the present application. DETAILED DESCRIPTION

[0015] Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0016] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0017] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0018] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0019] Figure 1 Schematic diagram of a block diagram of a conference summary automatic generation system based on natural language processing technology according to an embodiment of the present application. Figure 1 As shown, according to the embodiment of the present application, the automatic conference summary generation system 100 based on natural language processing technology includes: a conference information data acquisition module 110, which is used to acquire text information transcribed from the conference recording and conference image data containing conference information; a conference information data extraction module 120, which is used to extract the conference record text global semantic feature vector and the conference information tracking global feature vector from the text information transcribed from the conference recording and the conference image data containing the conference information; a conference summary generation module 130, which is used to generate a conference summary based on the conference record text global semantic feature vector and the conference information tracking global feature vector.

[0020] In the above-mentioned conference summary automatic generation system 100 based on natural language processing technology, the conference information data acquisition module 110 is used to obtain text information transcribed from conference recordings and conference image data containing conference information. It should be understood that a meeting is a structured discussion process led by organizers and leaders, aiming to achieve specific goals through collective communication and decision-making. Among them, the meeting is held at a predetermined time and place according to established procedures to ensure the effectiveness of the discussion and the achievement of goals. The content recorded during the meeting will be organized into meeting minutes, and a short text description, namely the meeting summary, will be extracted from it. In today's working environment, meetings are a key link in communication and decision-making. Although meeting minutes and voice transcription systems have been widely used to efficiently record and manage meeting content, manual organization is still required to generate meeting summaries. This process is not only time-consuming and labor-intensive, but also significantly affects the efficiency of generating meeting summaries. Therefore, in the technical solution of the present application, by obtaining text information transcribed from conference recordings and conference image data containing conference information, and combining deep learning technology, a conference summary is generated to reduce the time and demand for manual organization, and improve the accuracy of the record and overall work efficiency.

[0021] Specifically, the text information transcribed from the conference recording provides a detailed record of the conference discussion. Through voice transcription technology, all oral discussions in the meeting can be accurately converted into text. Such text information can not only preserve the complete discussion details, but also play a role in subsequent retrieval and analysis. Text records can help participants review the content of the meeting, track the progress of the discussion, and ensure that all decisions and action items are properly recorded. Secondly, conference image data, such as slides, whiteboard content or other visual materials displayed in the meeting, provide visual information supplements. These image data can convey details that are not fully expressed in text and oral discussions. For example, charts, diagrams and other visual materials are often used to supplement or enhance the key points in the meeting discussion, making the information clearer and easier to understand. The combined use of these two data sources, namely text and image information, can more comprehensively capture the core content and key information of the meeting, thereby ensuring that the generated meeting summary is both detailed and accurate, and can better reflect the actual situation and decision content of the meeting.

[0022] In the above-mentioned automatic meeting summary generation system 100 based on natural language processing technology, the meeting information data extraction module 120 is used to extract the meeting record text global semantic feature vector and the meeting information tracking global feature vector from the text information transcribed from the meeting recording and the meeting image data containing the meeting information. It should be understood that combining the global semantic feature vector of the text with the feature vector of the image data can provide a more complete meeting record analysis framework. Among them, the text feature vector provides the deep semantics of the oral discussion, while the image feature vector supplements the visual and graphical information. This combination can help the system fully understand the content and intention of the meeting, thereby generating a more accurate and comprehensive meeting summary. By comprehensively analyzing these global feature vectors, organizations can more efficiently capture the key points of the meeting, track the decision-making process, and improve the quality and practicality of meeting records.

[0023] Figure 2 FIG. 1 is a block diagram of a conference information data extraction module in a conference summary automatic generation system based on natural language processing technology according to an embodiment of the present application. Figure 2 As shown, in a specific embodiment of the present application, the conference information data extraction module 120 includes: a conference text information feature extraction unit 121, used to perform feature extraction on the text information transcribed from the conference recording to obtain the global semantic feature vector of the conference record text; a conference image information feature extraction unit 122, used to perform feature extraction on the conference image data containing the conference information to obtain the conference information tracking global feature vector.

[0024] It should be understood that the text of the conference recording transcription usually includes a lot of details and discussions, which contain various topics, opinions and details. By performing feature extraction on these texts, the original and lengthy text data can be converted into structured and compressed feature vectors that can capture the global semantic information of the text.

[0025] Furthermore, image data contains rich visual information. Meeting image data includes slides, charts, whiteboard notes, etc. These visual elements often contain text, graphics, and other key information that can supplement or enhance the content of the meeting discussion. For example, charts can show data trends, slides can summarize key points, and whiteboard notes may record impromptu opinions in the discussion. By performing feature extraction on these images, the key information in the images can be converted into digital feature vectors that can capture the important content and structure of the images.

[0026] In a specific embodiment of the present application, the conference text information feature extraction unit 121 includes: passing the text information transcribed from the conference recording through a conference record text semantic encoder to obtain multiple conference record text semantic feature vectors; and cascading the multiple conference record text semantic feature vectors into the conference record text global semantic feature vector.

[0027] It should be understood that the text transcribed from the conference recording may contain rich discussion content, including multiple topics, complex relationships, and subtle emotional changes. The text semantic encoder converts this complex semantic information into feature vectors through advanced natural language processing technology, which can capture the core concepts, context, and potential intentions of the text. Specifically, the text information transcribed from the conference recording is converted into an embedding vector using the embedding layer of the conference record text semantic encoder to obtain a sequence of conference record text embedding vectors; the sequence of conference record text embedding vectors is encoded based on global context semantics using the converter-based Bert model of the conference record text semantic encoder to obtain multiple conference record text semantic feature vectors.

[0028] Furthermore, cascading multiple feature vectors can comprehensively capture the multi-level information in the text. In the meeting minutes, different semantic feature vectors represent different topics, viewpoints, or details of the discussion in the meeting. By cascading these feature vectors, these different levels of information can be integrated into a unified global feature vector. This approach ensures that each topic and detail is fully considered, thereby providing a comprehensive semantic representation. This comprehensive representation can more accurately reflect the overall content and structure of the meeting discussion, avoiding omissions and one-sidedness of information.

[0029] Figure 3 FIG. 1 is a block diagram of a conference image information feature extraction unit in a conference summary automatic generation system based on natural language processing technology according to an embodiment of the present application. Figure 3 As shown, in a specific embodiment of the present application, the conference image information feature extraction unit 122 includes: a conference image target feature encoding subunit 1221, used to perform target feature encoding on the conference image data containing the conference information to obtain multiple conference information interest regions; a conference interest information convolution feature encoding subunit 1222, used to perform convolution feature encoding on the multiple conference information interest regions to obtain the conference information tracking global feature vector.

[0030] It should be understood that the visual information contained in conference image data is often complex and contains a large number of irrelevant or minor visual elements. Through target feature encoding, the most meaningful areas of the image can be identified and extracted. These areas usually include key information points, such as data in charts, key points in slides, key notes on whiteboards, etc. Focusing on these areas of interest can ensure that when analyzing and interpreting images, the focus is on the information that is most helpful to the content of the meeting. This method not only reduces the interference of irrelevant information, but also improves the accuracy and efficiency of data processing.

[0031] Furthermore, although the regions of interest provide a detailed description of the local content of the image, in order to fully understand the conference image, these local features need to be integrated into a global representation. By summarizing the convolutional features of multiple local regions, a global feature vector can be generated, which can comprehensively reflect the main information of all regions of interest in the image. This global feature vector not only retains local details, but also integrates overall information, providing a comprehensive view of the conference image.

[0032] In a specific embodiment of the present application, the conference image target feature encoding subunit 1221 includes: passing the conference image data containing conference information through a conference image denoising module based on an autoencoder to obtain a denoised conference image; passing the denoised conference image through a conference information target of interest detection model to obtain the multiple conference information regions of interest.

[0033] It should be understood that meeting image data may contain noise and interference. Meeting images such as slides, charts, or whiteboard notes may have noise, blur, or other visual interference due to poor lighting, camera quality problems, or unsatisfactory shooting angles. Noise affects the clarity of important information in the image, making it difficult to identify key information. The denoising module based on the autoencoder can effectively remove these noises and improve the readability of the image. Among them, the autoencoder can retain the core information of the image while removing unnecessary noise during the denoising process by learning the mapping relationship between the original image and the clean image. The denoising ability of the autoencoder comes from its feature learning and data reconstruction mechanism. The autoencoder is an unsupervised learning model that compresses the image into a low-dimensional feature representation through the encoder and then reconstructs the features into the original image through the decoder. In this process, the autoencoder can learn the main features and structure of the image, suppress noise, and reconstruct a clear image. In this way, the denoised image can better present the key information in the meeting, such as the data of the chart, the content of the slide, or the notes on the whiteboard, ensuring the accuracy and completeness of the information. Specifically, the conference image data containing the conference information is input into the encoder of the conference image denoising module based on the autoencoder, wherein the encoder uses a convolutional layer to perform explicit spatial encoding on the conference image data containing the conference information to obtain image features; and the image features are input into the decoder of the conference image denoising module based on the autoencoder, wherein the decoder uses a deconvolution layer to perform deconvolution processing on the image features to obtain the denoised conference image.

[0034] Furthermore, after the noise reduction process, the noise and interference in the image are effectively removed, making the important details in the image clearer. This clarity improvement provides more accurate input data for the target detection model, helping the model to better identify and locate the key information areas in the image. By applying the target detection model on a clear image, the detection results of the model can be ensured to be more reliable and accurate. Among them, the target detection model is specially designed to identify specific objects or areas in the image. It has learned how to detect and locate these key information areas through training, so it can automatically extract the most important part of the meeting content from the image. This automated detection process saves time for manual annotation and recognition and improves work efficiency. Specifically, the target anchor layer of the meeting information target detection model is used to slide the anchor frame B to process the denoised meeting image, so as to frame the meeting information target interest area of ​​the denoised meeting image, thereby obtaining the multiple meeting information interest areas. Accordingly, in a specific example of the present application, the meeting information interest target detection model is Fast R-CNN, Faster R-CNN or RetinaNet. More specifically, the conference information interest target detection model is used to process the denoised conference image using the following target detection formula to obtain the multiple conference information interest regions; wherein the target detection formula is: ;in, This is the conference image after noise reduction. is the anchor frame, Areas of interest for multiple conference information, Indicates classification, Indicates regression.

[0035] In a specific embodiment of the present application, the conference information of interest convolution feature encoding subunit 1222 includes: arranging the multiple conference information interest regions into a conference information interest tracking region input tensor; passing the conference information interest tracking region input tensor through a conference information interest tracking region feature encoder based on a three-dimensional convolutional neural network to obtain the conference information tracking global feature vector.

[0036] It should be understood that the regions of interest in the conference information image may have different sizes, positions, and shapes after being identified by the object detection model. In order for these regions to be further processed, such as feature extraction or information fusion, they need to be converted into a unified format. By arranging these regions of interest as input tensors, it can be ensured that all data is processed in the same structure and format, thereby simplifying subsequent operation steps and improving analysis efficiency.

[0037] Furthermore, the input tensor of the conference information tracking region of interest is processed by a feature encoder based on a three-dimensional convolutional neural network (3D-CNN) in order to deeply mine and understand the spatial and temporal features in the conference image, so as to more comprehensively describe and analyze the conference content. Among them, the three-dimensional convolutional neural network can capture spatial and temporal features. Unlike the traditional two-dimensional convolutional neural network, the 3D-CNN not only considers the spatial information of the image, but also can process changes in the temporal dimension. For example, in the conference image, the region of interest may contain dynamic content or time series data (such as different pages of slides, different stages in the presentation process). 3D-CNN can effectively extract these complex features by performing convolution operations in both spatial and temporal dimensions, so that the final feature encoding can better reflect the overall picture and dynamic changes of the conference content. By extracting and encoding the features of the region of interest through 3D-CNN, multiple local regions of interest can be integrated into a global feature vector. This global feature vector contains the comprehensive information of all regions of interest in the image and reflects the overall conference content. Compared with traditional feature extraction methods, 3D-CNN can capture more complex feature relationships and contextual information, thereby generating a more comprehensive and accurate global feature vector. Specifically, each layer of the conference information tracking region of interest feature encoder based on the three-dimensional convolutional neural network performs convolution processing, mean pooling processing based on the local feature matrix and nonlinear activation processing on the input data in the forward pass of the layer, so that the last layer of the conference information tracking region of interest feature encoder based on the three-dimensional convolutional neural network outputs the conference information tracking global feature vector, wherein the input of the conference information tracking region of interest feature encoder based on the three-dimensional convolutional neural network is the conference information tracking region input tensor.

[0038] In the above-mentioned conference summary automatic generation system 100 based on natural language processing technology, the conference summary generation module 130 is used to generate a conference summary based on the global semantic feature vector of the conference record text and the global feature vector of the conference information tracking. It should be understood that conference records usually contain a large amount of text information, which may include speech content, discussion points, and decision records. Conference images provide visual supplements, such as slide content, chart data, and handwritten notes. By extracting the global semantic feature vector of the text and the global feature vector of the image respectively, the key features of the conference content can be obtained from two different perspectives. This multimodal data fusion method can capture the full picture of the meeting more comprehensively, thereby generating a more accurate and rich conference summary. By combining text and image features to generate a summary, it can ensure that all important information is effectively summarized, reduce omissions and ambiguity, and support better information review and subsequent decision-making.

[0039] Figure 4FIG. 1 is a block diagram of a conference summary generation module in a conference summary automatic generation system based on natural language processing technology according to an embodiment of the present application. Figure 4 As shown, in a specific embodiment of the present application, the conference summary generation module 130 includes: a conference summary information feature fusion unit 131, which is used to perform fine-grained feature adjustment on the global semantic feature vector of the conference record text and the global feature vector of the conference information tracking based on local structure collapse reconstruction to obtain a conference summary generation feature vector; a conference summary information generation unit 132, which is used to pass the conference summary generation feature vector through a generator to generate a conference summary.

[0040] It should be understood that the fusion of the global semantic feature vector of the conference record text and the global feature vector of the conference information tracking to generate the conference summary feature vector is to comprehensively integrate the multi-dimensional information of the conference content, thereby generating an accurate and comprehensive conference summary. The core purpose of this process is to improve the quality and expressiveness of the summary by combining data features from different sources. Specifically, it is considered that a single data source may not be able to fully capture the full picture of the conference content. That is, the text feature vector may miss the key information presented in the image, while the image feature vector may not fully reflect the detailed discussion in the text. By fusing these two feature vectors, it can ensure that the summary covers all the important details in the text and visual information, thereby generating a more accurate and informative conference summary. This method can effectively reduce omissions and information bias, making the summary more complete and accurate.

[0041] In particular, in the technical solution of the present application, text features are usually generated by semantic encoders, emphasizing the vocabulary and contextual relationships in the text, and capturing high-level semantic information, which often has high dimensions and complex semantic structures. The conference information tracking features are derived from image data, processed by target detection models and three-dimensional convolutional neural networks, and mainly focus on the spatial structure and dynamic changes in the image. Image features are usually processed by denoising and spatial feature extraction during processing, showing more spatially concentrated and specific features, which is very different from the high-dimensional, abstract semantic structure of text features. Here, when text features and image features fail to be effectively aligned during the fusion process, the information of local features may be lost or diluted. For example, certain specific details in the text and specific information in the image may not be fully expressed in the simple fused features, so that the generated conference summary lacks key contextual information and details. This dimensional shift will cause the fused feature vector to have local structure collapse. This collapse of local structure means that although the global feature vector may integrate the information of text and image to some extent, the local details and semantic structure are not effectively retained, affecting the accuracy and completeness of summary generation. Therefore, in the technical solution of the present application, a feature vector for conference summary generation is obtained by performing fine-grained feature adjustment based on local structure collapse reconstruction on the global semantic feature vector of conference record text and the global feature vector of conference information tracking.

[0042] Among them, the feature fine-grained adjustment based on local structure collapse reconstruction is performed on the global semantic feature vector of the conference record text and the global feature vector of the conference information tracking to obtain the feature vector for conference summary generation, including: calculating the position difference between the global semantic feature vector of the conference record text and the global feature vector of the conference information tracking to obtain a fine-grained adjusted differential feature vector; calculating the position dot multiplication between the global semantic feature vector of the conference record text and the global feature vector of the conference information tracking to obtain a fine-grained adjusted position dot multiplication feature vector; calculating the position addition between the global semantic feature vector of the conference record text and the global feature vector of the conference information tracking to obtain a fine-grained adjusted position addition feature vector; adding the fine-grained adjusted differential feature vector and the fine-grained adjusted position The dot product feature vector and the fine-grained adjusted position-added feature vector are cascaded and input into a one-dimensional convolutional layer and maximum pooling is performed to obtain a fine-grained adjusted convolutional coding pooling feature vector; the square root of the two norm of the fine-grained adjusted convolutional coding pooling feature vector is weighted and added to the one norm of the fine-grained adjusted convolutional coding pooling feature vector to obtain a fine-grained adjustment factor; the fine-grained adjusted position-dot product feature vector and the square root of the length of the global semantic feature vector of the conference record text are added by position to obtain a fine-grained adjustment intermediate vector; the fine-grained adjustment factor is used as the first weighting coefficient and the second weighted hyperparameter is used as the second weighting coefficient to calculate the position-weighted sum of the fine-grained adjusted position-added feature vector and the fine-grained adjustment intermediate vector to obtain the conference summary generation feature vector.

[0043] Among them, the feature fine-grained adjustment based on local structure collapse reconstruction is performed on the global semantic feature vector of the conference record text and the global feature vector of the conference information tracking to obtain the feature vector for conference summary generation, including: ; ; ;in, represents the global semantic feature vector of the conference record text, represents the global feature vector of conference information tracking, It means subtracting by position. It means point multiplication by position. It means adding by position. represents a one-dimensional convolutional layer, represents maximum pooling, represents the fine-grained adjustment of the convolutional coding pooling feature vector, represents a norm, represents the two-norm, Indicates cascade, represents the first weighted hyperparameter, represents the fine-grained adjustment factor, represents the length of the global semantic feature vector of the conference record text, represents the second weighted hyperparameter, Represents the conference summary generation feature vector.

[0044] In the technical solution of the present application, in the process of fusing the global semantic feature vector of the conference record text and the global feature vector of the conference information tracking, the feature fine-grained dimension offset between the global semantic feature vector of the conference record text and the global feature vector of the conference information tracking is taken into account, which may cause the fused conference summary generation feature vector to have local structural collapse. Based on this, in the technical solution of the present application, the feature fine-grained adjustment based on local structural collapse reconstruction is performed on the global semantic feature vector of the conference record text and the global feature vector of the conference information tracking.

[0045] Specifically, based on the restricted expression of the foreground-background structure distinction of the high-dimensional manifold of the global semantic feature vector of the conference minutes text and the conference information tracking global feature vector by the amplitude of the semantic difference vector between the global semantic feature vector of the conference minutes text and the conference information tracking global feature vector, the fine-grained correspondence of key features between the global semantic feature vector of the conference minutes text and the conference information tracking global feature vector is modeled, and then active fine-grained correspondence adjustment propagation is performed through the back-propagation displacement compensation information between the corresponding eigenvalues ​​of the global semantic feature vector of the conference minutes text and the conference information tracking global feature vector, so as to avoid the imbalance at the feature vector level of the global semantic feature vector of the conference minutes text and the conference information tracking global feature vector in a focused manner, thereby synchronizing the micro-dimensions of the two feature vectors, ensuring that in the fusion process, the information between the feature vectors can be effectively aligned and combined, thereby avoiding abnormal accumulation of local structures.

[0046] Furthermore, the main reason for generating conference summaries by passing the conference summary feature vector through a generator is to convert the abstract feature vector into a concrete and coherent text, a process that can more effectively extract and present the core information of the conference. Considering that conference summary feature vectors provide the semantics and context of information, but they do not directly form complete sentences or paragraphs. The generator generates a coherent conference summary by processing the conference summary feature vector using deep learning models (such as neural networks), thereby converting the abstract high-dimensional feature vector into a concrete and fluent natural language text, saving the time and demand for manual summarization, thereby improving the accuracy of records and work efficiency. Among them, common generator types include recurrent neural networks (RNNs), which are suitable for processing sequence data; long short-term memory networks (LSTMs), which can effectively capture dependencies in long sequences; gated recurrent units (GRUs), which simplify the structure of LSTMs; transformers, which use self-attention mechanisms to process complex textual relationships; and BERT, which generates context-dependent feature vectors to improve semantic understanding.

[0047] As described above, the automatic conference summary generation system 100 based on natural language processing technology according to the embodiment of the present application can be implemented in various terminal devices. In one example, the automatic conference summary generation system 100 based on natural language processing technology can be integrated into the terminal device as a software module and / or a hardware module. For example, the automatic conference summary generation system 100 based on natural language processing technology can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the automatic conference summary generation system 100 based on natural language processing technology can also be one of the many hardware modules of the terminal device.

[0048] Alternatively, in another example, the conference summary automatic generation system 100 based on natural language processing technology and the terminal device may also be separate devices, and the conference summary automatic generation system 100 based on natural language processing technology may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0049] Figure 5 Flowchart of the method for automatically generating conference summaries based on natural language processing technology according to an embodiment of the present application. Figure 5As shown, according to the embodiment of the present application, the method for automatically generating conference summaries based on natural language processing technology includes: S110, obtaining text information transcribed from conference recordings and conference image data containing conference information; S120, extracting a conference record text global semantic feature vector and a conference information tracking global feature vector from the text information transcribed from the conference recordings and the conference image data containing conference information; S130, generating a conference summary based on the conference record text global semantic feature vector and the conference information tracking global feature vector.

[0050] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned method for automatically generating conference summaries based on natural language processing technology have been referred to above. Figures 1 to 4 The automatic generation system of conference summaries based on natural language processing technology has been introduced in detail, and therefore, its repeated description will be omitted.

[0051] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0052] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0053] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0054] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0055] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0056] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The second and other words are used to indicate names, but not to indicate any particular order.

[0057] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit of the technical solution of the present invention.

Claims

1. A conference summary automatic generation system based on natural language processing technology, It is characterized in that include: A conference information data acquisition module, used to acquire text information transcribed from conference recordings and conference image data containing conference information; A conference information data extraction module, used to extract the conference record text global semantic feature vector and the conference information tracking global feature vector from the text information transcribed from the conference recording and the conference image data containing the conference information; The conference summary generation module is used to generate a conference summary based on the conference record text global semantic feature vector and the conference information tracking global feature vector.

2. The automatic conference summary generation system based on natural language processing technology according to claim 1, It is characterized in that The conference information data extraction module includes: A conference text information feature extraction unit, used for extracting features from the text information transcribed from the conference recording to obtain a global semantic feature vector of the conference record text; The conference image information feature extraction unit is used to extract features from the conference image data containing the conference information to obtain the conference information tracking global feature vector.

3. The automatic conference summary generation system based on natural language processing technology according to claim 2, It is characterized in that The conference text information feature extraction unit includes: Passing the text information transcribed from the conference recording through a conference record text semantic encoder to obtain a plurality of conference record text semantic feature vectors; The multiple conference record text semantic feature vectors are cascaded into the conference record text global semantic feature vector.

4. The automatic conference summary generation system based on natural language processing technology according to claim 3, It is characterized in that The conference image information feature extraction unit includes: A conference image target feature encoding subunit, used for performing target feature encoding on the conference image data containing the conference information to obtain a plurality of conference information interest regions; The conference information of interest convolution feature encoding subunit is used to perform convolution feature encoding on the multiple conference information areas of interest to obtain the conference information tracking global feature vector.

5. The automatic conference summary generation system based on natural language processing technology according to claim 4, It is characterized in that The conference image target feature encoding subunit includes: Passing the conference image data containing the conference information through a conference image denoising module based on an autoencoder to obtain a denoised conference image; The denoised conference image is passed through a conference information object of interest detection model to obtain the multiple conference information regions of interest.

6. The automatic conference summary generation system based on natural language processing technology according to claim 5, It is characterized in that The conference interest information convolution feature encoding subunit includes: Arranging the plurality of conference information regions of interest into a conference information region of interest tracking input tensor; The conference information tracking region of interest input tensor is passed through a conference information tracking region of interest feature encoder based on a three-dimensional convolutional neural network to obtain the conference information tracking global feature vector.

7. The automatic conference summary generation system based on natural language processing technology according to claim 6, It is characterized in that The conference summary generation module includes: A conference summary information feature fusion unit is used to perform feature fine-grained adjustment on the global semantic feature vector of the conference record text and the global feature vector of the conference information tracking based on local structure collapse reconstruction to obtain a conference summary generation feature vector; The conference summary information generating unit is used to generate a conference summary by passing the conference summary generating feature vector through a generator.

8. The automatic conference summary generation system based on natural language processing technology according to claim 7, It is characterized in that The conference summary information feature fusion unit includes: Calculating the position difference between the conference record text global semantic feature vector and the conference information tracking global feature vector to obtain a fine-grained adjusted differential feature vector; Calculating the positional dot product between the conference record text global semantic feature vector and the conference information tracking global feature vector to obtain a fine-grained adjusted positional dot product feature vector; Calculating the positional addition between the conference record text global semantic feature vector and the conference information tracking global feature vector to obtain a fine-grained adjusted positional addition feature vector; The fine-grained adjusted differential feature vector, the fine-grained adjusted positional point product feature vector and the fine-grained adjusted positional addition feature vector are concatenated and input into a one-dimensional convolutional layer and subjected to maximum pooling to obtain a fine-grained adjusted convolutional coding pooling feature vector; Weighting the square root value of the two-norm of the fine-grained adjusted convolutional coding pooling feature vector and adding the square root value to the one-norm of the fine-grained adjusted convolutional coding pooling feature vector to obtain a fine-grained adjustment factor; Adding the fine-grained adjustment point product feature vector by position to the square root of the length of the global semantic feature vector of the conference record text by position to obtain a fine-grained adjustment intermediate vector; The fine-grained adjustment factor is used as the first weighting coefficient, the second weighted hyperparameter is used as the second weighting coefficient, and the position-weighted sum of the fine-grained adjustment position-added feature vector and the fine-grained adjustment intermediate vector is calculated to obtain the conference summary generation feature vector.

9. A method for automatically generating conference summaries based on natural language processing technology. It is characterized in that include: Obtaining text information transcribed from the conference recording and conference image data containing conference information; Extracting a conference record text global semantic feature vector and a conference information tracking global feature vector from the text information transcribed from the conference recording and the conference image data containing the conference information; A conference summary is generated based on the conference record text global semantic feature vector and the conference information tracking global feature vector.

10. The method for automatically generating conference summaries based on natural language processing technology according to claim 9, It is characterized in that Extracting a conference record text global semantic feature vector and a conference information tracking global feature vector from the text information transcribed from the conference recording and the conference image data containing the conference information includes: Performing feature extraction on the text information transcribed from the conference recording to obtain a global semantic feature vector of the conference record text; Feature extraction is performed on the conference image data containing the conference information to obtain the conference information tracking global feature vector.

Citation Information

Cited By

  • An innovative system and method for automatically generating meeting minutes and intelligently refining them

    CN119783644A