Conference record processing method, device, equipment, medium and program product

Abstract

The invention discloses a conference record processing method, device and equipment, a medium and a program product, and relates to the field of artificial intelligence, in particular to application of a large model in the field of financial science and technology. The conference record processing method comprises the following steps: inputting a conference record image to be recognized into an optical character recognition model for text recognition, and obtaining a text recognition result output by the optical character recognition model; wherein the optical character recognition model comprises a text detection network structure based on a connection preselection box network and a cyclic convolutional neural network structure based on a self-attention mechanism; the conference record image comprises a conference note in a handwritten form; and performing error correction on the text recognition result based on the first large language model, and performing abstract extraction on the text recognition result after error correction based on the second large language model to obtain a conference abstract. According to the technical scheme of the embodiment of the invention, the conference record processing efficiency can be improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, specifically to the application of large models in the field of financial technology, and particularly to a method, apparatus, device, medium, and program product for processing meeting minutes. Background Technology

[0002] In meeting venues with high confidentiality requirements, recorders typically rely on handwritten notes to capture meeting information. Handwritten notes often suffer from illegible handwriting, spelling errors, non-standard word choice, and scattered information distribution, requiring word-by-word correction and logical organization. Furthermore, paper documents cannot be directly retrieved, necessitating manual reorganization, making electronic archiving time-consuming and labor-intensive.

[0003] To address the aforementioned issues, various text recognition methods have emerged, such as optical character recognition (OCR). However, traditional OCR methods struggle with illegible handwriting, multilingual text, and graphic annotations, resulting in a high error rate and making them unsuitable for generating valid electronic documents. Therefore, efficient and accurate text recognition is crucial for subsequent processing of meeting minutes. Summary of the Invention

[0004] This invention provides a method, apparatus, device, medium, and program product for processing meeting minutes, in order to solve the problems of high error rate in recognizing handwritten meeting notes and low processing efficiency.

[0005] According to one aspect of the present invention, a meeting minutes processing method is provided, comprising:

[0006] The meeting record image to be recognized is input into the optical character recognition model for text recognition, and the text recognition result output by the optical character recognition model is obtained;

[0007] The optical character recognition model includes a text detection network structure based on a pre-selection box network and a recurrent convolutional neural network structure based on a self-attention mechanism; the meeting record image includes handwritten meeting notes;

[0008] Based on the first major language model, the text recognition results are corrected, and based on the second major language model, the corrected text recognition results are extracted to obtain the conference summary.

[0009] According to another aspect of the present invention, a meeting record processing apparatus is provided, comprising:

[0010] The text recognition module is used to input the meeting record image to be recognized into the optical character recognition model for text recognition, and to obtain the text recognition result output by the optical character recognition model;

[0011] The optical character recognition model includes a text detection network structure based on a pre-selection box network and a recurrent convolutional neural network structure based on a self-attention mechanism; the meeting record image includes handwritten meeting notes;

[0012] The meeting summary generation module is used to correct the text recognition results based on the first major language model, and to extract the summary from the corrected text recognition results based on the second major language model to obtain the meeting summary.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the meeting recording processing method according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the meeting recording processing method according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer program product is provided, including a computer program that, when executed by a processor, implements the meeting recording processing method of any embodiment of the present disclosure.

[0019] The technical solution of this invention involves inputting a meeting record image to be recognized into an optical character recognition (OCR) model for text recognition, and obtaining the text recognition result output by the OCR model. The OCR model includes a text detection network structure based on a pre-selection box network and a recurrent convolutional neural network structure based on a self-attention mechanism. The meeting record image includes handwritten meeting notes. Based on a first language model, the text recognition result is corrected, and based on a second language model, a summary is extracted from the corrected text recognition result to obtain a meeting summary. This improves the accuracy of meeting record recognition and the efficiency of meeting record processing.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a meeting record processing method provided in Embodiment 1 of the present invention;

[0023] Figure 2a This is a flowchart of a meeting record processing method according to Embodiment 2 of the present invention;

[0024] Figure 2b This is a flowchart of locating text candidate regions according to Embodiment 2 of the present invention;

[0025] Figure 2c This is a flowchart of text recognition provided according to Embodiment 2 of the present invention;

[0026] Figure 2d This is a flowchart of meeting summary generation according to Embodiment 2 of the present invention;

[0027] Figure 3 This is a schematic diagram of a meeting record processing device according to Embodiment 3 of the present invention;

[0028] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the meeting recording processing method of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] Example 1

[0032] Figure 1 This is a flowchart illustrating a meeting record processing method according to Embodiment 1 of the present invention. This embodiment is applicable to situations involving text recognition and summary generation of handwritten meeting records. The method can be executed by a meeting record processing device, which can be implemented in hardware and / or software and can be configured in various general-purpose computing devices. Figure 1 As shown, the method includes:

[0033] S110. Input the meeting record image to be recognized into the optical character recognition model for text recognition, and obtain the text recognition result output by the optical character recognition model.

[0034] The optical character recognition model includes a text detection network structure based on a connected pre-selection box network and a recurrent convolutional neural network structure based on a self-attention mechanism; the meeting record images include handwritten meeting notes.

[0035] Meeting minutes images are images obtained by photographing or scanning meeting notes taken by staff within a department. Because the notes contained in meeting minutes images are handwritten, there are problems such as illegible handwriting, mixing of multiple languages, and scattered information distribution.

[0036] The optical character recognition model includes a text detection network structure based on a pre-selection bounding box network and a recurrent convolutional neural network structure based on a self-attention mechanism. The text detection network structure based on the pre-selection bounding box network is used to locate text candidate regions in the meeting record image to be recognized; the recurrent convolutional neural network based on the self-attention mechanism is used to perform text recognition on the text candidate regions to obtain the text recognition result.

[0037] In this embodiment of the invention, the meeting record image to be recognized is input into an optical character recognition (OCR) model, which performs text recognition to determine the text information contained in the meeting record image. Specifically, after the meeting record image is input into the OCR model, a text detection network structure based on a pre-selection box network first identifies candidate text regions in the meeting record image. Further, a recurrent convolutional neural network based on a self-attention mechanism performs text recognition on the candidate text regions and outputs the text recognition result.

[0038] Optionally, before inputting the meeting transcript image to be recognized into the optical character recognition model for text recognition, the method further includes:

[0039] The original meeting transcript image is cropped to a set size, and then the cropped original meeting transcript image is converted to grayscale to obtain a grayscale image;

[0040] The grayscale image is noise-removed using a Gaussian filter.

[0041] The grayscale image after noise removal is binarized to obtain a black and white image, which is used as the meeting record image to be identified.

[0042] In this optional embodiment, before inputting the meeting record image to be recognized into the optical character recognition model for text recognition, the method further includes: cropping the original meeting record image to a set size. Specifically, the original meeting record image is obtained by photographing or scanning the meeting notes collected by Jia Ang, and the image information is in the format of ".jpg" or ".png". Cropping the initial meeting record image ensures that all images are of the same size, which facilitates its electronic storage.

[0043] Furthermore, to improve the accuracy of pixel identification in subsequent text recognition and highlight the differences between text and background pixels, the cropped original meeting transcript image is converted to grayscale to obtain a grayscale image. The specific grayscale conversion formula is as follows:

[0044] Gray = a*R + b*G + c*B

[0045] Where Gray is the pixel grayscale value, R is the red channel pixel value, G is the green channel pixel value, B is the blue channel pixel value, and a, b, and c are the weights of the red, green, and blue channels, respectively.

[0046] Furthermore, to remove noise from the original meeting transcript image, Gaussian filtering is used to filter out interfering pixels, resulting in a grayscale image with less interference, thus improving the accuracy of subsequent text recognition. Gaussian noise exists in the image in the form of speckles, and its probability density follows a normal distribution. The specific expression is shown below:

[0047]

[0048] Where z is the gray value of a pixel in the image, μ represents the mean or expected value of z, and a represents the standard deviation of z. This method uses a sliding window of odd size to perform weighted averaging on the pixels, that is, multiplying the pixel values ​​within the window by the corresponding Gaussian weight coefficients and then summing them. Finally, this result is used to replace the gray value of the target pixel, thereby achieving the purpose of noise reduction.

[0049] After obtaining the image through grayscale conversion and filtering, in order to reduce the interference caused by different ink colors, a binarization operation is used to process the image information, converting the grayscale image into a black and white image as the meeting record image to be recognized. The image pixels are converted into black or white by setting a pixel threshold.

[0050] S120. Based on the first language model, the text recognition results are corrected, and based on the second language model, the corrected text recognition results are extracted to obtain the conference summary.

[0051] In this embodiment of the invention, since the text in the meeting record image to be recognized is handwritten, it often suffers from irregular handwriting, making it more difficult to recognize than printed standard text. Therefore, the text recognition results output by the optical character recognition model are prone to errors. To improve recognition accuracy, the text recognition results are further input into a first large language model for error correction. Furthermore, based on the large language model's ability to jointly analyze text context, semantic-level error correction can be performed on the text recognition results.

[0052] Furthermore, the corrected text recognition results are input into the second language model, which extracts the core elements from the text recognition results and constructs a framework for these core elements to generate a conference summary, facilitating subsequent retrieval and use of conference information.

[0053] In a specific example, the text recognition results and prompts such as "correction" or "replace typos" are input into the first language model. The first language model corrects the text recognition results and extracts key elements from the corrected text. Further, the corrected text recognition results, key elements, and keywords such as "summary generation" or "extract summary" are used as prompts and input into the second language model to obtain the conference summary output by the second language model.

[0054] It is worth noting that, in order to enhance the security of meeting information, the optical character recognition model, the first language model, and the second language model can be deployed locally to ensure that the data is used on the local network, thereby meeting the confidentiality requirements of meeting information in special scenarios such as finance.

[0055] The technical solution of this invention involves inputting a meeting record image to be recognized into an optical character recognition (OCR) model for text recognition, and obtaining the text recognition result output by the OCR model. The OCR model includes a text detection network structure based on a pre-selection box network and a recurrent convolutional neural network structure based on a self-attention mechanism. The meeting record image includes handwritten meeting notes. Based on a first language model, the text recognition result is corrected, and based on a second language model, a summary is extracted from the corrected text recognition result to obtain a meeting summary. This improves the accuracy of meeting record recognition and the efficiency of meeting record processing.

[0056] Example 2

[0057] Figure 2a This is a flowchart of a meeting record processing method provided in Embodiment 2 of the present invention. This embodiment further refines the above embodiment, providing specific steps for inputting the meeting record image to be recognized into an optical character recognition model for text recognition, obtaining the text recognition result output by the optical character recognition model, and for correcting errors in the text recognition result based on a first language model, and extracting a summary from the corrected text recognition result based on a second language model to obtain a meeting summary. Figure 2a As shown, the method includes:

[0058] S210. Using the text detection network structure based on the connected pre-selection box network in the optical character recognition model, locate the text candidate region in the meeting record image to be recognized.

[0059] In this embodiment of the invention, after the meeting record image to be recognized is input into the optical character recognition model, the text candidate region in the meeting record image is first located by the text detection network structure based on the pre-selection box network in the optical character recognition model.

[0060] Optionally, the text detection network structure based on the connected preselection box network includes a visual geometry group neural network layer, a bidirectional long short-term memory network layer, and a fully connected layer;

[0061] Using a text detection network structure based on a connected preselection box network in an optical character recognition model, candidate text regions are located in the meeting transcript image to be recognized, including:

[0062] The image to be processed is input into the visual geometry group neural network layer for feature extraction to obtain image feature data;

[0063] Image feature data is converted into a vector sequence and input into a bidirectional long short-term memory network layer to establish the contextual relationship of the text sequence, thereby obtaining the continuous distribution feature data output by the bidirectional long short-term memory network.

[0064] Continuously distributed feature data is input into a fully connected layer to obtain the output text candidate regions and corresponding confidence scores.

[0065] In this optional embodiment, the text detection network structure based on the connected preselection box network is specifically defined to include a visual geometric group neural network layer, a bidirectional long short-term memory network layer, and a fully connected layer. Correspondingly, a specific method is provided for locating text candidate regions in a meeting record image to be recognized using the text detection network structure based on the connected preselection box network in an optical character recognition model, such as... Figure 2b As shown, the image to be processed is first input into the visual geometry group neural network layer for feature extraction, obtaining image feature data. The extracted feature width is W and the height is H, and feature extraction is performed using a 3*3 convolutional kernel. Further, the image feature data is converted into a vector sequence and input into a bidirectional long short-term memory network layer to establish the text sequence context, obtaining continuous distribution feature data output by the bidirectional long short-term memory network. By combining temporal and spatial information, the bidirectional long short-term memory network can effectively capture the continuous distribution features of text lines. Finally, the continuous distribution feature data is input into a fully connected layer to obtain the output text candidate regions and their corresponding confidence scores. The specific output of the fully connected layer includes 2k vertical coordinates, representing the ordinate of the center position of the k text candidate regions and the height of the text candidate regions; 2k scores, representing the confidence scores of each text candidate region for containing text and not containing text; and k side-refinement parameters, representing the horizontal offset corresponding to each text candidate region.

[0066] For example, after processing with a fully connected layer, the model can output 2000 sets of fixed-size text candidate region coordinates and their confidence scores. Finally, cluster analysis is used to horizontally stitch together adjacent candidate regions within the same text line to form a complete set of text line candidate boxes.

[0067] The text detection network structure based on the preselection box network is further optimized for fine-grained processing of small texts. Specifically, the network learns an additional parameter 'o', which represents a scaling factor, to accurately stretch the final result. 'o' and 'o' are used to further refine the text detection. * The actual value is calculated using the following formula:

[0068]

[0069] Where, x side This represents the x-coordinate of the text candidate region predicted by the network. Represents the true x-coordinate of the candidate text region. ω represents the x-coordinate of the center of the candidate box corresponding to the text candidate region. α That is the width of the candidate box.

[0070] The loss function is calculated by fine-tuning the candidate boxes in the horizontal direction as follows:

[0071]

[0072] Where λ2 is the loss weight, N O It is the regularization coefficient. This represents the regression error after refinement.

[0073] S220. By using the recurrent convolutional neural network structure based on the self-attention mechanism in the optical character recognition model, text recognition is performed on the text candidate region to obtain the text recognition result.

[0074] In this embodiment of the invention, after locating the text candidate region in the meeting record image to be recognized, text recognition is performed on the text candidate region using a recurrent convolutional neural network structure based on a self-attention mechanism in the optical character recognition model, resulting in a text recognition result. Specifically, the text candidate region can first be self-attention encoded using a self-attention mechanism encoder to obtain the feature encoding of the text candidate region. Further, a recurrent convolutional neural network is used to further extract features from the feature encoding, and finally, text recognition is performed based on the extracted features to obtain the text recognition result. The recurrent convolutional neural network structure using a self-attention mechanism can capture the overall structural relationships of Chinese characters through self-attention, making it suitable for long-range dependency modeling of complex Chinese character components and improving the accuracy of handwritten Chinese character recognition.

[0075] Optionally, the recurrent convolutional neural network structure based on the self-attention mechanism includes a self-attention encoding layer, a convolutional layer, and a bidirectional long short-term memory network layer;

[0076] The text candidate region is used to perform text recognition through a recurrent convolutional neural network structure based on a self-attention mechanism in an optical character recognition model, resulting in text recognition results, including:

[0077] The text candidate region is input into the self-attention coding layer to obtain the feature encoding of the text candidate region output by the self-attention coding layer;

[0078] The feature encoding is input into the convolutional layer to obtain multi-scale local feature information output by the convolutional layer;

[0079] The multi-scale local feature information is input into the bidirectional long short-term memory network layer to obtain the text recognition result output by the bidirectional long short-term memory network layer.

[0080] In this optional embodiment, the recurrent convolutional neural network structure based on the self-attention mechanism is specifically defined to include a self-attention encoding layer, a convolutional layer, and a bidirectional long short-term memory network layer. Correspondingly, a specific method is provided for performing text recognition on the text candidate region using the recurrent convolutional neural network structure based on the self-attention mechanism in the optical character recognition model to obtain the text recognition result, such as... Figure 2c As shown: First, the text candidate region is input into the self-attention encoding layer to obtain the feature encoding of the text candidate region output by the self-attention encoding layer. The self-attention mechanism encoding layer captures the overall structural relationship of Chinese characters through the self-attention mechanism, which is suitable for long-range dependency modeling of complex Chinese character components. In terms of the number of multi-head layers, 8 layers based on the self-attention mechanism encoder can be selected, with each layer containing 8 multi-head attention mechanisms.

[0081] After obtaining the self-attention encoded feature code as the output, the feature code is input into the convolutional layer to further mine the multi-scale local feature information of the Chinese characters in the image, resulting in the multi-scale local feature information output by the convolutional layer. Specifically, the convolutional layer includes a convolutional neural network and a max-pooling network.

[0082] Finally, multi-scale local feature information is input into a bidirectional long short-term memory (LSTM) network layer. This LSM learns the stroke order rules and outputs the character recognition result. The bidirectional LSM network layer comprises a long short-term memory network and a fully connected network. By using a recurrent convolutional neural network (RCN) structure based on a self-attention mechanism, feature encoding can be performed first, providing better learning results for complex Chinese characters. Then, the RCN continues to mine multi-scale local feature information, obtaining more text feature information and thus improving the accuracy of handwritten text recognition.

[0083] S230. Input the error-correcting prompts and the text recognition results together into the first language model to obtain the error-corrected text recognition results output by the first language model.

[0084] In this embodiment of the invention, the text recognition result is further processed, specifically as follows: Figure 2d As shown: First, the error-correction-related prompts and the text recognition results are input into the first language model to obtain the error-corrected text recognition results output by the first language model. The error-correction-related prompts include terms such as "correction," "correction," and "error correction." The first language model can perform semantic-level error correction based on context, reducing typos, grammatical errors, and terminological confusion, thereby improving the accuracy of meeting information recognition.

[0085] At the same time, key information and related prompts can be extracted and input into the first language model, so that the first language model can correct the text recognition results and output the corrected text recognition results, as well as the key information.

[0086] S240. Input the prompt words associated with the summary generation and the text recognition results after error correction into the second language model to obtain the conference summary output by the second language model.

[0087] In this embodiment of the invention, the prompts associated with summary generation and the corrected text recognition results are input into a second language model to obtain the conference summary output by the second language model. The prompts associated with summary generation include "generate summary," "extract summary," and "conference summary generation," etc. The second language model is used to construct the logical framework of the conference, extract structured information, and generate a conference summary, enhancing content reusability.

[0088] The technical solution of this invention locates text candidate regions in the meeting record image to be recognized through a text detection network structure based on a pre-selection box network in the optical character recognition model. Then, it performs text recognition on the text candidate regions through a recurrent convolutional neural network structure based on a self-attention mechanism in the optical character recognition model to obtain text recognition results. Based on a first language model, the text recognition results are corrected, and based on a second language model, a summary is extracted from the corrected text recognition results to obtain a meeting summary. This can improve the accuracy of meeting record recognition and the efficiency of meeting record processing.

[0089] Example 3

[0090] Figure 3 This is a schematic diagram of a meeting recording processing device provided in Embodiment 3 of the present invention.

[0091] like Figure 3 As shown, the device includes:

[0092] The text recognition module 310 is used to input the meeting record image to be recognized into the optical character recognition model for text recognition, and obtain the text recognition result output by the optical character recognition model;

[0093] The optical character recognition model includes a text detection network structure based on a pre-selection box network and a recurrent convolutional neural network structure based on a self-attention mechanism; the meeting record image includes handwritten meeting notes;

[0094] The meeting summary generation module 320 is used to correct the text recognition results based on the first major language model, and to extract the summary from the corrected text recognition results based on the second major language model to obtain the meeting summary.

[0095] The technical solution of this invention involves inputting a meeting record image to be recognized into an optical character recognition (OCR) model for text recognition, and obtaining the text recognition result output by the OCR model. The OCR model includes a text detection network structure based on a pre-selection box network and a recurrent convolutional neural network structure based on a self-attention mechanism. The meeting record image includes handwritten meeting notes. Based on a first language model, the text recognition result is corrected, and based on a second language model, a summary is extracted from the corrected text recognition result to obtain a meeting summary. This improves the accuracy of meeting record recognition and the efficiency of meeting record processing.

[0096] Optionally, the text recognition module 310 includes:

[0097] The candidate region localization unit is used to locate text candidate regions in the meeting record image to be recognized by means of the text detection network structure based on the connected pre-selection box network in the optical character recognition model.

[0098] The text recognition unit is used to perform text recognition on the text candidate region through the recurrent convolutional neural network structure based on the self-attention mechanism in the optical character recognition model, and obtain the text recognition result.

[0099] Optionally, the recurrent convolutional neural network structure based on the self-attention mechanism includes a self-attention encoding layer, a convolutional layer, and a bidirectional long short-term memory network layer;

[0100] The text recognition unit is specifically used for:

[0101] The text candidate region is input into the self-attention coding layer to obtain the feature encoding of the text candidate region output by the self-attention coding layer;

[0102] The feature encoding is input into the convolutional layer to obtain multi-scale local feature information output by the convolutional layer;

[0103] The multi-scale local feature information is input into the bidirectional long short-term memory network layer to obtain the text recognition result output by the bidirectional long short-term memory network layer.

[0104] Optionally, the text detection network structure based on the connected preselection box network includes a visual geometric group neural network layer, a bidirectional long short-term memory network layer, and a fully connected layer;

[0105] Candidate region positioning units are specifically used for:

[0106] The image to be processed is input into the visual geometry group neural network layer for feature extraction to obtain image feature data;

[0107] The image feature data is converted into a vector sequence and input into the bidirectional long short-term memory network layer to establish a text sequence context relationship, thereby obtaining the continuous distribution feature data output by the bidirectional long short-term memory network.

[0108] The continuous distribution feature data is input into the fully connected layer to obtain the output text candidate regions and corresponding confidence scores.

[0109] Optionally, the meeting recording processing device also includes:

[0110] The grayscale conversion module is used to crop the original meeting record image to a set size and convert the cropped original meeting record image to grayscale before inputting it into the optical character recognition model for text recognition.

[0111] The noise filtering module is used to filter noise from the grayscale image using a Gaussian filtering method.

[0112] The image determination module is used to binarize the noise-filtered grayscale image to obtain a black and white image as the meeting record image to be identified.

[0113] Optional, the meeting summary generation module 320 is specifically used for:

[0114] The error-correcting prompts and the text recognition results are input together into the first language model to obtain the error-corrected text recognition results output by the first language model.

[0115] The prompt words associated with the summary generation and the text recognition results after error correction are input together into the second language model to obtain the conference summary output by the second language model.

[0116] Optionally, the optical character recognition model, the first language model, and the second language model are deployed in a designated local area network.

[0117] The meeting record processing device provided in the embodiments of the present invention can execute the meeting record processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0118] In the technical solution of this invention, the information collected is information and data authorized by the user or fully authorized by all parties. The collection, storage, use, processing, transmission, provision, disclosure and application of related data all comply with the relevant laws, regulations and standards of relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0119] Example 4

[0120] According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.

[0121] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, application processors, blade application processors, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0122] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0123] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0124] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as meeting recording processing methods.

[0125] In some embodiments, the conference recording processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the conference recording processing method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the conference recording processing method by any other suitable means (e.g., by means of firmware).

[0126] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0127] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or application.

[0128] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0129] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0130] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data application processors), or computing systems that include middleware components (e.g., application application processors), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0131] A computing system can include clients and applications. Clients and applications are generally geographically separated and typically interact via a communication network. The client-application relationship is established by computer programs running on the respective computers and having a client-application relationship with each other. An application can be a cloud application, also known as a cloud computing application or cloud server, which is a hosting product within the cloud computing application ecosystem. It addresses the shortcomings of traditional physical servers and VPS applications, such as high management difficulty and weak business scalability.

[0132] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0133] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for processing meeting minutes, characterized in that, include: The meeting record image to be recognized is input into the optical character recognition model for text recognition, and the text recognition result output by the optical character recognition model is obtained; The optical character recognition model includes a text detection network structure based on a pre-selection box network and a recurrent convolutional neural network structure based on a self-attention mechanism; the meeting record image includes handwritten meeting notes; Based on the first major language model, the text recognition results are corrected, and based on the second major language model, the corrected text recognition results are extracted to obtain the conference summary.

2. The method according to claim 1, characterized in that, The meeting record image to be recognized is input into an optical character recognition model for text recognition, and the text recognition result output by the optical character recognition model is obtained, including: By using the text detection network structure based on the connected preselection box network in the optical character recognition model, candidate text regions are located in the meeting record image to be recognized; The text candidate region is used to perform text recognition by using a recurrent convolutional neural network structure based on the self-attention mechanism in the optical character recognition model, and the text recognition result is obtained.

3. The method according to claim 2, characterized in that, The recurrent convolutional neural network structure based on the self-attention mechanism includes a self-attention coding layer, a convolutional layer, and a bidirectional long short-term memory network layer; The text candidate region is used to perform text recognition through a recurrent convolutional neural network structure based on a self-attention mechanism in an optical character recognition model, resulting in text recognition results, including: The text candidate region is input into the self-attention coding layer to obtain the feature encoding of the text candidate region output by the self-attention coding layer; The feature encoding is input into the convolutional layer to obtain multi-scale local feature information output by the convolutional layer; The multi-scale local feature information is input into the bidirectional long short-term memory network layer to obtain the text recognition result output by the bidirectional long short-term memory network layer.

4. The method according to claim 2, characterized in that, The text detection network structure based on the connected preselection box network includes a visual geometric group neural network layer, a bidirectional long short-term memory network layer, and a fully connected layer; Using a text detection network structure based on a connected preselection box network in an optical character recognition model, candidate text regions are located in the meeting transcript image to be recognized, including: The image to be processed is input into the visual geometry group neural network layer for feature extraction to obtain image feature data; The image feature data is converted into a vector sequence and input into the bidirectional long short-term memory network layer to establish a text sequence context relationship, thereby obtaining the continuous distribution feature data output by the bidirectional long short-term memory network. The continuously distributed feature data is input into the fully connected layer to obtain the output text candidate regions and corresponding confidence scores.

5. The method according to claim 1, characterized in that, Before inputting the meeting transcript image to be recognized into the optical character recognition model for text recognition, the following steps are also included: The original meeting transcript image is cropped to a set size, and then the cropped original meeting transcript image is converted to grayscale to obtain a grayscale image; The grayscale image is noise-removed using a Gaussian filter. The grayscale image after noise removal is binarized to obtain a black and white image, which is used as the meeting record image to be identified.

6. The method according to claim 1, characterized in that, Based on the first major language model, the text recognition results are corrected, and based on the second major language model, a summary is extracted from the corrected text recognition results to obtain the conference summary, including: The error-correcting prompts and the text recognition results are input together into the first language model to obtain the error-corrected text recognition results output by the first language model. The prompts associated with the summary generation and the corrected text recognition results are input together into the second language model to obtain the conference summary output by the second language model.

7. The method according to any one of claims 1-6, characterized in that, The optical character recognition model, the first language model, and the second language model are deployed in a designated local area network.

8. A meeting record processing device, characterized in that, include: The text recognition module is used to input the meeting record image to be recognized into the optical character recognition model for text recognition, and to obtain the text recognition result output by the optical character recognition model; The optical character recognition model includes a text detection network structure based on a pre-selection box network and a recurrent convolutional neural network structure based on a self-attention mechanism; the meeting record image includes handwritten meeting notes; The meeting summary generation module is used to correct the text recognition results based on the first major language model, and to extract the summary from the corrected text recognition results based on the second major language model to obtain the meeting summary.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the meeting recording processing method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the meeting recording processing method according to any one of claims 1-7.

11. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the meeting recording processing method according to any one of claims 1-7.

Images