A manuscript preprocessing method, device, electronic device and storage medium

Abstract

The present invention discloses a manuscript preprocessing method, device, electronic device and storage medium. The manuscript preprocessing method comprises: converting the content of the manuscript to be processed into an image to obtain a first image, wherein the pixel value of the text area of ​​each paragraph of the content of the manuscript to be processed in the image is a specified pixel value; intercepting a second image from the first image, wherein the second image is an image corresponding to the minimum circumscribed rectangular area of ​​the first text of each paragraph. The first image and the second image are input into a pre-trained manuscript preprocessing model, and then the probability that each paragraph of the manuscript to be processed belongs to each text element and the probability that each paragraph belongs to each text format can be obtained. The preprocessing information of the manuscript to be processed is generated according to the probability that each paragraph belongs to each text element and the probability that each paragraph belongs to each text format. Compared with manual review and identification, it can be faster, simpler and more accurate, and can effectively improve the efficiency of manuscript review, while facilitating the reviewer to review.

Description

Technical Field

[0001] The present invention relates to the field of communication analysis technology, and in particular to a manuscript preprocessing method, device, electronic equipment and storage medium. Background Art

[0002] As scientific research is valued by everyone, many fund applications, achievement awards, talent evaluations, etc. are based on academic research results. Paper publication is one of the main ways to test academic achievements. Therefore, each submission platform receives a large number of submissions, while the number of reviewers is relatively small. A reviewer may be responsible for the review of several or more than a dozen manuscripts at the same time, or may be responsible for the review of multiple submission platforms at the same time. Each submission platform has different requirements for the format and content of manuscripts. If the reviewer also has to review whether the format, content missing, etc. meet the requirements, it will consume a lot of time and energy. In order to save reviewers' time and speed up the review process, it is necessary to pre-process the newly received manuscripts.

[0003] The current manuscript preprocessing method is: staff review the received manuscripts to determine whether the manuscripts are complete and whether they meet the preset format requirements, and mark the reviewed content. However, the manual review method is inefficient and time-consuming, and manual review is prone to omissions and errors, which can easily mislead and cause inconvenience to reviewers, affecting the review process and efficiency. Summary of the invention

[0004] The present invention provides a manuscript preprocessing method, device, electronic device and storage medium to solve the problem that manual manuscript review is prone to omissions and errors, which can easily mislead and cause inconvenience to reviewers and affect the review process and efficiency.

[0005] In a first aspect, the present invention provides a manuscript preprocessing method, comprising:

[0006] Converting the content of the manuscript to be processed into an image to obtain a first image, wherein the pixel value of the text area of ​​each paragraph of the content of the manuscript to be processed in the image is a specified pixel value;

[0007] intercepting a second image from the first image, where the second image is an image corresponding to the minimum circumscribed rectangular area of ​​the first text of each paragraph;

[0008] Inputting the first image and the second image into a pre-trained manuscript preprocessing model to obtain the probability that each paragraph of the manuscript to be processed belongs to each text element and the probability that each paragraph belongs to each text format;

[0009] The preprocessing information of the manuscript to be processed is generated according to the probability that each paragraph belongs to each text element and the probability that each paragraph belongs to each text format.

[0010] In a second aspect, the present invention provides a manuscript preprocessing device, comprising:

[0011] A content conversion module, used for converting the content of the manuscript to be processed into an image to obtain a first image, wherein the pixel value of the text area of ​​each paragraph of the content of the manuscript to be processed in the image is a specified pixel value;

[0012] An image capture module, used to capture a second image from the first image, where the second image is an image corresponding to the minimum circumscribed rectangular area of ​​the first text of each paragraph;

[0013] A probability calculation module, used for inputting the first image and the second image into a pre-trained manuscript preprocessing model to obtain the probability that each paragraph of the manuscript to be processed belongs to each text element and the probability that each paragraph belongs to each text format;

[0014] The preprocessing information generating module is used to generate the preprocessing information of the manuscript to be processed according to the probability that each paragraph belongs to each text element and the probability that each paragraph belongs to each text format.

[0015] In a third aspect, the present invention provides an electronic device, the electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the manuscript preprocessing method described in the first aspect of the present invention.

[0019] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a processor to implement the manuscript preprocessing method described in the first aspect of the present invention when executed.

[0020] The manuscript preprocessing method provided by the embodiment of the present invention converts the content of the manuscript to be processed into an image to obtain a first image, and the pixel value of the text area of ​​each paragraph of the content of the manuscript to be processed in the image is a specified pixel value, so that the text area and the non-text area are distinguished in the first image; the second image is intercepted from the first image, and the second image is an image corresponding to the minimum circumscribed rectangular area of ​​the first text of each paragraph. The first image and the second image are input into a pre-trained manuscript preprocessing model, and the probability of each paragraph of the manuscript to be processed belonging to each text element and the probability of belonging to each text format can be obtained. The preprocessing information of the manuscript to be processed is generated according to the probability of each paragraph belonging to each text element and the probability of belonging to each text format. Compared with manual review and identification, it can be faster, simpler and more accurate, and can effectively improve the efficiency of manuscript review, while facilitating the review of reviewers.

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a flowchart of a manuscript preprocessing method provided according to Embodiment 1 of the present invention;

[0024] Figure 2 is a schematic diagram of a first image provided according to Embodiment 1 of the present invention;

[0025] Figure 3 is a schematic diagram of a second image provided according to Embodiment 1 of the present invention;

[0026] Figure 4 is a schematic diagram of text comparison provided according to the first embodiment of the present invention;

[0027] Figure 5 is a flow chart of a manuscript preprocessing method provided according to Embodiment 2 of the present invention;

[0028] Figure 6 is a schematic diagram of marking each paragraph label according to the second embodiment of the present invention;

[0029] Figure 7 is a structural schematic diagram of a manuscript preprocessing device provided according to Embodiment 3 of the present invention;

[0030] Figure 8 It is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] Embodiment 1

[0033] Figure 1 This is a flowchart of a manuscript preprocessing method provided in the first embodiment of the present invention. This embodiment can be applied to the case of reviewing the manuscripts received by the submission platform for formal issues. The method can be executed by a manuscript preprocessing device, which can be implemented in the form of hardware and / or software, and can be configured in an electronic device. Figure 1 As shown, the manuscript preprocessing method includes:

[0034] S101, converting the content of the manuscript to be processed into an image to obtain a first image, wherein the pixel value of the text area of ​​each paragraph of the content of the manuscript to be processed in the image is a designated pixel value.

[0035] Manuscripts to be processed are original manuscripts received by the submission platform from contributors. The submission platform can be journals, newspapers, academic websites, etc. Given that the number of manuscripts received is quite different from the ratio of reviewers, reviewers must review not only the substance of the manuscripts, but also the form. In order to reduce the review tasks of reviewers and speed up the review process, when receiving manuscripts from contributors, in addition to the substance of the manuscripts, the form of the manuscripts can be reviewed in advance, for example, whether the text format and text elements of the manuscripts meet the requirements.

[0036] Since the formal issues of the manuscript to be processed are to be reviewed, the content of the manuscript to be processed can be converted into an image, and the formal issues of the manuscript to be processed can be analyzed based on the image features in the image. The first image can be obtained by first taking a screenshot of the content part of the manuscript to be processed in the reading mode, or by using text-to-image software to convert the content part of the manuscript to be processed into an image. After the content part of the manuscript to be processed is converted into an image, the text area in the image is identified, and the pixel value of the text area is set to a specified pixel value.

[0037] The process of converting the content of the manuscript to be processed into an image is a digital process. The pixel value is the value assigned by the computer when the content of the manuscript to be processed is digitized. It represents the average brightness information of a small square in the original manuscript to be processed, or the average reflection (transmission) density information of the small square. By assigning specific pixel values ​​to the text area of ​​each paragraph of the manuscript to be processed, the content of the manuscript to be processed can be quickly segmented, and the text part and non-text part in each paragraph can be distinguished, so as to facilitate the extraction and analysis of the image features of each paragraph.

[0038] In an example of the present embodiment, the content of the manuscript to be processed is converted into an image to obtain a first image, including: converting the content of the manuscript to be processed into an image to obtain an initial image; identifying multiple paragraphs of the content of the manuscript to be processed in the initial image; generating a circumscribed polygon for each paragraph; setting the pixel values ​​of the pixels within the circumscribed rectangle to a first pixel value, and setting the pixel values ​​of the pixels outside the circumscribed rectangle to a second pixel value to obtain the first image.

[0039] Generally speaking, there are obvious paragraph start and end marks in a paragraph. For example, paragraph start marks include first line indentation, line break, etc., and paragraph end marks include end-of-paragraph space, line spacing change, etc. Specifically, to identify multiple paragraphs of the content of the manuscript to be processed, the text area between two adjacent paragraph start and paragraph end marks can be regarded as a paragraph, or the text area between two adjacent paragraph start or paragraph end marks can be regarded as a paragraph.

[0040] For example, Figure 2 As shown, the circumscribed polygon is a closed area with the edge of the text area of ​​the paragraph as the reference line. For the first pixel value, R=G=B is set and RGB is a value between 0-255, and the part within the circumscribed rectangle in the first image appears in gray; for the second pixel value, R=G=B=255, that is, the part outside the circumscribed rectangle in the first image appears in white.

[0041] By distinguishing the text area from the non-text area of ​​each paragraph through different pixel values, the image features of the text part and the non-text part can be highlighted. In the subsequent steps, the manuscript preprocessing model can quickly identify and extract the image features of each paragraph.

[0042] S102: capturing a second image from the first image, where the second image is an image corresponding to the minimum circumscribed rectangular area of ​​the first text in each paragraph.

[0043] like Figure 3As shown in , for the first text in each paragraph, its minimum enclosing rectangle is the rectangle with the smallest area covering the text. The height and width of the minimum enclosing rectangle can represent the font size and font format of the text, or the font size and font format of the paragraph, such as Figure 4 The following is a comparison diagram of the height and width of texts with the same font size but different font formats. The texts with the same font size have the same height but different widths. Figure 4 As shown in the figure, when the text "Abstract" is in the same size 4 font, the width of the text is different in bold and non-bold, Songti and Heiti formats. Therefore, the minimum enclosing rectangle of the first text in the paragraph can be used to determine the font size and font format of the paragraph. Of course, since there are many cases where the fonts are very similar, it is difficult to distinguish different fonts based on the height and width of the text for texts of the same font size and different fonts, or the effect of distinguishing is poor. At this time, there are two optional methods, one is to obtain the probability that the text belongs to each font size and each font, and the other is to only obtain the probability that the text belongs to each font size. The specific method can be set according to the needs.

[0044] In an example of this embodiment, capturing the second image from the first image includes:

[0045] For the content area of ​​each paragraph in the first image, a text area of ​​the first text is identified from the text area, and the text area is intercepted to obtain a second image, and the pixel value of the second image is the third pixel value.

[0046] If there is a certain interval between adjacent texts, the texts in the text area can be distinguished, and the text area of ​​the first text can be easily identified to obtain the second image. In order to distinguish it from the text areas of other texts, the text area of ​​the first text is assigned a third pixel value corresponding to the second image, so as to facilitate the recognition of the manuscript preprocessing model. In addition, for some left-right structured characters, there is a certain interval even for one character, but generally speaking, the distance between the left and right parts of a left-right structured character is smaller than the distance between two adjacent characters. Therefore, when recognizing the first text alone, a preset interval can be set to avoid using the left part of a left-right structured character as the first text, thereby improving the accuracy of text recognition.

[0047] S103, inputting the first image and the second image into a pre-trained manuscript preprocessing model to obtain the probability that each paragraph of the manuscript to be processed belongs to each text element and the probability that each paragraph belongs to each text format.

[0048] Among them, text elements include title, abstract, keywords, main text, acknowledgments, references, etc., and text format includes font size, font format, text alignment, paragraph spacing, line spacing, etc. For each submission platform, its requirements for the text elements of the manuscript and the text format of each text element are relatively fixed, that is, the text elements and text formats correspond one to one. For example, when the text element is a title, the text format is required to be: Songti, size 2 font, bold, 2 times line spacing, and text is centered; and when the text element is the main text, the text format is required to be: Songti, size 4 font, 1.5 times line spacing, and text is aligned on both sides. Therefore, the image features of each paragraph of the manuscript to be processed can be used as the basis for text element classification and text format classification of each paragraph.

[0049] On the premise that the submission platform's requirements for text format, paragraph order, paragraph position relationship, number of spaces, etc. for each text element are known, the manuscript preprocessing model can convert the requirements of each text element into digital image features, and match the text elements with the image features of the text elements one by one. Similarly, based on the differences between texts in different text formats, the text formats can be matched with the image features of the text formats one by one.

[0050] The manuscript preprocessing model can not only extract the image features of the text elements of each paragraph in the manuscript to be processed from the first image, but also extract the image features of each paragraph such as the paragraph area, paragraph order, paragraph position relationship, text alignment, number of spaces, etc. from the first image, and can also extract the image features of the text format such as the height and width of the minimum circumscribed rectangle from the second image of each paragraph, and then obtain the probability of each paragraph in the manuscript to be processed belonging to each text element and the probability of belonging to each text format based on the above image features. Specifically, when the manuscript preprocessing model recognizes the image features of each paragraph based on the first image and the second image of the manuscript to be processed, the image features of each paragraph are compared with the image features of each text element and the image features of each text format to obtain the overlap between the image features and the image features of the text element and the image features of the text format. The higher the overlap, the greater the probability that the paragraph belongs to the text element or text format. Conversely, the lower the overlap, the smaller the probability that the paragraph belongs to the text element or text format.

[0051] For the image features such as paragraph area, paragraph order, paragraph position relationship, text alignment, number of spaces, etc., for example, Figure 2In the figure, for the paragraph "Abstract", the corresponding paragraph area is smaller than that of other paragraphs, and the paragraph order is 3rd. The previous and next paragraphs of this paragraph are the title and the abstract text respectively. The line spacing before and after the paragraph are both 2 times the line spacing. The text is centered and the number of spaces is Ns, where N is the number of characters that can be accommodated in each line when the font format and font size are determined, and s is the number of characters in the text in the image. For example, the number of characters occupied by the text "Abstract" is 2. Under the premise that other text formats are the same, if the "Abstract" is set to "Songti, Bold", the number of characters that can be accommodated in each line is 29, and the number of spaces is 27. When the "Abstract" is set to "Songti", the number of characters that can be accommodated in each line is 33, and the number of spaces is 31.

[0052] Of course, the image features of the text elements and text formats listed above are only examples. In fact, the image features of each text element and each text format are not limited to the parts listed above, but can also include page margins, etc., which can be set according to actual requirements.

[0053] S104, generating preprocessing information of the manuscript to be processed according to the probability that each paragraph belongs to each text element and the probability that each paragraph belongs to each text format.

[0054] According to the probability that each paragraph belongs to each text element and the probability that it belongs to each text format, the text element and text format to which each paragraph belongs can be determined, and this can be used to judge whether the text elements of the manuscript to be processed are complete, whether the text element positions are correct, and whether the text formats corresponding to the text elements of each paragraph are correct. If the text elements are complete, the text element positions are correct, and the text formats are correct, then it is determined that the manuscript to be processed can enter the substantive content review stage, that is, it can be sent to the reviewer. If the text elements are missing, the text elements are incorrectly positioned, or the text format is incorrect, the manuscript to be processed can be annotated according to the missing or incorrectly positioned text elements and the incorrect text formats, and then sent to the reviewer after annotating. Alternatively, the severity of the missing text elements, incorrect text element positions, or incorrect text formats of the manuscript can be evaluated. When the severity reaches the preset return condition, it will be returned to the submitter for supplement or modification. For example, when the abstract or references are missing, it can be returned to the submitter for supplement.

[0055] The manuscript preprocessing method provided by the embodiment of the present invention converts the content of the manuscript to be processed into an image to obtain a first image, and the pixel value of the text area of ​​each paragraph of the content of the manuscript to be processed in the image is a specified pixel value, so that the text area and the non-text area are distinguished in the first image, and the image features of each paragraph and the correlation features between each paragraph can also be displayed; the second image is intercepted from the first image, and the second image is the image corresponding to the minimum circumscribed rectangular area of ​​the first text of each paragraph. The first image and the second image are input into a pre-trained manuscript preprocessing model, and the probability of each paragraph of the manuscript to be processed belonging to each text element and the probability of belonging to each text format can be obtained. The preprocessing information of the manuscript to be processed is generated according to the probability of each paragraph belonging to each text element and the probability of belonging to each text format. Compared with manual review and identification, it can be faster, simpler and more accurate, and can effectively improve the efficiency of manuscript review, while facilitating the review of reviewers.

[0056] Embodiment 2

[0057] Figure 5 This is a flowchart of a manuscript preprocessing method provided in Embodiment 2 of the present invention. Embodiment 2 of the present invention is optimized based on Embodiment 1 above. Figure 5 As shown, the manuscript preprocessing method includes:

[0058] S201, converting the content of the manuscript to be processed into an image to obtain a first image, wherein the pixel value of the text area of ​​each paragraph of the content of the manuscript to be processed in the image is a specified pixel value.

[0059] S201 is the same as S101 in the first embodiment, and has the same functions and technical effects as S101, and will not be described here. For details, please refer to the relevant description part of the first embodiment.

[0060] S202: capturing a second image from the first image, where the second image is an image corresponding to the minimum circumscribed rectangular area of ​​the first text in each paragraph.

[0061] In this embodiment, the end position of each paragraph is identified in the first image, and the second image corresponding to each paragraph is superimposed at the end position to obtain a third image. In the third image, the beginning and end of each paragraph are the second image, and the text area between the second images (including the second image) is a paragraph. The manuscript preprocessing model can regard the text area between two identical second images (including the second image) as a paragraph, and the image feature of the second image as one of the image features of the paragraph. It should be noted here that the second image at the end of each paragraph in the third image is only used to identify and recognize each paragraph, and is not used as the content part of each paragraph.

[0062] When the manuscript preprocessing model requires the input image size to be of a preset size, the third image is scaled to the preset size to obtain a fourth image. Scaling refers to reducing or enlarging the third image to reach the preset size, and then the fourth image is input into the pre-trained manuscript preprocessing model to obtain the probability of each paragraph of the manuscript to be processed belonging to each text element and the probability of belonging to each text format.

[0063] S203: Input the first image and the second image into a pre-trained manuscript preprocessing model to obtain the probability that each paragraph of the manuscript to be processed belongs to each text element and the probability that each paragraph belongs to each text format.

[0064] Among them, the manuscript preprocessing model is trained in the following way:

[0065] Positive sample manuscripts and negative sample manuscripts are obtained, where the positive sample manuscripts are manuscripts with correct text elements and text formats, and the negative sample manuscripts are manuscripts with incorrect text elements and text formats. The contents of the positive sample manuscripts and the negative sample manuscripts are converted into images to obtain sample images. In the sample images, the regions corresponding to the paragraphs of the positive sample manuscripts and the negative sample manuscripts are annotated with text elements and text formats to obtain multiple text element label values ​​and text format label values ​​of the sample images, such as Figure 4 As shown, taking the positive sample manuscript as an example, the text elements and text formats are annotated in the areas corresponding to each paragraph of the positive sample manuscript. The negative sample manuscript can choose different text elements and manuscripts with incorrect text formats corresponding to the positive sample manuscript. For example, there is no abstract text after the abstract title, the text size is size 5, etc. The text element label value is the probability that each paragraph belongs to the annotated text element, and the text format label value is the probability that each paragraph belongs to the annotated text format. For the positive sample manuscript, its text element label value and text format label value are both 1, and for the negative sample manuscript, its text element label value and text format label value are both 0.

[0066] Initialize the manuscript preprocessing model, randomly extract sample images and input them into the manuscript preprocessing model to obtain multiple predicted text element label values ​​and multiple predicted text format label values, and use the predicted text element label values, predicted text format label values, text element label values, and text format label values ​​to calculate the loss rate. Among them, the predicted text element label value is the probability value of the text area of ​​each paragraph predicted by the manuscript preprocessing model belonging to each text element, and the predicted text format label value is the probability value of the text area of ​​each paragraph predicted by the manuscript preprocessing model belonging to each text format.

[0067] Determine whether the loss rate is less than a preset threshold. If so, stop training the manuscript preprocessing model to obtain a trained manuscript preprocessing model. If not, adjust the model parameters of the manuscript preprocessing model according to the loss rate, and return to the step of randomly extracting sample images and inputting them into the manuscript preprocessing model to obtain multiple predicted text element label values ​​and multiple text format label values. The preset threshold is used to determine whether the training effect of the manuscript preprocessing model meets the preset requirements. The smaller the loss rate, the better the training effect of the manuscript preprocessing model, and the higher the degree of consistency between the multiple predicted text element label values ​​and the multiple predicted text format label values ​​predicted by the manuscript preprocessing model and the corresponding text element label values ​​and the corresponding text format label values. The specific loss rate threshold can be set according to actual needs.

[0068] The loss rate is calculated by using the predicted text element label value, the predicted text format label value, the text element label value, and the text format label value, including:

[0069] The loss rate L is calculated by the following formula:

[0070]

[0071] Among them, a and b are text element weight and text format weight respectively, and w i is the weight of the i-th paragraph, Label1 i 、Label2 i are the predicted text element label value and text element label value of the i-th paragraph, respectively. Label3 i 、Label4 i are the predicted text format label value and text format label value of the ith paragraph respectively, and the weights of the 1-kth paragraphs and the pnth paragraph are greater than the weight of the kpth paragraph, and n is the preset number of manuscript paragraphs. Generally speaking, the preset number of paragraphs can be set to a larger value to take into account manuscript samples with different numbers of paragraphs.

[0072] The weight indicates the importance of each text element and text format, and also indicates the influence of each text element and text format in training the manuscript preprocessing model. Generally speaking, the manuscript has more body content and more corresponding paragraphs. In manuscript preprocessing, the focus is on reviewing the format of the manuscript, not the body content of the manuscript. In addition, the format of the body content is relatively loose and not necessarily completely unified. For example, some manuscripts contain tables, and some manuscripts are attached with experimental result graphs. Therefore, in order to facilitate the pre-examination of the manuscript, the weight of the body content of the manuscript can be weakened, that is, the text element weight of the paragraph where the body content is located can be reduced accordingly. For papers, the submission platform requires that there must be text elements such as title, author name, abstract, and abstract body, and the text format of the title must be "small four, bold, centered", then the text format weight of these paragraphs can be increased accordingly. The main body of the text is usually located in the middle paragraph of the manuscript and occupies a longer space, while each text element is usually located at the beginning or end of the manuscript. Therefore, the weights of the 1-kth and pnth paragraphs can be set greater than the weight of the kpth paragraph. The specific k and p values ​​can be determined according to the paragraph value of the main body of the text and the paragraph value of the text element.

[0073] By setting weights to adjust the influence of each paragraph on the training effect of the manuscript preprocessing model, the submission platform can set it according to the academic field, manuscript requirements, and the review habits of contract reviewers, or by regularly modifying the weights to regularly adjust the manuscript processing model so that the manuscript processing model meets the current manuscript preprocessing needs, thereby enhancing the practicality of the manuscript preprocessing method and being applicable to most manuscript processing platforms.

[0074] S204: For each paragraph, the text element with the highest probability is used as the text element of the paragraph.

[0075] The text element with the highest probability means that the paragraph is most likely to be the text element, and the text element with the highest probability can be determined as the text element of the paragraph.

[0076] S205: Determine the target text element of the information to be hidden from the text elements of the paragraph.

[0077] S206: Shield the text at a specified position in the paragraph corresponding to the target text element.

[0078] In order to prevent the reviewer's judgment and fairness from being affected by the identity and contact address of the contributor during the review process, after determining the text elements of each paragraph of the manuscript to be processed, the text involving the identity and address of the contributor in each paragraph can be shielded, for example, the text can be deleted, blacked out or added with a non-transparent watermark. For example, all the text in the paragraph corresponding to the text element (author's name) can be blacked out.

[0079] S207 , generating preprocessing information of the manuscript to be processed according to the probability that each paragraph belongs to each text element and the probability that each paragraph belongs to each text format.

[0080] After determining the text elements with the highest probability in each paragraph, the text element set of the manuscript to be processed can be obtained, and it is determined whether the text element set contains multiple preset specified text elements, which are the text elements required by the submission platform. If so, it is determined that the file structure of the manuscript to be processed is complete. If not, the text elements that do not exist in the file structure set are found from the multiple specified text elements as the missing text elements of the manuscript to be processed, and the text element missing information is generated. For example, when a text element (reference document) is missing, the missing reference document can be included in the text element missing information.

[0081] For each paragraph, a preset text format corresponding to the text element with the highest probability of the paragraph is obtained, the text format with the highest probability of the paragraph is obtained, and it is determined whether the text format with the highest probability is the preset text format. If so, it is determined that the text format of the paragraph is correct. If not, it is determined that the text format of the paragraph is wrong, and text format error information is generated. Similarly, when the text format of the referenced document is missing, the format error information may include the referenced document format error.

[0082] S208: Add the pre-processing information to the manuscript to be processed, and send the manuscript to be processed to the manual pre-examination terminal.

[0083] The manual pre-examination terminal is used to display the manuscripts to be processed and the pre-processing information in the manuscripts to be processed, so that the staff can further review and confirm the review results to ensure the accuracy of the manuscript pre-processing. The pre-processing information is added to the manuscripts to be processed, that is, the pre-processing information is marked at the corresponding position of the manuscripts to be processed, for example, "Title font size error" is marked in the title to facilitate the staff to review each error or missing content. After the staff completes the review, if the review is correct, they can click the corresponding confirmation button on the page of the manual pre-examination terminal to use the original pre-processing information as the target pre-processing information. If an error is found, the original pre-processing information is modified to obtain the target pre-processing information. The target pre-processing information is the information generated after the manual review terminal confirms the pre-processing information.

[0084] S209: Receive target preprocessing information sent by the manual pre-examination terminal.

[0085] S210: Send the target preprocessing information and the manuscript to be processed to the reviewer terminal.

[0086] After the manual pre-review terminal reviews and receives the target pre-processing information, the target pre-processing information and the manuscript to be processed can be sent to the reviewer terminal for the reviewer to review. Similarly, in order to facilitate the reviewer to review each error or missing content, the target pre-processing information can also be added to the corresponding position in the manuscript to be processed.

[0087] The manuscript preprocessing method provided in the embodiment of the present invention trains the manuscript preprocessing model through positive sample manuscripts, negative sample manuscripts and setting the weights of each paragraph in the manuscript, which can make the manuscript preprocessing model more practical and applicable to more submission platforms. In addition, after the manuscript preprocessing model identifies the text elements and text structures of each paragraph of the manuscript to be processed, it will also be sent for manual review to further ensure the accuracy of manuscript preprocessing, which can further effectively improve the efficiency of manuscript review and facilitate reviewers to review.

[0088] Embodiment 3

[0089] Figure 7 This is a schematic diagram of the structure of a manuscript preprocessing device provided in Embodiment 3 of the present invention. Figure 7 As shown, the manuscript preprocessing device comprises:

[0090] Content conversion module 701, used for converting the content of the manuscript to be processed into an image to obtain a first image, wherein the pixel value of the text area of ​​each paragraph of the content of the manuscript to be processed in the image is a specified pixel value;

[0091] An image capture module 702 is used to capture a second image from the first image, where the second image is an image corresponding to the minimum circumscribed rectangular area of ​​the first text of each paragraph;

[0092] The probability calculation module 703 is used to input the first image and the second image into a pre-trained manuscript pre-processing model to obtain the probability that each paragraph of the manuscript to be processed belongs to each text element and the probability that each paragraph belongs to each text format;

[0093] The preprocessing information generating module 704 is used to generate preprocessing information of the manuscript to be processed according to the probability that each paragraph belongs to each text element and the probability that each paragraph belongs to each text format.

[0094] Optionally, the probability calculation module 703 includes:

[0095] An end position identification submodule, used to identify the end position of each paragraph in the first image;

[0096] A second image superimposing submodule, used for superimposing the second image corresponding to each paragraph at the end position to obtain a third image;

[0097] A size scaling submodule, used for scaling the third image to a preset size to obtain a fourth image;

[0098] The probability calculation submodule is used to input the fourth image into a pre-trained manuscript preprocessing model to obtain the probability that each paragraph of the manuscript to be processed belongs to each text element and the probability that each paragraph belongs to each text format.

[0099] Optionally, the manuscript preprocessing model is trained with the following modules:

[0100] A manuscript sample acquisition module is used to acquire positive sample manuscripts and negative sample manuscripts. The positive sample manuscripts are manuscripts with correct text elements and text formats, and the negative sample manuscripts are manuscripts with incorrect text elements and text formats.

[0101] A sample image acquisition module is used to convert the contents of the positive sample manuscript and the negative sample manuscript into images to obtain sample images;

[0102] A label value acquisition module is used to annotate text elements and text formats in the areas corresponding to each paragraph of the positive sample manuscript and the negative sample manuscript in the sample image, and obtain multiple text element label values ​​and text format label values ​​of the sample image;

[0103] Model initialization module, used to initialize the manuscript preprocessing model;

[0104] A predicted label value acquisition module is used to randomly extract sample images and input them into the manuscript preprocessing model to obtain multiple predicted text element label values ​​and multiple predicted text format label values;

[0105] A loss rate calculation module, used to calculate the loss rate using the predicted text element label value, the predicted text format label value, the text element label value and the text format label value;

[0106] The loss rate judgment module is used to judge whether the loss rate is less than a preset threshold. If so, the content executed by the training completion module is executed; otherwise, the content executed by the parameter adjustment module is executed;

[0107] A training completion module is used to stop training the manuscript preprocessing model and obtain a trained manuscript preprocessing model;

[0108] The parameter adjustment module is used to adjust the model parameters of the manuscript preprocessing model according to the loss rate, and return the content executed by the predicted label value acquisition module.

[0109] Optionally, the loss rate calculation module includes:

[0110] The loss rate calculation submodule is used to calculate the loss rate L using the following formula:

[0111]

[0112] Among them, a and b are text element weight and text format weight respectively, and w i is the weight of the i-th paragraph, Label1 i 、Label2 i are the predicted text element label value and text element label value of the i-th paragraph, respectively. Label3 i 、Label4 i are the predicted text format label value and text format label value of the ith paragraph respectively. The weights of the 1-kth paragraph and the pnth paragraph are greater than the weight of the kpth paragraph. n is the preset number of manuscript paragraphs.

[0113] Optionally, the preprocessing information generating module 704 includes:

[0114] The text element determination submodule is used to determine the text element with the highest probability in each paragraph, and obtain the text element set of the manuscript to be processed; determine whether the text element set contains multiple preset specified text elements; if so, execute the content executed by the text element complete determination submodule, otherwise execute the content executed by the missing information generation submodule.

[0115] A text element completeness determination submodule is used to determine whether the text elements of the manuscript to be processed are complete;

[0116] A missing information generating submodule is used to find out the text elements that do not exist in the text element set from the multiple specified text elements as the missing text elements of the manuscript to be processed, and generate text element missing information;

[0117] A preset text format determination submodule is used to obtain, for each paragraph, a preset text format corresponding to a text element with the highest probability in the paragraph;

[0118] The maximum probability text format acquisition submodule is used to acquire the maximum probability text format of the paragraph;

[0119] The text format comparison submodule is used to determine whether the text format with the highest probability is the preset text format; if so, the content executed by the text format determination submodule is executed; if not, the content executed by the error information generation submodule is executed;

[0120] The text format determination submodule is used to determine whether the text format of the paragraph is correct;

[0121] The error message generation submodule is used to determine the text format error of the paragraph and generate text format error information.

[0122] Optionally, the manuscript preprocessing device further includes:

[0123] A manual pre-examination module is used to add pre-processing information to the manuscript to be processed and send the manuscript to be processed to a manual pre-examination terminal, which is used to display the manuscript to be processed and the pre-processing information;

[0124] The target preprocessing information receiving module is used to receive the target preprocessing information sent by the manual pre-examination terminal, and the target preprocessing information is the information generated after the manual pre-examination terminal performs a confirmation operation on the preprocessing information;

[0125] The processing information sending module is used to send the target pre-processing information and the manuscript to be processed to the reviewer terminal.

[0126] Optionally, the manuscript preprocessing device further includes:

[0127] A text element determination module, for determining, for each paragraph, the text element with the highest probability as the text element of the paragraph;

[0128] A target text element determination module is used to determine the target text element of the information to be hidden from the text elements of the paragraph;

[0129] The text masking module is used to mask the text at a specified position in the paragraph corresponding to the target text element.

[0130] The manuscript preprocessing device provided in the embodiment of the present invention can execute the manuscript preprocessing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0131] Embodiment 4

[0132] Figure 8 A schematic diagram of the structure of an electronic device 40 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0133] like Figure 8As shown, the electronic device 40 includes at least one processor 41, and a memory connected to the at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 to the random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the electronic device 40 can also be stored. The processor 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0134] A number of components in the electronic device 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0135] The processor 41 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 41 performs the various methods and processes described above, such as the manuscript preprocessing method.

[0136] In some embodiments, the manuscript preprocessing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the manuscript review task assignment method described above may be performed. Alternatively, in other embodiments, the processor 41 may be configured to execute the manuscript preprocessing method in any other appropriate manner (e.g., by means of firmware).

[0137] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0138] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0139] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0140] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0141] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0142] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0143] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0144] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A manuscript preprocessing method, characterized in that: include: Converting the content of the manuscript to be processed into an image to obtain a first image, wherein the pixel value of the text area of ​​each paragraph of the content of the manuscript to be processed in the image is a specified pixel value; intercepting a second image from the first image, where the second image is an image corresponding to the minimum circumscribed rectangular area of ​​the first text of each paragraph; Inputting the first image and the second image into a pre-trained manuscript preprocessing model to obtain the probability that each paragraph of the manuscript to be processed belongs to each text element and the probability that each paragraph belongs to each text format; Generating preprocessing information of the manuscript to be processed according to the probability that each paragraph belongs to each text element and the probability that each paragraph belongs to each text format; The step of generating the preprocessing information of the manuscript to be processed according to the probability that each paragraph belongs to each text element and the probability that each paragraph belongs to each text format includes: Determine the text element with the highest probability for each of the paragraphs to obtain a text element set of the manuscript to be processed; determine whether the text element set includes a plurality of preset specified text elements; If so, determining that the text elements of the manuscript to be processed are complete; If not, searching for text elements that do not exist in the text element set from a plurality of designated text elements as missing text elements of the manuscript to be processed, and generating text element missing information; For each of the paragraphs, obtaining a preset text format corresponding to a text element with the highest probability in the paragraph; Obtaining the text format with the highest probability for the paragraph; Determining whether the text format with the highest probability is the preset text format; If so, it is determined that the text format of the paragraph is correct; If not, it is determined that the text format of the paragraph is wrong, and text format error information is generated.

2. The manuscript preprocessing method according to claim 1, characterized in that: The step of inputting the first image and the second image into a pre-trained manuscript preprocessing model to obtain the probability that each paragraph of the manuscript to be processed belongs to each text element and the probability that each paragraph belongs to each text format includes: identifying the end positions of each paragraph in the first image; superimposing the second images corresponding to the respective paragraphs at the end position to obtain a third image; Scaling the third image to a preset size to obtain a fourth image; The fourth image is input into a pre-trained manuscript preprocessing model to obtain the probability that each paragraph of the manuscript to be processed belongs to each text element and the probability that each paragraph belongs to each text format.

3. The manuscript preprocessing method according to any one of claims 1 to 2, characterized in that: The manuscript preprocessing model is trained in the following way: Obtaining positive sample manuscripts and negative sample manuscripts, wherein the positive sample manuscripts are manuscripts with correct text elements and text formats, and the negative sample manuscripts are manuscripts with incorrect text elements and text formats; Convert the contents of the positive sample manuscript and the negative sample manuscript into images to obtain sample images; In the sample image, marking text elements and text formats in the regions corresponding to the paragraphs of the positive sample manuscript and the negative sample manuscript to obtain a plurality of text element label values ​​and text format label values ​​of the sample image; Initialize the manuscript preprocessing model; Randomly extracting the sample image and inputting it into the manuscript preprocessing model to obtain a plurality of predicted text element label values ​​and a plurality of predicted text format label values; Calculating a loss rate using the predicted text element label value, the predicted text format label value, the text element label value, and the text format label value; Determining whether the loss rate is less than a preset threshold; If yes, stop training the manuscript preprocessing model to obtain a trained manuscript preprocessing model; If not, the model parameters of the manuscript preprocessing model are adjusted according to the loss rate, and the process returns to the step of randomly extracting the sample image and inputting it into the manuscript preprocessing model to obtain a plurality of predicted text element label values ​​and a plurality of text format label values.

4. The manuscript preprocessing method according to claim 3, characterized in that: The calculating of the loss rate by using the predicted text element label value, the predicted text format label value, the text element label value and the text format label value comprises: The loss rate L is calculated by the following formula: Among them, a and b are text element weight and text format weight respectively, and w i is the weight of the i-th paragraph, Label1 i 、Label2 i are the predicted text element label value and text element label value of the i-th paragraph, respectively. Label3 i 、Label4 i are the predicted text format label value and text format label value of the ith paragraph respectively, and the weights of the 1-kth paragraph and the pnth paragraph are greater than the weight of the kpth paragraph, and n is the preset number of manuscript paragraphs.

5. The manuscript preprocessing method according to claim 1, characterized in that: After generating the preprocessing information of the manuscript to be processed according to the probability that each paragraph belongs to each text element and the probability that each paragraph belongs to each text format, the method further includes: Adding the preprocessing information to the manuscript to be processed, and sending the manuscript to be processed to a manual pre-examination terminal, wherein the manual pre-examination terminal is used to display the manuscript to be processed and the preprocessing information; Receiving target preprocessing information sent by the manual pre-examination terminal, the target pre-processing information is information generated after the manual pre-examination terminal performs a confirmation operation on the pre-processing information; The target preprocessing information and the manuscript to be processed are sent to the reviewer terminal.

6. The manuscript preprocessing method according to claim 1, characterized in that: After inputting the first image and the second image into a pre-trained manuscript preprocessing model to obtain the probability that each paragraph of the manuscript to be processed belongs to each text element and the probability that each paragraph belongs to each text format, the method further includes: For each of the paragraphs, taking the text element with the highest probability as the text element of the paragraph; Determining a target text element of information to be hidden from the text elements of the paragraph; The text at a specified position in the paragraph corresponding to the target text element is masked.

7. A manuscript preprocessing device, characterized in that: include: A content conversion module, used for converting the content of the manuscript to be processed into an image to obtain a first image, wherein the pixel value of the text area of ​​each paragraph of the content of the manuscript to be processed in the image is a specified pixel value; An image capture module, used to capture a second image from the first image, where the second image is an image corresponding to the minimum circumscribed rectangular area of ​​the first text of each paragraph; A probability calculation module, used for inputting the first image and the second image into a pre-trained manuscript preprocessing model to obtain the probability that each paragraph of the manuscript to be processed belongs to each text element and the probability that each paragraph belongs to each text format; A preprocessing information generating module, used for generating preprocessing information of the manuscript to be processed according to the probability that each paragraph belongs to each text element and the probability that each paragraph belongs to each text format; The preprocessing information generating module comprises: The text element determination submodule is used to determine the text element with the highest probability in each paragraph, and obtain the text element set of the manuscript to be processed; determine whether the text element set contains multiple preset specified text elements; if so, execute the content executed by the text element complete determination submodule, if not, execute the content executed by the missing information generation submodule; A text element completeness determination submodule is used to determine whether the text elements of the manuscript to be processed are complete; A missing information generating submodule is used to find out the text elements that do not exist in the text element set from the multiple specified text elements as the missing text elements of the manuscript to be processed, and generate text element missing information; A preset text format determination submodule is used to obtain, for each paragraph, a preset text format corresponding to a text element with the highest probability in the paragraph; The maximum probability text format acquisition submodule is used to acquire the maximum probability text format of the paragraph; The text format comparison submodule is used to determine whether the text format with the highest probability is the preset text format; if so, the content executed by the text format determination submodule is executed; if not, the content executed by the error information generation submodule is executed; The text format determination submodule is used to determine whether the text format of the paragraph is correct; The error message generation submodule is used to determine the text format error of the paragraph and generate text format error information.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the manuscript preprocessing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the manuscript preprocessing method according to any one of claims 1 to 6 when executed.

Images