Work screening method and device based on mailbox, computer equipment and readable storage medium

Abstract

The invention discloses a work screening method and device based on a mailbox, computer equipment and a readable storage medium, and the method comprises the steps: obtaining an original mail from a target mailbox server, filtering the original mail to obtain an undetermined mail work, carrying out the compliance detection based on the work type, carrying out the multi-dimensional scoring after the compliance is passed, and obtaining a scoring result, and finally, determining a final screening result according to the size of the result after the scoring result is rechecked by the user, thereby realizing efficient and accurate work screening.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a mailbox-based work screening method, device, computer equipment and readable storage medium. Background Art

[0002] In today's digital age, it is increasingly common to receive submissions of various works (such as literary works, artistic works, etc.) through email. However, faced with a large number of submission emails, how to efficiently and accurately screen out works that meet the requirements has become a challenge. Traditional manual screening methods are inefficient and prone to subjective errors. With the development of information technology, a screening method with a high degree of automation that can comprehensively evaluate works based on the type of work is needed. Summary of the invention

[0003] The object of the present invention is to provide a method, device, computer equipment and readable storage medium for selecting works based on a mailbox.

[0004] In a first aspect, an embodiment of the present invention provides a method for selecting works based on a mailbox, comprising:

[0005] Get the original email from the target mailbox server and filter it to get the pending email works;

[0006] Performing work compliance detection based on the work type of the pending email work;

[0007] In the case where the work passes the compliance detection, multi-dimensional scoring is performed on the pending email work to obtain a multi-dimensional scoring result of the pending email work;

[0008] In response to a user review approval indication for the multi-dimensional scoring result, a final screening result of the pending email works is obtained according to the magnitude of the multi-dimensional scoring result.

[0009] In a possible implementation, obtaining the original email from the target mailbox server and filtering to obtain the pending email works includes:

[0010] Get the IMAP server address and port number of the target mailbox server;

[0011] Establishing a connection with the target mailbox server according to the IMAP server address and the port number in combination with the login credentials;

[0012] Download the original email in the target mailbox server according to the preset submission time range, wherein the original email includes the original email body and the original email attachment;

[0013] The original email body and the original email attachment are filtered according to preset filtering rules, and when the filtering result is characterized as meeting the filtering rules, the original email attachment is used as the pending email work.

[0014] In a possible implementation manner, the work compliance detection based on the work type of the pending email work includes:

[0015] Determining a detection standard for the pending email work based on the work type of the pending email work;

[0016] Based on the detection criteria, the pending email work is subjected to sensitive information detection, duplicate submission detection and subject consistency detection.

[0017] In a possible implementation, performing sensitive information detection on the pending email work includes:

[0018] In the case where the pending email work is a text work, preprocessing and sentiment analysis are performed on the text work, and sensitive information detection is performed in combination with a preset sensitive word library and a preset sensitive word detection model;

[0019] If the pending email work is an image work, the image work is segmented and sensitive information detection is performed in combination with a preset scene recognition model; if the image work includes text content, text is extracted from the image work, and the text content is preprocessed and sentiment analysis is performed, and sensitive information detection is performed in combination with a preset sensitive word library and a preset sensitive word detection model;

[0020] In the case where the pending email work is an audio work, the video sound is converted into audio text using ASR, and the audio text is preprocessed and sentiment analysis is performed, and sensitive information detection is performed in combination with a preset sensitive word library and a preset sensitive word detection model;

[0021] In the case where the pending email work is a video work, ASR is used to convert the video sound into audio text, and the audio text is preprocessed and subjected to sentiment analysis, and sensitive information detection is performed in combination with a preset sensitive word library and a preset sensitive word detection model; the video work is extracted at preset time intervals to obtain multiple video images, and the video images are segmented, and sensitive information detection is performed in combination with a preset scene recognition model; in the case where the video image includes text content, text is extracted from the video image, and the text content is preprocessed and subjected to sentiment analysis, and sensitive information detection is performed in combination with a preset sensitive word library and a preset sensitive word detection model.

[0022] In a possible implementation, performing duplicate submission detection on the pending email work includes:

[0023] In the case where the pending email work is a text work, the text work is segmented to obtain a plurality of vocabulary units; stop words in the plurality of vocabulary units are removed, and the plurality of vocabulary units are converted into a plurality of text vectors; text similarity comparison and semantic similarity comparison are performed based on the plurality of text vectors and the archived text vectors in the work database, and the duplicate submission detection is performed;

[0024] In the case that the pending email work is an image work, the SURF algorithm is used to extract the key points and descriptors of the image work; deep feature extraction is performed based on the key points and descriptors to obtain the deep features of the image work; FLANN and Brute-Force algorithms are used to perform feature matching similarity comparison in combination with the archived image works in the work database to perform the duplicate submission detection.

[0025] In a possible implementation, performing a subject consistency check on the pending email work includes:

[0026] Extracting keywords and phrases from the pending email works;

[0027] Use the pre-trained LDA topic model to determine the current topic distribution;

[0028] Performing grammatical analysis on the keywords and phrases to obtain deep features of the pending email work;

[0029] The topic consistency detection is achieved by performing a similarity comparison based on the current topic distribution and the deep features.

[0030] In a possible implementation manner, performing multi-dimensional scoring on the pending email work to obtain the multi-dimensional scoring result of the pending email work includes:

[0031] In the case where the pending email work is a written work, multi-dimensional scoring is performed based on the BERT model using part-of-speech tagging, syntactic analysis, and sentiment analysis as scoring indicators to obtain a multi-dimensional scoring result of the written work;

[0032] In the case where the pending email work is an image work or a video work, a multi-dimensional scoring is performed based on the LSTM and Transformer models using the painting style, painting skills and artistic elements as scoring indicators to obtain a multi-dimensional scoring result of the image work or the video work;

[0033] When the pending email work is a calligraphy work, a multi-dimensional scoring is performed based on the area difference, aspect ratio difference, distance difference and orientation difference between the calligraphy work and the standard characters to obtain a multi-dimensional scoring result of the calligraphy work.

[0034] In a second aspect, an embodiment of the present invention provides a device for selecting works based on a mailbox, comprising:

[0035] The acquisition module is used to obtain the original email from the target mailbox server and filter it to obtain the pending email works;

[0036] A screening module is used to perform a compliance check on the pending email work based on the work type of the pending email work; if the work passes the compliance check, multi-dimensional scoring is performed on the pending email work to obtain the multi-dimensional scoring result of the pending email work; in response to a user review pass indication for the multi-dimensional scoring result, a final screening result of the pending email work is obtained according to the size of the multi-dimensional scoring result.

[0037] In a third aspect, an embodiment of the present invention provides a computer device, comprising a processor and a non-volatile memory storing computer instructions, wherein when the computer instructions are executed by the processor, the computer device executes the method described in the first aspect.

[0038] In a fourth aspect, an embodiment of the present invention provides a readable storage medium, wherein the readable storage medium includes a computer program, and when the computer program is executed, the computer device where the readable storage medium is located is controlled to execute the method described in the first aspect.

[0039] Compared with the prior art, the beneficial effects provided by the present invention include: adopting a mailbox-based work screening method, device, computer equipment and readable storage medium disclosed in the present invention, obtaining the original email from the target mailbox server and filtering to obtain the pending email works, and then performing compliance detection based on the work type, and performing multi-dimensional scoring to obtain the scoring result after the compliance is passed, and finally after the user reviews the scoring result and passes it, determining the final screening result according to the result size, thereby achieving efficient and accurate work screening. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can also be obtained based on these drawings without creative work.

[0041] Figure 1 A schematic diagram of the steps of the mailbox-based works screening method provided by an embodiment of the present invention;

[0042] Figure 2 A schematic block diagram of the structure of a mailbox-based works screening device provided by an embodiment of the present invention;

[0043] Figure 3 A schematic block diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0045] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings.

[0046] In order to solve the technical problems in the aforementioned background technology, Figure 1 This is a flow chart of a mailbox-based works screening method provided in an embodiment of the present disclosure. The mailbox-based works screening method is introduced in detail below.

[0047] Step S201, obtaining the original email from the target mailbox server, and filtering to obtain the pending email works;

[0048] Step S202, performing work compliance detection based on the work type of the pending email work;

[0049] Step S203, when the work passes the compliance detection, multi-dimensional scoring is performed on the pending email work to obtain the multi-dimensional scoring result of the pending email work;

[0050] Step S204, in response to the user's review approval indication for the multi-dimensional scoring result, obtaining the final screening result of the pending email works according to the magnitude of the multi-dimensional scoring result.

[0051] In the embodiment of the present invention, for example, it is assumed that the server wants to screen works for a specific submission mailbox, which is set up by an art competition organization. The mailbox uses the IMAP (Internet Message Access Protocol) protocol. The server knows in advance that the IMAP server address corresponding to this submission mailbox is "imap.example.com" and the port number is 143 (this is a common non-encrypted IMAP port number. If it is an encrypted connection, the port number may be 993). This address and port number are like the door address and entrance number leading to the submission mailbox. The server can only find the location of the target mailbox server and try to establish a connection through this information.

[0052] After obtaining the IMAP server address and port number, the server still needs login credentials to establish a connection. Assume that the login account of the art competition submission mailbox is "contest@example.com" and the password is "Contest1234". The server uses these login credentials and sends a connection request to the target mailbox server in accordance with the provisions of the IMAP protocol. The server and the target mailbox server are like two people confirming their identities with each other. The target mailbox server will check whether the received account and password are correct. If they are correct, the server is allowed to establish a connection, just like opening the door to the submission mailbox, and the server can enter to view the email content inside.

[0053] The art competition stipulates that the submission period is from January 1, 2023 to March 31, 2023. After successfully connecting to the target mailbox server, the server will filter the emails to be downloaded according to this preset submission time range. The server will traverse each email in the mailbox and check the time when the email was received. For example, if an email is received on February 15, 2023, this time is within the preset submission time range, the server will download the original email body and original email attachments of this email. The original email body may contain some text information such as the author's description of the work and creative ideas, while the original email attachment may be the real entry, such as an image file of a painting, a document file of a literary work, etc.

[0054] Assume that the preset filtering rules are: the body of the email cannot contain malicious advertising links, and the size of the attachment cannot exceed 10MB. After downloading the original email, the server starts filtering it. For the body of the email, the server will use a special link detection algorithm to check whether there is a malicious advertising link. If a suspicious link is found in the body of an email, and it is compared with the malicious link database and found that this link is a malicious advertising link, then this email does not meet the filtering rules. For attachments, the server will check their file size. If there is an attachment with a size of 12MB, which exceeds the limit of 10MB, this email also does not meet the filtering rules. Only when there is no malicious advertising link in the body of the email and the size of the attachment is within 10MB, the email is considered to meet the filtering rules, and the original email attachment is treated as a pending email work. For example, there is an email whose body is just a normal work description without malicious links, and the attachment is an 8MB painting image file, then this painting image file will be treated as a pending email work.

[0055] After receiving the pending email work, the server must first determine its type. Assuming that the pending email work is an image file, the detection standards for image works may include: it cannot contain sensitive scenes such as pornography, violence, and terror; it cannot be highly similar to existing entries (repeated submission detection); the content of the work must be consistent with the theme of the competition, etc. If the pending email work is a text file, the detection standards may include: it cannot contain sensitive words such as insults and discrimination; it cannot be plagiarized from existing works (repeated submission detection); the topic of the article must be related to the theme of the competition, etc.

[0056] Suppose this text work is a short story. The server first preprocesses the short story, such as converting all letters to lowercase letters and removing punctuation marks. Then, sentiment analysis is performed using a pre-trained sentiment analysis model, such as the TextBlob sentiment analysis model. If the sentiment analysis results show that the overall sentiment of the novel is positive. Next, sensitive information detection is performed by combining the preset sensitive word library and the preset sensitive word detection model. The preset sensitive word library contains words such as "discriminatory words" and "abusive words". Suppose there is a sentence in the novel "He is a fool". Although this is not a very serious abusive word, it may be regarded as an expression that does not meet the requirements within the scope of civilized language required by the competition. If such words are found after detection, it is determined that this text work contains sensitive information.

[0057] Suppose this image work is a landscape painting. The server segments this landscape painting and uses an image segmentation algorithm, such as the U-Net algorithm based on deep learning, to segment the image into different areas such as the sky, mountains, and rivers. Then, sensitive information detection is performed in combination with a preset scene recognition model. This preset scene recognition model is trained with a large amount of data and can identify scenes such as pornography and violence. If these bad scenes are not recognized in the image, but the image contains some text content, such as a sign with some text in the picture. The server will extract text from the image work using OCR (optical character recognition) technology, such as Tesseract-OCR. After extracting the text, the text content is preprocessed, such as removing special characters, and then sentiment analysis is performed and detection is performed in combination with the sensitive word library and sensitive word detection model, just like the detection of text works.

[0058] Suppose this audio work is an audio of a poem recitation. The server uses ASR (Automatic Speech Recognition) technology, such as Baidu's speech recognition API, to convert the video sound into audio text. Suppose the converted audio text is "In that faraway place, there is beautiful scenery...". Then, preprocess this audio text, such as removing some meaningless modal particles and other operations, and then conduct sentiment analysis using a suitable sentiment analysis algorithm. At the same time, combine the preset sensitive word library and the preset sensitive word detection model to detect sensitive information. If no sensitive information is found in the audio text, it means that this audio work has passed the sensitive information detection process.

[0059] If the pending email work is a video work

[0060] Suppose this video work is a creative short video. The server first uses ASR to convert the video sound into audio text, then preprocesses and conducts sentiment analysis on the audio text, combines the preset sensitive word library and the preset sensitive word detection model to detect sensitive information, and the operation process is similar to that of the audio work. At the same time, extract multiple video images from this video work at preset time intervals (such as every 5 seconds). For example, for a 60-second video, extracting at 5-second intervals will result in 12 video images. Segment these video images using an image segmentation algorithm, such as the U-Net algorithm mentioned above, and then combine the preset scene recognition model to detect sensitive information. If there is text content in the video image, such as subtitles in the video, extract the text from the video image using OCR technology, and then preprocess, conduct sentiment analysis on the text content, and combine the sensitive word library and the sensitive word detection model to detect.

[0061] Suppose this text work is a scientific and technological paper. The server performs word segmentation on this paper using tools such as Jieba Segmentation to obtain multiple lexical units, such as "artificial intelligence", "algorithm optimization", etc. Then remove the stop words in these lexical units, such as "de" (of), "shi" (is), etc., and convert the remaining lexical units into multiple text vectors. Models such as Word2Vec can be used to convert words into vectors. Then, compare the text similarity and semantic similarity between these text vectors and the archived text vectors in the work database. Suppose there is an archived paper in the work database. After calculation, the text similarity between these two papers reaches 80%, and the semantic similarity is also very high, then it may be determined that this text work has the suspicion of duplicate submission.

[0062] If the pending email work is an image work

[0063] Suppose this image work is a portrait. The server uses the SURF (Speeded Up Robust Features) algorithm to extract the key points and descriptors of the portrait, which can characterize the features of the image. Then, deep feature extraction is performed based on these key points and descriptors to obtain the deep features of the image work. Then, the FLANN (Fast Approximate Nearest Neighbor Search Library) and Brute-Force algorithms are used to compare the feature matching similarity in combination with the archived image works in the work database. If a work with a feature matching similarity of more than 90% with the current portrait is found in the archived image works, it may be determined that the portrait has been submitted repeatedly.

[0064] Suppose the pending email work is an essay on the theme of environmental protection. The server first extracts keywords and phrases from the essay, and uses natural language processing technology to extract keywords and phrases such as "environmental protection" and "energy conservation and emission reduction". Then, the pre-trained LDA (latent Dirichlet allocation) topic model is used to determine the current topic distribution. This topic model can analyze the distribution of these keywords and phrases in the entire article, thereby determining the topic structure of the article. Then, these keywords and phrases are subjected to grammatical analysis to obtain the deep features of the essay, such as sentence structure, part of speech and other information. Finally, a similarity comparison is performed based on the current topic distribution and deep features, and compared with the environmental protection theme set by the competition. If it is found that the topic structure, keywords and phrases of the article are highly relevant to the environmental protection theme, it is determined that the theme of this work is consistent.

[0065] Assume that the pending email work is a poem. Based on the BERT (Bidirectional Encoder Representations from Transformers) model, part-of-speech tagging, syntactic analysis, and sentiment analysis are used as scoring indicators for multi-dimensional scoring.

[0066] Part-of-speech tagging: If the poem correctly uses various parts of speech, such as nouns, verbs, adjectives, etc., and uses them creatively and accurately, such as using some uncommon but appropriate adjectives to describe the scenery, the score for this part will be higher.

[0067] In terms of syntactic analysis: If the sentence structure of the poem is reasonable and the rhythm is harmonious, for example, the parallelism between the verses is neat and the long and short sentences are reasonably matched, this will get a good score in the syntactic analysis scoring indicator.

[0068] Emotional analysis: If the emotions expressed in the poem are sincere and profound and can resonate with readers, such as expressing a passionate yearning for love or a deep longing for home, the poem will have a higher score in the emotional analysis indicator. Combining the scores of these three aspects, we can get the multi-dimensional score of the poem.

[0069] Assume that the work to be determined in the email is an oil painting. Based on the LSTM (Long Short-Term Memory) and Transformer models, the painting style, painting skills and artistic elements are used as scoring indicators for multi-dimensional scoring.

[0070] Painting style: If the painting is in the Impressionist style and embodies the blurred and colorful characteristics of Impressionism, such as bold colors and loose brushstrokes, it will get a higher score in the painting style indicator.

[0071] In terms of painting skills: If the color transition in the oil painting is natural, and the depiction of the characters or scenery is delicate and realistic, for example, the facial expressions of the characters are lifelike, this shows that the painting skills are sophisticated and will get a good score on this indicator.

[0072] Artistic elements: If the oil painting contains rich artistic elements, such as unique composition, clever use of light and shadow, etc., it will get a higher score in the indicator of artistic elements. Combining the scores of these three aspects, we can get the multi-dimensional score result of this oil painting.

[0073] Assume that the email work to be determined is a running script calligraphy work. Multi-dimensional scoring is performed based on the area difference, aspect ratio difference, distance difference, and orientation difference between the calligraphy work and the standard character.

[0074] In terms of area difference: if the size of the characters in the calligraphy work is not much different from the size of the standard characters, for example, the area of ​​the standard characters is 10 square centimeters, and the area of ​​the characters in the calligraphy work is between 8-12 square centimeters, then it will get a better score on the area difference indicator.

[0075] In terms of the difference in aspect ratio: if the aspect ratio of the characters in the calligraphy work is close to that of the standard characters, for example, the aspect ratio of the standard characters is 1:1.2, and the aspect ratio of the characters in the calligraphy work is between 1:1-1:1.4, then a better score will be obtained on this indicator.

[0076] In terms of distance difference: if the spacing between characters and lines in a calligraphy work is close to the standard calligraphy layout spacing, a better score will be obtained on the distance difference indicator.

[0077] Aspects of orientation difference: If the writing orientation of the characters in the calligraphy work is not obviously skewed, the orientation difference will get a good score. Combining the scores of these four aspects, we get the multi-dimensional score result of this running script calligraphy work.

[0078] Assume that the server has obtained the multi-dimensional scoring results of multiple entries (pending email entries), for example, the multi-dimensional scoring results of 10 essays have been calculated. These results are displayed on a management interface, and the administrator (user) reviews these scoring results. The administrator will carefully check the scoring basis of each work, for example, for an essay, check the specific scores in terms of part-of-speech tagging, syntactic analysis, and sentiment analysis. If the administrator has no objection to these scoring results, he will give the user a review and approval instruction.

[0079] After receiving this instruction, the server sorts these prose works according to the size of the multi-dimensional scoring results. Assume that one of the prose works has the highest comprehensive score in the three indicators of part-of-speech tagging, syntactic analysis and sentiment analysis, and ranks first. Then this prose will become the first work in the final screening results, and the other works will be ranked in order according to their scores. In this way, the final screening results of the pending email works are obtained, which can be used to determine the winning works, outstanding works, etc. of the competition.

[0080] In the embodiment of the present invention, the process of obtaining the original email from the target mailbox server and filtering to obtain the pending email works may be implemented through the following example.

[0081] Get the IMAP server address and port number of the target mailbox server;

[0082] Establishing a connection with the target mailbox server according to the IMAP server address and the port number in combination with the login credentials;

[0083] Download the original email in the target mailbox server according to the preset submission time range, wherein the original email includes the original email body and the original email attachment;

[0084] The original email body and the original email attachment are filtered according to preset filtering rules, and when the filtering result is characterized as meeting the filtering rules, the original email attachment is used as the pending email work.

[0085] In the embodiment of the present invention, for example, in a large-scale art collection activity, the server is responsible for screening the works in the submission mailbox. The submission mailbox is set by the event organizer on a specific mail server, and the IMAP protocol is used to enable the server to obtain the mail content.

[0086] Assume that this art collection event is global. In order to facilitate submissions from contestants from different regions, the organizer has selected a well-known email service provider. The email service provider has assigned a specific IMAP server address "imap.artcontestmail.com" to the submission mailbox, and the port number is 143 (a common port number for non-encrypted connections). The server needs to accurately obtain this address and port number information at the initial stage of the entire work screening process. This information is like part of a key and is an important basis for establishing a connection with the target email server.

[0087] After obtaining the IMAP server address "imap.artcontestmail.com" and port number 143, the server also needs login credentials to establish a connection. The event organizer has created a special login account "contestserver@artcontestmail.com" and the corresponding password "ServerContestPass123" for the server.

[0088] The server starts to establish a connection according to the specifications of the IMAP protocol. It first sends a connection request to the target mailbox server, which contains information such as the IMAP server address, port number, login account and password. After receiving this request, the target mailbox server will verify the account and password. This process is similar to the access control system checking the identity of the visitor. If the account and password match correctly, the target mailbox server will allow the server to establish a connection. At this point, a data transmission channel is established between the server and the target mailbox server, just like opening the door to the submission mailbox, preparing for the subsequent acquisition of the email content.

[0089] In this art collection activity, the organizer stipulated that the submission time range is from 00:00:00 on May 1, 2023 to 23:59:59 on August 31, 2023. After the server successfully establishes a connection with the target mailbox server, it starts to download the original email according to this preset submission time range.

[0090] The server traverses each email in the target mailbox. For each email, it first checks the time when the email was received. For example, the receiving timestamp of an email is 2023-06-15-10:30:00, which is within the preset submission time range, so this email meets the download conditions. The original email contains two important parts, namely the original email body and the original email attachment. The original email body may contain text information such as the contestant's explanation of the creation concept of the work and the introduction of the work. For example, in the body of an original email submission, the contestant wrote: "I was inspired to create this painting from the tranquility and harmony of nature, and I hope to convey this beauty through colors and lines." The original email attachment is the real entry, which may be various types of files, such as image files (.jpg, .png, etc.), document files (.docx, .pdf, etc.), etc. If it is a painting submission, the attachment may be a beautiful painting stored in .jpg format; if it is a literary work submission, the attachment may be a .docx document containing novels or poems.

[0091] In this art collection activity, a series of strict filtering rules are preset.

[0092] Filtering of the original email body:

[0093] Rule 1: Do not include malicious links:

[0094] The server scans the body of every email that meets the submission time range. It uses a special network link analysis tool to detect whether there are malicious links. This tool parses each link in the email body and compares it with the global malicious link database. For example, if the body of an email mentions "Click here to see more examples of works: http: / / maliciouslink.example.com", the server will find that this link exists in the malicious link database through comparison, and this email does not meet the requirements.

[0095] Rule 2: Limit the use of specific keywords:

[0096] The organizer of the event stipulates that the body of the email cannot contain certain specific keywords related to bad content, such as discriminatory words, words related to banned drugs, etc. The server will perform a full-text search on the body of the email. Suppose the discriminatory word "inferior race" appears in the body of an email. After the server detects this keyword, it determines that the email does not meet the filtering rules.

[0097] Filtering of original email attachments:

[0098] Rule 1: Limit attachment type and size:

[0099] The activity stipulates that only certain types of attachments are accepted, such as only .jpg and .png image files for paintings, and only .docx and .pdf document files for literary works. The server will check the file format of the attachment. If an attachment is in .exe (executable file format), this obviously does not meet the requirements. At the same time, there are also restrictions on the size of attachments, such as the maximum cannot exceed 20MB. If one of the attachments is a 30MB image file, the server will determine that the email does not meet the filtering rules.

[0100] Rule 2: Preliminary inspection of attachment content (for specific types of attachments):

[0101] For image attachments, if it is a painting entry, the server will do a preliminary image integrity check. It will check if the image is damaged and whether it can be opened and displayed normally. If an image attachment is damaged due to problems during transmission and the image content cannot be displayed normally, then this email will not meet the requirements.

[0102] Only when the original email body and the original email attachment pass all the preset filtering rules, the server will treat the original email attachment as a pending email work. For example, if there is an email whose body does not contain malicious links and prohibited keywords, and the attachment is a 15MB .jpg format painting image file, and the image can be displayed normally, then this painting image file will be treated as a pending email work by the server, waiting for further work compliance detection and other operations.

[0103] In the embodiment of the present invention, the work compliance detection based on the work type of the pending email work can be implemented through the following examples.

[0104] Determining a detection standard for the pending email work based on the work type of the pending email work;

[0105] Based on the detection criteria, the pending email work is subjected to sensitive information detection, duplicate submission detection and subject consistency detection.

[0106] In the embodiment of the present invention, illustratively, the detection standard of the pending email work is determined based on the work type of the pending email work:

[0107] Paintings:

[0108] In a painting collection activity, the server received multiple painting image attachments as pending email works. For paintings, the detection criteria mainly focus on the following aspects:

[0109] Content:

[0110] The images cannot contain any violent, pornographic, horrific or other objectionable content. For example, there cannot be depictions of bloody and violent scenes, nor can there be images of characters or images with pornographic implications.

[0111] The content of the paintings should conform to public order and good morals and should not contain content that violates social moral ethics, such as malicious slander or spoof images of specific groups.

[0112] Technical aspects:

[0113] The painting should have a certain integrity. For example, if it is an oil painting, there should not be large unfinished areas or messy smears of paint that make the picture unrecognizable.

[0114] The use of colors should be reasonable. For example, there should not be a serious lack of harmony in color matching that affects the overall visual effect, unless it is a deliberate artistic expression style and can be reasonably judged as artistic innovation.

[0115] Literary works:

[0116] In the case of literary works collection, such as poetry, novels, and essays, after being received by the server as pending email works, the detection criteria are as follows:

[0117] Content:

[0118] Text content must be positive and healthy and must not contain discriminatory language, whether based on race, gender, religion, or other factors. For example, no derogatory descriptions of a particular race are allowed.

[0119] It cannot contain content that promotes violence, terrorism or bad values, such as glorifying criminal acts.

[0120] Creation specifications:

[0121] If it is poetry, it should follow certain rhythmic rules or have a unique poetic structure and expression. For example, classical poetry should meet the requirements of meter (if it is a collection of ancient poems), and modern poetry should also have a unique rhythm and sense of rhythm.

[0122] Novels and essays should have a clear logical structure, vivid characters, and reasonable and coherent plot development.

[0123] Musical works:

[0124] When a musical work (such as a pending email work in the form of an audio file) is processed by the server, the detection criteria include:

[0125] Content:

[0126] Lyrics (if any) must not contain offensive information, such as discriminatory, violent or sexually suggestive words.

[0127] The musical melody cannot plagiarize existing well-known works and should have a certain degree of originality.

[0128] Technical aspects:

[0129] The audio quality should meet certain standards without serious noise, distortion or chaotic audio clipping.

[0130] If it is a work performed by multiple instruments, the coordination between the instruments should be harmonious and the rhythm should be stable.

[0131] Based on the detection standards, sensitive information detection, duplicate submission detection and subject consistency detection are performed on the pending email works:

[0132] Sensitive information detection:

[0133] Paintings:

[0134] The server will use image recognition technology to detect sensitive information. For paintings, the image is first converted into a digital matrix for processing.

[0135] When detecting whether a picture contains pornographic content, the server uses a deep learning-based image classification model that has been trained with a large amount of image data that contains and does not contain pornographic content. For example, after the server inputs an image of a painting into the model, the model will analyze the image of the person, posture, and screen elements in the image. If there is an overly revealing or suggestive posture of a person in the picture, the model will identify it and determine that it contains sensitive information.

[0136] In terms of detecting horror and violence, the server will identify scene elements in the picture, such as whether there are bloody scenes, scary creatures, or inappropriate depictions of weapons. For example, if there is a figure holding a sharp knife and covered in blood in the picture, the server may determine that the painting contains sensitive information.

[0137] Literary works:

[0138] For literary works, the server will detect them word by word.

[0139] When detecting discriminatory language, the server uses a predefined discriminatory vocabulary library. When detecting content that promotes violence and terrorism, the server analyzes the semantics of the sentence. If there is a text that describes a crime in detail and has a glorified or encouraging attitude, such as "He brutally killed the man with a knife, which is really a cool act", the server will determine that this literary work contains sensitive information.

[0140] Musical works:

[0141] When the music piece has lyrics, the server processes the lyrics.

[0142] The server will perform word segmentation on the lyrics and then compare them with a predefined sensitive vocabulary library. For example, if the word "drug" appears in the lyrics and is in a context that promotes the use of drugs, such as "Drugs make me feel really high", the server will determine that sensitive information exists.

[0143] If the music work has no lyrics, the server will analyze the melody style of the music. Although it is difficult for the music melody itself to directly contain sensitive information, if the melody style is highly similar to certain specific bad music (such as music that promotes violence or horror), the server will also mark it for further manual review.

[0144] Duplicate submission detection:

[0145] Painting works:

[0146] The server will extract the feature vector of the painting work. First, an image feature extraction algorithm such as the SIFT (Scale-Invariant Feature Transform) algorithm is used.

[0147] The server processes the painting work and extracts features such as color distribution features, line direction features, and object shape features. For example, for a painting work mainly in blue tones with many curved lines depicting waves, these features will be extracted and converted into feature vectors.

[0148] Then, the server compares this feature vector with the feature vectors of the painting works stored in the work database. The similarity is judged by calculating the distance between the vectors (such as the Euclidean distance). If this distance is less than a preset threshold (for example, after testing, 0.3 indicates a high similarity), it is determined that the work may be a duplicate submission.

[0149] Literary works:

[0150] The server performs text processing on the literary work. First, lexical analysis is carried out to break down the article into words and phrases.

[0151] For example, for an essay, the server breaks it down into words and phrases such as "early morning", "sunshine", "woods", etc. Then, some common stop words such as "de", "shi", "zai" are removed.

[0152] Next, the server uses text fingerprint technology to generate a unique fingerprint (a digital representation form) for the processed text. This fingerprint is compared with the text fingerprints of the archived literary works. If the two fingerprints are highly similar, for example, the similarity reaches more than 90%, it is determined that there may be a duplicate submission.

[0153] Music works:

[0154] For music works, the server extracts the features of the audio. First, the audio is converted into a spectrogram, and then rhythm features, pitch features, timbre features, etc. are extracted from the spectrogram.

[0155] For example, a piece of pop music has a rhythm of 4 / 4, a main pitch near C, and a bright timbre (obtained through spectral analysis), and these features will be combined into a feature vector.

[0156] The server compares this feature vector with the feature vectors of existing music works. If the similarity between the two feature vectors exceeds a set threshold (such as 0.8), it is determined that the work may be a duplicate submission.

[0157] Subject match check:

[0158] Paintings:

[0159] The server first performs content analysis on the paintings. For example, in a painting collection activity with the theme of "the beauty of nature".

[0160] The server will identify the elements in the picture and determine whether they are related to nature. If the picture mainly depicts urban buildings and has only a few natural elements (such as a small piece of sky), then it is less consistent with the theme of "natural beauty".

[0161] The server also analyzes whether the style of the painting reflects the theme. If the theme is "serene nature" and the painting style uses strong contrasting colors and chaotic lines, showing a sense of restlessness, it may not meet the theme requirements.

[0162] Literary works:

[0163] The server performs keyword and semantic analysis on literary works. In an essay contest with the theme of "friendship".

[0164] The server extracts keywords from the works, such as "friends", "company", "trust", etc. Then analyzes the distribution and weight of these keywords in the article. If an article rarely mentions keywords related to friendship, but instead describes more about personal loneliness, then the topic fit is poor.

[0165] The server also analyzes the semantics of the article. If the article mentions the word "friend", but the overall semantics is describing betrayal and hatred between friends, this does not meet the theme requirement of "friendship".

[0166] Musical works:

[0167] In a music collection activity with the theme of "Happy Festival", the server will analyze the music.

[0168] From the perspective of music rhythm, if the music's rhythm is slow and heavy, such as a slow rhythm of only 40 beats per minute, while usually joyful festival music has a lighter rhythm (such as around 120 beats per minute), then the rhythm does not quite meet the theme requirements.

[0169] From the perspective of melody, if the melody is low and sad, rather than cheerful and bright, it does not meet the theme requirements of "Happy Festival". If there are lyrics in the music, the server will also analyze whether the lyrics are related to happy festivals, such as whether the lyrics mention festive elements and a happy atmosphere.

[0170] In the embodiment of the present invention, sensitive information detection is performed on the pending email work, which can be implemented through the following example.

[0171] In the case where the pending email work is a text work, preprocessing and sentiment analysis are performed on the text work, and sensitive information detection is performed in combination with a preset sensitive word library and a preset sensitive word detection model;

[0172] If the pending email work is an image work, the image work is segmented and sensitive information detection is performed in combination with a preset scene recognition model; if the image work includes text content, text is extracted from the image work, and the text content is preprocessed and sentiment analysis is performed, and sensitive information detection is performed in combination with a preset sensitive word library and a preset sensitive word detection model;

[0173] In the case where the pending email work is an audio work, the video sound is converted into audio text using ASR, and the audio text is preprocessed and sentiment analysis is performed, and sensitive information detection is performed in combination with a preset sensitive word library and a preset sensitive word detection model;

[0174] In the case where the pending email work is a video work, ASR is used to convert the video sound into audio text, and the audio text is preprocessed and subjected to sentiment analysis, and sensitive information detection is performed in combination with a preset sensitive word library and a preset sensitive word detection model; the video work is extracted at preset time intervals to obtain multiple video images, and the video images are segmented, and sensitive information detection is performed in combination with a preset scene recognition model; in the case where the video image includes text content, text is extracted from the video image, and the text content is preprocessed and subjected to sentiment analysis, and sensitive information detection is performed in combination with a preset sensitive word library and a preset sensitive word detection model.

[0175] In the embodiment of the present invention, illustratively, when the pending email work is a text work:

[0176] In a literary work submission activity, the server receives numerous text works as pending email works.

[0177] Preprocessing:

[0178] The server first performs preprocessing operations on the text works. This includes converting the text to a standard format, such as converting all uppercase letters to lowercase letters, so that the subsequent processing is consistent. For example, for an article that originally contains sentences starting with uppercase letters and a mixture of uppercase proper nouns, after conversion, all letters become lowercase.

[0179] The server will then remove punctuation marks from the text, as they may interfere with subsequent analysis in some cases. For example, "Hello, world!" will become "hello world". At the same time, some special characters, such as tabs and line breaks, will also be properly processed to organize the text into a continuous string.

[0180] Next, the server will perform lexical analysis on the text and segment the text into individual words, which is the basis for subsequent analysis. For example, "I love reading books" will be segmented into "I", "love", "reading", "books" and other words.

[0181] Sentiment analysis:

[0182] The server uses a pre-trained sentiment analysis model to perform sentiment analysis. This model is trained based on a large amount of text data with sentiment annotations (such as positive, negative, or neutral). For example, for an article describing natural scenery, which contains words such as "beautiful scenery", "fresh air", and "pleasant sunshine", the model determines that the overall sentiment of the article is positive by analyzing these words and the overall structure of the sentence.

[0183] If an article describes the cruelty of war, and contains words such as "bloody battlefield" and "people's grief", the model will determine it as a negative sentiment tendency. Although this kind of sentiment tendency analysis cannot directly detect sensitive information, it can be used as auxiliary information, because some texts with extremely negative or bad sentiment tendencies may be more likely to contain sensitive information.

[0184] Combine the preset sensitive word library and the preset sensitive word detection model to detect sensitive information:

[0185] The server has a huge preset sensitive word library, which contains various types of sensitive words, such as discriminatory words (such as racist words, sexist words, etc.), abusive words, words involving pornography and violence, etc.

[0186] At the same time, the server also uses a preset sensitive word detection model, which can identify some content that has been transformed or euphemistically expressed but is still sensitive information in nature. For example, some cases where homophones, abbreviations, or special symbols are used to replace sensitive words. For an article, the server will compare each word with the sensitive word library, and perform in-depth detection through the sensitive word detection model.

[0187] For example, the word "f*ck" (a veiled insult with asterisks replacing some letters) appears in an article. Although the word is not in its complete form, it can be identified by the sensitive word detection model. If an article contains multiple such sensitive words or suspected sensitive words, the server will determine that the text contains sensitive information.

[0188] In the case where the pending email work is an image work:

[0189] Segment the image artwork:

[0190] In a painting submission scenario, after receiving the image work, the server starts the image segmentation operation. The server uses an image segmentation algorithm based on deep learning, such as the Mask R-CNN algorithm.

[0191] Suppose you receive a landscape image that contains elements such as the sky, mountains, rivers, and trees. Using the image segmentation algorithm, the server can accurately segment the image into different regions, each of which corresponds to an element in the image. For example, the sky is segmented into one region, the mountain is segmented into another region, and so on. This segmentation helps to conduct a detailed analysis of each region in the future to detect whether there is sensitive information.

[0192] Combine the preset scene recognition model to detect sensitive information:

[0193] The server has a preset scene recognition model, which is trained with a large number of annotated scene images (such as normal scenes, violent scenes, pornographic scenes, etc.). The model will identify the segmented image area.

[0194] For example, if a segmented image region contains scenes that resemble human organs exposed and poses suggestive of pornography, the model will identify this as pornographic content and thus determine that the image contains sensitive information. Or if an area looks like a bloody battle scene that contains many violent elements, such as weapons, injured people, etc., the model will also determine that sensitive information exists.

[0195] In the case of textual content included in the graphic work:

[0196] If in this landscape image, there is a sign with some text, such as "No Trespassing" or something like that.

[0197] The server will first perform text extraction on the image. It will use OCR (Optical Character Recognition) technology, such as Tesseract-OCR tool. This tool can recognize the text in the image and convert it into a text format that can be processed by the computer.

[0198] Then, the extracted text content is preprocessed, and the preprocessing process is similar to that of the text work, including operations such as unifying the uppercase and lowercase letters and removing punctuation marks. For example, "NO ENTRY" is converted to "no entry".

[0199] Then, we conduct sentiment analysis to determine the sentiment of these words. If the text content is relatively neutral, such as "no entry", the sentiment is neutral. But if it is "get out you idiot" (including the insulting word "idiot"), the sentiment is negative and contains sensitive information. Finally, we combine the preset sensitive word library and the preset sensitive word detection model to perform detection, just like treating pure text works. If sensitive words exist, it is determined that this image work contains sensitive information.

[0200] In the case where the pending email work is an audio work:

[0201] In a music work submission activity, there may be some audio works with singing content or narration.

[0202] Use ASR to convert video sound into audio text:

[0203] The server uses ASR (automatic speech recognition) technology, such as Baidu's speech recognition service. After receiving the audio work, the sound in the audio is converted into audio text. For example, the lyrics of the singing part of a piece of music are "We run on sunny days", which are converted into corresponding text content through ASR technology.

[0204] Preprocessing and sentiment analysis of audio text:

[0205] Preprocess the converted audio text. Similar to the preprocessing of the text work, convert all letters to lowercase letters, remove punctuation marks, etc. For example, change "We run on sunny days." to "We run on sunny days."

[0206] Then, we conduct sentiment analysis using a pre-trained sentiment analysis model. If the lyrics are positive, such as describing a good life, happy times, etc., the sentiment is positive. If the lyrics describe sadness, pain, or contain negative emotions, the sentiment is negative.

[0207] Combine the preset sensitive word library and the preset sensitive word detection model to detect sensitive information

[0208] The server compares the pre-processed audio text with the preset sensitive word library and uses the sensitive word detection model for detection. For example, if the word "damn" (an abusive word) appears in the audio text, it will be determined that the audio work contains sensitive information through comparison with the sensitive word library and model detection.

[0209] If the pending email work is a video work:

[0210] Use ASR to convert video sound into audio text, preprocess and analyze the sentiment of the audio text, and detect sensitive information by combining the preset sensitive word library and preset sensitive word detection model.

[0211] In a video submission activity, the server receives the video. First, the ASR technology is used to convert the sound in the video into audio text. For example, a video has a narration saying "This is a place full of hope", which is converted into the corresponding text content through ASR.

[0212] Then the audio text is preprocessed, such as converting uppercase letters to lowercase letters, removing punctuation marks, etc., to become "This is a place full of hope." Then sentiment analysis is performed to determine the sentiment based on the text content. If it is a positive statement, the sentiment is positive.

[0213] Finally, the preset sensitive word library and the preset sensitive word detection model are combined for detection. If sensitive words, such as "fool" (abusive word), appear in the audio text, it is determined that the video work contains sensitive information.

[0214] Extract video works at preset time intervals to obtain multiple video images, segment the video images, and perform sensitive information detection in combination with preset scene recognition models:

[0215] The server extracts a frame of video images from the video work at a preset time interval, for example, every 5 seconds. For a video work with a duration of 60 seconds, 12 video images will be extracted.

[0216] Perform segmentation operations on each extracted video image using an image segmentation algorithm, such as the Mask R-CNN algorithm mentioned above. Assuming that there are elements such as people and background buildings in the video image, the segmentation algorithm is used to divide it into different areas.

[0217] Then, the detection is performed in combination with the preset scene recognition model. If a violent scene, such as fighting, weapon display, etc., or a pornographic scene, such as inappropriate human exposure, is found in the segmented area of ​​a video image, the video work is determined to contain sensitive information.

[0218] When the video image includes text content:

[0219] If there are text contents such as subtitles or signs in a video image, for example, the subtitles show "Danger Zone".

[0220] The server first extracts text from the video image, uses OCR technology to extract the text and converts it into text format.

[0221] The extracted text content is then preprocessed, such as unifying uppercase and lowercase letters, removing punctuation marks, etc., to turn it into a "dangerous area". Then, sentiment analysis is performed. In this example, the sentiment of the text is neutral.

[0222] Finally, the preset sensitive word library and the preset sensitive word detection model are combined for detection. If sensitive words exist, it is determined that the video work contains sensitive information.

[0223] In the embodiment of the present invention, the duplicate submission detection of the pending email work may be implemented through the following example.

[0224] In the case where the pending email work is a text work, the text work is segmented to obtain a plurality of vocabulary units; stop words in the plurality of vocabulary units are removed, and the plurality of vocabulary units are converted into a plurality of text vectors; text similarity comparison and semantic similarity comparison are performed based on the plurality of text vectors and the archived text vectors in the work database, and the duplicate submission detection is performed;

[0225] In the case that the pending email work is an image work, the SURF algorithm is used to extract the key points and descriptors of the image work; deep feature extraction is performed based on the key points and descriptors to obtain the deep features of the image work; FLANN and Brute-Force algorithms are used to perform feature matching similarity comparison in combination with the archived image works in the work database to perform the duplicate submission detection.

[0226] In the embodiment of the present invention, illustratively, when the pending email work is a text work:

[0227] Tokenize the text works to obtain multiple lexical units:

[0228] In the scenario of a large-scale literary work submission platform, the server receives numerous text works as pending email works. For each text work, the server will perform tokenization. For example, for a prose work "Early in the morning, the sunlight shines through the gaps in the leaves and forms patches of light on the ground. The birds are singing merrily on the branches, as if telling the beauty of the new day.", the server uses a professional tokenization tool, such as Jieba Tokenizer, to tokenize this article and obtain lexical units: "Early in the morning", "sunlight", "through", "leaves", "gaps", "shines on", "ground", "forms", "patches of", "light", "birds", "branches", "merrily", "singing", "as if", "telling", "new day", "beauty", etc. These lexical units are the basis for subsequent processing.

[0229] Remove the stop words from the multiple lexical units and convert the multiple lexical units into multiple text vectors:

[0230] Remove stop words:

[0231] After obtaining the lexical units, the server will remove the stop words among them. Stop words are words that frequently appear in the text but do not contribute substantially to semantic expression, such as "de", "di", "de", "shi", "zai", etc. After removing the stop words, the lexical units in the above prose may remain "Early in the morning", "sunlight", "leaves", "gaps", "light", "birds", "branches", "merrily", "singing", "new day", "beauty", etc.

[0232] Convert to text vectors:

[0233] The server will then convert these lexical units into text vectors. This process usually relies on a pre-trained word vector model, such as the Word2Vec model. The Word2Vec model is trained based on a large amount of text data, and it can map each lexical unit to a vector space of a fixed dimension. For example, the word "Early in the morning" may be mapped to a 100-dimensional vector [0.1, 0.2, -0.3,...], and "sunlight" is also mapped to a 100-dimensional vector [0.3, -0.1, 0.2,...], etc. In this way, the entire text work is converted into a series of text vectors.

[0234] Perform duplicate submission detection by comparing the text similarity and semantic similarity between multiple text vectors and the archived text vectors in the work database:

[0235] Text similarity comparison:

[0236] The server has a huge database of works, which stores the text vectors corresponding to the text documents that have been received and processed before. For the text vector of the current pending email work, the server will calculate its text similarity with the archived text vectors in the database. There are many ways to calculate text similarity, such as cosine similarity. Suppose the text vector of the current pending email work after processing is vector set A, and an archived text vector in the database is vector set B. By calculating the cosine similarity formula: \(cosine\_similarity=\frac{A\cdot B}{\vert A\vert\times\vert B\vert}\), a similarity value is obtained. If this similarity value is very high, such as exceeding 0.8 (this threshold is set based on experience or platform regulations), it indicates that the two articles have a high similarity at the lexical level.

[0237] Semantic similarity comparison:

[0238] In addition to text similarity comparison, the server also performs semantic similarity comparison. Semantic similarity comparison considers not only the surface matching of vocabulary, but also the semantic meaning of sentences and articles. For example, the server may use a semantic analysis model based on deep learning, such as the BERT (Bidirectional Encoder Representations from Transformers) model. The current pending email work and the archived text in the database are input into the BERT model respectively to obtain their semantic representation vectors. Then the similarity between the two semantic representation vectors is calculated, also using methods such as cosine similarity. If the semantic similarity is also high, for example, exceeding 0.7, then it is more likely to be a duplicate submission. Combining the results of text similarity and semantic similarity, if both exceed the corresponding thresholds, the server determines that the text work is suspected of being submitted repeatedly.

[0239] In the case where the pending email work is an image work:

[0240] Use SURF algorithm to extract key points and descriptors of image works:

[0241] In a painting submission system, the server receives image works as pending email works. For each image work, the server uses the SURF (Speeded-Up Robust Features) algorithm for processing. For example, for an oil painting depicting a landscape, the SURF algorithm will detect some representative key points in the image. These key points are usually places where the grayscale changes in the image are obvious, such as the junction of the sky and the mountains, the outline edges of trees, etc. At the same time, the SURF algorithm will generate a descriptor for each key point. This descriptor is a vector that describes the pixel information around the key point, such as the gradient direction and gradient size of the pixel. Through the SURF algorithm, this oil painting may get hundreds of key points and corresponding descriptors. These key points and descriptors can characterize the characteristics of this image to a certain extent.

[0242] Deep feature extraction is performed based on key points and descriptors to obtain the deep features of the image work:

[0243] After obtaining the key points and descriptors, the server will perform deep feature extraction. This process may involve more complex neural network models or feature combination algorithms. For example, the server can use a convolutional neural network (CNN) to further process these key points and descriptors. CNN can automatically learn the high-level semantic relationship between key points and descriptors, thereby extracting more representative deep features. These deep features can better reflect the overall characteristics of the image, such as the style of the image, the shape combination of objects, etc. For the landscape oil painting mentioned earlier, after deep feature extraction, the obtained deep features may contain information about the layout of the scenery in the picture, color matching, etc.

[0244] Using FLANN and Brute-Force algorithms, combined with archived image works in the work database, feature matching similarity comparison is performed to perform duplicate submission detection:

[0245] FLANN (Fast Library for Approximate Nearest Neighbors, fast approximate nearest neighbor search library) algorithm:

[0246] The server compares the depth features of the current pending email work with the depth features of the archived image works in the work database. First, the FLANN algorithm is used. The FLANN algorithm is an algorithm for quickly searching for approximate nearest neighbors in high-dimensional space. It can quickly find the archived image features that are most similar to the current image features in large-scale image feature data. For example, for the depth features of the current landscape oil painting, the FLANN algorithm searches the database for the depth features of the archived image works that are closest to it.

[0247] Brute-Force algorithm (brute force matching algorithm):

[0248] In addition to the FLANN algorithm, the server also uses the Brute-Force algorithm for feature matching similarity comparison. The Brute-Force algorithm is a simple and direct method that compares the depth features of the current image work with the depth features of all archived image works in the database one by one. Although this method is computationally intensive, it can obtain more accurate matching results. The similarity is measured by calculating the distance between the depth features (such as the Euclidean distance). For example, if the Euclidean distance between the depth features of the current landscape oil painting and the depth features of an archived landscape oil painting in the database is very small, such as less than a set threshold (based on experience or platform regulations, such as 0.1), it indicates that the two images are very similar in features.

[0249] Comprehensive judgment:

[0250] Combining the results of the FLANN and Brute-Force algorithms, if an archived image work is found that has a high degree of similarity (based on a set threshold) with the deep features of the current pending email work, the server determines that there is a possibility that this image work has been submitted repeatedly.

[0251] In the embodiment of the present invention, subject consistency detection is performed on the pending email work, which can be implemented through the following example.

[0252] Extracting keywords and phrases from the pending email works;

[0253] Use the pre-trained LDA topic model to determine the current topic distribution;

[0254] Performing grammatical analysis on the keywords and phrases to obtain deep features of the pending email work;

[0255] The topic consistency detection is achieved by performing a similarity comparison based on the current topic distribution and the deep features.

[0256] In the embodiment of the present invention, for example, keywords and phrases are extracted from the pending email works:

[0257] In the scenario of a news article submission system, the server receives numerous pending email works, which contain various types of news reports.

[0258] For text-based news articles:

[0259] The server first performs word segmentation on the text using professional word segmentation tools such as Jieba Word Segmentation (for Chinese text) or NLTK (for English text). For example, for a news article about a new breakthrough in the field of science and technology: "Recently, scientists have made a major breakthrough in the field of artificial intelligence. They have developed a new algorithm that can significantly improve the accuracy of image recognition." After word segmentation, the word units obtained are "recently", "scientists", "artificial intelligence", "field", "achieved", "a", "major", "breakthrough", "research and development", "a", "new", "algorithm", "can", "substantially", "improve", "image recognition", "accuracy" and so on.

[0260] The server will then identify the keywords and phrases based on certain rules. These rules may include word frequency statistics, part-of-speech tagging, etc. In this example, words such as "artificial intelligence", "algorithm", and "image recognition" are identified as keywords because they are of high importance in the field of science and technology news and have relatively high word frequencies; "major breakthrough" is identified as a phrase.

[0261] For pending emails that are images (such as images with captions or labels):

[0262] If the image is accompanied by some descriptive text, for example, a picture showing a new solar panel, the description is "new solar panels, efficient conversion of solar energy, contributing to the development of environmentally friendly energy." The server also performs word segmentation on this description text to obtain vocabulary units such as "new", "solar panels", "efficient", "conversion", "solar energy", "environmentally friendly energy", "development", and "help". Then, keywords such as "solar panels", "environmentally friendly energy" and phrases such as "efficient conversion" are extracted from it.

[0263] For pending email works of audio or video (if they contain text information such as audio transcription or subtitles in videos):

[0264] Take a video about history and culture as an example. The subtitles in the video include "In ancient Egyptian civilization, the pyramid is its most representative building, which contains rich historical and cultural connotations." The server segmented the subtitle text to obtain vocabulary units such as "ancient", "Egyptian civilization", "pyramid", "most", "representative", "architecture", "contains", "rich", "historical culture", and "connotation", and then extracted keywords such as "Egyptian civilization", "pyramid", "historical culture" and phrases such as "most representative".

[0265] Use the pre-trained LDA topic model to determine the current topic distribution:

[0266] The server has pre-trained the LDA (Latent Dirichlet Allocation) topic model using a large amount of text data covering various subject areas such as technology, culture, entertainment, sports, etc.

[0267] For the previously extracted keywords and phrases, the server inputs them into the LDA topic model. Taking the science and technology news article just now as an example, keywords such as "artificial intelligence", "algorithm", and "image recognition" are input into the LDA model.

[0268] The LDA model will determine the topic distribution of the current pending email works based on the distribution probability of these keywords under different topics. Assuming that in the training data, the topic that is highly related to words such as "artificial intelligence", "algorithm", and "image recognition" is the "technology-computer technology" topic, the LDA model will calculate the probability of this topic in the current work, for example, it may be 0.6, indicating that there is a 60% probability that this work belongs to the "technology-computer technology" topic. At the same time, the model will also calculate the probabilities of other related topics, such as the probability of the "technology-data processing" topic may be 0.2, the probability of the "technology-automation" topic may be 0.1, and so on.

[0269] Perform grammatical analysis on keywords and phrases to obtain the deep features of the pending email works:

[0270] The server performs grammatical analysis on the previously extracted keywords and phrases. Take the keyword "new algorithm" in the previous science and technology news article as an example.

[0271] Part-of-speech tagging:

[0272] The server first performs part-of-speech tagging to determine that "novel" is an adjective and "algorithm" is a noun. This part-of-speech tagging can help the server understand the role of keywords in a sentence. For example, the combination of adjective + noun is often used to describe a specific thing or concept.

[0273] Dependency Analysis:

[0274] The server also performs dependency analysis. In the sentence "They developed a new algorithm", "development" is the predicate verb, "algorithm" is the object, and "new" is the attributive of "algorithm". This dependency relationship shows that "new" is used to modify "algorithm", and the action of "development" is aimed at the object "algorithm". Through this dependency analysis, the server can further understand the semantic relationship between keywords and phrases in the sentence.

[0275] Named Entity Recognition (if applicable):

[0276] If the keyword contains a named entity, such as "Egyptian civilization" in the subtitles of a video about history and culture, the server will recognize that this is a named entity, a specific name belonging to the cultural category. This helps to grasp the subject content of the work more accurately. Through these grammatical analysis operations, the server obtains the deep features of the pending email work, which are not only the keywords and phrases themselves, but also the grammatical relationships and semantic roles between them.

[0277] Based on the current topic distribution and deep features, similarity comparison is performed to achieve topic consistency detection:

[0278] The server compares the subject distribution and deep features of the current pending mail works with the predefined subject requirements. In a science and technology news submission platform, it is assumed that the platform stipulates some specific subject classifications, such as "computer technology innovation", "biotechnology progress", "new energy development", etc.

[0279] For previous science and technology news articles, the topic distribution shows that there is a high probability that they belong to the "Science and Technology - Computer Technology" topic. At the same time, the deep features obtained through grammatical analysis show that "algorithm" is a core concept and is related to actions such as "research and development". If the platform's "Computer Technology Innovation" topic requires content about new achievements and new methods in the field of computer technology, then the server will determine that this work is consistent with the "Computer Technology Innovation" topic.

[0280] However, if there is a news article whose keywords and phrases are extracted as "stars", "concerts", etc., and the subject distribution shows that it belongs to the "Entertainment-Performing Arts" theme, but the platform requires articles on technology-related topics, then the server will determine that this work does not match the theme required by the platform.

[0281] In the image category, let's assume that it is an art work submission platform, and the theme requirement is "natural landscape painting". For an image work, the keywords extracted from its description text are "high-rise buildings" and "city streets". The theme distribution shows that it is related to the theme of "urban architecture". The deep features of grammatical analysis also show that these keywords describe the urban environment. The server will determine that this image work does not match the theme of "natural landscape painting".

[0282] The same is true for audio or video works. For example, on a historical and cultural documentary submission platform, the required theme is "Ancient Civilization Exploration". If the subtitle keywords and phrases of a video work are extracted as "Modern Urban Life" and "Fashion Trends", the theme distribution is related to the theme of "Modern Lifestyle", and the deep features of grammatical analysis also indicate that it is about modern life, then the server will determine that this video work does not match the theme required by the platform. Through this comparison process, the server realizes the theme consistency detection of the pending email works.

[0283] In the embodiment of the present invention, the multi-dimensional scoring of the pending email work to obtain the multi-dimensional scoring result of the pending email work can be implemented through the following examples.

[0284] In the case where the pending email work is a written work, multi-dimensional scoring is performed based on the BERT model using part-of-speech tagging, syntactic analysis, and sentiment analysis as scoring indicators to obtain a multi-dimensional scoring result of the written work;

[0285] In the case where the pending email work is an image work or a video work, a multi-dimensional scoring is performed based on the LSTM and Transformer models using the painting style, painting skills and artistic elements as scoring indicators to obtain a multi-dimensional scoring result of the image work or the video work;

[0286] When the pending email work is a calligraphy work, a multi-dimensional scoring is performed based on the area difference, aspect ratio difference, distance difference and orientation difference between the calligraphy work and the standard characters to obtain a multi-dimensional scoring result of the calligraphy work.

[0287] In the embodiment of the present invention, exemplarily, when the pending email work is a writing work, part-of-speech tagging, syntactic analysis and sentiment analysis are used as scoring indicators for multi-dimensional scoring based on the BERT model to obtain a multi-dimensional scoring result for the writing work.

[0288] Part-of-speech tagging scoring:

[0289] In a literary work submission platform, the server receives many writing works as pending email works. For each writing work, the server first performs part-of-speech tagging. For example, for a prose: "In the early morning, the sun shines gently on the earth, and the birds sing happily." The server uses the BERT model to perform part-of-speech tagging.

[0290] The BERT model recognizes that "morning" is a noun, "sunshine" is a noun, "gently" is an adverb, "sprinkle" is a verb, "on" is a preposition, "earth" is a noun phrase, "birds" is a noun, "joyfully" is an adverb, and "singing" is a verb.

[0291] Then the score is given based on the accuracy and diversity of the part-of-speech tagging. If the part-of-speech tagging in the article is accurate and uses a rich variety of parts of speech, such as more adjectives and adverbs to vividly describe scenes and actions, then a higher score will be obtained on the part-of-speech tagging indicator. For example, if an article cleverly uses many expressive adjectives such as "gorgeous sunset" and "quiet forest" in addition to common nouns and verbs, this shows that the author is more proficient in the use of parts of speech and may get 8-10 points on this indicator (assuming a full score of 10 points). On the contrary, if there are errors in part-of-speech tagging, such as mislabeling verbs as nouns, or the use of parts of speech is very simple, it may only get 3-5 points.

[0292] Syntactic Analysis Rating:

[0293] Next, the server performs syntactic analysis on the writing. Taking the above prose as an example, the BERT model will analyze the sentence structure. For example, "In the early morning, the sun shines gently on the earth" is a sentence with a subject-predicate structure, "sunshine" is the subject, "sprinkle" is the predicate, "on the earth" is the complement, and "gently" is the adverbial; "The birds sing happily" is also a subject-predicate structure, "birds" is the subject, "singing" is the predicate, and "joyfully" is the adverbial.

[0294] Scoring is based on the rationality and complexity of syntactic analysis. If the sentence structure is complete, reasonable, and has a certain degree of complexity, such as using multiple clauses and compound sentence structures to express complex ideas, it will get a higher score. For example, "When night falls, the elves hidden in the darkness shuttle through the woods, as if telling an ancient story, and I, standing there quietly, attracted by this mysterious atmosphere." This sentence contains multiple structures such as time adverbial clauses and parallel sentences, showing that the author has a good ability to use syntax, and may get 7-9 points in the syntactic analysis indicator. If the sentence structure is simple and there are many grammatical errors, such as incomplete sentence components and chaotic word order, it may only get 2-4 points.

[0295] Sentiment Analysis Scoring:

[0296] The server also performs sentiment analysis on the writing. For a description in a prose, such as "the sun shines gently on the earth, and the birds sing happily", the BERT model can determine that the overall sentiment is positive.

[0297] Scoring is based on the depth, authenticity, and coherence of the emotional expression. If an article can express a positive or negative emotion deeply, authentically, and coherently, such as an article describing the grief of losing a loved one, through delicate descriptions and progressive layers of emotions, the reader can deeply feel the author's pain, then it will get a higher score on the sentiment analysis indicator, perhaps 8-10 points. If the emotional expression is vague, incoherent, or inconsistent with the content of the article, such as describing a beautiful scenery at the beginning of the article, and then suddenly expressing sadness without logic at the end, it may only get 3-5 points.

[0298] Finally, the scores of part-of-speech tagging, syntactic analysis, and sentiment analysis are combined to obtain the multidimensional score of the writing work. For example, if an article gets 7 points in part-of-speech tagging, 6 points in syntactic analysis, and 8 points in sentiment analysis, then its multidimensional score may be (7+6+8) / 3=7 points (assuming that the weights of each indicator are the same).

[0299] When the pending email work is an image work or a video work, a multi-dimensional scoring is performed based on the LSTM and Transformer models using the painting style, painting techniques and artistic elements as scoring indicators to obtain a multi-dimensional scoring result for the image work or the video work.

[0300] Painting style rating (taking image works as an example):

[0301] In an art collection activity, the server receives image works. For an oil painting, the server first analyzes its painting style. If the painting is in the Impressionist style, the LSTM and Transformer models will identify its typical features, such as loose brushstrokes, emphasis on light and shadow effects, and the mixing and interweaving of colors to create a hazy visual effect.

[0302] Scoring is based on the typicality, uniqueness and innovation of the painting style. If this oil painting is a typical example of the Impressionist style, such as Monet's "Water Lilies" that captures the feeling of momentary light and shadow through colors and brushstrokes, and has some unique innovations based on this style, such as bolder use of colors, then it may get 8-10 points in the painting style indicator. If the painting style is unclear, it looks like both Impressionism and Realism, but neither of them reflects their characteristics well, it may only get 3-5 points.

[0303] Drawing skills scoring (taking image works as an example):

[0304] Next, we analyze the painting techniques. For oil paintings, painting techniques include color matching, brushstroke application, object shape depiction, etc. For example, in terms of color matching, if the color transition in the picture is natural and harmonious, and there is no obvious color conflict (unless it is a deliberate artistic effect), such as the natural transition from light blue to dark blue sky color, then it will get a good score in this aspect.

[0305] In terms of brushstroke application, if the brushstrokes are delicate and expressive, and can vividly depict the texture of the object, such as soft and delicate brushstrokes when depicting human skin, and rough and powerful brushstrokes when depicting rocks, this will also increase the score of painting skills. Scoring is based on the comprehensive performance of these painting skills. If the painting skills are skillful and exquisite, it may get 7-9 points on this indicator; if the color matching is not coordinated, the brushstrokes are stiff and lack of expression, it may only get 2-4 points.

[0306] Artistic element scoring (taking image works as an example):

[0307] Then analyze the artistic elements. Artistic elements include composition, lines, light and shadow, etc. For example, in terms of composition, if the picture adopts a unique composition method, such as golden ratio composition, places the subject in a key position in the picture, and makes the whole picture balanced and attractive, then it will get a good score in the artistic element of composition.

[0308] In terms of the use of lines, if the lines are smooth and rhythmic, and can well outline the contours and shapes of objects, such as using smooth curves to depict the contours of a woman's body, the score will also increase. Scoring is based on the comprehensive use of artistic elements. If the artistic elements are used properly and richly, this indicator may get 7-9 points; if the composition is messy and the lines lack beauty, it may only get 2-4 points.

[0309] For video works, the analysis of painting style, painting techniques and artistic elements is similar, but the dynamic elements of the video need to be considered, such as the switching of shots, the rhythm of the picture, etc. For example, in terms of shot switching, if the switching is natural and smooth, and can guide the audience's sight well, it will get a good score in the artistic element of shot switching. The scores of painting style, painting techniques and artistic elements are combined to obtain the multi-dimensional scoring results of the image work or video work. For example, if an image work gets 8 points in painting style, 7 points in painting techniques, and 8 points in artistic elements, then its multi-dimensional scoring result may be (8+7+8) / 3=7.67 points (assuming that the weights of each indicator are the same).

[0310] When the pending email work is a calligraphy work, a multi-dimensional score is performed based on the area difference, aspect ratio difference, distance difference and orientation difference between the calligraphy work and the standard characters to obtain a multi-dimensional score result of the calligraphy work.

[0311] Area Difference Rating:

[0312] In a calligraphy work submission platform, the server receives calligraphy works. For each calligraphy work, the server first compares it with the standard characters to analyze the area difference. For example, the area of ​​the standard characters is set to 10 square centimeters (this is an assumed standard, which can be set according to the font type and requirements in practice).

[0313] If the average area of ​​the characters in the calligraphy work is 9-11 square centimeters, it means that the area difference with the standard characters is small, and it may get 8-10 points on this indicator. Because the small area difference means that the font size is more in line with the standard and the writing is more standardized. If the average area of ​​the characters is 6-8 square centimeters or 12-14 square centimeters, that is, the area difference with the standard characters is large, it may only get 3-5 points, because this may affect the overall visual effect and standardization.

[0314] Aspect Ratio Difference Rating:

[0315] Next, we analyze the difference in aspect ratio. Assuming that the aspect ratio of a standard character is 1:1.2, for characters in calligraphy works, if the aspect ratio of most characters is between 0.9:1.1-1.1:1.3, it means that the aspect ratio difference is small, and the character may get 7-9 points on this indicator. This shows that the shape ratio of the character is close to the standard and meets the aesthetic standards of calligraphy.

[0316] If the aspect ratio is 0.7:1.5 or 1.3:1.0, which is significantly different from the standard aspect ratio, you may only get 2-4 points, because such characters may appear uncoordinated and not conform to the proportional beauty of traditional calligraphy.

[0317] Distance Difference Rating:

[0318] Then analyze the distance difference, where the distance refers to the spacing between characters and lines. If the spacing between characters and lines in the calligraphy work is close to the standard calligraphy layout spacing (assuming the standard spacing is 1 / 2 of the character height), for example, the character spacing is between 0.4-0.6 times the character height, and the line spacing is between 1.4-1.6 times the character height, it may get 8-10 points on this indicator. This shows that the layout is reasonable and the density is appropriate.

[0319] If the character spacing is too large or too small, and the line spacing does not meet the standard, such as the character spacing is 0.2 times or 0.8 times the character height, and the line spacing is 1.0 times or 2.0 times the character height, you may only get 3-5 points, because such a layout will make the work look crowded or loose, affecting the overall beauty.

[0320] Azimuth difference score:

[0321] Finally, the position difference is analyzed, that is, whether the writing position of the characters is correct. If the characters in the calligraphy work basically maintain the horizontal and vertical directions without obvious skewness, it may get 8-10 points on this indicator. This reflects the writer's ability to control the structure and layout of the characters.

[0322] If the characters are obviously skewed, such as if part of the characters are tilted more than 5 degrees (this is an assumed threshold), they may only get 2-4 points, as this will affect the overall neatness and beauty of the calligraphy work.

[0323] The scores of area difference, aspect ratio difference, distance difference and orientation difference are combined to obtain the multi-dimensional scoring result of the calligraphy work. For example, if a calligraphy work gets 8 points in area difference, 7 points in aspect ratio difference, 9 points in distance difference and 8 points in orientation difference, then its multi-dimensional scoring result may be (8+7+9+8) / 4=8 points (assuming that the weights of each indicator are the same).

[0324] In order to more clearly describe the solution provided by the embodiment of the present invention, a relatively complete implementation method is provided below.

[0325] 1. A method for collecting works from email boxes:

[0326] The organizing committee organizes competitions and collects submissions in the following ways:

[0327] a) Set up a dedicated mailbox: Participants submit their works via email, and the organizing committee can batch download or manage these works through the mailbox.

[0328] b) Third-party competition management platform: Use a third-party competition management platform, which generally provides integrated services for submission, review and display of works.

[0329] c) Official website: Integrate the work submission function in the official website or dedicated application of the organizing committee, so that contestants can upload their works directly through the website or application. Among them, collecting contestants' works by email is the most widely used method.

[0330] The present invention provides an efficient method for collecting works from mailboxes, so as to efficiently download and manage a large number of entries from emails, and automatically filter out the works that obviously do not meet the entry requirements, saving time and reducing human errors. The following are some key steps and methods:

[0331] S1 configures the mail server parameters:

[0332] Get the IMAP server address and port number of the target mailbox server.

[0333] Configure SSL / TLS secure connection to ensure the security of data transmission.

[0334] S2 connects to the mail server:

[0335] A connection is established through the IMAP address and port of the mail server configured in step S1, and the login credentials.

[0336] S3 download email:

[0337] S3-1 defines the submission start time T_start and submission deadline T_end according to the submission time requirements of the current competition activity, filters out emails with submission times greater than T_start and less than T_end, and uses the FETCH command to obtain the email body and attachments.

[0338] S3-2 stores the email content in a relational database and file server in the required format.

[0339] S4 filtering does not meet the conditions of the mail works:

[0340] S4-1 Configure filtering rules according to the submission requirements of the current competition theme, such as submission email title S_title, attachment name F_name, attachment file format F_type, attachment file size F_size, attachment size F_spec, etc.

[0341] S4-2 performs filtering on the downloaded email works according to the filtering rules configured in step S4-1, and determines whether the submission meets the conditions by defining rules:

[0342] a) Whether the title complies with the regulations

[0343] The system presets the email delivery title rules for this event, such as: "School_Class_Name_Contact Number", which can be verified by the regular expression: ^(\w+)_(\w+)_([\u4e00-\u9fa5]+)_1[3-9]\d{9}$.

[0344] The following is an explanation of each part of the regular expression: (\w+): matches the school name, \w matches letters, numbers and underscores, + means one or more. _: A literal underscore character. (\w+): matches the class name, the rules are the same as above. _: A literal underscore character. ([\u4e00-\u9fa5]+): matches one or more Chinese characters, \u4e00-\u9fa5 are Chinese characters in the Unicode encoding range. _: A literal underscore character. 1[3-9]\d{9}: matches a mobile phone number in the preset area, starting with 1, the second digit is a number between 3 and 9, \d{9} means it is followed by 9 digits.

[0345] By matching regular expressions, we can quickly filter out submissions whose S_titles meet the standards. For some submissions that are slightly non-compliant with the standards, the system can also make some compatible processing. For example, if the separator is mistakenly written as a hyphen "-", in order to be compatible with such email titles, we can adjust the regular expression appropriately and change it to: ^(\w+)[_-](\w+)[_-]([\u4e00-\u9fa5\w]+)[_-]1[3-9]\d{9}$.

[0346] b) Whether the attachments meet the specifications

[0347] The system presets the attachment naming standard for this event, such as: "School_Class_Name_Work Name.docx". The attachment name F_name can be verified by regular expression, such as: ^(\w+)[_-](\w+)[_-]([\u4e00-\u9fa5\w]+)[_-][\u4e00-\u9fa5\w]+)\.docx$.

[0348] The system automatically downloads the attachments and parses them accordingly to obtain the file type and file size of the attachments; based on the file type, it obtains additional parameter information. For example, for document files such as .doc, .docx, .txt, etc., the system can automatically obtain the number of words in the file; for image files such as .jpg, .jpeg, .png, .bmp, .gif, .svg, etc., it can obtain the corresponding image size, aspect ratio, resolution and other parameter information; for audio and video files such as .mp3, .mp4 file types, it can obtain the corresponding duration and resolution information;

[0349] Combined with the competition rules, we will filter the attachments. For example, if the video length must not exceed 3 minutes, the works that are longer than 3 minutes will be automatically marked as "Exceeded" and archived. If the file size cannot exceed 50MB, the works with attachment size F_size>50Mb will be automatically marked as "Oversized Attachment" and archived.

[0350] S5 further analyzes and processes:

[0351] S5-1 After step S4, the works that obviously do not meet the requirements are filtered out, and the works that meet the requirements are further processed, the content is analyzed, and useful information is extracted, such as title, attachment, email address, mobile phone number, school, grade, class, work name, etc.

[0352] S5-2 classifies the works according to title rules, such as painting, music, writing, calligraphy, etc. Each category is configured with different review and scoring standards and review and scoring models.

[0353] 2. A method for reviewing the compliance of works:

[0354] The compliance check of works mainly checks whether the works comply with laws and regulations and whether they comply with moral standards and social norms. Check whether the works contain risky content or elements such as pornography, violence, political sensitivity, and insults; whether there are duplicate submissions; whether the content of the works is consistent with the theme, etc.

[0355] The present invention provides a method for compliance review of works, which helps competition organizers reduce manual review costs and improve preliminary screening efficiency and accuracy in academic and creative competitions (such as calligraphy competitions, composition competitions, creative design competitions, etc.). The following are some key steps and methods:

[0356] S1 default rules:

[0357] According to laws, regulations and social norms, set predefined rules and standards for preliminary content screening. Configure sensitive word library dictionary to detect whether the work hits sensitive and illegal words.

[0358] S2 Works Category:

[0359] There are various forms of entries for academic and creative competitions, but they can be summarized into the following formats:

[0360] Document type: such as text documents (txt, docx), PDF documents;

[0361] Presentation: presentations in PPT and PPTX formats;

[0362] Image category: bitmap images (JPEG, PNG, GIF, etc.), vector graphics (SVG, AI, EPS, etc.);

[0363] Audio: lossless audio files (WAV), compressed audio files (MP3, AAC, etc.);

[0364] Video: standard video files (MP4, MOV, AVI), high-definition video files (HD, 4K);

[0365] The works are classified by type by parsing the titles and attachment suffixes, and different detection methods and detection models are configured for different categories.

[0366] S3 content compliance detection:

[0367] Compliance testing includes a) sensitive information testing; b) duplicate submission testing; c) work integrity testing; c) subject consistency testing;

[0368] According to the classification results obtained in step S2, different models are used for detection according to different classifications. a) Sensitive information detection:

[0369] S3-1, use NLP technology for text categories. Use natural language processing technology for works to check language style, sensitive words, political sensitivity, etc. Segment continuous text strings into meaningful units and assign grammatical categories to each word in the text. Identify specific entities in the text, analyze the dependencies between words in the text, and build a grammatical tree structure of sentences. Use machine learning models to quantitatively analyze the sentiment of the text based on the vocabulary and context in the text. Use pattern matching and machine learning models to identify whether the text hits the sensitive words or inappropriate words in step S1. These words may include political topics or speeches or inappropriate content such as pornographic violence. If the text contains image links or descriptions, use computer vision technology to analyze the image content. Map the vocabulary to vectors in high-dimensional space to capture the semantic relationship between words. Apply algorithms such as logistic regression, support vector machine (SVM), and random forest to train the model to identify text features. Use sentiment analysis to determine the sentiment of the text and identify text that may contain negative emotions or offensive language.

[0370] S3-2, use computer vision technology for image classes to detect inappropriate content. Use deep learning models and convolutional neural networks (CNN) to identify specific objects or scenes in images. Train deep learning models using a large number of labeled image data sets so that they can recognize and distinguish different types of content. Segment images to identify different areas or objects in the image for more accurate analysis and identification. Use facial recognition technology to detect faces in images and identify content that may involve personal privacy. Identify scenes in images, such as violent scenes, pornographic scenes, etc., by training specific scene recognition models. If the image contains text, use OCR (optical character recognition) technology to extract the text, and use step S3-1 to further analyze the text content and detect the compliance of the text.

[0371] S3-3, for audio, use ASR technology to convert the video sound into text, and then perform content detection on the text in step S3-1;

[0372] S3-4, for video, use ASR technology to convert the video sound into text, and then perform content detection on the text in step S3-1; use video decoding technology to extract frames at a certain time interval and convert them into images, and then perform detection on the extracted frame images in step S3-2 respectively;

[0373] b) Duplicate submission / plagiarism detection:

[0374] S3-5. Use data fingerprint technology and hash algorithms to identify and prevent the same work from being submitted repeatedly. By comparing existing works in the database, text matching and image comparison algorithms are used to detect plagiarism issues.

[0375] The steps for document detection are as follows:

[0376] S3-5.1 Word Segmentation and Stop Word Filtering:

[0377] Perform word segmentation on the text to split continuous text into discrete lexical units. Remove stop words, such as common words like "of", "and", "is", etc., which contribute less to the analysis.

[0378] S3-5.2 Feature Extraction:

[0379] Use the TF-IDF method to convert the text into a vectorized representation for easy calculation of similarity.

[0380] S3-5.3 Similarity Calculation:

[0381] Apply cosine similarity and Jaccard similarity algorithms to calculate the similarity between texts.

[0382] S3-5.4 Machine Learning Model Training:

[0383] Use the SVM machine learning algorithm to train a model to identify plagiarized text.

[0384] Use the BERT deep learning model for deeper semantic comparison.

[0385] S3-5.5 Threshold Setting and Plagiarism Judgment:

[0386] Set a similarity threshold, and texts exceeding this threshold are considered potential plagiarism. Mark the texts with similarity exceeding the threshold.

[0387] The steps for image detection are as follows:

[0388] S3-5.6 Feature Extraction:

[0389] Use the SURF algorithm to extract key points and descriptors of the image. Use the CNN deep learning model to extract deep features of the image.

[0390] S3-5.7 Feature Matching:

[0391] Use the FLANN and Brute-Force algorithms for feature matching.

[0392] S3-5.8 Similarity Calculation:

[0393] The similarity of matching features is calculated mainly by using the Euclidean distance formula to measure the similarity between data points. For two points x and y in the data set, the Euclidean distance between them can be calculated by the following formula: d(x,y) = sqrt((x1-y1)^2+(x2-y2)^2+...+(xn-yn)^2) where x1, x2,..., xn and y1, y2,..., yn are the coordinate values ​​of points x and y in each dimension respectively, and n is the dimension of the point data.

[0394] S3-5.9 Similar image comparison:

[0395] Compare the image to be detected with the images in the database, and find similar images based on the similarity results calculated in step S3-5.8.

[0396] S3-5.10 Threshold setting and decision making:

[0397] Set a similarity threshold T to determine whether an image is plagiarized. Mark works with similarity exceeding the threshold.

[0398] c) Subject consistency check:

[0399] For essay and keynote speech competitions, you can check whether the content is consistent with the competition theme. The specific steps are as follows:

[0400] S3-5.11 Keyword and phrase extraction

[0401] Extract keywords and phrases from the text using TF-IDF and TextRank techniques. Identify named entities in the text as indicators of topic relevance.

[0402] S3-5.12 Building a Topic Model

[0403] The LDA topic model algorithm is used to extract topic distribution from the reference data.

[0404] Based on the vocabulary and topic assignments in the document, the probability of each word appearing on each topic is calculated.

[0405] Update topic assignments: Update the topic assignments of the document based on the probability of each word appearing on each topic.

[0406] Repeat the above steps until the topic assignment and the probability of vocabulary occurrence on the topic are stable.

[0407] The core formula of the mathematical model of the LDA topic model is as follows:

[0408] p(w,z|α,β)=p(w|z,β)*p(z|α). Where w represents the word in the document; z represents the topic to which the word belongs; α and β are the hyperparameters of the model, which control the prior distribution of the document topic distribution and the topic word distribution respectively; p(w|z,β) represents the probability of word w appearing under the condition of given topic z and parameter β; p(z|α) represents the topic distribution of the document under the condition of given parameter α; the extracted topic is compared with the keywords of the work.

[0409] S3-5.13 Semantic Understanding and Dependency Syntax Analysis

[0410] Use the dependency parsing tool (spaCy library) to understand the semantic structure of the text. The dependency parsing in spaCy relies on the weights and activation functions of the neural network model. The key steps are as follows:

[0411] Feature extraction: xt = f(w, ct), where xt is the feature representation of the tth word, w is the model parameter, and ct is the context information of the word. Neural network forward propagation: ht = σ(W·xt+b), where ht is the hidden state of the tth word, and σ is the activation function, such as ReLU. Dependency and label prediction: yt = g(V·ht) where yt is the predicted probability distribution of the dependency and label of the tth word, V is the weight matrix of the output layer, and g is the softmax function.

[0412] S3-5.14 Similarity calculation

[0413] The similarity between text vectors is calculated using a metric such as cosine similarity. The cosine value between two vectors can be found using the Euclidean dot product formula: . Given two attribute vectors, A and B, the cosine similarity θ is given by the dot product and the length of the vector, as shown below:

[0414]

[0415] Here Ai and Bi represent the components of vectors A and B respectively.

[0416] The similarity given ranges from -1 to 1: -1 means that the two vectors point in opposite directions, 1 means that they point in exactly the same direction, 0 generally means that they are independent, and values ​​in between indicate intermediate similarities or dissimilarities.

[0417] S3-5.15 Threshold setting:

[0418] Set a similarity threshold T as the standard for judging whether the text meets the topic.

[0419] S3-5.16 Automated testing and scoring:

[0420] Automatically detect artworks using a trained model and a set threshold.

[0421] Score Socre = Similarity calculated in step S3-5.14 * 100. Score Score and threshold T are compared by algorithm. When Score>=T, it means that the topic is met.

[0422] Generate a rating or classification result that matches the topic for each work.

[0423] Through this multi-level and multi-dimensional review method, the compliance of the works can be effectively ensured while improving the efficiency and fairness of the review process.

[0424] 3. A method for preliminary scoring of works:

[0425] The present invention provides a preliminary scoring method for works, which can perform preliminary scoring on entries (calligraphy, literary works, creative writing, paintings, etc.), provide data support and decision-making suggestions for the review team, and improve the efficiency of the review process. The following are some key steps and methods:

[0426] The work scoring module uses AI technology model training to conduct multi-dimensional evaluations of different types of entries based on defined scoring criteria and come up with a comprehensive reference score.

[0427] S1 sets the scoring criteria and weights:

[0428] Clarify the quantitative dimensions of work scoring, including but not limited to key indicators such as creativity, technical skills and originality.

[0429] Define the scoring dimensions DIM and weights W, such as creativity, technicality, and expressiveness. Set quantitative indicators and scoring rules for each dimension, such as grammar, spelling, content, logic, and creativity for writing works; and whether there are writing errors, neatness, pen strokes, strength, and other indicators for calligraphy works;

[0430] S2 feature annotation:

[0431] The eigenvalues ​​are normalized to a uniform range of 0 to 1 to eliminate the effects of different dimensions and magnitudes.

[0432] S3 Feature Extraction:

[0433] Digitalize the works, clean and standardize the data, and extract key features based on different types of works.

[0434] a) For writing: Use NLP technology to analyze the content of the article, including part-of-speech tagging, syntactic analysis, sentiment analysis, etc. Evaluate grammatical errors in the article, such as subject-verb agreement, preposition collocation, etc. Analyze the use of vocabulary in the article, including the richness and appropriateness of vocabulary. Evaluate the complexity and diversity of sentences, such as the use of simple sentences, parallel sentences, and complex sentences. Check whether the content of the article is relevant and whether it revolves around the given theme. Analyze the logical relationship between paragraphs in the article and the overall coherence. Evaluate the writing style and innovation of the article, such as the use of metaphors, similes, and other rhetorical devices. Construct scoring dimensions based on multiple features such as text coherence, literary talent, intention analysis, and paragraph structure.

[0435] b) For image and video categories:

[0436] Use image processing technology to perform preprocessing operations such as denoising and normalization on the input painting images. To make the data more uniformly distributed in various dimensions, the optimization algorithm converges faster. Avoid overfitting of the model in one feature dimension and improve the generalization ability of the model. Use convolutional neural network (CNN) to extract key visual features in paintings.

[0437] c) For calligraphy:

[0438] Use pattern recognition technology to extract strokes, structure, layout, size, and component data from the works, and conduct multi-dimensional analysis of calligraphy characters, including component area, component aspect ratio, distance between components, and orientation between components.

[0439] S4 dimension score calculation:

[0440] According to the scoring dimensions and weights defined for different work types, the dimension scores are calculated based on the extracted key features.

[0441] For writing:

[0442] Based on the pre-trained language model BERT, the last layer output corresponding to [CLS] is used to represent the article and input into the classifier for scoring: Score = Max(L,MIN(B+F,U)).

[0443] In the above formula, Score represents the final score, the MAX function represents taking the larger value of the two, the MIN function represents taking the smaller value of the two, L represents the lowest score of a certain level, U represents the highest score of a certain level, B represents the basic score of a certain level, and F represents a floating score.

[0444] For image and video categories:

[0445] Based on the extracted features, the style, technique and artistic elements of the painting are evaluated. One million images are selected from the historical works library. To ensure the universality of the training model for pictures of various categories and styles, these images must be evenly distributed in various styles and themes, such as: people, still life, color, black and white, city, scientific creativity, humanistic photography, etc. A score distribution map that is as complete as possible is selected as the data label (to prevent the influence of extreme scores and subjective scores, the score of each picture is the median score of all scores).

[0446] Train LSTM and Transformer deep learning models to identify and evaluate the creativity and uniqueness of paintings. Use a multi-task learning framework to simultaneously evaluate the technical and artistic qualities of paintings. Use models pre-trained on large-scale datasets to migrate to painting scoring tasks to improve learning efficiency.

[0447] Based on Early Stopping, reduce the training time as much as possible to avoid overfitting, and stop increasing the training cycle in time when the accuracy has not been significantly improved. Test different optimizers and learning rates to ensure that the error between the final model score and the manual score is within an acceptable range.

[0448] The deep residual network ResNet50 is selected for training, the number of output units is changed to 1, and the output prediction score S is limited by the activation function to ensure that it is between 0 and 1. The final score calculation formula is as follows: Score = S*100. Through the above model training method, the image work score Score is finally obtained.

[0449] For calligraphy:

[0450] Calculate the difference between the area ratio of the two parts of the calligraphy character and the area ratio of the two parts of the standard character: D_area = ((area_hw^(i+1) / area_hw^i)) / ((area_gt^(i+1) / area_gt^i)-1). Wherein, area represents the area dimension, the subscript hw represents the calligraphy handwritten character, gt represents the corresponding standard character, and i represents the component number.

[0451] Calculate the score coefficient of the area ratio difference between the current components of the handwritten character: Score_area = 1-((|D_area|-|α_area|)) / β_area. Among them, α represents the area threshold, which is used to judge whether the area ratio between components is reasonable. It is obtained by kmeans clustering based on the pre-classified data. represents the maximum tolerance threshold, which is set manually.

[0452] The total score of calligraphy evaluation is the average score of multiple component dimensions:

[0453] Score_total=(Score_area+Score_aspect+Score_distance+Score_angle) / 4.

[0454] Among them, Score_area is the score of area dimension, Score_aspect is the score of aspect ratio dimension, Score_distance is the score of distance dimension, and Score_angle is the score of orientation dimension. Through the above method, calligraphy works can be automatically evaluated and initial score data can be output.

[0455] S6 final score calculation:

[0456] Design an automated scoring process to ensure scoring consistency and repeatability.

[0457] Integrate the evaluation and scoring data of multiple dimensions into a single score, set thresholds based on the distribution of scoring results, and distinguish works of different levels.

[0458] The scoring result of each dimension is multiplied by its corresponding weight, and then the weighted results are added together to obtain the preliminary score of the work.

[0459] Final_Score=w1×s1+w2×s2+…+wn×sn+Post-Processing.

[0460] Where: wi is the weight of the i-th feature, si is the score of the i-th feature, n is the total number of features, and Post-Processing represents any necessary post-processing steps such as normalization or adjustment.

[0461] Through this method, multiple dimensions can be comprehensively considered to give a final score data for a work.

[0462] S7 manual review feedback:

[0463] Introduce a manual review mechanism to review the preliminary AI scoring results to ensure the fairness and accuracy of the scoring.

[0464] Collect review feedback for further training and optimization of the model.

[0465] S8 model iteration and performance optimization:

[0466] Based on feedback and review results, we continuously iterate and optimize the model to improve scoring accuracy. We regularly update the training set to incorporate new scoring data and expert review opinions. Through the above multi-dimensional and multi-indicator scoring method and deep learning and self-optimization capabilities, we provide an efficient and fair scoring method for the evaluation of entries.

[0467] Please refer to Figure 2 , Figure 2 A mailbox-based work screening device 110 provided in an embodiment of the present invention includes:

[0468] The acquisition module 1101 is used to obtain the original email from the target mailbox server and filter it to obtain the pending email works;

[0469] The screening module 1102 is used to perform a compliance check on the pending email work based on the work type of the pending email work; if the work passes the compliance check, the pending email work is scored in multiple dimensions to obtain the multidimensional scoring result of the pending email work; in response to a user review approval indication for the multidimensional scoring result, the final screening result of the pending email work is obtained according to the size of the multidimensional scoring result.

[0470] It should be noted that the implementation principle of the aforementioned mailbox-based work screening device 110 can refer to the implementation principle of the aforementioned mailbox-based work screening method, and will not be repeated here. It should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into a physical entity, or physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the mailbox-based work screening device 110 can be a separate processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0471] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more microprocessors (digital signal processors, DSP), or one or more field programmable gate arrays (FPGA), etc. For another example, when a module above is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0472] The embodiment of the present invention provides a computer device 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the aforementioned mailbox-based work screening device 110. Figure 3 As shown, Figure 3 The computer device 100 is a block diagram of a structure of a computer device 100 provided in an embodiment of the present invention. The computer device 100 includes a mailbox-based work screening device 110 , a memory 111 , a processor 112 and a communication unit 113 .

[0473] In order to realize data transmission or interaction, the memory 111, the processor 112 and the communication unit 113 are electrically connected to each other directly or indirectly. For example, the electrical connection between these elements can be realized through one or more communication buses or signal lines. The mailbox-based work screening device 110 includes at least one software function module that can be stored in the memory 111 in the form of software or firmware or solidified in the operating system (OS) of the computer device 100. The processor 112 is used to execute the mailbox-based work screening device 110 stored in the memory 111, such as the software function modules and computer programs included in the mailbox-based work screening device 110.

[0474] An embodiment of the present invention provides a readable storage medium, which includes a computer program. When the computer program is executed, the computer device where the readable storage medium is located is controlled to execute the aforementioned mailbox-based work screening device 110.

[0475] For illustrative purposes, the foregoing description is made with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. Numerous modifications and variations are possible in accordance with the above teachings. These embodiments are selected and described in order to best illustrate the principles of the present disclosure and its practical application, so that those skilled in the art can best utilize the present disclosure and utilize various embodiments with different modifications to suit the intended specific application.

Claims

1. A method for screening works based on mailboxes, characterized in that: include: Get the original email from the target mailbox server and filter it to get the pending email works; Performing work compliance detection based on the work type of the pending email work; In the case where the work passes the compliance detection, multi-dimensional scoring is performed on the pending email work to obtain a multi-dimensional scoring result of the pending email work; In response to a user review approval indication for the multi-dimensional scoring result, a final screening result of the pending email works is obtained according to the magnitude of the multi-dimensional scoring result.

2. The method according to claim 1, characterized in that The process of obtaining the original email from the target mailbox server and filtering the original email to obtain the pending email works includes: Get the IMAP server address and port number of the target mailbox server; Establishing a connection with the target mailbox server according to the IMAP server address and the port number in combination with the login credentials; Download the original email in the target mailbox server according to the preset submission time range, wherein the original email includes the original email body and the original email attachment; The original email body and the original email attachment are filtered according to preset filtering rules, and when the filtering result indicates that the filtering rules are met, the original email attachment is used as the pending email work.

3. The method according to claim 1, characterized in that The work compliance detection based on the work type of the pending email work includes: Determining a detection standard for the pending email work based on the work type of the pending email work; Based on the detection criteria, the pending email work is subjected to sensitive information detection, duplicate submission detection and subject consistency detection.

4. The method according to claim 3, characterized in that Perform sensitive information detection on the pending email works, including: In the case where the pending email work is a text work, preprocessing and sentiment analysis are performed on the text work, and sensitive information detection is performed in combination with a preset sensitive word library and a preset sensitive word detection model; If the pending email work is an image work, the image work is segmented and sensitive information detection is performed in combination with a preset scene recognition model; if the image work includes text content, text is extracted from the image work, and the text content is preprocessed and sentiment analysis is performed, and sensitive information detection is performed in combination with a preset sensitive word library and a preset sensitive word detection model; In the case where the pending email work is an audio work, the video sound is converted into audio text using ASR, and the audio text is preprocessed and sentiment analysis is performed, and sensitive information detection is performed in combination with a preset sensitive word library and a preset sensitive word detection model; In the case where the pending email work is a video work, ASR is used to convert the video sound into audio text, and the audio text is preprocessed and subjected to sentiment analysis, and sensitive information detection is performed in combination with a preset sensitive word library and a preset sensitive word detection model; the video work is extracted at preset time intervals to obtain multiple video images, and the video images are segmented, and sensitive information detection is performed in combination with a preset scene recognition model; in the case where the video image includes text content, text is extracted from the video image, and the text content is preprocessed and subjected to sentiment analysis, and sensitive information detection is performed in combination with a preset sensitive word library and a preset sensitive word detection model.

5. The method according to claim 3, characterized in that: Perform duplicate submission detection on the pending email works, including: In the case where the pending email work is a text work, the text work is segmented to obtain a plurality of vocabulary units; stop words in the plurality of vocabulary units are removed, and the plurality of vocabulary units are converted into a plurality of text vectors; text similarity comparison and semantic similarity comparison are performed based on the plurality of text vectors and the archived text vectors in the work database, and the duplicate submission detection is performed; In the case that the pending email work is an image work, the SURF algorithm is used to extract the key points and descriptors of the image work; deep feature extraction is performed based on the key points and descriptors to obtain the deep features of the image work; FLANN and Brute-Force algorithms are used to perform feature matching similarity comparison in combination with the archived image works in the work database to perform the duplicate submission detection.

6. The method according to claim 3, characterized in that: Performing a subject match check on the pending email work, including: Extracting keywords and phrases from the pending email works; Use the pre-trained LDA topic model to determine the current topic distribution; Performing grammatical analysis on the keywords and phrases to obtain deep features of the pending email work; The topic consistency detection is achieved by performing a similarity comparison based on the current topic distribution and the deep features.

7. The method according to claim 1, characterized in that The multi-dimensional scoring of the pending email work to obtain the multi-dimensional scoring result of the pending email work includes: In the case where the pending email work is a written work, multi-dimensional scoring is performed based on the BERT model using part-of-speech tagging, syntactic analysis, and sentiment analysis as scoring indicators to obtain a multi-dimensional scoring result of the written work; In the case where the pending email work is an image work or a video work, a multi-dimensional scoring is performed based on the LSTM and Transformer models using the painting style, painting skills and artistic elements as scoring indicators to obtain a multi-dimensional scoring result of the image work or the video work; When the pending email work is a calligraphy work, a multi-dimensional scoring is performed based on the area difference, aspect ratio difference, distance difference and orientation difference between the calligraphy work and the standard characters to obtain a multi-dimensional scoring result of the calligraphy work.

8. A device for selecting works based on mailboxes, characterized in that: include: The acquisition module is used to obtain the original email from the target mailbox server and filter it to obtain the pending email works; A screening module, used for performing work compliance detection based on the work type of the pending email work; In the case where the work passes the compliance detection, multi-dimensional scoring is performed on the pending email work to obtain a multi-dimensional scoring result of the pending email work; In response to a user review approval indication for the multi-dimensional scoring result, a final screening result of the pending email works is obtained according to the magnitude of the multi-dimensional scoring result.

9. A computer device, characterized in that: The computer device comprises a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device executes the method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium includes a computer program, and when the computer program is executed, the computer device where the readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

Images