Data processing method and device

Abstract

The embodiment of the invention provides a data processing method and device, and the method comprises the steps: obtaining initial display information and target content of a first file in response to the obtained first file; obtaining a target material matched with the target content based on an intention recognition result of the first file; and fusing the target material into the first file based on the initial display information to obtain a target file.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of computer technology, and in particular to a data processing method and device. Background Art

[0002] At present, the solutions for text beautification are relatively traditional and can only provide fixed image material tags. Users need to manually select and arrange material tags, which is cumbersome. In addition, when the number of note pages is large, users need to spend more time, which greatly reduces user experience and operation enthusiasm. Summary of the invention

[0003] The embodiment of the present application provides a data processing method and device. The technical solution of the embodiment of the present application is implemented as follows:

[0004] In a first aspect, an embodiment of the present application provides a data processing method, the method comprising: in response to obtaining a first file, obtaining initial display information and target content of the first file; acquiring target material matching the target content based on an intent recognition result of the first file; and integrating the target material into the first file based on the initial display information to obtain a target file.

[0005] In a second aspect, an embodiment of the present application provides a data processing device, which includes: a response module, configured to obtain initial display information and target content of a first file in response to obtaining the first file; an acquisition module, configured to obtain target material matching the target content based on an intent recognition result of the first file; and a fusion module, configured to fuse the target material into the first file based on the initial display information to obtain a target file.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the method described in the embodiment of the present application is implemented.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided in the embodiment of the present application.

[0008] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0010] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0011] Figure 1 A schematic diagram of an implementation process of a data processing method provided in an embodiment of the present application Figure 1 ;

[0012] Figure 2 A schematic diagram of a process for obtaining target material provided in an embodiment of the present application;

[0013] Figure 3 A schematic diagram of a process for obtaining a target file provided in an embodiment of the present application;

[0014] Figure 4 A schematic diagram of an implementation process for determining a tag range provided in an embodiment of the present application Figure 1 ;

[0015] Figure 5 A schematic diagram of an implementation process for determining a tag range provided in an embodiment of the present application Figure 2 ;

[0016] Figure 6 A schematic diagram of an implementation flow of determining the coordinate position of a label image provided in an embodiment of the present application;

[0017] Figure 7 A schematic diagram of an implementation process for determining titles and paragraphs provided in an embodiment of the present application Figure 1 ;

[0018] Figure 8 A schematic diagram of an implementation process for determining titles and paragraphs provided in an embodiment of the present application Figure 2 ;

[0019] Fig. 9 A schematic diagram of an implementation flow of determining the coordinate position of a pendant placed at a title provided in an embodiment of the present application;

[0020] Fig.10A schematic diagram of an implementation flow of determining the coordinate position of a pendant placed at a paragraph provided in an embodiment of the present application;

[0021] Fig.11 A schematic diagram of an implementation process of a data processing method provided in an embodiment of the present application Figure 2 ;

[0022] Fig.12 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0023] Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0025] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0026] It should be pointed out that the terms "first\second\third" involved in the embodiments of the present application are merely to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0027] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those skilled in the art in the field to which the embodiments of the present application belong. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0028] The present application embodiment provides a data processing method. Figure 1 A schematic diagram of an implementation process of a data processing method provided in an embodiment of the present application Figure 1 ,like Figure 1 As shown, the method includes the following steps 101 to 103:

[0029] Step 101, in response to obtaining a first file, obtaining initial display information and target content of the first file;

[0030] Step 102, acquiring a target material matching the target content based on the intent recognition result of the first file;

[0031] Step 103: merge the target material into the first file based on the initial display information to obtain the target file.

[0032] It can be understood that in the embodiment of the present application, the initial display information and target content of the first file are obtained; the target material matching the target content is obtained based on the intention recognition result of the first file; the target material is merged into the first file based on the initial display information to obtain the target file. Since the target material is merged into the first file based on the initial display information of the first file, rather than providing the target material directly to the user for merging by the user, the target material is not merged with the first file at will. In this way, it is beneficial to ensure that the original format or display layout of the first file will not be destroyed after the target material is inserted, while improving the display effect of the target file and reducing the user's operating costs.

[0033] As a possible implementation method, if the display content of the first file is "This mountain stands on a vast land, as if it is a masterpiece of nature. It is located on the edge of a lush forest, echoing the mountains in the distance. The mountain is tall and majestic. The peak reaches the sky. The rocks on the mountain are hard and rough. The mountains are crisscrossed with gullies and streams, adding a bit of agility and vitality to the mountain.", based on the intent recognition result of the first file, the target material matching the target content is obtained, which can be an image of the mountain; among them, the image of the mountain can be used as a pendant of the first file, or as a background watermark of the first file, etc.

[0034] The following describes further optional implementations and related terms of each of the above steps.

[0035] In step 101, in response to obtaining a first file, initial display information and target content of the first file are obtained.

[0036] It should be understood that in the embodiments of the present application, the first file is not limited. In some embodiments, the first file is a file generated by an artificial intelligence (AI) generation model; in other embodiments, the first file is a file uploaded by a user. The file type of the first file is also not limited. In some embodiments, the first file is a text file. In other embodiments, the first file is an image file. In some other embodiments, the first file is a multimodal data file.

[0037] In an embodiment of the present application, for the first file in "obtaining the initial display information and target content of the first file", it includes: obtaining the display parameters and display content of the image converted from the first file; or, obtaining the display parameters and content information included for each frame of the multiple images converted from the first file.

[0038] In the embodiment of the present application, the initial display information is not limited. The initial display information includes at least one of the following: display layout parameters, layout design information, etc. The target content is also not limited. The target content includes at least one of the following: keywords, key image features.

[0039] In some embodiments, the initial display information may be information such as a user-defined text generation layout or display format, or may be different templates or layouts for different scenarios provided by default by an application (eg, a learning zone application or an AI learning application).

[0040] In some embodiments, in response to obtaining a first file, obtaining initial display information and target content of the first file includes:

[0041] In response to obtaining a first document file generated by the AI ​​generation model, obtaining display parameters and display content of a first image converted from the first document file, so as to obtain display layout parameters of the first document file, layout design information, keywords corresponding to the display content, and key image features corresponding to the first image;

[0042] In response to obtaining a first document file generated by the AI ​​generation model, obtaining display parameters and display content of each frame of the plurality of first images converted from the first document file, so as to obtain display layout parameters of the first document file, layout design information, keywords corresponding to the display content, and key image features corresponding to each frame of the plurality of first images;

[0043] In response to obtaining a second document file uploaded by a user, obtaining display parameters and display content of a second image converted from the second document file, so as to obtain display layout parameters of the second document file, layout design information, keywords corresponding to the display content, and key image features corresponding to the second image;

[0044] In response to obtaining a second document file uploaded by a user, obtaining display parameters and display content of each frame of the plurality of second images converted from the second document file, so as to obtain display layout parameters of the second document file, layout design information, keywords corresponding to the display content, and key image features corresponding to each frame of the plurality of second images;

[0045] In response to obtaining a third image generated by the AI ​​generation model, obtaining display parameters and display content of the third image to obtain display layout parameters of the third image, layout design information, keywords corresponding to the display content of the third image, and key image features corresponding to the third image;

[0046] In response to obtaining a plurality of third images generated by the AI ​​generation model, display parameters and display content of each frame of the plurality of third images are obtained to obtain display layout parameters of the plurality of third images, layout design information, keywords corresponding to the display content, and key image features corresponding to each frame of the plurality of third images;

[0047] In response to obtaining a fourth image uploaded by a user, obtaining display parameters and display content of the fourth image, so as to obtain display layout parameters of the fourth image, layout design information, keywords corresponding to the display content of the fourth image, and key image features corresponding to the fourth image;

[0048] In response to obtaining multiple fourth images uploaded by a user, display parameters and display content of each frame of the multiple fourth images are obtained to obtain display layout parameters, layouts, design information of the multiple fourth images, keywords corresponding to the display content, and corresponding key image features of each frame of the multiple fourth images.

[0049] In some embodiments, obtaining the initial display information and target content of the first file includes: identifying image parameters of an image file corresponding to the first file to obtain the initial display information; and determining the target content from text content and / or image content included in the first file based on target reference information.

[0050] Regarding "identifying the image file corresponding to the first file", in an embodiment of the present application, if the first file is an image file, the identified image file can be the first file itself, or it can be an image file after noise reduction or preprocessing of the first file; if the first file is a text file, the first file needs to be converted into an image file for identification to obtain display output parameters of fixed text content.

[0051] It should be understood that in the embodiments of the present application, the image parameters are not limited. In some embodiments, the image parameters include: attribute information of image elements and association information between image elements; wherein the attribute information includes: type information, size, shape, color and other information; the association information includes: primary and secondary relationships, symmetry, hierarchy, logical order, display rules and other information.

[0052] In the embodiments of the present application, the initial display information is not limited. In some embodiments, the initial display information includes: initial display layout or typesetting design information.

[0053] In the embodiments of the present application, the target reference information is not limited. In some embodiments, the target reference information includes: full-text word frequency statistics, usage information of the first file, user portrait information of the user who performs the generation task, theme and type of the first file, and other information.

[0054] In the embodiment of the present application, the text content and / or image content included in the first file is not limited. As a possible real-time method, the first file can be a text file or an image file or a file including text and images in multimedia, such as a web page file; the determined target content can be a word with a frequency of 10 or 20 times; it can also be a word or image feature that is highly relevant to the user's occupation or needs; it can also be text or image content that matches the purpose or adapts to a specific scene.

[0055] As a possible implementation method, if the user inputs a PDF document converted from a recording file, the PDF document is converted into a first image according to a preset text generation layout; the first image includes the voice content in the recording file; the voice content is "The mountain stands on a vast land, as if it is a masterpiece of nature. It is located on the edge of a lush forest, echoing the mountains in the distance. The mountain is tall and majestic. The peak reaches the sky. The rock of the mountain is hard and rough. The gullies and streams in the mountains add a bit of agility and vitality to the mountain." Based on the intention recognition result of the first file, the target material matching the target content is obtained, which can be an image of the mountain; wherein the image of the mountain can be used as a pendant of the first file, or as a background watermark of the first file, etc.; the size information of the first image and the arrangement information of the text content in the first image (for example: the coordinate position of the characters in the text content, the area occupied, the line spacing, whether it is bold, and other feature information) are obtained. According to the full-text word frequency statistics in the text content, it is determined that the target content of the text content is the mountain.

[0056] In some embodiments, identifying the image parameters of the image file corresponding to the first file to obtain the initial display information includes at least one of the following:

[0057] In the case where the first file is an image file, identifying attribute information of image elements in the first file and association information therebetween, so as to obtain the initial display information based on the attribute information and the association information;

[0058] In the case where the first file is a text file, converting the text file into an image file, and identifying display layout information between text contents in the image file as the initial display information;

[0059] In the case where the first file is a multimodal data file, converting the multimodal data file into a multi-layer image file, identifying attribute information of each modal data in the multi-layer image file and association information therebetween, so as to obtain the initial display information based on the attribute information and the association information;

[0060] In the case where the first file is a text file and is converted into a plurality of frames of images, display layout information between text contents in each frame of images is identified as the initial display information.

[0061] In the embodiment of the present application, the image elements may be understood as element objects constituting an image file, such as various objects, graphics, and the like.

[0062] Regarding “when the first file is an image file, identifying the attribute information of the image elements in the first file and the association information therebetween, so as to obtain the initial display information based on the attribute information and the association information”, in an embodiment of the present application, when the first file is an image file, directly identifying the type information, size, shape, color, primary and secondary relationship, symmetry, hierarchy, logical order, display rules, etc. of the element objects (e.g., various objects or graphics in the image file) constituting the image file in the image file, so as to determine the display layout of the image elements in the image based on the type information, size, shape, color, primary and secondary relationship, symmetry, hierarchy, logical order, display rules of the element objects (e.g., various objects or graphics in the image), and determining the display layout as the initial display information;

[0063] Alternatively, the first file is subjected to image preprocessing (for example, noise reduction, grayscale processing, etc.), and the type information, size, shape, color, primary and secondary relationships, symmetry, hierarchy, logical order, display rules, etc. of the element objects (for example, various objects or graphics in the image) of the preprocessed image file are identified, so as to determine the display layout of the image in the image based on the type information, size, shape, color, primary and secondary relationships, symmetry, hierarchy, logical order, and display rules of the element objects (for example, various objects or graphics in the image), and determine the display layout as the initial display information.

[0064] It should be understood that in the embodiment of the present application, the format of the first text file is not limited. For example, the format of the first text file includes at least one of the following: word document, txt document, excel document, PDF document, etc.; the display layout information is also not limited, and the display layout information can be typesetting information or format information between text contents in the text file, etc.

[0065] Regarding “when the first file is a text file, converting the text file into an image file, and identifying the display layout information between the text contents in the image file as the initial display information”, in an embodiment of the present application, when the first file is a text file, the text file (for example: word document, txt document, excel document, PDF document) is converted into an image file, and identifying the typesetting information or format information between the text contents in the image file as the initial display information.

[0066] It should be understood that in the embodiment of the present application, the multimodal data file is not limited, and the multimodal data file can be web page data, PPT data, data including text data, image data, audio data, video data, etc. In the embodiment of the present application, the multi-layer image file can be understood as storing data of different modalities in different layers to form a multi-layer file.

[0067] Regarding "when the first file is a multimodal data file, converting the multimodal data file into a multi-layer image file, identifying the attribute information of each modal data in the multi-layer image file and the association information therebetween, so as to obtain the initial display information based on the attribute information and the association information", in an embodiment of the present application, when the first file is web page data, PPT data, video data, or any one of data including text data, image data, and audio data, the multimodal data file is saved in different layers to form a multi-layer file, identifying the attribute information of each modal data in the multi-layer image file and the association information therebetween, so as to obtain the initial display information based on the attribute information and the association information.

[0068] Regarding “when the first file is a text file and is converted into multiple frames of images, identifying the display layout information between the text contents in each frame of the image as the initial display information”, it can be understood that when the first file is a text file and is converted into multiple frames of images, the initial display information of each frame of the image is obtained.

[0069] As a possible implementation manner, if the first file is an image file, the image file includes: mountains, trees, water, sun, birds, and clouds, wherein the initial display information of the first file includes: the size, shape, color, and primary and secondary relationships, symmetry, hierarchy, logical order, display rules, etc. of the mountains, trees, water, sun, birds, and clouds, thereby determining the display layout of the image file, that is, the mountains are the main body, and the trees, water, sun, birds, clouds, etc. are auxiliary information.

[0070] In some embodiments, determining the target content from the text content and / or image content included in the first file based on the target reference information includes at least one of the following:

[0071] In the case where the first file is a text file, performing word frequency statistics processing on the first file, and determining text content with a word frequency greater than a first threshold as the target content;

[0072] In the case where the first file is image content, performing area percentage statistics on the first file, and determining image content whose area percentage is greater than a second threshold or less than a third threshold as the target content;

[0073] In the case where the first file is image content, determining the image content in the first file whose resolution is within a fourth threshold range as the target content;

[0074] Obtaining usage information of the target file, and determining text content and / or image content in the first file that matches the usage information as the target content;

[0075] Obtaining user portrait information of a target user, and determining text content and / or image content in the first file that matches the user portrait information as the target content, wherein the target user includes a provider of the first file and / or a recipient of the target file;

[0076] The document template information selected by the target user is obtained, and the target content is determined based on the document template information.

[0077] It should be understood that in the embodiment of the present application, "when the first file is a text file, the first file is subjected to word frequency statistical processing, and the text content with a word frequency greater than a first threshold is determined as the target content" can be understood as filtering the words in the text of the first file according to the word frequency corresponding to the words. In the embodiment of the present application, the first threshold is not limited.

[0078] As a possible implementation, if the image converted from the first file includes multiple images, the text content corresponding to each frame of the multiple images is segmented and named entity recognized, and relevant parts of speech such as nouns and gerunds are extracted, and stop words are removed. The extracted word set is counted for word frequency to obtain statistical results. Based on the statistical results, the words with the highest word frequency or the words with the highest word frequency are selected as target content.

[0079] In the embodiment of the present application, "when the first file is image content, the area ratio of the first file is counted, and the image content with an area ratio greater than the second threshold or less than the third threshold is determined as the target content" can be understood as, for the image file, the image elements in the image file are screened according to the area ratio of the image elements to beautify or modify the first file. In this embodiment of the present application, the second threshold and the third threshold are not limited.

[0080] In the embodiment of the present application, "when the first file is image content, the image content in the first file whose resolution is within the fourth threshold range is determined as the target content" can be understood as screening the target content from the first file according to the resolution, so as to beautify or modify the target content with too low or too high resolution to improve the overall image display effect. In the embodiment of the present application, the fourth threshold range is not limited.

[0081] In some embodiments, target content may also be filtered from the first file according to pixel color value or style.

[0082] In an embodiment of the present application, "obtaining usage information of the target file, and determining the text content and / or image content in the first file that matches the usage information as the target content" can be understood as screening according to the specific usage scenario or purpose of the target file to obtain materials or decorations that match the user information, thereby improving the recognition or prominence of specific content.

[0083] As a possible implementation method, when doing environmental protection publicity, you can add relevant materials or pendants to green-related content (content related to environmental protection) to highlight the theme of green environmental protection.

[0084] In an embodiment of the present application, the user portrait information in "obtaining user portrait information of a target user, determining the text content and / or image content in the first file that matches the user portrait information as the target content, and the target user includes the provider of the first file and / or the recipient of the target file" can be understood as user habits or user preferences, and user habits or user preferences can be determined based on historical data; that is, the target content is determined based on historical data to screen out materials or widgets that conform to user habits or preferences.

[0085] In the embodiment of the present application, the file template information in "obtaining the file template information selected by the target user, and determining the target content based on the file template information" can be understood as the beautification template selected by the user. The embodiment of the present application takes the beautification template into consideration because the beautification template may cause the content that the user wants to beautify to shift, such as the color tone or style of the template will affect the selection of materials or widgets. In this way, it is beneficial to make the determined target content more in line with the actual situation.

[0086] As a possible implementation method, if the first file includes the content "result of today's football match, B lost to C with a big score of 0 to 7, the number of people watching today's football match is 500, and the athletes participating in the football match are D, E, F, etc.", then, the first file is processed with word frequency statistics, and the football match is taken as the target content.

[0087] In step 102, a target material matching the target content is acquired based on the intent recognition result of the first file.

[0088] It should be understood that in the embodiment of the present application, obtaining the target material that matches the target content based on the intent recognition result of the first file can be understood as screening the target material that matches the target content within the material range limited by the intent recognition result.

[0089] In some embodiments, Figure 2 A schematic diagram of a process for obtaining target material provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the target material can be obtained through the following steps 201 to 203:

[0090] Step 201, performing intent recognition processing on the text content and / or image content included in the first file to obtain intent recognition keywords;

[0091] Step 202, filtering out a first material set from a target material library based on the intent recognition keyword;

[0092] Step 203: Acquire target material matching the target content from the first material set.

[0093] It should be understood that in the embodiments of the present application, the specific implementation of the intent recognition process is not limited. The intent recognition process can be implemented by a deep learning method (for example: a neural network model, a Transformer algorithm model, etc.) or by combining a deep learning model with a traditional algorithm. The traditional algorithm can be: regular processing or fuzzy matching of text.

[0094] As a possible implementation method, the text content extracted from the text file or recording file input by the user is combined with the Transformer algorithm model to perform text content intent recognition, and the recognized intent keywords are converted into vectors by text embedding, and the similarity between the vectorized tags and the vectorized tags of the material library is calculated to locate the extraction range of the material tags (i.e., an example of the first material set). Then, the target material matching the target content is obtained from the first material set.

[0095] As a possible implementation, if the first file includes the content "Today's football match result, B lost to C with a score of 0 to 7, the number of spectators of today's football match is 500, and the athletes participating in the football match are D, E, F, etc.", then the intent recognition keywords of the first file are game, sports, and exercise. Based on game, sports, and exercise, the materials corresponding to the game and the materials corresponding to the exercise are screened out from the target material library, and the materials corresponding to the game and the materials corresponding to the exercise are used as the first material set, and the target material matching the football match is obtained from the material set corresponding to the game and the material set corresponding to the exercise.

[0096] It should be understood that in the embodiment of the present application, the materials corresponding to the game and the materials corresponding to the sports include multiple materials, not just one.

[0097] In some embodiments, the method further includes: filtering out a second material set from the target material library based on the target content, so as to determine the target material from the second material set when the first material set cannot match the target content.

[0098] It can be understood that in the embodiment of the present application, not only the target material matching the target content is obtained from the first material set, but also the second material set is screened out from the target material library based on the target content, so that when the first material set cannot match the target content, the target material is determined from the second material set. This is beneficial to ensure that the target material matches the target content, thereby improving the display effect of the target file and reducing the user's operating cost.

[0099] It should be understood that in the embodiment of the present application, filtering out a second material set from the target material library based on the target content includes: filtering out materials from the target material library whose similarity with the target content is greater than or equal to a fourth threshold to obtain a second material set; wherein, the fourth threshold is not limited in the embodiment of the present application.

[0100] As a possible implementation method, if the display content of the first file also includes "In the passionate football game, the cheerleading team is undoubtedly the most beautiful and dynamic scenery. Following the rhythm of the game, the cheerleaders cheer for the athletes on the field with uniform, rhythmic dances and movements. The cheerleading team's slogans are loud and contagious", then the target content also includes the cheerleading team, and the second material set is selected from the target material library based on the cheerleading team, for example: the material corresponding to the cheerleading team: for example: pictures of the cheerleading team, artistic words of the cheerleading team, etc.

[0101] In step 103, the target material is merged into the first file based on the initial display information to obtain a target file.

[0102] It should be understood that in the embodiment of the present application, the reason why the target material is integrated into the first file based on the initial display information to obtain the target object is because the initial display information determines the embedding position of the target material. Therefore, based on the initial display information, the target material is integrated into the first file, so that the original format or display layout of the first file will not be destroyed after the target material is inserted.

[0103] In some embodiments, Figure 3 A schematic diagram of a process flow for obtaining a target file provided in an embodiment of the present application is as follows: Figure 3 As shown, the target file can be obtained through the following steps 301 to 303:

[0104] Step 301, determining fusion position information of a target material based on initial display information;

[0105] Step 302, processing the first file into a first intermediate image with a first transparency;

[0106] Step 303: fuse the target material to the corresponding position of the first intermediate image based on the fusion position information to obtain the target file.

[0107] As a possible implementation method, the saved note image (i.e., an example of the first file) is converted into a transparent image (i.e., an example of the first intermediate image) through Python image processing library PIL (Python Imaging Library, PIL) image processing technology, and the matched pendant image (i.e., an example of the target material) is displayed at the corresponding position of the converted transparent image according to the calculated coordinates (i.e., an example of fusion position information).

[0108] As a possible implementation, if the background of the first image is green, the background in the first image is processed to be transparent to obtain a first intermediate image with a transparent background, and then the target material is merged into the corresponding position of the first intermediate image.

[0109] In some embodiments, the method further comprises at least one of the following:

[0110] In response to obtaining an operation of a target user selecting a target file template, fusing the first intermediate image with a template image corresponding to the target file template into a second intermediate image, and fusing the target material to a corresponding position of the second intermediate image to obtain the target file;

[0111] In the case where the first file is converted into multiple frames of images, the material fusion position and the material fusion quantity of each frame of image are determined based on the initial display information and the text content of each frame of image.

[0112] As a possible implementation method, the saved note image (i.e., an example of the first file) is converted into a transparent image (i.e., an example of the first intermediate image) through PIL image processing technology, the transparent image and the beautification template image (i.e., an example of the template image corresponding to the target file template) are superimposed to synthesize a new image with a user-selected background (i.e., an example of the second intermediate image), and the matched pendant image (i.e., an example of the target material) is displayed at the corresponding position of each image according to the calculated coordinates (i.e., an example of fusion position information), which can support user secondary editing.

[0113] As a possible implementation, the template image corresponding to the target file template selected by the user is left-aligned in the title, and the font size is 2, the subtitle is displayed below the title, the font size of the subtitle is 4, the main text is displayed three lines below the subtitle, and the paragraph spacing of the main text is double spacing. The first intermediate image with a transparent background is merged with the template image described above to form a second intermediate image, and then the target material is merged to the corresponding position of the second intermediate image.

[0114] In some embodiments, determining the fusion position information of the target material based on the initial display information includes: obtaining appearance parameters of the target material; and determining the fusion position of the target material in the image corresponding to the first file based on image parameters of the image corresponding to the first file and the appearance parameters.

[0115] It should be understood that in the embodiments of the present application, the appearance parameters of the target material are not limited. In some embodiments, the appearance parameters of the target material include one or more of the following: material shape, width, height and other parameters.

[0116] In the embodiments of the present application, the image parameters of the image corresponding to the first document are not limited. In some embodiments, the image parameters of the image corresponding to the first document include one or more of the following: the first width of the image corresponding to the first document, the position of the data content of the image corresponding to the first document, the line spacing of the data content of the image corresponding to the first document, the paragraph information of the data content of the image corresponding to the first document, and other parameters.

[0117] In some embodiments, the image parameters of the image corresponding to the first document include: the first width of the image corresponding to the first document, the position of the data content, and the line spacing of the data content; the appearance parameters of the target material include: the second width of the target material and the first height of the target material; determining the fusion position of the target material in the image corresponding to the first document based on the image parameters of the image corresponding to the first document and the appearance parameters includes:

[0118] Determine a third width based on the first width and the position of the data content;

[0119] If the second width is less than or equal to the third width and the first height is less than or equal to the line spacing of the data content, determine the first position of the target material as the position in the first region.

[0120] It should be understood that in the embodiments of the present application, the third width is the boundary of the image file and the width of the character closest to the boundary.

[0121] As a possible implementation, assume that the upper left coordinate of the title is (100, 100), the lower right coordinate is (300, 150), the width of the image is 500, the width of the pendant is 50, and the height is 30. Calculation of the pendant placement position at the title: The width of the title is Wt = 300 - 100 = 200. The blank width on the left side of the title is WI = 100, and the blank width on the right side is Wr = 500 - 300 = 200. Since Wl > Wg and Wr > Wg, the pendant is placed on the right side of the title, and the upper left coordinate of the pendant is (300, 100). Let the upper left coordinate of the title be (X1, Y1), the lower right coordinate be (X2, Y2), the width of the title be Wt = X2 - X1, and the height be Ht = Y2 - Y1. Let the width of the pendant be Wg and the height be Hg. The blank width on the left side of the title is Wl = X1, and the blank width on the right side is Wr = lw - X2, where lw is the width of the image. If Wl > wg and Wr > wg, the pendant is placed on the right side of the title, and the upper left coordinate of the pendant is (X2, Y1). If Wr < Wg, the pendant is placed on the left side of the title, and the upper left coordinate of the pendant is (X1 - Wg, Y1). If neither side satisfies the width of the pendant, the pendant is not placed.

[0122] As a possible implementation, an example is given: Suppose the lower left corner coordinates of the last line in the main text paragraph are (200, 300), and the line spacing is 40. The width of the blank at the end of the main text paragraph is Wb = 500 - 200 = 300. Since Wb > wg and Hg < d, the pendant is placed at the end of the main text paragraph, and the lower left corner coordinates of the pendant are (200, 300 + 150) = (200, 450). Let the lower left corner coordinates of the last line in the main text paragraph be (X3, Y3), and the line spacing be d. The width of the blank at the end of the main text paragraph is Wb = lw - X3, where lw is the width of the image. If Wb > Wg and Hg < d, the pendant is placed at the end of the main text paragraph, and the lower left corner coordinates of the pendant are (X3, Y3 + Ht). If the above conditions are not met, the pendant is not placed.

[0123] The following describes an exemplary application of the processing method provided in the embodiments of the present application in an actual scenario.

[0124] The embodiments of the present application provide a data processing method, which can automatically beautify personal notes with text or pictures (i.e., an example of the first file) as the main body. This method can use document parsing and optical character recognition (OCR) technology to extract the text content and the text content involved in the pictures. After extraction, the note format is uniformly converted into pictures. The text content described in the note is classified through an intention recognition model, and the label pictures (i.e., an example of the target material) that best match the intention classification in the material library (i.e., an example of the target material library) are matched through vector calculation. The label pictures and the note pictures (i.e., an example of the first file) are synthesized and superimposed according to certain rules, so as to achieve the effect of automatic beautification.

[0125] The data processing method provided in the embodiments of the present application extracts the text content by obtaining the text file or recording file uploaded by the user, and performs processing such as intention recognition, text segmentation, named entity recognition, word frequency statistics, OCR recognition, text embedding, similarity calculation, and image synthesis on the extracted text content, and realizes the synthesis of the background template, so as to automatically match the label pictures that match the text content, and automatically place the label pictures in reasonable positions, so as to achieve the beautification effect.

[0126] Among them, the intention recognition algorithm recognizes the note intention and extracts keywords, including:

[0127] Obtain text content extracted from a text file or recording file input by the user, input the extracted text content into the Transformer algorithm model for content intent recognition, and obtain intent keywords; convert the obtained intent keywords into vectors by text embedding, and perform similarity calculation with the vectorized labels of the material library to locate the extraction range of the label image (i.e., an example of the first material set, referred to as the label range).

[0128] Figure 4 A schematic diagram of an implementation process for determining a tag range provided in an embodiment of the present application Figure 1 ,like Figure 4 As shown, the tag range can be determined through the following steps 401 to 403:

[0129] Step 401, receiving a text file uploaded by a user, and extracting the text content by text parsing means;

[0130] Step 402: Input the text content into the Transformer model for intent recognition and extract the intent keywords;

[0131] As a possible implementation method, if the extracted text content is "Today's football game results, B lost to C with a big score of 0 to 7", the intent keywords extracted according to the intent algorithm model may be "football", "sports", and "athletes".

[0132] Step 403, embed the keywords into text and convert them into vectors, perform similarity calculation with the label vectors in the material library, and determine the label range.

[0133] Figure 5 A schematic diagram of an implementation process for determining a tag range provided in an embodiment of the present application Figure 2 ,like Figure 5 As shown, the first material library can be determined through the following steps 501 to 506:

[0134] Step 501, embed "football", "sports", and "athlete" into text; for example, "football": (a1, a2, a3...an), "sports": (b1, b2, b3...bn);

[0135] Step 502, label vectors in the material library, "sports": (c1, c2, c3...cn), "playing football": (d1, d2, d3...dn), "shooting": (e1, e2, e3...en)...

[0136] Step 503, calculating the vector similarity between the keyword and the tag by using cosine similarity;

[0137]

[0138] Step 504, the calculated results are as follows: "shooting": 0.763, "playing football": 0.982, "sports": 0.936...;

[0139] Step 505, introducing a reordering algorithm for the similarity results to reorder the results; S = w1·S1+w2·S2;

[0140] Step 506, after rearrangement, the final result is "playing football", "exercise".

[0141] Among them, determining the coordinate position of the label image includes: using OCR recognition technology to distinguish the title and text in the note after converting the note content into an image, extracting the coordinate position, and reasonably placing the matching material image at the corresponding coordinate position.

[0142] Figure 6 A schematic diagram of an implementation flow of determining the coordinate position of a label image provided in an embodiment of the present application, such as Figure 6 As shown, the coordinate position of the label image can be determined through the following steps 601 to 604:

[0143] Step 601, grayscale the input image to reduce computational complexity;

[0144] The input image is the image that the text file or recording file input by the user is converted into;

[0145] Step 602, identifying titles and paragraphs in the image;

[0146] Step 603, calculating the coordinate position of the hanging widget at the title;

[0147] Among them, the pendant is an example of a label image;

[0148] Step 604, calculate the coordinate position of the pendant at the paragraph.

[0149] In some embodiments, suppose there is a picture, and after OCR recognition, the following text regions are obtained: Text region 1: The coordinates are (10, 10, 100, 30), the font is large and bold, and it is determined to be the title "Example Title". Text region 2: The coordinates are (10, 40, 200, 60), and the paragraph content is "This is an example paragraph".

[0150] Figure 7 A schematic diagram of an implementation process for determining titles and paragraphs provided in an embodiment of the present application Figure 1 ,like Figure 7 As shown, the title and paragraph can be determined through the following steps 701 to 702:

[0151] Step 701, using paddleOCR technology to extract the coordinates and content of the text from the pre-processed image;

[0152] Step 702: Analyze the text features in the OCR recognition result, such as font size, bold, centering, etc., to confirm the title and text coordinates.

[0153] Figure 8 A schematic diagram of an implementation process for determining titles and paragraphs provided in an embodiment of the present application Figure 2 ,like Figure 8 As shown, the title and paragraph can be determined through the following steps 801 to 805:

[0154] Step 801, determining the boundaries of titles and paragraphs by analyzing information such as character coordinates and line spacing;

[0155] Step 802, a function G(C) may be defined, where C is a set of characters, and the output value of the function represents the possibility that the set of characters is a title;

[0156] Step 803, determining a group of characters with similar ordinates, continuous abscissas, large fonts, bold fonts and other obvious features as a title;

[0157] Step 804, determining the paragraphs by line spacing and character continuity;

[0158] Step 805, obtaining the coordinates of the title and paragraph.

[0159] Fig. 9 A schematic diagram of a process for determining the coordinate position of a pendant placed at a title provided in an embodiment of the present application, such as Fig. 9 As shown, the coordinate position of the widget placed at the title can be determined through the following steps 901 to 905:

[0160] Step 901, the coordinates of the title and the width and height of the widget are substituted into the formula for calculation;

[0161] Step 902, determine whether the right blank is greater than the widget width; if so, execute step 903, otherwise execute step 904;

[0162] Step 903, placing the pendant on the right side;

[0163] Step 904, determine whether the left side is larger than the width of the pendant; if so, execute step 905, otherwise end;

[0164] Step 905, place the pendant on the left side.

[0165] As a possible implementation, assume that the upper left corner coordinates of the title are (100, 100), the lower right corner coordinates are (300, 150), the width of the image is 500, the width of the widget is 50, and the height is 30. Calculation of the widget placement position at the title: The width of the title is Wt = 300 - 100 = 200. The blank width on the left side of the title is WI = 100, and the blank width on the right side is Wr = 500 - 300 = 200. Since Wl > Wg and Wr > Wg, the widget is placed on the right side of the title, and the upper left corner coordinates of the widget are (300, 100). Let the upper left corner coordinates of the title be (X1, Y1), the lower right corner coordinates be (X2, Y2), the width of the title be Wt = X2 - X1, and the height be Ht = Y2 - Y1. Let the width of the widget be Wg and the height be Hg. The blank width on the left side of the title is Wl = X1, and the blank width on the right side is Wr = lw - X2, where lw is the width of the image. If Wl > wg and Wr > wg, the widget is placed on the right side of the title, and the upper left corner coordinates of the widget are (X2, Y1). If Wr < Wg, the widget is placed on the left side of the title, and the upper left corner coordinates of the widget are (X1 - Wg, Y1). If the width of the widget is not satisfied on both sides, the widget is not placed.

[0166] Fig.10 The following is a schematic diagram of the implementation process for determining the coordinate position of the widget placement at the paragraph provided by the embodiment of the present application. As Fig.10 shown, the coordinate position of the widget placement at the paragraph can be determined through the following steps 1001 to 1004:

[0167] Step 1001, calculate by substituting the coordinates of the main text and the width and height of the widget into the formula;

[0168] Step 1002, determine whether the blank width at the end of the paragraph is greater than the width of the widget; if so, execute step 1002, otherwise end;

[0169] Step 1003, determine whether the height of the widget is less than the line spacing; if so, execute step 1004, otherwise end;

[0170] Step 1004, place the widget at the end of the paragraph.

[0171] As a possible implementation, assume that the lower left corner coordinates of the last line in the body text paragraph are (200, 300), and the line spacing is 40. The width of the blank at the end of the body text paragraph is Wb = 500 - 200 = 300. Since Wb > wg and Hg < d, the pendant is placed at the end of the body text paragraph, and the lower left corner coordinates of the pendant are (200, 300 + 150) = (200, 450). Let the lower left corner coordinates of the last line in the body text paragraph be (X3, Y3), and the line spacing be d. The width of the blank at the end of the body text paragraph is Wb = lw - X3, where lw is the width of the image. If Wb > Wg and Hg < d, the pendant is placed at the end of the body text paragraph, and the lower left corner coordinates of the pendant are (X3, Y3 + Ht). If the above conditions are not met, the pendant is not placed.

[0172] It can be understood that in the embodiments of the present application, intent recognition can be performed according to the note content uploaded by the user, and relevant tag images can be automatically matched, improving the relevance between the tag images and the text content, reducing the user's screening cost, and also reducing the problem of low matching rate between the content input during search and the material library. And it can automatically identify a reasonable placement position according to the format of the content, increasing the aesthetics, reducing the user's operations, solving the problem of user operation fatigue caused by too many document pages, thus exhausting the user's enthusiasm for beautification and making the beautification function a "burden".

[0173] In some embodiments, the data processing method can be implemented through the following steps 1 to 8:

[0174] Step 1, obtain the note content uploaded by the user. After extracting the text content, convert it into a picture according to the original content format in the user's note and save it. Each page of the multi-page content will be saved as a picture.

[0175] Step 2, intent recognition: Establish an intent recognition model using a method that combines "rule matching" and "Transformer (GPT)". Input the extracted note content into the intent recognition model to obtain the intent of the content expressed by the note, and circle the range of tag images required for beautifying the note through the intent.

[0176] Step 3, perform OCR recognition technology to distinguish the title and body text parts, and calculate the positions and quantities where pendants can be placed on each page of the picture.

[0177] Step 4, text tokenization: Tokenize the text content of each page, perform named entity recognition, extract nouns, gerunds and other related parts of speech, and remove stop words.

[0178] Step 5, word frequency statistics: Perform word frequency statistics on the set of extracted words and save the results.

[0179] Step 6, keyword extraction: match the words in the word frequency statistics results with the tags locked by intent recognition in the subordinate library, and put the directly matched results into set A. If the length of set A does not meet the number of widgets, the remaining part is taken according to the number of widgets that can be placed on each page, and TOP (len (number of widgets) - len (set A)) is stored in set B for subsequent tag matching calculations. And ensure that the widget tags on each page are not repeated.

[0180] Step 7, text embedding and similarity calculation: The extracted keyword set B is embedded in the text, and the cosine similarity calculation of the vector is performed with the text embedding of the image name under the intent recognition confirmation label image to obtain the similarity score. Finally, the final pendant material is obtained according to the rearrangement model.

[0181] Step 8, image synthesis and overlay: convert the saved note image into a transparent image through PIL image processing technology, overlay the transparent image with the beautification template image to synthesize a new image with a user-selected background, and finally display the matched widget image in the front section at the corresponding position of each image according to the calculated coordinates, which supports secondary editing by users.

[0182] Fig.11 A schematic diagram of an implementation process of a data processing method provided in an embodiment of the present application Figure 2 ,like Fig.11 As shown, the method includes the following steps 1101 to 1109:

[0183] Step 1101, parse the content of the note uploaded by the user, and save it as a picture (multiple pages, one picture per page) in the original format;

[0184] Step 1102, intent recognition (rule matching + Transformer (GPT)) locks the tag range;

[0185] Step 1103, OCR recognizes and distinguishes the title and the text, and calculates the position and quantity of the pendants;

[0186] Step 1104, text segmentation, named entity recognition, and stop word removal;

[0187] Step 1105, word frequency statistics, extracting high-frequency words;

[0188] Step 1106, extracting intent keywords (matching with intent identification tags, subsets A and B);

[0189] Among them, the label pictures in set A are used as the widgets to be output;

[0190] Step 1107, text embedding and similarity calculation (for set B, cosine similarity calculation is performed with the text embedding of the picture name under the label picture) to obtain the final widget to be output;

[0191] Step 1108: Convert the saved note image into a transparent image (PIL image processing) and overlay it with the user-specified background template

[0192] Step 1109, the synthesized image and the widget image are displayed on the front end according to the calculated coordinate positions, which can be edited by the user.

[0193] It should be noted that although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.; or, steps in different embodiments may be combined into a new technical solution.

[0194] Based on the foregoing embodiments, an embodiment of the present application provides a data processing device, which includes the modules included and the units included in the modules, which can be implemented by a processor; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be an AI acceleration engine (such as NPU, etc.), GPU, central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP) or field programmable gate array (FPGA), etc.

[0195] Fig.12 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application is shown in FIG. Fig.12 As shown, the data processing device 120 includes a response module 1201, an acquisition module and a fusion module 1203, wherein:

[0196] A response module 1201 is configured to obtain initial display information and target content of the first file in response to obtaining the first file;

[0197] An acquisition module 1202 is configured to acquire a target material matching the target content based on the intent recognition result of the first file;

[0198] The fusion module 1203 is configured to fuse the target material into the first file based on the initial display information to obtain a target file.

[0199] In some embodiments, the response module 1201 is configured to: identify image parameters of the image file corresponding to the first file to obtain the initial display information; and determine target content from text content and / or image content included in the first file based on target reference information.

[0200] In some embodiments, the response module 1201 is configured to: when the first file is an image file, identify the attribute information of the image elements in the first file and the association information therebetween, so as to obtain the initial display information based on the attribute information and the association information; when the first file is a text file, convert the text file into an image file, and identify the display layout information between the text contents in the image file as the initial display information; when the first file is a multimodal data file, convert the multimodal data file into a multi-layer image file, identify the attribute information of each modal data in the multi-layer image file and the association information therebetween, so as to obtain the initial display information based on the attribute information and the association information; when the first file is a text file and is converted into a multi-frame image, identify the display layout information between the text contents in each frame of the image as the initial display information.

[0201] In some embodiments, the response module 1201 is configured to: when the first file is a text file, perform word frequency statistics on the first file, and determine the text content with a word frequency greater than a first threshold as the target content; when the first file is image content, perform area ratio statistics on the first file, and determine the image content with an area ratio greater than a second threshold or less than a third threshold as the target content; when the first file is image content, determine the image content in the first file whose resolution is within a fourth threshold range as the target content; obtain usage information of the target file, and determine the text content and / or image content in the first file that matches the usage information as the target content; obtain user portrait information of the target user, and determine the text content and / or image content in the first file that matches the user portrait information as the target content, the target user includes the provider of the first file and / or the recipient of the target file; obtain file template information selected by the target user, and determine the target content based on the file template information.

[0202] In some embodiments, the acquisition module 1202 is configured to: perform intent recognition processing on the text content and / or image content included in the first file to obtain intent recognition keywords; filter out a first material set from a target material library based on the intent recognition keywords; and obtain target materials that match the target content from the first material set.

[0203] In some embodiments, the data processing device 120 further includes a determination module; the determination module is configured to: filter out a second material set from the target material library based on the target content, so as to determine the target material from the second material set when the first material set cannot match the target content.

[0204] In some embodiments, the fusion module 1203 is configured to: determine the fusion position information of the target material based on the initial display information; process the first file into a first intermediate image with a first transparency; and fuse the target material to a corresponding position of the first intermediate image based on the fusion position information to obtain the target file.

[0205] In some embodiments, the fusion module 1203 is configured to: in response to obtaining an operation of a target user selecting a target file template, fuse the first intermediate image with a template image corresponding to the target file template into a second intermediate image, and fuse the target material to a corresponding position of the second intermediate image to obtain the target file;

[0206] In some embodiments, the determination module is configured to: when the first file is converted into multiple frames of images, determine the material fusion position and material fusion quantity of each frame of image based on the initial display information and the text content of each frame of image.

[0207] In some embodiments, the determination module is configured to: obtain appearance parameters of the target material; and determine a fusion position of the target material in the image corresponding to the first file based on image parameters of the image corresponding to the first file and the appearance parameters.

[0208] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0209] It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit. It may also be implemented in the form of a combination of software and hardware.

[0210] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0211] An embodiment of the present application provides an electronic device, Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Fig.13 As shown, the electronic device 130 includes a memory 1301 and a processor 1302 . The memory 1301 stores a computer program that can be run on the processor 1302 . When the processor 1302 executes the program, the steps in the method provided in the above embodiment are implemented.

[0212] It should be noted that the memory 1301 is configured to store instructions and applications executable by the processor 1302, and can also cache data to be processed or processed by the processor 1302 and various modules in the electronic device 130, which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0213] An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the method provided in the above embodiment are implemented.

[0214] An embodiment of the present application provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the steps of the method provided in the above method embodiment.

[0215] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium, storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0216] It should be understood that the "one embodiment" or "an embodiment" or "some embodiments" or "in a possible implementation" or "example" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" or "in some embodiments" or "in a possible implementation" or "example" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. The above description of the various embodiments tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. For the sake of brevity, this article will not repeat them.

[0217] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist at the same time, and object B exists alone.

[0218] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In the several embodiments provided in the present application, it should be understood that the disclosed equipment and devices, etc., can be implemented in other ways. The embodiments described above are merely illustrative. The features disclosed in the several product embodiments provided in the present application can be arbitrarily combined without conflict to obtain new product embodiments.

[0219] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data processing method, comprising: In response to obtaining a first file, obtaining initial display information and target content of the first file; Acquire a target material matching the target content based on the intent recognition result of the first file; The target material is merged into the first file based on the initial display information to obtain a target file.

2. The method according to claim 1, wherein: Obtaining initial display information and target content of the first file, including: identifying image parameters of an image file corresponding to the first file to obtain the initial display information; and, The target content is determined from the text content and / or image content included in the first file based on the target reference information.

3. The method according to claim 2, wherein: Identifying the image parameters of the image file corresponding to the first file to obtain the initial display information includes at least one of the following: In the case where the first file is an image file, identifying attribute information of image elements in the first file and association information therebetween, so as to obtain the initial display information based on the attribute information and the association information; In the case where the first file is a text file, converting the text file into an image file, and identifying display layout information between text contents in the image file as the initial display information; In the case where the first file is a multimodal data file, converting the multimodal data file into a multi-layer image file, identifying attribute information of each modal data in the multi-layer image file and association information therebetween, so as to obtain the initial display information based on the attribute information and the association information; In the case where the first file is a text file and is converted into a plurality of frames of images, display layout information between text contents in each frame of images is identified as the initial display information.

4. The method according to claim 2, wherein: Determining target content from text content and / or image content included in the first file based on the target reference information includes at least one of the following: In the case where the first file is a text file, performing word frequency statistics processing on the first file, and determining text content with a word frequency greater than a first threshold as the target content; In the case where the first file is image content, performing area percentage statistics on the first file, and determining image content whose area percentage is greater than a second threshold or less than a third threshold as the target content; In the case where the first file is image content, determining the image content in the first file whose resolution is within a fourth threshold range as the target content; Obtaining usage information of the target file, and determining text content and / or image content in the first file that matches the usage information as the target content; Obtaining user portrait information of a target user, and determining text content and / or image content in the first file that matches the user portrait information as the target content, wherein the target user includes a provider of the first file and / or a recipient of the target file; The document template information selected by the target user is obtained, and the target content is determined based on the document template information.

5. The method according to claim 1, wherein: Acquiring a target material matching the target content based on the intent recognition result of the first file includes: Performing intent recognition processing on the text content and / or image content included in the first file to obtain intent recognition keywords; Filtering a first material set from a target material library based on the intention recognition keyword; A target material matching the target content is acquired from the first material set.

6. The method according to claim 5, further comprising: A second material set is screened out from the target material library based on the target content, so that the target material is determined from the second material set when the first material set cannot match the target content.

7. The method according to claim 1, wherein: Merging the target material into the first file based on the initial display information to obtain a target file includes: Determine fusion position information of the target material based on the initial display information; Processing the first file into a first intermediate image having a first transparency; The target material is fused to a corresponding position of the first intermediate image based on the fusion position information to obtain the target file.

8. The method according to claim 7, further comprising at least one of the following: In response to obtaining an operation of a target user selecting a target file template, fusing the first intermediate image with a template image corresponding to the target file template into a second intermediate image, and fusing the target material to a corresponding position of the second intermediate image to obtain the target file; In the case where the first file is converted into multiple frames of images, the material fusion position and the material fusion quantity of each frame of image are determined based on the initial display information and the text content of each frame of image.

9. The method according to claim 7 or 8, wherein: Determining the fusion position information of the target material based on the initial display information includes: Obtaining appearance parameters of the target material; The fusion position of the target material in the image corresponding to the first file is determined based on the image parameters of the image corresponding to the first file and the appearance parameters.

10. A data processing device, comprising: a response module, configured to obtain initial display information and target content of the first file in response to obtaining the first file; an acquisition module, configured to acquire a target material matching the target content based on the intent recognition result of the first file; The fusion module is configured to fuse the target material into the first file based on the initial display information to obtain a target file.

Images