A file processing method and device

Abstract

The embodiment of the present application discloses a file processing method and device, the method comprising: displaying text data in the first display page of the media file, and displaying M candidate images in the data recommendation area for the text data; M is a positive integer; in response to an image selection operation for a target image among the M candidate images, displaying N layout pages in the data recommendation area; each layout page includes the target image and the text data displayed in the first display page of the media file; N is a positive integer; in response to a page selection operation for a target layout page among the N layout pages, switching the first display page to display the target layout page. The present application can improve the generation efficiency of media files and reduce the generation time of media files.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a file processing method and device. Background Art

[0002] When creating a media file, users often insert pictures into the media file to improve the page richness and display effect of the media file. That is to say, in the process of creating a media file, the user needs to search for the required pictures according to the content of the media file, insert the selected pictures into the media file, and then rearrange the media file so that the inserted pictures can adapt to the display of the media file. This requires the user to frequently open a third-party application to search for pictures in the process of creating a media file. The process of searching and finding the pictures and the layout of the pictures after inserting them into the media file will consume a lot of time, and the layout of the media file with the inserted pictures needs to be constantly adjusted, resulting in low efficiency in generating the media file. Summary of the invention

[0003] The embodiments of the present application provide a file processing method and device, which can improve the generation efficiency of media files and save the generation time of media files.

[0004] On the one hand, an embodiment of the present application provides a file processing method, the method comprising:

[0005] Displaying text data in a first display page of a media file, and displaying M candidate images in a data recommendation area for the text data, where M is a positive integer;

[0006] In response to an image selection operation for a target image among the M candidate images, N layout pages are displayed in the data recommendation area; each layout page includes the target image and text data displayed in the first display page of the media file; N is a positive integer;

[0007] In response to a page selection operation for a target layout page among the N layout pages, the first display page is switched to be displayed as the target layout page.

[0008] An embodiment of the present application provides a file processing device, the device comprising:

[0009] A candidate image display module is used to display text data in the first display page of the media file, and for the text data, display M candidate images in the data recommendation area; M is a positive integer;

[0010] The layout display module is used to respond to the image selection operation for the target image among the M candidate images and display N layout pages in the data recommendation area; each layout page includes the target image and the text data displayed in the first display page of the media file; N is a positive integer;

[0011] The layout selection module is used to respond to a page selection operation for a target layout page among the N layout pages, and switch the first display page to be displayed as the target layout page.

[0012] The candidate image display module includes:

[0013] A text input unit, for responding to an input operation of target text data for a media file, and displaying the target text data in the media file;

[0014] The candidate picture display unit is used to respond to the picture recommendation operation for the target text data and display M candidate pictures associated with the target text data in the data recommendation area.

[0015] The candidate image display module includes:

[0016] The candidate image display unit further includes displaying the target text data in the first display page of the media file and displaying M candidate images associated with the target text data in the data recommendation area; or,

[0017] The candidate image display unit further includes displaying the target text data in the first display page of the media file, and when a trigger operation for the image recommendation component is received, displaying M candidate images associated with the target text data in the data recommendation area; or,

[0018] A component display unit, used to display target text data and a picture recommendation component in a first display page of a media file;

[0019] The component triggering unit is used to respond to the triggering operation on the picture recommendation component and display M candidate pictures associated with the target text data in the data recommendation area.

[0020] The device further comprises:

[0021] The utilization rate display module is used to respond to the viewing operation of the target image among the M candidate images and display the historical utilization rate of the target image; the historical utilization rate is used to indicate the probability of the target image being selected.

[0022] Wherein, the layout page also includes a text box to be edited; the device also includes:

[0023] The text box display module is used to display content prompt text in the target to-be-edited text box on the target layout page, and in response to an input operation on the target to-be-edited text box, switch the content prompt text in the target to-be-edited text box to display the text content corresponding to the input operation.

[0024] The media file includes at least two display pages, and the at least two display pages include a first display page; and the device further includes:

[0025] The marking display module is used to mark the illustration of the second display page having the same semantic information as the first display page among the at least two display pages; the illustration mark is used to indicate that M candidate images are recommended for the second display page.

[0026] The candidate image display module includes:

[0027] A text acquisition unit, configured to respond to a request for recommending an image of a media file and acquire text data displayed in a first display page of the media file based on the request for recommending the image;

[0028] A keyword extraction unit, used to extract keywords from text data to obtain text keywords corresponding to the text data;

[0029] The image acquisition unit is used to acquire M candidate images matching the text keywords from the candidate recommendation library and display the M candidate images in the data recommendation area.

[0030] The keyword extraction unit includes:

[0031] The text segmentation subunit is used to perform segmentation processing on the text data to obtain f1 segmentation phrases; f1 is a positive integer;

[0032] The word frequency acquisition subunit is used to obtain the phrase frequencies corresponding to the f1 word segmentation phrases respectively;

[0033] The inverse frequency acquisition subunit is used to obtain the inverse document frequencies corresponding to the f1 word groups;

[0034] The importance determination subunit is used to determine the importance of the phrases corresponding to the f1 segmented word phrases according to the phrase frequencies and inverse document frequencies corresponding to the f1 segmented word phrases;

[0035] The keyword selection subunit is used to determine the text keywords corresponding to the text data from the f1 segmented word groups based on the word group importance.

[0036] The word frequency acquisition subunit includes:

[0037] The phrase division subunit is used to divide the same segmentation phrases in the f1 segmentation phrases into f2 phrase sets; the segmentation phrases included in each phrase set are the same; f2 is a positive integer;

[0038] The word frequency determination subunit is used to count the number of phrases of the segmented phrases included in the i-th phrase set, and determine the sum of the number of phrases corresponding to the i-th phrase set and the number of phrases of the segmented phrases respectively included in the f2 phrase sets as the phrase frequency corresponding to the segmented phrases included in the i-th phrase set.

[0039] Wherein, the inverse frequency acquisition subunit includes:

[0040] The sample word segmentation subunit is used to obtain at least two sample text data included in the corpus, perform word segmentation processing on the at least two sample text data respectively, and obtain sample word segmentation phrases corresponding to the at least two sample text data respectively;

[0041] The associated statistical subunit is used to determine the number of sample text data associated with the sample segmentation phrase as the number of associated texts of the sample segmentation phrase;

[0042] The inverse frequency determination subunit is used to obtain the total number of sample texts of at least two sample text data, and determine the inverse document frequency of the sample segmentation phrase according to the total number of sample texts and the associated text data of the sample segmentation phrase.

[0043] The device further comprises:

[0044] The invalid phrase classification module is used to determine the word segmentation phrases whose inverse document frequency is less than the phrase validity threshold as invalid word segmentation phrases, and record the word segmentation phrases other than the invalid word segmentation phrases in the f1 word segmentation phrases as valid word segmentation phrases;

[0045] This keyword selects subunits, specifically for:

[0046] Based on the phrase importance, text keywords corresponding to the text data are determined from the valid segmented phrases.

[0047] The text segmentation subunit includes:

[0048] A word graph generation subunit is used to split the text data to obtain at least two characters constituting the text data, and to form a directed word graph with the at least two characters; the at least two characters are nodes of the directed word graph;

[0049] A path acquisition subunit, used to obtain at least two character paths according to the directed word graph and the association between adjacent characters in the directed word graph;

[0050] A path screening subunit, used to obtain the path lengths corresponding to at least two character paths respectively, and determine the shortest character path from the at least two character paths according to the path lengths;

[0051] The phrase generation subunit is used to group the characters corresponding to the shortest character path into segmented phrases to obtain f1 segmented phrases.

[0052] The image acquisition unit includes:

[0053] A first vector conversion subunit is used to perform vector conversion on text keywords to obtain keyword vectors;

[0054] The tag matching subunit is used to obtain at least two associated pictures whose picture tags are associated with the text keywords from the candidate recommendation picture library;

[0055] A second vector conversion subunit is used to perform picture encoding on at least two associated pictures respectively to obtain picture vectors corresponding to the at least two associated pictures respectively;

[0056] A similarity acquisition subunit, used to determine the semantic similarity between at least two associated pictures and the text keywords respectively according to the vector distance between the picture vectors and the keyword vectors respectively corresponding to the at least two associated pictures;

[0057] The picture selection subunit is used to obtain M candidate pictures from at least two related pictures based on semantic similarity.

[0058] The first vector conversion subunit is specifically used for:

[0059] Map text keywords to the target semantic space to obtain the first vector of the text keywords;

[0060] Performing dimensionality reduction processing on the first vector to obtain a keyword vector having a target vector length;

[0061] The second vector conversion subunit is specifically used for:

[0062] Mapping at least two associated images to the target semantic space respectively to obtain second vectors corresponding to the at least two associated images respectively;

[0063] Dimensionality reduction processing is performed on at least two second vectors to obtain picture vectors corresponding to at least two associated pictures respectively; the length of the picture vector is the target vector length.

[0064] Wherein, the layout display module includes:

[0065] A first quantity acquisition unit is used to respond to the picture selection operation for the target picture among the M candidate pictures, acquire the number of layout selections and the layout recommendation ratio, and determine the first layout recommendation number and the second layout recommendation number according to the number of layout selections and the layout recommendation ratio; the sum of the first layout recommendation number and the second layout recommendation number is the layout selection number;

[0066] A template acquisition unit, configured to acquire a first layout template from a cold start layout pool based on the number of first layout recommendations, and acquire a second layout template from a hot start layout pool based on the number of second layout recommendations; a recommendation weight value of the second layout template is greater than a recommendation weight value of the first layout template;

[0067] A layout generation unit, used to combine the first layout template and the second layout template with the target image respectively to generate N layout pages, where N is the number of layout selections;

[0068] The layout display unit is used to display N layout pages in the data recommendation area.

[0069] The device further comprises:

[0070] A first number updating module, used for updating the target history recommendation number and the target history selection number of the target layout page based on the page selection operation for the target layout page;

[0071] The weight updating module is used to update the recommendation weight value of the target layout template corresponding to the target layout page according to the target historical recommendation times and the target historical selection times.

[0072] The device further comprises:

[0073] A first weight acquisition module, used to acquire at least two cold start layout templates included in the cold start layout pool and a first recommended weight value corresponding to each cold start layout template;

[0074] A storage update module, configured to add a cold start layout template whose first recommended weight value is greater than or equal to the weight threshold to a hot start layout pool;

[0075] A second weight acquisition module is used to acquire at least two hot start layout templates included in the hot start layout pool and a second recommended weight value corresponding to each hot start layout template;

[0076] The template removal module is used to remove the hot start layout template whose second recommended weight value is less than the weight threshold from the hot start layout pool.

[0077] The device further comprises:

[0078] A second times updating module is used to update a first historical times of recommendation of the first layout template if the time that the layout page corresponding to the first layout template is in the page display state is greater than or equal to the recommended exposure time threshold;

[0079] A utilization rate acquisition module, used to acquire the layout utilization rate of the first layout template according to the updated first historical recommendation number and the first historical selection number of the first layout template;

[0080] The template deletion module is used to delete the first layout template in the cold start layout pool if the layout utilization rate is less than the layout retention threshold.

[0081] Wherein, the layout display module includes:

[0082] A layout acquisition unit, used to respond to an image selection operation for a target image among the M candidate images and acquire N layout templates;

[0083] A picture adding unit, used for acquiring picture display areas respectively included in N layout templates, adding a target picture to the picture display areas in the N layout templates, and generating N layout pages;

[0084] The page display unit is used to display N layout pages in the data recommendation area.

[0085] The layout display module includes:

[0086] A second quantity acquisition unit, configured to respond to a picture selection operation for a target picture among the M candidate pictures and acquire a displayed text quantity of text data displayed in a first display page of the media file and a displayed picture quantity of file pictures;

[0087] A target layout acquisition unit is used to acquire a target layout pool corresponding to the number of displayed texts and the number of displayed images, and acquire N layout templates from the target layout pool;

[0088] A page generation unit, used for writing target images, text data and file images into N layout templates to generate N layout pages;

[0089] The page display unit is also used to display N layout pages in the data recommendation area.

[0090] On the one hand, an embodiment of the present application provides a computer device, including a processor, a memory, and an input and output interface;

[0091] The processor is connected to the memory and the input / output interface respectively, wherein the input / output interface is used to receive and output data, the memory is used to store a computer program, and the processor is used to call the computer program so that a computer device including the processor executes the file processing method in one aspect of an embodiment of the present application.

[0092] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is suitable for being loaded and executed by a processor so that a computer device having the processor executes the file processing method in one aspect of the embodiment of the present application.

[0093] In one aspect, an embodiment of the present application provides a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in various optional ways in one aspect of the embodiment of the present application.

[0094] Implementing the embodiments of the present application will have the following beneficial effects:

[0095] In an embodiment of the present application, a user device can display text data in a first display page of a media file, and for the text data, display M candidate images in a data recommendation area; M is a positive integer; in response to an image selection operation for a target image among the M candidate images, display N layout pages in the data recommendation area; each layout page includes a target image and text data displayed in the first display page of the media file; N is a positive integer; in response to a page selection operation for a target layout page among the N layout pages, the first display page is switched to display the target layout page. Through the above process, relevant images can be recommended for the media file according to the content in the media file for the user to select, without the user having to search for images by himself, which can reduce the cost of image acquisition when the user retrieves and produces the media file; and when the user selects the target image to be used, a layout page including the target image is provided to the user based on the target image, and the layout page is a page that has been typeset, that is, the user can directly select the desired target layout page from the layout pages provided by the user device, realize direct typesetting of the page, reduce the typesetting cost of the media file, and thus improve the generation efficiency of the media file, improve the flexibility of the media file generation and the display effect, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0097] Figure 1 This is a network interaction architecture diagram for file processing provided by an embodiment of the present application;

[0098] Figure 2 This is a schematic diagram of a file processing scenario provided by an embodiment of the present application;

[0099] Figure 3 It is a flow chart of a method for file processing provided by an embodiment of the present application;

[0100] Figure 4a This is a schematic diagram of a picture recommendation scenario based on media files provided in an embodiment of the present application;

[0101] Figure 4b This is a schematic diagram of a picture recommendation scenario based on elements in a media file provided in an embodiment of the present application;

[0102] Figure 5 This is a schematic diagram of a utilization display scenario provided by an embodiment of the present application;

[0103] Figure 6 This is a schematic diagram of a progress display scenario provided by an embodiment of the present application;

[0104] Figure 7 This is a flowchart of a specific method for file processing scenario provided by an embodiment of the present application;

[0105] Figure 8 This is a schematic diagram of a text splitting scenario provided by an embodiment of the present application;

[0106] Fig. 9 This is a schematic diagram of a distance acquisition scenario provided by an embodiment of the present application;

[0107] Fig.10 It is a schematic diagram of a text processing scenario provided by an embodiment of the present application;

[0108] Fig.11 This is a schematic diagram of an image processing scenario provided by an embodiment of the present application;

[0109] Fig.12 This is a schematic diagram of a layout recommendation scenario provided in an embodiment of the present application;

[0110] Fig.13 This is a flow chart of a format pool updating method provided in an embodiment of the present application;

[0111] Fig.14 This is a flow chart of a method for updating a recommendation weight value provided in an embodiment of the present application;

[0112] Fig.15 It is a schematic diagram of a format dictionary tree provided in an embodiment of the present application;

[0113] Fig.16 It is a schematic diagram of a subject data structure provided in an embodiment of the present application;

[0114] Fig.17 is a schematic diagram of a file processing device provided in an embodiment of the present application;

[0115] Fig.18It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0116] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0117] In the examples of this application, see Figure 1 , Figure 1 It is a network interaction architecture diagram of file processing provided by an embodiment of the present application, and the embodiment of the present application can be implemented by a user device. Among them, a media application is an application that can edit or view files, etc., and can realize the management of media files in one or more media formats, such as document files (such as media files in .doc or .docx formats), video files or presentation files, etc., wherein a presentation file (PowerPoint, PPT) is a file that makes static file content into a dynamically browseable file, and turns complex content into easy-to-understand visual information, making it more vivid and leaving a more impressive slide. The presentation file is a file composed of one or more slides. Among them, the media application can be an online media application or a conventional media application, etc., and there is no limitation here. Among them, an online media application refers to an application that can be collaborated by multiple users to share or edit the same media file online; a conventional media application can refer to an application that is used by users to edit media files locally, etc.

[0118] For example, when d user devices collaborate based on an online media application, data can be exchanged between the d user devices. If any one of the d user devices makes a change to an online media file (such as adding content, deleting content, or modifying content, etc.), the changed data for the online media file will be shared with other user devices, where d is a positive integer. The d user devices can exchange data through a server, or directly, or can simultaneously implement data exchange between different user devices based on a server and direct communication. Figure 1As shown, the present application can be implemented by any user device (such as user device 102a, user device 102b or user device 102c, etc.), and the user device can obtain data from the computer device 101, or can obtain data based on cloud storage technology or blockchain technology. In other words, the data involved in the present application (such as candidate images and layout pages, etc.) can be stored in the computer device 101, or can be stored in the cloud space based on cloud storage technology, or can be stored in the blockchain network based on blockchain technology, etc., without limitation. Optionally, if the media application is an online media application, after the user device updates the media file, the updated media file can be synchronized to other user devices. For example, after the user device 102a updates the media file, the updated media file can be synchronized to other user devices (such as user device 102b and user device 102c, etc.), wherein the user device that receives the media file synchronized by the user device 102a refers to a user device that collaborates with the user device 102a and can edit the media file.

[0119] For details, see Figure 2 , Figure 2 Schematic diagram of a file processing scenario provided by an embodiment of the present application. Figure 2As shown, the user device 201 can respond to a picture recommendation request for the media file 202, and based on the picture recommendation request, display M candidate pictures 203 in the data recommendation area 2021, where M is a positive integer, wherein the M candidate pictures 203 are obtained based on the media file 202 and match the semantics of the media file 202, so that the user can obtain pictures that are more strongly related to the content in the media file without having to search for the required pictures, thereby saving the retrieval cost of the pictures (including time and resources, etc.). Furthermore, the user device 201 can respond to an image selection operation for a target image 204 among the M candidate images 203, and display N layout pages 205 in the data recommendation area 2021, each layout page including the target image 204 and the text data displayed in the first display page of the media file, where N is a positive integer. Through this process, N typeset layout pages are provided to the user, and the user can select the layout page he or she needs based on needs, and can use it directly without typesetting, so that the content typesetting of the first display page in the media file is equivalent to one-click generation. When the user device 201 responds to a page selection operation for a target layout page 206 among the N layout pages 205, the first display page can be switched to display the target layout page 206, thereby saving time and resources consumed in the process of image search, finding, and typesetting after the image is inserted into the media file during the media file generation process, saving the media file generation cost, and thereby improving the media file generation efficiency, and enhancing the flexibility of media file generation and the display effect.

[0120] Among them, when the media application is an online media application, if the d user devices perform editing operations on the media file at the same time, it may happen that multiple user devices insert pictures in the first display page. Through the scheme in this application, the d user devices will all display M candidate pictures associated with the text data displayed in the media file, so that the selection range of the pictures required to be inserted in the first display page by the d user devices is the same, which reduces the time spent by each user device to search for pictures separately, and the pictures searched by different user devices are quite different, resulting in different opinions on the changes to the first display page, making it impossible to insert pictures and layout the page in real time on the first display page, reducing the layout efficiency of the first display page. By recommending the same range of candidate pictures to each user device, the selection range of each user device is relatively consistent, which reduces the possibility of different selections by each user device, and improves the editing efficiency of the media file. Optionally, if each user device selects a target image from the M candidate images, the server may, based on the time sequence of the image selection operations corresponding to each user device, use the target image with the smallest trigger time of the corresponding image selection operation as the image to be inserted into the first display page, and generate N layout pages according to the target image with the smallest trigger time, and each user device may display the N layout pages in the data recommendation area. For example, assuming that d is 3, the trigger time of the image selection operation of user device 1 for target image 1 is trigger time 1, the trigger time of the image selection operation of user device 2 for target image 2 is trigger time 2, and the trigger time of the image selection operation of user device 3 for target image 3 is trigger time 3, assuming that trigger time 1 is less than trigger time 3, and trigger time 3 is less than trigger time 2, then N layout pages are generated according to target image 1 corresponding to trigger time 1. Alternatively, when receiving an image selection operation for a target image among the M candidate images, the target images selected by d user devices (wherein there may be a user device that has not selected a target image) may be obtained within the operation receiving cycle, and N layout pages may be generated according to the target image with the largest number of selections, etc., without limitation here. The operation receiving cycle is to avoid the situation where the picture insertion process for the media file cannot be continued when the user equipment does not insert the picture into the first display page.

[0121] It is understandable that the user device (such as the first user device or the second user device, etc.) mentioned in the embodiments of the present application can be a computer device, and the computer device in the embodiments of the present application includes but is not limited to a terminal device or a server. In other words, the computer device can be a server or a terminal device, or a system composed of a server and a terminal device. Among them, the terminal device mentioned above can be an electronic device, including but not limited to a mobile phone, a tablet computer, a desktop computer, a laptop computer, a PDA, a vehicle-mounted device, an augmented reality / virtual reality (Augmented Reality / Virtual Reality, AR / VR) device, a helmet display, a smart TV, a wearable device, a smart speaker, a digital camera, a camera and other mobile Internet devices (mobile internet device, MID) with network access capabilities, or terminal devices in scenarios such as trains, ships, and flights. Among them, the servers mentioned above can be independent physical servers, or they can be server clusters or distributed systems composed of multiple physical servers. They can also be cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, cars, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0122] Optionally, the data involved in the embodiments of the present application can be stored in a computer device, or the data involved in the present application can be stored based on cloud storage technology or blockchain technology, which is not limited here.

[0123] For further information, see Figure 3 , Figure 3 This is a flowchart of a method for file processing provided by an embodiment of the present application. Figure 3 As shown, the file processing process includes the following steps:

[0124] Step S301: display text data in a first display page of a media file, and display M candidate images in a data recommendation area for the text data.

[0125] In an embodiment of the present application, the user device can display text data in the first display page of the media file, obtain M candidate images associated with the text data in the media file, and display the M candidate images in the data recommendation area, where M is a positive integer. Optionally, the media file mentioned in the present application can be a document file, a video file, or a presentation file, etc.; the media file can be an online media file or a local media file, wherein an online media file refers to a media file that can be collaboratively edited with other users, and a local media file can refer to a media file edited by the user himself, and the local media file needs to be sent to other users so that other users can edit the local media file, etc. Optionally, the user device can display an image recommendation component, which is associated with a media file, and respond to a trigger operation on the image recommendation component to obtain M candidate images associated with the media file, and display the M candidate images in the data recommendation area, wherein the image recommendation component can be recorded as a first image recommendation component, and the first image recommendation component is an image recommendation component associated with the entire media file. Optionally, the user device can obtain a first display page of the media file and obtain M candidate images associated with the first display page, and the first display page refers to the page where the media file is being displayed, that is, the page where the user can directly see the page content.

[0126] Specifically, the user device can generate a picture recommendation request for the media file in the media application based on the triggering operation of the picture recommendation component in the media file. The picture recommendation component can be located at any position in the media application, and the picture recommendation component can have different triggering modes.

[0127] For example, see Figure 4a to Figure 4b , Figure 4a to Figure 4b Schematic diagram of a picture recommendation scenario provided by an embodiment of the present application. Figure 4a to Figure 4b As shown, the media application 401 may include, but is not limited to, application areas such as an editing component area 402, a thumbnail display area 403, and a file display area 404, wherein the relative position of the application area displayed in the media application 401 is not limited. When managing media files in the media application 401, the media files may be considered to be associated with the application area, and the relevant data of the media files may be displayed through the application area. Figure 4a As shown, Figure 4a 4 is a schematic diagram of a media file-based picture recommendation scenario provided by an embodiment of the present application. The user device can display a picture recommendation component in the editing component area 402 of the media application 401; or, the picture recommendation component can be displayed in the application function component list associated with any file editing component (such as a file component or an editing component, etc.) displayed in the editing component area 402. Figure 4a For example, the user device may respond to the trigger operation for the image recommendation component 4011 and display M candidate images in the data recommendation area 405. In response to the trigger operation for the image recommendation component 4011, the user device may obtain the text data in the first display page, obtain the M candidate images associated with the text data, and display the M candidate images in the data recommendation area 405; or, the user device may obtain the target text data in the text data displayed on the first display page, obtain the M candidate images associated with the target text data, and display the M candidate images in the data recommendation area 405. In other words, the user device may recommend candidate images for the text data in the entire page of the first display page, or may recommend candidate images for part of the target text data in the first display page (i.e., one or at least two of the text data displayed on the first display page). In the data recommendation area 405 may be displayed in the media application 401 all the time, or may be displayed in the media application 401 when the candidate images need to be displayed, without limitation here. Optionally, the data recommendation area 405 may be displayed on the same page as the first display page, or may be displayed independently on the first display page, etc., which is not limited here.

[0128] For example, the user device may recommend images for elements in the first display page. Specifically, the user device may respond to an input operation on target text data of a media file and display the target text data in the media file; and respond to an image recommendation operation on the target text data and display M candidate images associated with the target text data in the data recommendation area. The input operation on the target text data and the image recommendation operation on the target text data may be triggered adjacently, that is, after the input operation on the target text data is triggered, the image recommendation operation on the target text data is triggered; or they may not be triggered adjacently, that is, after the input operation on the target text data is triggered, other operations may be performed, and then the image recommendation operation on the target text data is triggered. Specifically, the user device may display the target text data in the first display page of the media file and display M candidate images associated with the target text data in the data recommendation area. That is, when the target text data is displayed in the first display page, the image recommendation operation on the target text data may be triggered, and the M candidate images associated with the target text data may be obtained. Alternatively, the user device may display the target text data in the first display page of the media file, and when a request for an image recommendation component (such as Figure 4aIn response to a trigger operation on the image recommendation component shown in the figure, M candidate images associated with the target text data are displayed in the data recommendation area. Alternatively, the user device may display the target text data in the first display page of the media file, and in response to a trigger operation on the target text data, display the image recommendation component; in response to a trigger operation on the image recommendation component, M candidate images associated with the target text data are displayed in the data recommendation area.

[0129] For example, see Figure 4b , Figure 4b is a schematic diagram of a picture recommendation scenario based on elements in a media file provided by an embodiment of the present application. Figure 4b As shown, the user device responds to the input operation of the target text data of the media file and displays the target text data in the media file. Optionally, a target text box 4041 may be displayed in the file display area 404. In response to the input operation on the target text box 4041, the target text data "nursing of the elderly" input in the target text box 4041 is obtained and displayed in the media file. Further, the user device responds to the trigger operation on the target text data "nursing of the elderly" and displays the image recommendation component 4042. In response to the trigger operation on the image recommendation component 4042, M candidate images 406 associated with the target text data "nursing of the elderly" are displayed in the data recommendation area 405.

[0130] Optionally, the user device may respond to a viewing operation on a target image among the M candidate images and display a historical utilization rate of the target image, where the historical utilization rate is used to indicate the probability of the target image being selected, that is, the probability of the target image being selected by the user after being recommended to the user. The user device may respond to a viewing operation on any one of the M candidate images and display the historical utilization rate of the candidate image corresponding to the viewing operation.

[0131] For example, see Figure 5 , Figure 5 Schematic diagram of a utilization display scenario provided by an embodiment of the present application. Figure 5 As shown, the user device 501 can display M candidate images 502 in the data recommendation area, and in response to a viewing operation on a candidate image 5021 among the M candidate images 502, display the historical utilization rate of the candidate image 5021. Assuming that the historical utilization rate of the candidate image 5021 is 63%, the user device 501 can display the historical utilization rate of the candidate image 5021 in the data recommendation area, or in the utilization rate display area, and the utilization rate display area can be displayed independently from the data recommendation area, etc., without limitation. For example, in Figure 5In display mode ①, the historical utilization rate “63%” of the candidate image 5021 is displayed in the utilization display area 503, and the utilization display area 503 is associated with the candidate image 5021; in display mode ②, the historical utilization rate “63%” of the candidate image 5021 is displayed in the data recommendation area, as shown in area 504, and the area 504 is associated with the candidate image 5021. The user device 501 can display the area 504 in an area adjacent to the area where the candidate image 5021 is displayed, etc.

[0132] Furthermore, in response to a request for image recommendation for a media file, text data displayed in a first display page in the media file is obtained based on the image recommendation request; keywords are extracted from the text data to obtain text keywords corresponding to the text data; M candidate images matching the text keywords are obtained from a candidate recommendation gallery, and the M candidate images are displayed in a data recommendation area.

[0133] Step S302, in response to a picture selection operation for a target picture among the M candidate pictures, N layout pages are displayed in the data recommendation area.

[0134] In the embodiment of the present application, in response to the image selection operation for the target image among the M candidate images, N layout templates associated with the target image are obtained, the target image and the text data displayed in the first display page are written into the N layout templates, N layout pages are generated, and the N layout pages are displayed in the data recommendation area. Optionally, the user device can obtain the number of displayed texts of the text data displayed in the first display page of the media file and the number of displayed images of the file image, and obtain N layout templates based on the number of displayed texts and the number of displayed images; or the user device can obtain N layout templates from the hot start layout pool and the cold start layout pool based on the layout recommendation ratio, etc., which are not displayed here.

[0135] Optionally, when the user device generates N layout pages, it can display the generation progress message of each layout page. Specifically, in the media file, the user device responds to the image selection operation for the target image and displays the generation progress message of each layout page in the data recommendation area; if there is a layout page generation failure (the layout page that failed to generate can be recorded as an abnormal layout page), the generation failure prompt message of the abnormal layout page can be displayed in the data recommendation area, or the abnormal layout page can be directly deleted; if there is a layout page generation success (the layout page that successfully generated can be recorded as a normal layout page), the user device can display the normal layout page in the data recommendation area, and optionally, the user device can also display the generation success prompt message of the normal layout page.

[0136] For details, see Figure 6 , Figure 6 Schematic diagram of a progress display scenario provided by an embodiment of the present application. Figure 6 As shown, the user device 601 can respond to the image selection operation for the target image 6021 in the M candidate images 602, obtain M layout templates, and generate M layout pages based on the M layout templates, the target image and the first display page. Optionally, the user device 601 can display the generation progress message of each layout page in the data recommendation area 603, for example, the generation progress message 6031 ("4%") of the layout page 1, the generation progress message 6032 ("5%") of the layout page 2, and the generation progress message 6033 ("3%") of the layout page 3, etc. Furthermore, the user device 601 can display the generation results of each layout page. For example, if the layout page 1 is generated successfully, the user device 601 displays the generated layout page 1 in the data recommendation area 603 (i.e., the content indicated by area 6041); if the layout page 2 is generated successfully, the user device 601 displays the generated layout page 2 in the data recommendation area 603 (i.e., the content indicated by area 6042); if the layout page 3 fails to be generated, the user device 601 displays a prompt message 6043 indicating the failure to generate the layout page 3 in the data recommendation area, etc.

[0137] Step S303, in response to a page selection operation for a target layout page among the N layout pages, the first display page is switched to be displayed as the target layout page.

[0138] Further, the user device can respond to the page selection operation for the target layout page in the N layout pages, switch the first display page to the target layout page, implement the picture insertion and page layout of the first display page, and improve the page generation efficiency. Further optionally, the layout page can also include a text box to be edited, that is, the target layout page includes a target text box to be edited. The user device can display content prompt text in the target text box to be edited in the target layout page, and in response to the input operation for the target text box to be edited, switch the content prompt text in the target text box to be edited to the text content corresponding to the input operation. For example, the content prompt text displayed in the target text box to be edited is "Speaker Information", and in response to the input operation for the target text box to be edited, the content prompt text "Speaker Information" in the target text box to be edited is switched to the text content corresponding to the input operation, such as "Speaker 1", etc.

[0139] In an embodiment of the present application, a user device can respond to a request for image recommendation for a media file, and display M candidate images in a data recommendation area; M is a positive integer; in response to an image selection operation for a target image among the M candidate images, N layout pages are displayed in the data recommendation area; each layout page includes the target image and text data displayed in the first display page of the media file; N is a positive integer; in response to a page selection operation for a target layout page among the N layout pages, the first display page is switched to display the target layout page. Through the above process, relevant images can be recommended for the media file according to the content in the media file for the user to select, without the user having to search for images by himself, which can reduce the cost of image acquisition when the user retrieves and produces the media file; and when the user selects the target image to be used, a layout page including the target image is provided to the user based on the target image, and the layout page is a page that has been typeset, that is, the user can directly select the desired target layout page from the layout pages provided by the user device, realize direct typesetting of the page, reduce the typesetting cost of the media file, and thus improve the generation efficiency of the media file, improve the flexibility of the media file generation and the display effect, etc.

[0140] Further, see Figure 7 , Figure 7 This is a flowchart of a specific method for file processing scenario provided by an embodiment of the present application. Figure 7 As shown, the method comprises the following steps:

[0141] Step S701, obtaining text data in a media file.

[0142] In an embodiment of the present application, the user device may display text data in the first display page of the media file and obtain the text data in the media file. Optionally, the user device may obtain the text data displayed in the first display page of the media file, such as Figure 4a As shown; or, the user device can obtain the target text data corresponding to the image recommendation request, such as Figure 4bAs shown, no limitation is made here. It is specifically determined by the object to which the candidate image needs to be recommended. For example, the user device can respond to the image recommendation request for the media file and obtain the text data displayed in the first display page of the media file; or, when the user device displays the target text data in the first display page of the media file, it obtains the target text data and executes step S702; or, when the user device displays the target text data in the first display page of the media file, when a trigger operation for the image recommendation component is received, it obtains the target text data and executes step S702. Optionally, the target text data can be in a selected state at this time; or, when the user device displays the target text data in the first display page of the media file, it responds to the trigger operation for the target text data, displays the image recommendation component, responds to the trigger operation for the image recommendation component, obtains the target text data, and executes step S702. For details, see Figure 3 The specific description of step S301.

[0143] Step S702: extract text keywords from the text data.

[0144] In an embodiment of the present application, the user device can perform word segmentation processing on text data to obtain f1 word segmentation phrases; f1 is a positive integer; and text keywords are obtained from the f1 word segmentation phrases. Optionally, the user device can delete meaningless word segmentation phrases from the f1 word segmentation phrases to obtain text keywords, wherein a dictionary including word segmentation phrases without actual meaning can be established in advance, and the dictionary can include one or at least two meaningless word segmentation phrases, and the user device can delete the meaningless word segmentation phrases from the f1 word segmentation phrases that belong to the dictionary, wherein the meaningless word segmentation phrase refers to a phrase that has no actual meaning or has no semantic or actual impact on the sentence, such as an auxiliary word, etc.; or, the user device can obtain the phrase importance corresponding to each of the f1 word segmentation phrases, and determine the text keywords based on the phrase importance. Specifically, the user device can perform word segmentation processing on the text data to obtain f1 word segmentation phrases; f1 is a positive integer; obtain the phrase frequencies corresponding to the f1 word segmentation phrases, obtain the inverse document frequencies corresponding to the f1 word segmentation phrases, and determine the phrase importance corresponding to the f1 word segmentation phrases according to the phrase frequencies and inverse document frequencies corresponding to the f1 word segmentation phrases; determine the text keywords corresponding to the text data from the f1 word segmentation phrases based on the phrase importance. Among them, the user device can use a text word segmentation algorithm to perform word segmentation processing on the text data, and the text word segmentation algorithm includes but is not limited to an algorithm corresponding to a text word segmentation tool or a shortest path word segmentation algorithm. For example, the user device can split the text data to obtain at least two characters constituting the text data, and form a directed word graph with the at least two characters; the at least two characters are nodes of the directed word graph. At least two character paths are obtained according to the directed word graph and the association between adjacent characters in the directed word graph; the path lengths corresponding to the at least two character paths are obtained, and the shortest character path is determined from the at least two character paths according to the path lengths; the characters corresponding to the shortest character path are grouped into word segmentation phrases to obtain f1 word segmentation phrases.

[0145] For example, see Figure 8 , Figure 8 Schematic diagram of a text splitting scenario provided by an embodiment of the present application. Figure 8As shown, the user device can divide the text data into at least two characters, for example, split the text data "What he said is indeed reasonable" into at least two characters such as "he", "said", "of", "indeed", "really", "in" and "reason", and form a directed word graph with at least two characters. The directed word graph can be a directed acyclic graph, wherein the weight of each character in the directed word graph can be considered to be equal. For example, the weight of each edge in the directed word graph can be recorded as 1 or other default weight values. Among them, when obtaining the shortest path of a directed acyclic graph, the shortest path between two points in the directed acyclic graph also includes the shortest paths between other vertices on the path. For example, assuming that S—>A—>B—>E is the shortest path from S to E, then S—>A—>B must be the shortest path from S to B, otherwise there will be a point C such that d(S—>C—>B) is less than d(S—>A—>B), then the shortest path from S to E will also become S—>C—>B—>E, which contradicts the assumption. The properties of this optimal substructure are used to obtain the shortest path in the directed word graph. Among them, d(*) is used to represent the path length of "*", and the path length can be obtained by the weight of the edge corresponding to "*". Optionally, the user device can obtain at least two character paths based on the directed word graph and the correlation between adjacent characters in the directed word graph, such as Figure 8 In the text “indeed, really, indeed, and in reason, etc.”, a path resolution algorithm is used to obtain the path lengths corresponding to each character path, determine the shortest path from at least two character paths based on the path length, and group the characters corresponding to the shortest path into segmentation phrases, such as “indeed, in reason, etc.” Optionally, the path resolution algorithm can be a greedy algorithm or other algorithms that can resolve paths, which are not limited here. For example, the user device performs segmentation processing on the text data “nursing of the elderly” and obtains that “elderly” constitutes a shortest path and “nursing” constitutes a shortest path. Therefore, the user device can perform segmentation processing on the text data “nursing of the elderly” and obtain f1 segmentation phrases such as “elderly”, “of” and “nursing”. Furthermore, the user device can delete the meaningless segmentation phrase “of” from the f1 segmentation phrases to obtain the text keywords “elderly” and “nursing” corresponding to the text data.

[0146] Further optionally, when obtaining the phrase frequencies corresponding to the f1 segmented word phrases, the user device may divide the same segmented word phrases in the f1 segmented word phrases into f2 phrase sets; the segmented word phrases included in each phrase set are the same; f2 is a positive integer. For example, the f1 segmented word phrases include "elderly, nursing, nursing, diet and cultivation", etc., and the f1 segmented word phrases may be divided to obtain f2 phrase sets such as "elderly", "nursing, nursing", "diet" and "cultivation". Optionally, the phrase set is obtained only for the convenience of counting the number of times each segmented word phrase appears, and the f1 segmented word phrases may not be divided, and the number of times each segmented word phrase appears may be directly counted, which is not limited here. Further, the number of phrases of the segmented word phrases included in the i-th phrase set is counted, and the sum of the number of phrases corresponding to the i-th phrase set and the number of phrases of the segmented word phrases included in the f2 phrase sets is determined as the phrase frequency corresponding to the segmented word phrases included in the i-th phrase set. That is to say, the term frequency (TF) can represent the number of times the corresponding segmentation phrase appears in the text data, which can usually be expressed as TF = the number of phrases in which a segmentation phrase appears in the text data / the total number of segmentation phrases in the text data. The importance of a segmentation phrase in text data usually increases in direct proportion to the number of times the segmentation phrase appears in the text data, but at the same time decreases in inverse proportion to the frequency of the segmentation phrase in the corpus. For example, for the f1 segmentation phrases "elderly", "of" and "nursing" obtained above, it can be obtained that the TF of each segmentation phrase is 1 / 3, but it is obvious that using "of" for picture recommendation is meaningless, so the abnormal influence of "of" can be eliminated by using the Inverse Document Frequency (IDF).

[0147] Specifically, when obtaining the inverse document frequencies corresponding to f1 segmented phrase groups, the user device can obtain at least two sample text data included in the corpus, perform word segmentation processing on the at least two sample text data respectively to obtain sample segmented phrase groups corresponding to the at least two sample text data respectively; determine the number of sample text data associated with the sample segmented phrase group as the associated text number of the sample segmented phrase group; obtain the total number of sample text of the at least two sample text data, and determine the inverse document frequency of the sample segmented phrase group according to the total number of sample text and the associated text data of the sample segmented phrase group. Among them, IDF is an index expressing the importance of a word. The user device can use "IDF = log(total number of sample text in the corpus / (associated text data containing this segmented phrase group + 1))". If a certain segmented phrase group is included in all sample text data, it can be considered that the IDF of this segmented phrase group = log(1) = 0, that is, the importance of the phrase is 0. Optionally, to avoid the denominator being 0, so "+1" is added to the denominator. Optionally, IDF can be obtained through the above formula; it can also use the above formula when the associated text data is 0, and use "IDF = log(total number of sample text in the corpus / associated text data containing this segmented phrase group)" when the associated text data is not 0, which is not limited here.

[0148] Further optionally, since the smaller the inverse document frequency, the more times the segmented phrase group appears in the corpus and the smaller the possibility of having a practical meaning, therefore, it is possible to remove segmented phrase groups such as "de" that appear very frequently in each sample text data based on the inverse document frequency. Specifically, the user device can determine the segmented phrase groups with an inverse document frequency less than the phrase effective threshold as invalid segmented phrase groups, and record the segmented phrase groups other than the invalid segmented phrase groups among the f1 segmented phrase groups as valid segmented phrase groups. Further, when determining the text keywords corresponding to the text data from the f1 segmented phrase groups based on the phrase importance, the user device can determine the text keywords corresponding to the text data from the valid segmented phrase groups based on the phrase importance.

[0149] Optionally, the user device can determine the phrase importance of the segmented phrase group based on the phrase frequency and inverse document frequency of the segmented phrase group. Among them, the user device can use the product of the phrase frequency and inverse document frequency of the segmented phrase group as the phrase importance of the segmented phrase group, or can use the weighted product of the phrase frequency and inverse document frequency of the segmented phrase group as the phrase importance of the segmented phrase group, etc., which is not limited here. Optionally, if candidate pictures are recommended for the target text data, the text keywords of the target text data can be extracted, and step S702 can be executed for the target text data.

[0150] Step S703, obtain M candidate pictures that match the text keywords from the candidate recommendation picture library, and display the M candidate pictures in the data recommendation area.

[0151] In an embodiment of the present application, the user device can obtain M candidate images that match the text keywords from the candidate recommendation gallery, and display the M candidate images in the data recommendation area. Optionally, when adding the candidate image to the candidate recommendation gallery, a picture tag can be added to the candidate image, and the user device can obtain M candidate images whose picture tags match the text keywords from the candidate recommendation gallery. Furthermore, the user device can perform vector conversion on the text keywords to obtain a keyword vector; obtain at least two associated images whose picture tags are associated with the text keywords from the candidate recommendation gallery, perform picture encoding on the at least two associated images respectively, and obtain picture vectors corresponding to the at least two associated images respectively; determine the semantic similarity between the at least two associated images and the text keywords respectively based on the vector distance between the picture vectors corresponding to the at least two associated images and the keyword vector; obtain M candidate images from the at least two associated images based on the semantic similarity.

[0152] For example, see Fig. 9 , Fig. 9 Schematic diagram of a distance acquisition scenario provided by an embodiment of the present application. Fig. 9 As shown, the user device can obtain the associated image 901, perform image encoding on the associated image 901, and obtain the image vector of the associated image 901; perform vector conversion on the text keyword to obtain the keyword vector of the text keyword. The distance between the image vector of the associated image 901 and the keyword vector is measured to obtain the vector distance between the image vector of the associated image 901 and the keyword vector. The vector distance can be Euclidean distance or cosine distance, etc., wherein the Euclidean distance refers to the distance between two points, and the cosine distance refers to the distance obtained according to the cosine value of the angle between the two vectors.

[0153] Furthermore, the user device can map the text keywords to the target semantic space to obtain the first vector of the text keywords; perform dimensionality reduction processing on the first vector to obtain a keyword vector with the target vector length. At least two related pictures are respectively mapped to the target semantic space to obtain second vectors corresponding to at least two related pictures; perform dimensionality reduction processing on at least two second vectors to obtain picture vectors corresponding to at least two related pictures; the length of the picture vector is the target vector length. In other words, the picture (i.e., the related picture) and the text (text keywords) are mapped to the same semantic space, and then the semantic similarity between the related picture and the text keyword can be represented by the vector distance between the picture vector and the keyword vector.

[0154] For example, see Fig.10 , Fig.10 Schematic diagram of a text processing scenario provided by an embodiment of the present application. Fig.10 As shown, the user device obtains the text data "nursing of the elderly", performs word segmentation processing on the text data, and obtains f1 word segmentation phrases "elderly, of, nursing", determines the text keyword "elderly|nursing" from the f1 word segmentation phrases, maps the text keyword to the target semantic space, performs vector conversion, and obtains the first vector of the text keyword. Assuming that the first vector is "0 0 0 0.12 0 0 0 0 0.56 0.23 0 0 ... ", assuming that the vector length of the first vector is 30000, performs dimensionality reduction processing on the first vector, and obtains a keyword vector with a target vector length. Assuming that the target vector length is 1000, performs dimensionality reduction processing on the first vector, and obtains the keyword vector "0.12 -0.33 0.15 -0.54 -0.01 0.04 0.31...". Among them, the user device may determine that the text keyword is converted into a sparse first vector according to the phrase frequency and inverse document frequency of the text keyword. For example, the text keyword may be converted into a vector by a text vector conversion model to obtain a first vector. The text vector conversion model may be, but is not limited to, a bag of words model, a vector space model, etc. Optionally, the user device may use a dimensionality reduction model to reduce the dimensionality of the first vector. The dimensionality reduction model may include, but is not limited to, a principal component analysis (PCA) model, a singular value decomposition (SVD) or a linear discriminant analysis (LDA), etc., which are not limited here.

[0155] For example, see Fig.11 , Fig.11 Schematic diagram of a picture processing scenario provided by an embodiment of the present application. Fig.11As shown, the user device obtains the associated image 1101, maps the associated image to the target semantic space, and obtains the second vector 1104 corresponding to the associated image 1101, wherein the vector conversion process can be implemented based on the image vector conversion model 1102. Further, the user device can use a fully connected layer to perform dimensionality reduction processing on the second vector 1104 to obtain a picture vector 1105 with a target vector length. Optionally, the picture vector conversion model 1102 includes a convolutional neural network 1103, and the convolutional neural network 1103 can extract semantic information of the associated image. For example, the convolutional neural network 1103 can use a 3*3 convolution kernel and 16 network layers, including 13 convolution layers, etc., which are not limited here, that is, the size of the convolution kernel used by the convolutional neural network 1103, the number of network layers included, and the number of convolution layers, etc. can be changed according to actual needs. Optionally, the convolutional neural network 1103 can also include a maximum pooling layer, which can reduce the size of the associated image, such as Fig.11 In the example, the maximum pooling layer is used to perform maximum pooling on the first matrix 1106 to obtain a second matrix 1107, wherein the maximum pooling is performed with a size of 2*2 and a step size of 2 (the size and step size of the maximum pooling can be changed as needed), that is, Perform maximum pooling to obtain 9 in the second matrix 1107, and Perform maximum pooling to obtain 5 in the second matrix 1107, and Perform maximum pooling to obtain 6 in the second matrix 1107, and The maximum pooling is performed to obtain 8 in the second matrix 1107, etc. Based on this principle, the scale of the associated image is transformed through the maximum pooling layer.

[0156] For example, the convolutional neural network can be composed of two or three convolutional layers with a maximum pooling layer. For example, a 224*224 related image is input. The architecture of a convolutional neural network can be seen in Table 1. Table 1 is a table of the architecture of the convolutional neural network and the scale change of the image, as shown below:

[0157] Table 1

[0158]

[0159] Among them, the number associated with the fully connected layer is used to represent the dimension of the vector obtained after processing the associated image, so that the length of the obtained image vector is the target vector length.

[0160] Furthermore, the user device may obtain M candidate images from at least two associated images based on semantic similarity. Specifically, the user device may directly display the M candidate images in the data recommendation area; or, based on the semantic similarities corresponding to the M candidate images, the M candidate images may be displayed in sequence in the data recommendation area, that is, the M candidate images may be displayed in sequence in the data recommendation area in descending order of semantic similarity, etc., without limitation. Optionally, since the size of the recommendation area of ​​the data recommendation area may make it impossible to display the M candidate images, a first sliding control may be displayed in the data recommendation area, and the user device may respond to a sliding operation on the first sliding control to display the M candidate images for the user.

[0161] For details, see Fig.12 , Fig.12 Schematic diagram of a layout recommendation scenario provided by an embodiment of the present application. Fig.12 As shown, the user device 1201 responds to a request for image recommendation for a media file, obtains text data in the media file, such as text data 1203 (nursing of the elderly) displayed in the first display page 1202 of the media file, extracts the text keywords "elderly, nursing" in the text data 1203, obtains M candidate images matching the text keywords, and displays M candidate images 1205 in a data recommendation area 1204. Optionally, a first sliding control may be displayed in the data recommendation area 1204. When responding to a trigger operation on the first sliding control, the user device 1201 may update the candidate images in the data recommendation area 1204 that are in a page display state, wherein the page display state means that the candidate images can be seen by the user.

[0162] Step S704, in response to the picture selection operation for the target picture among the M candidate pictures, N layout templates are obtained, N layout pages are generated based on the N layout templates, and the N layout pages are displayed in the data recommendation area.

[0163] In an embodiment of the present application, the user device can respond to an image selection operation for a target image among M candidate images, obtain N layout templates; obtain image display areas respectively included in the N layout templates, add the target image to the image display areas in the N layout templates, generate N layout pages; and display the N layout pages in the data recommendation area. The number of the target images can be one or at least two.

[0164] Optional, such as Fig.12As shown, the user device 1201 can respond to the image selection operation for the target image 1206, obtain N layout templates, write the target image and text data in each layout template, generate N layout pages, and display N layout pages 1207 in the data recommendation area 1204. Optionally, the user device 1201 can also display the target image 1206 in the first display page 1202, which is an optional process. Optionally, the user device 1201 can also display a second sliding control in the data recommendation area 1204, and when responding to the trigger operation for the second sliding control, the layout page in the page display state in the data recommendation area 1204 can be updated.

[0165] Specifically, the user device may respond to the image selection operation for the target image among the M candidate images, obtain the number of layout selections and the layout recommendation ratio, and determine the first layout recommendation number and the second layout recommendation number according to the number of layout selections and the layout recommendation ratio; the sum of the first layout recommendation number and the second layout recommendation number is the number of layout selections. For example, if the number of layout selections is N, and the layout recommendation ratio is 1:9 for the ratio of the cold start layout pool to the hot start layout pool, then the first layout recommendation number is determined to be N*10%, and the second layout recommendation number is determined to be N*90%. Among them, the hot start layout template included in the hot start layout pool refers to the high-quality layout template with a larger recommendation weight value, and the cold start layout template included in the cold start layout pool has a smaller recommendation weight value. Since the cold start layout template in the cold start layout pool may be a newly extracted layout template, that is, the recommendation weight value of the cold start layout template may be the initial weight value. Therefore, if the layout template is directly obtained from the hot start layout pool, the newly extracted layout template cannot be used. Therefore, the layout template can be obtained from both the hot start layout pool and the cold start layout pool based on the layout recommendation ratio, so that the newly extracted layout template may also be recommended to the user, thereby improving the recommendation accuracy of the layout template. Furthermore, the user device can obtain the first layout template from the cold start layout pool based on the number of first layout recommendations, and obtain the second layout template from the hot start layout pool based on the number of second layout recommendations; the recommendation weight value of the second layout template is greater than the recommendation weight value of the first layout template; the first layout template and the second layout template are respectively combined with the target image to generate N layout pages; N is the number of layout selections; and N layout pages are displayed in the data recommendation area.

[0166] Optionally, the user device can update the target history recommendation times and the target history selection times of the target layout page based on the page selection operation for the target layout page. That is to say, if the target layout page is selected, it means that the recommendation times and the selection times of the target layout template corresponding to the target layout page are increased, and the target history recommendation times and the target history selection times of the target layout template corresponding to the target layout page can be updated. According to the target history recommendation times and the target history selection times, the recommendation weight value of the target layout template corresponding to the target layout page is updated. Optionally, it can be considered that the smaller the ratio of the target history selection times to the target history recommendation times, the smaller the recommendation weight value of the target layout template. That is, the ratio of the target history selection times to the target history recommendation times can be directly obtained, and the ratio is used as the target history utilization rate of the target layout template, and the target history utilization rate is determined as the recommendation weight value of the target layout template; or, the target history selection times and the target history recommendation times can be weighted, and then the ratio of the weighted target history selection times to the weighted target history recommendation times can be obtained, and the ratio is determined as the recommendation weight value of the target layout template, etc., and there is no restriction here.

[0167] Optionally, the user device can obtain at least two cold start layout templates included in the cold start layout pool and the first recommended weight value corresponding to each cold start layout template; add the cold start layout template whose first recommended weight value is greater than or equal to the weight threshold to the hot start layout pool. That is to say, if the first recommended weight value of the cold start layout template in the cold start layout pool is greater than or equal to the weight threshold, it means that the cold start layout template is a high-quality template, and the cold start layout template whose first recommended weight value is greater than or equal to the weight threshold can be added to the hot start layout pool, and the cold start layout template whose first recommended weight value is greater than or equal to the weight threshold can be removed from the cold start layout pool. Furthermore, at least two hot start layout templates included in the hot start layout pool and the second recommended weight value corresponding to each hot start layout template can be obtained; remove the hot start layout template whose second recommended weight value is less than the weight threshold from the hot start layout pool. Optionally, the hot start layout template whose second recommended weight value is less than the weight threshold can be directly deleted; or the hot start layout template whose second recommended weight value is less than the weight threshold can be added to the cold start layout pool, and the hot start layout template whose second recommended weight value is less than the weight threshold in the hot start layout pool can be deleted.

[0168] If the time that the layout page corresponding to the first layout template is in the page display state is greater than or equal to the recommended exposure time threshold, the first historical recommendation number of the first layout template is updated. The layout page corresponding to the layout template has a page display state and a page hidden state. When the layout page is in the page hidden state, it means that the user cannot see the layout page. That is to say, among the N layout pages displayed in the data recommendation area, the user can see the layout page in the page display state, while the layout page in the page hidden state cannot be triggered by the page selection operation because the user cannot see it. Therefore, directly updating the historical recommendation number of the layout templates corresponding to the N layout pages respectively will make the statistical results of the layout templates corresponding to the layout pages that are not seen by the user inaccurate. By counting the time that the layout page corresponding to the layout template is in the page display state, the accuracy of determining the recommendation weight value of the layout template is improved. Further, according to the updated first historical recommendation number and the first historical selection number of the first layout template, the layout utilization rate of the first layout template is obtained. Optionally, if the layout page corresponding to the first layout template is triggered to select a page, the first historical selection count of the first layout template may also be updated; and the layout utilization rate of the first layout template is obtained according to the updated first historical recommendation count and the updated first historical selection count. If the layout utilization rate is less than the layout retention threshold, the first layout template in the cold start layout pool is deleted.

[0169] Optionally, the media file includes at least two display pages, and the at least two display pages include a first display page. An illustration mark is performed on a second display page having the same semantic information as the first display page among the at least two display pages; the illustration mark is used to indicate that M candidate images are recommended for the second display page. This allows the user to directly select images for the second display page having the same semantic information without having to re-analyze the semantics of the second display page to recommend images, thereby improving the flexibility and efficiency of image recommendation.

[0170] Optionally, the user device may obtain the display position of the first display page in the media file, which includes the cover position, chapter position, text position, and end position, etc., and obtain N layout templates based on the display position of the first display page. For example, at the cover position, a picture is generally used as the background; at the text position, a picture is generally used as the text content for explanation, etc. Therefore, the layout template recommended can be further accurately determined by the display position of the first display page, which can improve the accuracy of layout recommendation.

[0171] Optionally, the user device may respond to the image selection operation for the target image among the M candidate images and obtain the number of displayed texts of the text data and the number of displayed images of the file image displayed in the first display page of the media file. Obtain the target layout pool corresponding to the number of displayed texts and the number of displayed images, and obtain N layout templates from the target layout pool, specifically obtaining the target layout pool corresponding to the number of displayed texts and the number of displayed images and the number of target images. That is, there may be multiple layout pools, and the number of texts and the number of images corresponding to each layout pool are different. For example, the number of displayed texts is 1, the number of displayed images is 1, and the number of target images is 2. Then the user device may obtain a target layout pool with a number of texts of 1 and a number of images of 3. Write the target image, text data, and file image into N layout templates to generate N layout pages, that is, write the target image, text data, and file image into each layout template to generate N layout pages; and display the N layout pages in the data recommendation area.

[0172] Optionally, the layout pool can also be divided by the number of texts and images, cold start and hot start, etc., and there is no restriction here.

[0173] Step S705 , in response to a page selection operation for a target layout page among the N layout pages, the first display page is switched to be displayed as the target layout page.

[0174] In the embodiments of the present application, Fig.12 As shown, in response to a page selection operation for a target layout page 1208 among the N layout pages 1207, the first display page 1202 is switched to be displayed as the target layout page 1208, so as to implement the layout of each element included in the first display page 1202. Optionally, when displaying the target layout page 1208, the user device may cancel displaying the data recommendation area 1204, or may continue to display the data recommendation area 1204.

[0175] Optionally, when the user generates a third display page, the user device may respond to a layout extraction request for the third display page, convert the third display page into a third layout template, and add the third layout template to the cold start layout pool. Optionally, the user device may respond to a layout extraction request for the third display page, display a shared attribute selection message, and when responding to a confirmation operation for the shared attribute selection message, convert the third display page into a third layout template, and add the third layout template to a layout pool (such as a cold start layout pool); when responding to a cancel operation for the shared attribute selection message, the third display page may be converted into a third layout template, and the third layout template may be stored in the user device (i.e., locally).

[0176] Further, see Fig.13 , Fig.13 This is a flow chart of a format pool updating method provided by an embodiment of the present application. Fig.13 As shown, the method comprises the following steps:

[0177] Step S130a: extracting sample layout templates from sample media files in the existing corpus.

[0178] In the embodiment of the present application, sample layout templates may be extracted from sample media files in an existing corpus.

[0179] Step S130b: adding the sample layout template to the cold start layout pool.

[0180] Step S130d, obtaining N layout templates.

[0181] In the embodiment of the present application, when responding to the image selection operation for the target image, N layout templates are obtained from the cold start layout pool and the hot start layout pool 130c. Figure 7 Specific description shown in step S704.

[0182] Step S130e, providing positive feedback to the selected layout template.

[0183] In an embodiment of the present application, positive feedback is performed on the selected layout template, that is, positive feedback is performed on the target layout template corresponding to the target layout page that triggers the page selection operation among the N layout templates, that is, the recommendation weight value of the target layout template is increased.

[0184] For the cold start layout template in the cold start layout pool, please refer to the following steps:

[0185] Step S130g, whether it is selected after being recommended.

[0186] In an embodiment of the present application, it is detected whether the cold start layout template in the cold start layout pool is selected after being recommended, that is, whether the first layout template belonging to the cold start layout pool among the recommended N layout templates is selected. If so, step S130h is executed; if not, step S130i is executed.

[0187] Step S130h: updating the recommended weight value of the cold start layout template, and determining the storage location of the cold start layout template based on the recommended weight value.

[0188] In the embodiment of the present application, if the layout page corresponding to the first layout template is triggered to select a page, the first historical selection times of the first layout template can be updated, and the recommended weight value of the first layout template can be increased. If the recommended weight value of the first layout template is greater than or equal to the weight threshold, the first layout template is added to the hot start layout pool; if the recommended weight value of the first layout template is less than the weight threshold, the first layout template is still stored in the cold start layout pool.

[0189] Step S130i: Process the cold start layout template based on the layout utilization rate.

[0190] In an embodiment of the present application, the layout utilization of the first layout template can be obtained. If the layout utilization of the first layout template is less than the layout retention threshold, the first layout template in the cold start layout pool is deleted; if the layout utilization of the first layout template is greater than or equal to the layout retention threshold, the first layout template is not processed.

[0191] For the hot start layout pool, perform the following steps:

[0192] Step S130k: after being recommended, update the recommendation weight value based on user feedback.

[0193] In an embodiment of the present application, after a hot start layout template in the hot start layout pool is recommended, that is, the second layout template among N layout templates, the recommendation weight value can be updated based on user feedback, that is, detecting whether the layout page corresponding to the second layout template is triggered by the page selection operation, updating the second historical selection times of the second layout template based on the detection result, and determining the recommendation weight value of the second layout template based on the second historical recommendation times and the second historical selection times of the second layout template.

[0194] Step S130 l , whether the recommended weight value is lower than the weight threshold.

[0195] In the embodiment of the present application, it is detected whether the recommendation weight of the hot start layout template in the hot start layout pool is lower than the weight threshold. If it is lower than the weight threshold, step S130m is executed.

[0196] Step S130m: remove the hot start layout templates with a weight lower than the threshold from the hot start layout pool.

[0197] In an embodiment of the present application, a hot start layout template having a recommended weight value less than a weight threshold may be removed from the hot start layout pool. Optionally, the removed hot start layout template may also be added to the cold start layout pool.

[0198] Further, see Fig.14 , Fig.14: is a flow chart of a method for updating a recommendation weight value provided by an embodiment of the present application. Fig.14 As shown, the method comprises the following steps:

[0199] Step S140a: extracting a sample layout template from sample text data in the corpus.

[0200] Step S140b, generating a layout pool.

[0201] In an embodiment of the present application, a layout pool can be generated based on a sample layout template, and the layout pool can include a hot start layout pool and a cold start layout pool, wherein the cold start layout pool is used to store newly extracted layout templates. Optionally, both the hot start layout pool and the cold start layout pool can be divided into multiple layout pools based on the number of texts and the number of pictures, that is, the hot start layout pool can include one or at least two hot start sub-layout pools, and the number of texts and the number of pictures corresponding to each hot start sub-layout pool are different from each other; the cold start layout pool can include one or at least two cold start sub-layout pools, and the number of texts and the number of pictures corresponding to each cold start sub-layout pool are different from each other. Optionally, there can also be one or at least two layout pools, and the number of texts and the number of pictures corresponding to each layout pool are different from each other, that is, the number of texts and the number of pictures of layout pool 1 are different from the number of texts and the number of pictures of layout pool 2, and each layout pool can include a hot start sub-layout pool and a cold start sub-layout pool.

[0202] Step S140c, editing text data.

[0203] In the embodiments of this application, please refer to Figure 7 The specific description of step S701 in will not be repeated here.

[0204] Step S140d, semantic analysis.

[0205] In the embodiments of this application, please refer to Figure 7 The specific description of step S702 in will not be repeated here.

[0206] Step S140e: recommend M candidate images based on the semantic analysis result.

[0207] In the embodiments of this application, please refer to Figure 7 The specific description of step S703 in will not be repeated here.

[0208] Step S140f, responding to the picture selection operation for the target picture.

[0209] Step S140g, start typesetting recommendation.

[0210] Step S140h: Obtain N layout templates from the layout pool.

[0211] In the embodiment of the present application, step S140f, step S140g and step S140h can refer to Figure 7 The specific description of step S704 in the above description will not be repeated here. The user device can generate N layout pages according to the N layout templates and display the N layout pages in the data recommendation area.

[0212] Step S140i, in response to a page selection operation for a target layout page, applying the target layout page.

[0213] In the embodiments of this application, please refer to Figure 7 The specific description of step S705 in will not be repeated here.

[0214] Step S140j, performing weighted feedback on the layout template.

[0215] In the embodiment of the present application, based on the page selection operation of the target layout page, weighted feedback is performed on N layout templates.

[0216] Step S140k: updating the recommended weight values ​​of the N layout templates.

[0217] In an embodiment of the present application, based on the page selection operation of the target layout page, the historical recommendation times and historical selection times of N layout templates are updated, and the recommendation weight values ​​of the N layout templates are updated according to the historical recommendation times and historical selection times of each layout template.

[0218] Further, see Fig.15 , Fig.15 is a schematic diagram of a layout dictionary tree provided in an embodiment of the present application, such as Fig.15 As shown in FIG. 1 , the layout dictionary tree is a dictionary tree constructed based on the number of texts and the number of images. For example, assuming that the first display page includes a file image and two paragraphs of text data, the following will be triggered: Fig.15In the layout pool C of the "single-image dual-text layout pool" indicated in the layout dictionary tree shown, N layout templates are obtained from the "single-image dual-text layout pool". Specifically, when the target image is inserted into the first display page, the target image is searched, and the first display page may include the existence of a file image, text data, or no other content. If there is no other content in the first display page, N layout templates can be obtained from the single-image layout pool; if there is text data in the first display page, and there is no other content on this basis, N layout templates can be obtained from the single-image single-text layout pool; if there are two paragraphs of text data in the first display page, N layout templates can be obtained from the single-image dual-text layout pool; if there is a file image and a paragraph of text data in the first display page, N layout templates can be obtained from the dual-image single-text layout pool; if there is a file image in the first display page, and there is no other content on this basis, N layout templates can be obtained from the dual-image no-text layout pool, etc.

[0219] Among them, see Fig.16 , Fig.16 This is a schematic diagram of a subject data structure provided by an embodiment of the present application. Fig.16 As shown, assuming that the media file is a presentation file, the theme tree corresponding to the media file can be a tree composed of a presentation node (Presentation) as the root node, and the theme tree includes one or more style nodes, such as a master node, a layout node, a file page node, a note master node, a note file page node, a handout master node, a theme (Themes) node, a code (Code) node, a presentation file property node, a view property (View Properties) node and a font node, etc. Among them, the root node and the style node in the theme tree, as well as the edges between the style nodes and the style nodes, are used to represent the association relationship between the nodes connected by the edges.

[0220] For further information, see Fig.17 , Fig.17 Schematic diagram of a file processing device provided in an embodiment of the present application. The file processing device may be a computer program (including program code, etc.) running in a computer device, for example, the file processing device may be an application software; the device may be used to execute the corresponding steps in the method provided in an embodiment of the present application. Fig.17 As shown, the file processing device 1700 can be used to Figure 3 The user equipment in the corresponding embodiment may specifically include: a candidate image display module 11 , a layout display module 12 and a layout selection module 13 .

[0221] The candidate image display module 11 is used to display text data in the first display page of the media file, and for the text data, display M candidate images in the data recommendation area; M is a positive integer;

[0222] The layout display module 12 is used to respond to the image selection operation for the target image among the M candidate images and display N layout pages in the data recommendation area; each layout page includes the target image and the text data displayed in the first display page of the media file; N is a positive integer;

[0223] The layout selection module 13 is used to respond to a page selection operation for a target layout page among the N layout pages and switch the first display page to be displayed as the target layout page.

[0224] The candidate image display module 11 includes:

[0225] The text input unit 111 is used to respond to the input operation of target text data for the media file and display the target text data in the media file;

[0226] The candidate picture display unit 112 is used to respond to the picture recommendation operation for the target text data and display M candidate pictures associated with the target text data in the data recommendation area.

[0227] The candidate image display module 11 includes:

[0228] The candidate image display unit 112 further includes displaying the target text data in the first display page of the media file and displaying M candidate images associated with the target text data in the data recommendation area; or,

[0229] The candidate image display unit 112 further includes displaying the target text data in the first display page of the media file, and when a trigger operation for the image recommendation component is received, displaying M candidate images associated with the target text data in the data recommendation area; or,

[0230] A component display unit 113, configured to display a picture recommendation component in response to a trigger operation on target text data;

[0231] The component triggering unit 114 is used to respond to the triggering operation on the picture recommendation component and display M candidate pictures associated with the target text data in the data recommendation area.

[0232] The device 1700 further includes:

[0233] The utilization rate display module 14 is used to respond to the viewing operation of the target picture among the M candidate pictures and display the historical utilization rate of the target picture; the historical utilization rate is used to indicate the probability of the target picture being selected.

[0234] Wherein, the layout page also includes a text box to be edited; the device also includes:

[0235] The text box display module 15 is used to display content prompt text in the target to-be-edited text box on the target layout page, and in response to an input operation on the target to-be-edited text box, switch the content prompt text in the target to-be-edited text box to display the text content corresponding to the input operation.

[0236] The media file includes at least two display pages, and the at least two display pages include a first display page; the device 1700 also includes:

[0237] The marking display module 16 is used to mark the second display page having the same semantic information as the first display page among the at least two display pages; the illustration mark is used to indicate that M candidate images are recommended for the second display page.

[0238] The candidate image display module 11 includes:

[0239] The text acquisition unit 115 is used to respond to the image recommendation request for the media file and acquire the text data displayed in the first display page of the media file based on the image recommendation request;

[0240] A keyword extraction unit 116 is used to extract keywords from the text data to obtain text keywords corresponding to the text data;

[0241] The picture acquisition unit 117 is used to acquire M candidate pictures matching the text keywords from the candidate recommendation library, and display the M candidate pictures in the data recommendation area.

[0242] The keyword extraction unit 116 includes:

[0243] The text segmentation subunit 1161 is used to perform segmentation processing on the text data to obtain f1 segmentation phrases; f1 is a positive integer;

[0244] The word frequency acquisition subunit 1162 is used to acquire the phrase frequencies corresponding to the f1 segmented word phrases;

[0245] The inverse frequency acquisition subunit 1163 is used to obtain the inverse document frequencies corresponding to the f1 segmented word groups;

[0246] The importance determination subunit 1164 is used to determine the importance of the phrases corresponding to the f1 segmented word phrases according to the phrase frequencies and inverse document frequencies corresponding to the f1 segmented word phrases;

[0247] The keyword selection subunit 1165 is used to determine the text keywords corresponding to the text data from the f1 segmented word groups based on the word group importance.

[0248] The word frequency acquisition subunit 1162 includes:

[0249] The phrase division subunit 111a is used to divide the same segmented phrases in the f1 segmented phrases into f2 phrase sets; the segmented phrases included in each phrase set are the same; f2 is a positive integer;

[0250] The word frequency determination subunit 111b is used to count the number of phrases of the segmented phrases included in the i-th phrase set, and determine the sum of the number of phrases corresponding to the i-th phrase set and the number of phrases of the segmented phrases respectively included in the f2 phrase sets as the phrase frequency corresponding to the segmented phrases included in the i-th phrase set.

[0251] The inverse frequency acquisition subunit 1163 includes:

[0252] The sample word segmentation subunit 112a is used to obtain at least two sample text data included in the corpus, perform word segmentation processing on the at least two sample text data respectively, and obtain sample word segmentation phrases corresponding to the at least two sample text data respectively;

[0253] The associated statistical subunit 112b is used to determine the number of sample text data associated with the sample segmentation phrase as the number of associated texts of the sample segmentation phrase;

[0254] The inverse frequency determination subunit 112c is used to obtain the total number of sample texts of at least two sample text data, and determine the inverse document frequency of the sample segmentation phrase according to the total number of sample texts and the associated text data of the sample segmentation phrase.

[0255] The device 1700 further includes:

[0256] The invalid phrase classification module 17 is used to determine the segmentation phrases whose inverse document frequency is less than the valid phrase threshold as invalid segmentation phrases, and record the segmentation phrases other than the invalid segmentation phrases in the f1 segmentation phrases as valid segmentation phrases;

[0257] The keyword selection subunit 1165 is specifically used for:

[0258] Based on the phrase importance, text keywords corresponding to the text data are determined from the valid segmented phrases.

[0259] The text segmentation subunit 1161 includes:

[0260] The word graph generation subunit 113a is used to split the text data to obtain at least two characters constituting the text data, and to form a directed word graph with the at least two characters; the at least two characters are nodes of the directed word graph;

[0261] A path acquisition subunit 113b is used to obtain at least two character paths according to the directed word graph and the association between adjacent characters in the directed word graph;

[0262] A path screening subunit 113c is used to obtain the path lengths corresponding to at least two character paths respectively, and determine the shortest character path from the at least two character paths according to the path lengths;

[0263] The phrase generation subunit 113d is used to group the characters corresponding to the shortest character path into segmented phrases to obtain f1 segmented phrases.

[0264] The image acquisition unit 117 includes:

[0265] The first vector conversion subunit 1171 is used to perform vector conversion on text keywords to obtain keyword vectors;

[0266] The tag matching subunit 1172 is used to obtain at least two associated pictures whose picture tags are associated with the text keywords from the candidate recommendation picture library;

[0267] The second vector conversion subunit 1173 is used to perform picture encoding on at least two associated pictures respectively to obtain picture vectors corresponding to the at least two associated pictures respectively;

[0268] A similarity acquisition subunit 1174 is used to determine the semantic similarity between the at least two associated pictures and the text keywords according to the vector distance between the picture vectors and the keyword vectors corresponding to the at least two associated pictures respectively;

[0269] The picture selection subunit 1175 is used to obtain M candidate pictures from at least two related pictures based on semantic similarity.

[0270] The first vector conversion subunit 1171 is specifically used for:

[0271] Map text keywords to the target semantic space to obtain the first vector of the text keywords;

[0272] Performing dimensionality reduction processing on the first vector to obtain a keyword vector having a target vector length;

[0273] The second vector conversion subunit 1173 is specifically used for:

[0274] Mapping at least two associated images to the target semantic space respectively to obtain second vectors corresponding to the at least two associated images respectively;

[0275] Dimensionality reduction processing is performed on at least two second vectors to obtain picture vectors corresponding to at least two associated pictures respectively; the length of the picture vector is the target vector length.

[0276] The layout display module 12 includes:

[0277] The first quantity acquisition unit 12a is used to respond to the picture selection operation for the target picture among the M candidate pictures, acquire the number of layout selections and the layout recommendation ratio, and determine the first layout recommendation number and the second layout recommendation number according to the number of layout selections and the layout recommendation ratio; the sum of the first layout recommendation number and the second layout recommendation number is the layout selection number;

[0278] The template acquisition unit 12b is used to acquire the first layout template from the cold start layout pool based on the first layout recommendation quantity, and acquire the second layout template from the hot start layout pool based on the second layout recommendation quantity; the recommendation weight value of the second layout template is greater than the recommendation weight value of the first layout template;

[0279] The layout generation unit 12c is used to combine the first layout template and the second layout template with the target image respectively to generate N layout pages, where N is the number of layout selections;

[0280] The layout display unit 12d is used to display N layout pages in the data recommendation area.

[0281] The device further comprises:

[0282] A first number updating module 18 is used to update the target history recommendation number and the target history selection number of the target layout page based on the page selection operation for the target layout page;

[0283] The weight updating module 19 is used to update the recommendation weight value of the target layout template corresponding to the target layout page according to the target historical recommendation times and the target historical selection times.

[0284] The device 1700 further includes:

[0285] A first weight acquisition module 20 is used to acquire at least two cold start layout templates included in the cold start layout pool and a first recommended weight value corresponding to each cold start layout template;

[0286] The storage update module 21 is used to add the cold start layout template whose first recommended weight value is greater than or equal to the weight threshold to the hot start layout pool;

[0287] A second weight acquisition module 22 is used to acquire at least two hot start layout templates included in the hot start layout pool and a second recommended weight value corresponding to each hot start layout template;

[0288] The template removing module 23 is used to remove the hot start layout template whose second recommended weight value is less than the weight threshold from the hot start layout pool.

[0289] The device 1700 further includes:

[0290] A second times updating module 24 is configured to update a first historical times of recommendation of the first layout template if the time for which the layout page corresponding to the first layout template is in the page display state is greater than or equal to the recommended exposure time threshold;

[0291] A utilization rate acquisition module 25, configured to acquire a layout utilization rate of the first layout template according to the updated first historical recommendation number and the first historical selection number of the first layout template;

[0292] The template deletion module 26 is used to delete the first layout template in the cold start layout pool if the layout utilization rate is less than the layout retention threshold.

[0293] The layout display module 12 includes:

[0294] A layout acquisition unit 12e, configured to acquire N layout templates in response to a picture selection operation for a target picture among the M candidate pictures;

[0295] The picture adding unit 12f is used to obtain the picture display areas respectively included in the N layout templates, add the target picture to the picture display areas in the N layout templates, and generate N layout pages;

[0296] The page display unit 12g is used to display N layout pages in the data recommendation area.

[0297] The layout display module 12 includes:

[0298] A second quantity acquisition unit 12h is used to respond to the picture selection operation for the target picture among the M candidate pictures, and acquire the displayed text quantity of the text data displayed in the first display page of the media file and the displayed picture quantity of the file picture;

[0299] The target layout acquisition unit 12i is used to acquire a target layout pool corresponding to the number of displayed texts and the number of displayed pictures, and acquire N layout templates from the target layout pool;

[0300] The page generation unit 12j is used to write the target image, text data and file image into N layout templates to generate N layout pages;

[0301] The page display unit 12g is also used to display N layout pages in the data recommendation area.

[0302] An embodiment of the present application provides a file processing device, which can be run in a user device. In a media application, the user device can display text data in a first display page of a media file, and for the text data, display M candidate images in a data recommendation area; M is a positive integer; in response to an image selection operation for a target image among the M candidate images, N layout pages are displayed in the data recommendation area; each layout page includes the target image and the text data displayed in the first display page of the media file; N is a positive integer; in response to a page selection operation for a target layout page among the N layout pages, the first display page is switched to be displayed as the target layout page. Through the above process, relevant pictures can be recommended for the media file according to the content in the media file for the user to choose, without the user having to search for pictures by himself, which can reduce the cost of obtaining pictures when the user retrieves and produces media files; and when the user selects the target picture he wants to use, a layout page including the target picture is provided to the user based on the target picture. The layout page is a page that has been typeset, that is, the user can directly select the desired target layout page from the layout pages provided by the user device, realize direct typesetting of the page, reduce the typesetting cost of the media file, and thereby improve the generation efficiency of the media file, improve the flexibility of media file generation and the display effect, etc.

[0303] See also Fig.18 , Fig.18 Schematic diagram of the structure of a computer device provided in an embodiment of the present application. Fig.18 As shown, the computer device in the embodiment of the present application may include: one or more processors 1801, a memory 1802, and an input-output interface 1803. The processor 1801, the memory 1802, and the input-output interface 1803 are connected via a bus 1804. The memory 1802 is used to store a computer program, which includes program instructions, and the input-output interface 1803 is used to receive and output data, such as for data interaction between a user device and a server; the processor 1801 is used to execute the program instructions stored in the memory 1802.

[0304] The processor 1801 may perform the following operations:

[0305] Displaying text data in a first display page of a media file, and displaying M candidate images in a data recommendation area for the text data, where M is a positive integer;

[0306] In response to an image selection operation for a target image among the M candidate images, N layout pages are displayed in the data recommendation area; each layout page includes the target image and text data displayed in the first display page of the media file; N is a positive integer;

[0307] In response to a page selection operation for a target layout page among the N layout pages, the first display page is switched to be displayed as the target layout page.

[0308] In some feasible implementations, the processor 1801 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0309] The memory 1802 may include a read-only memory and a random access memory, and provide instructions and data to the processor 1801 and the input / output interface 1803. A portion of the memory 1802 may also include a non-volatile random access memory. For example, the memory 1802 may also store information about the device type.

[0310] In a specific implementation, the computer device can execute the following operations through its built-in functional modules: Figure 3 For details on the implementation methods provided in each step, please refer to the Figure 3 The implementation methods provided in each step will not be repeated here.

[0311] The embodiment of the present application provides a computer device, including: a processor, an input / output interface, and a memory, wherein the processor obtains a computer program in the memory and executes the computer program. Figure 3The steps of the method shown in the embodiment of the present application are used to perform file processing operations. The embodiment of the present application realizes recommending relevant pictures for the media file according to the content of the media file for the user to choose, without the user having to search for pictures by himself, which can reduce the cost of obtaining pictures when the user retrieves and makes media files; and when the user selects the target picture he wants to use, the layout page including the target picture is provided to the user based on the target picture. The layout page is a page that has been typeset, that is, the user can directly select the desired target layout page from the layout page provided by the user device, realize direct typesetting of the page, reduce the typesetting cost of the media file, and then improve the generation efficiency of the media file, improve the flexibility of the media file generation and the display effect, etc.

[0312] The present application also provides a computer-readable storage medium storing a computer program suitable for being loaded and executed by the processor. Figure 3 For details on the file processing methods provided in each step, please refer to the Figure 3 The implementation methods provided in each step are not repeated here. In addition, the description of the beneficial effects of adopting the same method is not repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, the computer program can be deployed to execute on one computer device, or on multiple computer devices located in one place, or on multiple computer devices distributed in multiple locations and interconnected by a communication network.

[0313] The computer-readable storage medium may be the file processing device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Further, the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0314] The present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes Figure 3 The method provided in the various optional methods realizes recommending relevant pictures for the media file according to the content in the media file for the user to choose, without the user having to search for pictures by himself, which can reduce the cost of obtaining pictures when the user retrieves and produces media files; and when the user selects the target picture he wants to use, based on the target picture, the user is provided with a layout page including the target picture, which is a page that has been typeset, that is, the user can directly select the desired target layout page from the layout pages provided by the user device, realize direct typesetting of the page, reduce the typesetting cost of the media file, and thus improve the generation efficiency of the media file, improve the flexibility of the media file generation and the display effect, etc.

[0315] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, device, product, or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally includes steps or modules that are not listed, or optionally includes other step units inherent to these processes, methods, devices, products, or equipment.

[0316] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in this description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0317] The method and related apparatus provided by the embodiment of the present application are described with reference to the method flow chart and / or structural diagram provided by the embodiment of the present application. Specifically, each process and / or box in the method flow chart and / or structural diagram, as well as the combination of the processes and / or boxes in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable file processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable file processing device generate instructions for implementing the process in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable file processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable file processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A flow or multiple flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0318] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0319] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0320] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A file processing method, characterized in that: The method comprises: Displaying text data in a first display page of a media file, and displaying M candidate images associated with target text data of the text data in a data recommendation area; including: displaying the target text data in the first display page of the media file, and displaying M candidate images associated with the target text data in the data recommendation area; or, displaying the target text data in the first display page of the media file, and when a trigger operation for a picture recommendation component is received, displaying the M candidate images associated with the target text data in the data recommendation area; or, displaying the target text data in the first display page of the media file, displaying a picture recommendation component in response to a trigger operation for the target text data, and displaying the M candidate images associated with the target text data in the data recommendation area in response to a trigger operation for the picture recommendation component; M is a positive integer; the media file includes one or more of a document file, a video file, or a presentation file; In response to an image selection operation for a target image among the M candidate images, N layout pages are displayed in the data recommendation area; including: in response to an image selection operation for a target image among the M candidate images, the number of displayed texts of the text data and the number of displayed images of the text images displayed in the first display page of the media file are obtained, a target layout pool corresponding to the number of displayed texts and the number of displayed images is obtained, N layout templates are obtained from the target layout pool, the target image, the text data and the text images are written into the N layout templates, N layout pages are generated, and the N layout pages are displayed in the data recommendation area; each layout page includes the target image and the text data displayed in the first display page of the media file; N is a positive integer; In response to a page selection operation for a target layout page among the N layout pages, the first display page is switched to be displayed as the target layout page.

2. The method according to claim 1, characterized in that The step of displaying text data in a first display page of a media file and displaying M candidate images associated with target text data in a data recommendation area includes: In response to an input operation on target text data of a media file, displaying the target text data in the media file; In response to the picture recommendation operation for the target text data, M candidate pictures associated with the target text data are displayed in the data recommendation area.

3. The method according to claim 1, characterized in that The method further comprises: In response to a viewing operation on the target picture among the M candidate pictures, a historical utilization rate of the target picture is displayed; the historical utilization rate is used to indicate a probability of the target picture being selected.

4. The method according to claim 1, characterized in that The layout page also includes a text box to be edited; the method also includes: Displaying content prompt text in a target to-be-edited text box on the target layout page, and in response to an input operation on the target to-be-edited text box, switching the content prompt text in the target to-be-edited text box to display text content corresponding to the input operation.

5. The method according to claim 1, characterized in that The media file includes at least two display pages, and the at least two display pages include the first display page; the method further includes: An illustration mark is performed on a second display page of the at least two display pages that has the same semantic information as the first display page; the illustration mark is used to indicate that the M candidate images are recommended for the second display page.

6. The method according to claim 1, characterized in that In response to the image recommendation request for the media file, displaying M candidate images in the data recommendation area includes: In response to a request for recommending an image of the media file, obtaining text data displayed in the first display page of the media file based on the request for recommending the image; Performing keyword extraction on the text data to obtain text keywords corresponding to the text data; M candidate images matching the text keyword are obtained from a candidate recommendation library, and the M candidate images are displayed in the data recommendation area.

7. The method according to claim 6, characterized in that The step of extracting keywords from the text data to obtain text keywords corresponding to the text data includes: Perform word segmentation processing on the text data to obtain f1 word segmentation phrases; f1 is a positive integer; Obtaining the phrase frequencies corresponding to the f1 segmented word groups, obtaining the inverse document frequencies corresponding to the f1 segmented word groups, and determining the phrase importances corresponding to the f1 segmented word groups according to the phrase frequencies and inverse document frequencies corresponding to the f1 segmented word groups; Based on the phrase importance, text keywords corresponding to the text data are determined from the f1 segmented phrases.

8. The method according to claim 7, characterized in that The obtaining of the phrase frequencies corresponding to the f1 segmented phrases respectively includes: The same segmentation phrases in the f1 segmentation phrases are divided into f2 phrase sets; the segmentation phrases included in each phrase set are the same; f2 is a positive integer; The number of phrases of the segmented phrases included in the i-th phrase set is counted, and the sum of the number of phrases corresponding to the i-th phrase set and the number of phrases of the segmented phrases respectively included in the f2 phrase sets is determined as the phrase frequency corresponding to the segmented phrases included in the i-th phrase set.

9. The method according to claim 7, characterized in that The obtaining of the inverse document frequencies corresponding to the f1 segmented word groups respectively includes: Acquire at least two sample text data included in the corpus, perform word segmentation processing on the at least two sample text data respectively, and obtain sample word segmentation phrases corresponding to the at least two sample text data respectively; Determine the number of sample text data associated with the sample segmentation phrase as the number of associated texts of the sample segmentation phrase; The total number of sample texts of the at least two sample text data is obtained, and the inverse document frequency of the sample segmentation phrase is determined according to the total number of sample texts and the associated text data of the sample segmentation phrase.

10. The method according to claim 7, characterized in that The method further comprises: Determine the segmentation phrases whose inverse document frequency is less than the valid threshold value of the phrase as invalid segmentation phrases, and record the segmentation phrases other than the invalid segmentation phrases in the f1 segmentation phrases as valid segmentation phrases; The determining of the text keywords corresponding to the text data from the f1 segmented word phrases based on the phrase importance includes: Based on the phrase importance, text keywords corresponding to the text data are determined from the valid segmented word phrases.

11. The method according to claim 7, characterized in that The text data is segmented to obtain f1 segmented phrases, including: Splitting the text data to obtain at least two characters constituting the text data, and forming a directed word graph with the at least two characters; the at least two characters are nodes of the directed word graph; Obtaining at least two character paths according to the directed word graph and the association between adjacent characters in the directed word graph; Obtaining path lengths respectively corresponding to the at least two character paths, and determining the shortest character path from the at least two character paths according to the path lengths; The characters corresponding to the shortest character path are grouped into segmentation phrases to obtain f1 segmentation phrases.

12. The method according to claim 6, characterized in that The step of obtaining M candidate images matching the text keyword from the candidate recommendation gallery includes: Performing vector conversion on the text keywords to obtain keyword vectors; Acquire at least two associated pictures whose picture labels are associated with the text keyword from the candidate recommendation picture library, perform picture encoding on the at least two associated pictures respectively, and obtain picture vectors corresponding to the at least two associated pictures respectively; Determining semantic similarities between the at least two associated pictures and the text keywords respectively according to vector distances between picture vectors corresponding to the at least two associated pictures and the keyword vector; Based on the semantic similarity, M candidate pictures are obtained from the at least two associated pictures.

13. The method according to claim 12, characterized in that The step of performing vector conversion on the text keywords to obtain keyword vectors includes: Mapping the text keywords to a target semantic space to obtain a first vector of the text keywords; Performing dimensionality reduction processing on the first vector to obtain a keyword vector having a target vector length; The performing picture encoding on the at least two associated pictures respectively to obtain picture vectors corresponding to the at least two associated pictures respectively includes: Mapping the at least two associated pictures to the target semantic space respectively, to obtain second vectors corresponding to the at least two associated pictures respectively; Perform dimensionality reduction processing on at least two second vectors to obtain picture vectors corresponding to the at least two associated pictures respectively; the length of the picture vector is the length of the target vector.

14. The method according to claim 1, wherein: In response to the picture selection operation of the target picture among the M candidate pictures, displaying N layout pages in the data recommendation area includes: In response to an image selection operation for a target image among the M candidate images, a layout selection quantity and a layout recommendation ratio are obtained, and a first layout recommendation quantity and a second layout recommendation quantity are determined according to the layout selection quantity and the layout recommendation ratio; the sum of the first layout recommendation quantity and the second layout recommendation quantity is the layout selection quantity; Based on the first layout recommendation quantity, a first layout template is obtained from a cold start layout pool; based on the second layout recommendation quantity, a second layout template is obtained from a hot start layout pool; a recommendation weight value of the second layout template is greater than a recommendation weight value of the first layout template; The first layout template and the second layout template are respectively combined with the target image to generate N layout pages, where N is the number of layout selections; The N layout pages are displayed in the data recommendation area.

15. The method according to claim 14, characterized in that The method further comprises: Acquire at least two cold start layout templates included in the cold start layout pool and a first recommendation weight value corresponding to each cold start layout template; Adding the cold start layout template whose first recommendation weight value is greater than or equal to the weight threshold to the hot start layout pool; Acquire at least two hot start layout templates included in the hot start layout pool and a second recommendation weight value corresponding to each hot start layout template; The hot start layout template whose second recommendation weight value is less than the weight threshold is removed from the hot start layout pool.

16. The method according to claim 14, characterized in that The method further comprises: If the time that the layout page corresponding to the first layout template is in the page display state is greater than or equal to the recommended exposure time threshold, updating the first historical recommendation number of the first layout template; Acquire a layout utilization rate of the first layout template according to the updated first historical recommendation number and the first historical selection number of the first layout template; If the layout utilization rate is less than the layout retention threshold, the first layout template in the cold start layout pool is deleted.

17. The method according to claim 1, wherein: In response to the picture selection operation of the target picture among the M candidate pictures, displaying N layout pages in the data recommendation area includes: In response to an image selection operation for a target image among the M candidate images, obtaining N layout templates; Acquire the picture display areas respectively included in the N layout templates, add the target picture to the picture display areas in the N layout templates, and generate N layout pages; The N layout pages are displayed in the data recommendation area.

18. A file processing device, characterized in that: The device comprises: A candidate image display module, for displaying text data in a first display page of a media file, and for target text data of the text data, displaying M candidate images associated with the target text data in a data recommendation area; comprising: displaying the target text data in the first display page of the media file, and displaying M candidate images associated with the target text data in the data recommendation area; or, displaying the target text data in the first display page of the media file, and when a trigger operation for an image recommendation component is received, displaying the M candidate images associated with the target text data in the data recommendation area; or, displaying the target text data in the first display page of the media file, displaying an image recommendation component in response to a trigger operation for the target text data, and displaying the M candidate images associated with the target text data in the data recommendation area in response to a trigger operation for the image recommendation component; M is a positive integer; the media file comprises one or more of a document file, a video file or a presentation file; A layout display module, for responding to an image selection operation for a target image among the M candidate images, and displaying N layout pages in the data recommendation area; comprising: in response to an image selection operation for a target image among the M candidate images, obtaining the number of displayed texts of the text data and the number of displayed images of the text images displayed in the first display page of the media file, obtaining a target layout pool corresponding to the number of displayed texts and the number of displayed images, obtaining N layout templates from the target layout pool, writing the target image, the text data and the text image into the N layout templates, generating N layout pages, and displaying the N layout pages in the data recommendation area; each layout page includes the target image and the text data displayed in the first display page of the media file; N is a positive integer; The layout selection module is used to respond to a page selection operation on a target layout page among the N layout pages, and switch the first display page to display the target layout page.

Images