Method and device for generating sharing material, equipment, storage medium

Abstract

Embodiments of the present application disclose a method and device for generating sharing materials, equipment and storage medium. The method can generate sharing materials according to filtered photos, material template information and a preset AI model, improve the efficiency of generating sharing materials, and no longer rely on third-party software to process photos. The method comprises the following steps: obtaining personalized configuration information of a user, wherein the personalized configuration information comprises filtering conditions and material template information; selecting filtered photos according to the filtering conditions; generating sharing materials according to the filtered photos, the material template information and a preset artificial intelligence (AI) model, wherein the quality of the photos in the sharing materials is greater than a preset photo quality; wherein the preset AI model is a model obtained by model training on sample photos, or a model obtained by model training on sample photos and sample material template information.

Description

Technical Field

[0001] This application relates to neural network technology, and to, but is not limited to, a method, apparatus, device, and storage medium for generating and sharing materials. Background Technology

[0002] Some photo albums have a "Memories" feature that generates a short music video (MV) based on the user's timeline at a specific location. There's also a "Featured Photos" feature that selects a certain number of photos from all timelines according to preset rules to display to the user. The generated featured photos have no templates; they are displayed as single images, requiring users to edit them before sharing them on social media platforms. Summary of the Invention

[0003] In view of this, the method, apparatus, device, and storage medium for generating shareable materials provided in this application can generate shareable materials based on selected photos, material template information, and preset AI models, improving the efficiency of generating shareable materials and eliminating reliance on third-party software for photo processing. The method, apparatus, device, and storage medium for generating shareable materials provided in this application are implemented as follows:

[0004] The method for generating shareable materials provided in this application embodiment may include: obtaining a user's personalized configuration information, the personalized configuration information including filtering conditions and material template information; selecting a filter photo according to the filtering conditions; generating shareable materials according to the filter photo, the material template information and a preset artificial intelligence (AI) model, wherein the quality of the photo in the shareable materials is greater than the preset photo quality; wherein the preset AI model is a model obtained by training the sampled photo, or a model obtained by training the sampled photo and the sampled material template information.

[0005] The apparatus for generating and sharing materials provided in this application embodiment may include:

[0006] The acquisition module is used to acquire the user's personalized configuration information, which includes filtering conditions and material template information.

[0007] The selection module is used to select photos for filtering based on the filtering criteria.

[0008] The generation module is used to generate sharing materials based on the selected photos, the material template information, and the preset artificial intelligence AI model, wherein the quality of the photos in the sharing materials is greater than the preset photo quality;

[0009] The preset artificial intelligence (AI) model is a model obtained by training the sampled photos, or a model obtained by training the sampled photos and sampled material template information.

[0010] The computer device provided in this application includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the method described in this application.

[0011] The computer-readable storage medium provided in this application embodiment stores a computer program thereon, which, when executed by a processor, implements the method described in this application embodiment.

[0012] This application discloses a computer program product that, when run on a computer, causes the computer to execute the method described in the first aspect of this application.

[0013] This application discloses an application publishing platform for publishing computer program products. When the computer program product is run on a computer, the computer executes the method described in the first aspect of this application.

[0014] The method, apparatus, computer device, and computer-readable storage medium for generating shareable materials provided in this application embodiment obtain personalized configuration information from users, including filtering conditions and material template information; selects filtered photos according to the filtering conditions; and generates shareable materials based on the filtered photos, the material template information, and a preset artificial intelligence (AI) model, wherein the quality of the photos in the shareable materials is greater than the preset photo quality; wherein the preset AI model is a model obtained by training the sampled photos, or a model obtained by training the sampled photos and sampled material template information. That is, shareable materials can be directly generated based on the filtered photos, material template information, and the preset AI model, without the need for third-party software to process the photos, thus improving the efficiency of generating shareable materials, saving system resources, and solving the technical problems mentioned in the background art. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0016] Figure 1 This is a schematic diagram of one embodiment of the method for generating sharing materials in this application.

[0017] Figure 2AThis is a schematic diagram of a user opening a photo album in an embodiment of this application;

[0018] Figure 2B This is a schematic diagram illustrating whether the mobile phone system prompts users on the display interface whether they need to generate materials to share on social media, as described in this application embodiment.

[0019] Figure 2C This is a schematic diagram of a user performing configuration operations on a personalized configuration interface in an embodiment of this application.

[0020] Figure 2D This is a schematic diagram of one embodiment of the shared materials generated in the embodiments of this application;

[0021] Figure 3A A flowchart illustrating the overall process for generating shareable materials provided in this application embodiment;

[0022] Figure 3B A schematic diagram illustrating a process for generating video sharing materials and photo sharing materials provided in an embodiment of this application;

[0023] Figure 4 A schematic diagram of the apparatus for generating and sharing materials provided in an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of one embodiment of the terminal device in this application;

[0025] Figure 6 This is a schematic diagram of another embodiment of the terminal device in this application;

[0026] Figure 7 This application provides a schematic diagram of the structure of a computer device. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0029] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0030] It should be noted that the terms "first, second, third" used in the embodiments of this application are used to distinguish similar or different objects and do not represent a specific order of objects. It can be understood that "first, second, third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0031] The following is a brief explanation of some of the terms used in this application:

[0032] EXIF: an abbreviation for Exchangeable Image File format, is a format specifically designed for digital camera photos. It can record the attribute information and shooting data of digital photos.

[0033] ROI (Region of Interest) is a region of interest defined in machine vision and image processing. It is a region of interest that needs to be processed within an image, often outlined using shapes such as rectangles, circles, ellipses, or irregular polygons. Various operators and functions are commonly used in machine vision software such as Halcon, OpenCV, and Matlab to determine the ROI and perform further image processing.

[0034] In the "Memories" feature, for example, in Changsha, photos taken between January and June will be selected according to a set strategy to generate a short music video with audio. However, regardless of whether it's in the "Memories" or "Featured Photos" feature, the generated music videos lack any special effects, resulting in a poor user experience. Furthermore, the image selection is not intelligent enough, sometimes including meaningless screenshots or blurry photos.

[0035] The technical solution of this application will be further described below by way of embodiments, such as... Figure 1 The diagram shown is a schematic representation of an embodiment of a method for generating shareable materials according to this application, which may include:

[0036] 101. Obtain the user's personalized configuration information, which includes filtering conditions and material template information.

[0037] Optionally, the filtering criteria may include, but are not limited to, time range, geographical location, and photo type.

[0038] Optionally, the personalized configuration information may also include the total number of target photos, which is the total number of photos in the final generated sharing material.

[0039] Optionally, the personalized configuration information may also include the number of photos selected for each filter condition.

[0040] Optionally, the photo types may include, but are not limited to, selfies, panoramic photos, screenshots, portraits, landscapes, snapshots, children's photos, and spring / summer / autumn / winter photos.

[0041] Optionally, the material template information includes material templates.

[0042] Optionally, the material template information may also include at least one of voice-over and subtitles.

[0043] Optionally, the voiceover can be user-defined or selected from preset music. The subtitles can be user-defined or selected from preset subtitles.

[0044] Optionally, the configured time range can include at least one time range. For example, the user configures a time range of January 1, 2020 to June 30, 2020, corresponding to a selected number of photos of 5; and a time range of May 1, 2023 to May 5, 2023, corresponding to a selected number of photos of 10.

[0045] Optionally, the configured geographic location may also include at least one geographic location, such as Xi'an, Chongqing, and Beijing.

[0046] Optionally, the configured photo types may include at least one photo type.

[0047] For example, in response to a user opening the photo album, the phone system prompts the user on the display whether they want to generate content to share on social media (referred to as sharing content). If the user clicks "Yes," a personalized configuration interface pops up. This interface displays filter criteria, video templates, photo templates, suggested voiceovers, and suggested subtitles for the user to configure. The system then receives the user's personalized configuration information in response to their configuration actions. Finally, if the user clicks "No," the system exits. Figure 2A The image shown is a schematic diagram of a user opening a photo album in an embodiment of this application. Figure 2B The image shown is a schematic diagram illustrating how the mobile phone system prompts users on the display interface whether they need to generate materials for sharing on social media, as described in this application embodiment. Figure 2C The image shown is a schematic diagram illustrating a user's configuration operation in the personalized configuration interface in an embodiment of this application.

[0048] 102. Select photos to filter based on the filtering criteria.

[0049] For example, if the time range in the filter criteria is from August 1, 2023 to August 31, 2023, photos from August 1 to August 31, 2023 can be selected from the photo library as the filter photos.

[0050] If the geographic location in the filter criteria is Chongqing, then you can select photos located in Chongqing from the photo library as the filter photos.

[0051] If the filter is set to selfie, then you can select selfie photos from the photo library to use as the filter photos.

[0052] If the filter is set to panorama, then you can select panorama photos from the photo library to use as the filter photos.

[0053] If the filter is set to selfie, then you can select screenshots from the photo library to use as the filter photos.

[0054] 103. Based on the selected photos, the material template information, and the preset artificial intelligence (AI) model, generate sharing materials, wherein the quality of the photos in the sharing materials is greater than the preset photo quality.

[0055] The preset artificial intelligence (AI) model is a model obtained by training the sampled photos, or a model obtained by training the sampled photos and sampled material template information.

[0056] The following explains the preset AI model in two different scenarios:

[0057] 1. The preset artificial intelligence (AI) model is a model obtained by training the sampled photos.

[0058] Optionally, generating sharing materials based on the selected photos, the material template information, and the preset artificial intelligence (AI) model may include: inputting the selected photos into the preset AI model and outputting a target photo; generating sharing materials based on the target photo and the material template information.

[0059] Optionally, the method may further include: acquiring the sampled photo; training a model on the sampled photo to obtain the preset AI model.

[0060] Optionally, training the model on the sampled photos to obtain the preset AI model may include: evaluating the image quality of the sampled photos to obtain an image quality score; selecting the sampled photos based on the image quality score to obtain actual selected photos; obtaining the preset AI model when the actual selected photo is the same as the reference selected photo, or when the error between the actual selected photo and the reference selected photo is less than a first preset error threshold, or when a first preset number of learning iterations has been reached.

[0061] Optionally, selecting the sampled photos based on the image quality score to obtain the actual selected photos may include: selecting the sampled photos based on the image quality score and a preset photo quality to obtain the actual selected photos, wherein the actual selected photos have a higher quality than the preset photo quality.

[0062] Optionally, the step of selecting the sampled photos based on the image quality score to obtain the actual selected photos may include: selecting the sampled photos from high to low image quality scores according to a preset number of photos to obtain the actual selected photos.

[0063] Optionally, the step of evaluating the image quality of the sampled photos to obtain an image quality score may include: acquiring image quality parameters of the sampled photos, wherein the image quality parameters include at least two of brightness, contrast, highlight / shadow ratio, saturation, white balance, color difference, and sharpness; and calculating an image quality score for each sampled photo based on the image quality parameters. It is understood that the image quality parameters may also include other parameters, such as face parameters, which are not specifically limited here.

[0064] Optionally, calculating the image quality score of each sampled photo based on the image quality parameters may include: calculating the score of each image quality parameter based on whether the value of each image quality parameter is within the corresponding range; and calculating the image quality score of each sampled photo based on the score of each image quality parameter.

[0065] The image quality parameters are briefly explained below:

[0066] (1) Brightness: The image is converted from the RGB color space to the Lab color space (an open-source technology), and then the average value α of the L channel is calculated. This value represents the average brightness of the entire image. Image brightness can be simply understood as the lightness or darkness of the image.

[0067] RGB represents the colors of the three channels: red, green, and blue. This standard covers almost all colors that human vision can perceive and is one of the most widely used color systems.

[0068] The Lab color model consists of three elements: lightness (L), a, and b, which are two color channels. A includes colors ranging from dark green (low lightness value) to gray (medium lightness value) to bright pink (high lightness value); b ranges from bright blue (low lightness value) to gray (medium lightness value) to yellow (high lightness value). Therefore, mixing these colors produces a bright and luminous effect.

[0069] (2) Contrast:

[0070] This refers to the difference between the dark and bright areas of an image, specifically the difference between the maximum and minimum gray levels. Analyzing the square root of the difference between the average of the squared brightness of each pixel and the squared average brightness of all pixels is equivalent to calculating the dispersion of the pixel brightness distribution across the entire image.

[0071] (3) Highlight ratio / Shadow ratio:

[0072] When converting an image from the RGB color space to the Lab color space, a statistical method is used. For example, count the number of pixels with L>240 (count1). Assuming the total number of pixels is count, the highlight ratio is count1 / count. Similarly, count the number of pixels with L<20 (count2). Then count2 / count is the shadow ratio.

[0073] Highlight ratio / Shadow ratio = (count1 / count) / (count2 / count).

[0074] (4) Saturation: Convert RGB to HSV color space and extract the S channel to calculate the average value.

[0075] The HSV color space, also known as the hexagonal pyramid color space, has three parameters: H represents hue, S represents saturation, and V represents value. The HSV color space model corresponds to a cone-shaped subset in a cylindrical coordinate system. The apex of the cone corresponds to V=1, and it contains the three faces of the RGB model: R=1, G=1, and B=1, representing brighter colors. The color H is given by the rotation angle around the V-axis. Red corresponds to an angle of 0°, green to 120°, and blue to 240°. In the HSV color space, each color differs from its complementary color by 180°. Saturation S ranges from 0 to 1, so the radius of the cone's apex is 1.

[0076] (5) White balance: Statistical analysis of the ratio of the three RGB channels of the entire screen.

[0077] (6) Color difference: Used to determine whether the image has a color cast, achieved by statistical image averaging Lab.

[0078] The code example is as follows:

[0079] def delta_E(img):

[0080] img_Lab=cv2.cvtColor(img,cv2.COLOR_BGR2Lab)

[0081] L=np.mean(img_Lab[:,:,0]) / 2.55

[0082] L=np.mean(img_Lab[:,:,1]) / -128

[0083] L=np.mean(img_Lab[:,:,2]) / -128

[0084] Return((L,a,b))

[0085] (7) Clarity:

[0086] It can extract the edges of the image using the Sobel edge extraction method; perform edge statistics; the sharper the image, the higher the sharpness value.

[0087] The code example is as follows:

[0088] def Tenengrad(img):

[0089] import filters from skimage

[0090] imgGray=cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)

[0091] tmp = filters.sobel(imgGray)

[0092] out = np.sum(tmp**2)

[0093] out = np.sqrt(out)

[0094] return out

[0095] If a face is detected, the following information about the face can be analyzed based on the ROI information written into the EXIF ​​data:

[0096] Brightness, contrast, skin tone, and skin saturation are calculated using the same methods as described earlier, except that the ROI is limited to the face region. If there are multiple faces, a maximum of, for example, three face regions will be selected for calculation based on the size of the faces.

[0097] For example, suppose a good photograph, measured in a laboratory, yields the following range for each parameter:

[0098] Brightness: (a1, b1); Contrast: (a2, b2); Highlight ratio / Shadow ratio: (a3, b3); Saturation: (a4, b4); White balance: (a5, b5); Color difference: (a6, b6); Sharpness: (a7, b7).

[0099] If it falls outside the above range, it is considered to be at a disadvantage in that item, and its weight will be reduced.

[0100] Each photo will be assigned a score, or image quality score, and the higher the score, the greater the chance of it being selected. Let's assume we have the following data for a photo:

[0101] Brightness is L1, contrast is L2, highlight ratio / shadow ratio is L3, saturation is L4, white balance is L5, color difference is L6, and sharpness is L7.

[0102] If the item does not fit the corresponding range, the score for that item is 0, meaning the weight is 0.

[0103] If L1 is closer to b1 and farther from a1, the score is calculated as: [(L1-a1) / (b1-a1)]*b1, which gives the brightness score; for example: (a1, b1) is (100, 120), and the current brightness L1 is 115, then the brightness score is: [(115-100) / (120-100)]*120=90.

[0104] If L1 is closer to a1 and farther from b1, the score is calculated as: [(L1-a1) / (b1-a1)]*a1, which gives the brightness score.

[0105] If L2 is closer to b2 and farther from a2, the score is calculated as: [(L2-a2) / (b2-a2)]*b2, which gives the contrast score; if L2 is closer to a2 and farther from b2, the score is calculated as: [(L2-a2) / (b2-a2)]*a2, which gives the contrast score.

[0106] If L3 is closer to b3 and farther from a3, the score is calculated as: [(L3-a3) / (b3-a3)]*b3, which gives the highlight ratio / shadow ratio score; if L3 is closer to a3 and farther from b3, the score is calculated as: [(L3-a3) / (b3-a3)]*a3, which gives the highlight ratio / shadow ratio score.

[0107] If L4 is closer to b4 and farther from a4, the score is calculated as: [(L4-a4) / (b4-a4)]*b4, which gives the saturation score; if L4 is closer to a4 and farther from b4, the score is calculated as: [(L4-a4) / (b4-a4)]*a4, which gives the saturation score.

[0108] If L5 is closer to b5 and farther from a5, the score is calculated as: [(L5-a5) / (b5-a5)]*b5, which gives the white balance score; if L5 is closer to a5 and farther from b5, the score is calculated as: [(L5-a5) / (b5-a5)]*a5, which gives the white balance score.

[0109] If L6 is closer to b6 and farther from a6, the score is calculated as: [(L6-a6) / (b6-a6)]*b6, which gives the difference score; if L6 is closer to a6 and farther from b6, the score is calculated as: [(L6-a6) / (b6-a6)]*a6, which gives the difference score.

[0110] If L7 is closer to b7 and farther from a7, the score is calculated as: [(L7-a7) / (b7-a7)]*b7, which gives the sharpness score; if L7 is closer to a7 and farther from b7, the score is calculated as: [(L7-a7) / (b7-a7)]*a7, which gives the sharpness score.

[0111] Finally, an image quality score is calculated for each image (the sum of scores for each image quality parameter). The images are then sorted from highest to lowest quality score, with higher scores increasing the likelihood of selection. Optionally, if a user marks an image as "liked" or "favorited," it can be excluded from the image quality score calculation and directly added to the selected list. Regarding the selection of portrait photos and other photo categories, portrait photos receive an additional face score, but this score is not significantly different from that of landscape photos; selection ultimately depends on the final image quality score of each photo.

[0112] 2. The preset AI model is a model obtained by training the sampled photos and sampled material template information.

[0113] Optionally, generating sharing materials based on the selected photos, the material template information, and the preset artificial intelligence (AI) model may include: inputting the selected photos and the material template information into the preset AI model and outputting sharing materials.

[0114] Optionally, the method may further include: acquiring the sampled photos and sampled material template information; and training a model based on the sampled photos and sampled material template information to obtain the preset AI model.

[0115] Optionally, the step of training the model based on the sampled photos and the sampled material template information to obtain the preset AI model may include: evaluating the image quality of the sampled photos to obtain an image quality score; selecting the sampled photos based on the image quality score to obtain actual selected photos; obtaining actual sharing materials based on the actual selected photos and the sampled material template information; obtaining the preset AI model when the actual sharing materials are the same as the reference sharing materials, or when the error between the actual sharing materials and the reference sharing materials is less than a second preset error threshold, or when a second preset number of learning iterations has been reached.

[0116] It should be noted that the implementation method for selecting photos based on image quality scores can be found in the above explanation, and will not be repeated here.

[0117] Optionally, the step of evaluating the image quality of the sampled photos to obtain an image quality score may include: classifying the sampled photos, evaluating the image quality of the classified sampled photos, and obtaining an image quality score.

[0118] Optionally, classifying the sampled photos and evaluating the image quality of the classified sampled photos to obtain an image quality score may include: classifying the sampled photos according to the photo type, evaluating the image quality of the classified sampled photos, and obtaining an image quality score.

[0119] Understandably, the camera's recognition function can be used to identify information such as geographical location, shooting time, people, age, and scene. Based on this information, the sampled photos can be classified to obtain photos of various types.

[0120] For example, if the user configures two photo types and the total number of target photos is 10, then the final selection of target photos for each photo type could be 5. Alternatively, the selection can be proportional to the number of photos in each of the two photo types. For instance, if the first photo type has 30 photos and the second photo type has 70 photos, then 3 target photos would be selected from the first photo type and 7 target photos would be selected from the second photo type.

[0121] If the user has not configured the photo type but has configured the total number of target photos, the target photos that meet the total number of target photos can be selected based on the image quality score, from highest to lowest.

[0122] If the user has not configured the photo type or the total number of target photos, the target photos can be selected based on the preset number of photos or the preset photo quality.

[0123] like Figure 2D The image shown is a schematic diagram of one embodiment of the shared materials generated in this application.

[0124] Optionally, the target photos include photos containing a favorite icon and / or a like icon.

[0125] Optionally, the shared material includes at least one of photos and videos.

[0126] Optionally, the photos in the shared materials are image-processed photos.

[0127] Optionally, the image processing may include, but is not limited to, filters, beautification, and other similar effects. That is, the preset AI model can also automatically perform image processing on the selected target photo.

[0128] Optionally, the method may further include: responding to the user's selection of the resolution of the shared material, and generating shared material with a corresponding resolution.

[0129] The method, apparatus, computer device, and computer-readable storage medium for generating shareable materials provided in this application embodiment obtain personalized configuration information from users, including filtering conditions and material template information; selects filtered photos according to the filtering conditions; and generates shareable materials based on the filtered photos, the material template information, and a preset artificial intelligence (AI) model, wherein the quality of the photos in the shareable materials is greater than the preset photo quality; wherein the preset AI model is a model obtained by training on sampled photos, or a model obtained by training on sampled photos and sampled material template information. This provides users with a more convenient method for generating shareable materials on social platforms, eliminating the need for users to rely on third-party image processing software to edit images to obtain shareable materials. This application can directly generate shareable materials based on filtered photos, material template information, and a preset AI model, thus improving the efficiency of generating shareable materials and saving system resources.

[0130] like Figure 3A The diagram shown is a general flowchart of the method for generating sharing materials provided in this application embodiment.

[0131] This application's technical solution is based on an Artificial Intelligence (AI) model to intelligently recognize photos in an album. For example, it extracts filtered photos from one or more time periods (the time range can be set by the user) and intelligently generates materials for sharing on social media (which can be customized according to the style of different social platforms, such as WeChat Moments: editing and generating a music video based on user-configured material templates. User-configured material templates may include common nine-grid collages, six-grid collages, three-grid collages, or single images with special effects, or other material templates (user-defined or selected from preset material templates).

[0132] After the user opens the photo album, the system will prompt whether they want to generate content to share on social media. If the user clicks "Yes," the phone will respond and display a personalized configuration interface. In this interface, the user can select options such as video editing templates, whether to add voiceover, whether to add subtitles, and the photo time range (the specifics need to be defined by the product team; these functions are collectively referred to as user personalized configuration information). After configuring the personalized configuration information, photos will be read according to the user's selected time range and used as selected photos for processing by a preset AI model.

[0133] like Figure 3BThe diagram illustrates a process for generating video and photo sharing materials according to an embodiment of this application. After receiving the selected photos, the AI ​​model parses the encrypted Exchangeable Image File Format (EXIF) to obtain information such as scene recognition and geographic location, and then intelligently categorizes them using a classification algorithm. A brief description of the photo type classification rules: This process can divide photos into different subcategories, such as landscapes, portraits, or types with a large proportion of landscapes and a small proportion of portraits. Then, photos can be selected based on the total number of target photos configured by the user, the preset number of photos, the preset photo quality, or the number of photos of each photo type configured by the user. In some optional implementations, weights can be allocated based on the quantity of each photo type; the more photo types there are, the more photos may be selected (there is also a special condition here: if a user has tagged an image as "favorite" while viewing it, that image must be selected; refer to the heart icon in the Photos app on Apple (iPhone) used to mark images saved by the user). If photos are generated, they will be stitched together according to the user-configured photo template, such as a nine-grid image or a three-image sequence. If a video is generated, it will be edited, dubbed, and subtitled according to the user's preset video template. Finally, a caption will be intelligently generated based on the EXIF ​​data. For example, if a user took many photos of natural scenery in Zhangjiajie Scenic Area, the caption could be: "A suspension bridge spans a precipice, a bottomless abyss lies beneath your feet. Knights come to test their courage, their souls trembling and legs weak as they cling to the railing." Optionally, if the user is not satisfied with the generated caption, it can be regenerated, or the user can customize the caption. Finally, the shared material is saved to the album, and the user can also choose the resolution of the shared material.

[0134] In this embodiment, users no longer rely on third-party image processing software to edit and process images to generate sharing materials. They can directly generate sharing materials based on selected photos, material template information, and preset AI models. Therefore, the efficiency of generating sharing materials can be improved, system resources can be saved, and time and effort can be saved.

[0135] It should be understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0136] Based on the foregoing embodiments, this application provides an apparatus for generating and sharing materials. The modules and units included in the apparatus can be implemented by a processor; of course, they can also be implemented by specific logic circuits. In the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field-programmable gate array (FPGA), etc.

[0137] like Figure 4 The diagram shown is a structural schematic of an apparatus for generating and sharing materials according to an embodiment of this application. The apparatus may include:

[0138] The acquisition module 401 is used to acquire the user's personalized configuration information, which includes filtering conditions and material template information.

[0139] Selection module 402 is used to select photos for filtering according to the filtering conditions;

[0140] The generation module 403 is used to generate sharing materials based on the filtered photos, the material template information and the preset artificial intelligence AI model, wherein the quality of the photos in the sharing materials is greater than the preset photo quality;

[0141] The preset artificial intelligence (AI) model is a model obtained by training the sampled photos, or a model obtained by training the sampled photos and sampled material template information.

[0142] Optionally, the generation module 403 is specifically used to input the selected photos into the preset AI model and output the target photos; and generate sharing materials based on the target photos and the material template information.

[0143] Optionally, the device may also include: a model training module 404;

[0144] The acquisition module 401 is also used to acquire the sampled photo;

[0145] The model training module 404 is used to train the model on the sampled photos to obtain the preset AI model.

[0146] Optionally, the generation module 403 is specifically used to input the selected photos and the material template information into the preset AI model and output the sharing material.

[0147] Optionally, the device may also include: a model training module 404;

[0148] The acquisition module 401 is also used to acquire the sampled photos and sampled material template information;

[0149] The model training module 404 is used to train the model based on the sampled photos and the sampled material template information to obtain the preset AI model.

[0150] Optionally, the model training module 404 is specifically used to evaluate the image quality of the sampled photos to obtain an image quality score; select the sampled photos according to the image quality score to obtain an actual selected photo; obtain the preset AI model when the actual selected photo is the same as the reference selected photo, or when the error between the actual selected photo and the reference selected photo is less than a preset error threshold, or when a preset number of learning iterations are reached.

[0151] Optionally, the model training module 404 is specifically used to obtain image quality parameters of the sampled photos, including at least two of brightness, contrast, highlight / shadow ratio, saturation, white balance, color difference, and sharpness; and to calculate an image quality score for each sampled photo based on the image quality parameters.

[0152] Optionally, the model training module 404 is specifically used to calculate the score of each image quality parameter based on whether the value of each image quality parameter is within the corresponding range; and to calculate the image quality score of each sampled photo based on the score of each image quality parameter.

[0153] Optionally, the device may also include:

[0154] The classification module 405 is also used to classify the sampled photos, evaluate the image quality of the classified sampled photos, and obtain an image quality score.

[0155] Optionally, target photos may include photos containing a favorite tag and / or a like tag;

[0156] The shared materials include at least one of the following: photos and videos.

[0157] like Figure 5 The diagram shown is a schematic representation of an embodiment of the terminal device described in this application, which may include, for example: Figure 4 The apparatus shown is for generating and sharing materials.

[0158] In the embodiments of this application, the terminal device may be a mobile phone, a tablet computer, a computer with wireless transceiver capabilities, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal device in industrial control, a wireless terminal device in self-driving, a wireless terminal device in remote medical care, a wireless terminal device in a smart grid, a wireless terminal device in transportation safety, a wireless terminal device in a smart city, or a wireless terminal device in a smart home, etc.

[0159] By way of example and not limitation, in this embodiment, the terminal device can also be a wearable device. Wearable devices, also known as wearable smart devices, are a general term for devices that utilize wearable technology to intelligently design and develop everyday wearables, such as glasses, gloves, watches, clothing, and shoes. Wearable devices are portable devices that are worn directly on the body or integrated into the user's clothing or accessories. Wearable devices are not merely hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include those that are feature-rich, large in size, and can achieve complete or partial functions without relying on a smartphone, such as smartwatches or smart glasses, as well as those that focus on a specific type of application function and require the use of other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.

[0160] like Figure 6 The diagram shown is a schematic representation of another embodiment of the terminal device in this application, which may include:

[0161] The following is combined Figure 6 A detailed introduction to the various components of a mobile phone in a terminal device:

[0162] RF circuit 610 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with processor 680; additionally, it transmits uplink data to the base station. Typically, RF circuit 610 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, RF circuit 610 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0163] The memory 620 can be used to store software programs and modules. The processor 680 executes various functions and data processing of the mobile phone by running the software programs and modules stored in the memory 620. The memory 620 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 620 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0164] The input unit 630 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 630 may include a touch panel 631 and other input devices 632. The touch panel 631, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 631), and drive the corresponding connection devices according to a pre-set program. Optionally, the touch panel 631 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor 680, and can receive and execute commands sent by the processor 680. In addition, the touch panel 631 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 631, the input unit 630 may also include other input devices 632. Specifically, other input devices 632 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0165] The display unit 640 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 640 may include a display panel 641, which may optionally be configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar display. Furthermore, a touch panel 631 may cover the display panel 641. When the touch panel 631 detects a touch operation on or near it, it transmits the information to the processor 680 to determine the type of touch event. Subsequently, the processor 680 provides corresponding visual output on the display panel 641 based on the type of touch event. Although in Figure 6 In this embodiment, the touch panel 631 and the display panel 641 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch panel 631 and the display panel 641 can be integrated to realize the input and output functions of the mobile phone.

[0166] The mobile phone may also include at least one sensor 650, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 641 according to the ambient light level, and the proximity sensor can turn off the display panel 641 and / or backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity, and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, taps), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0167] Audio circuit 660, speaker 661, and microphone 662 provide an audio interface between the user and the mobile phone. Audio circuit 660 converts received audio data into electrical signals and transmits them to speaker 661, where speaker 661 converts them into sound signals for output. On the other hand, microphone 662 converts collected sound signals into electrical signals, which are received by audio circuit 660, converted into audio data, and then output to processor 680 for processing. The audio data is then transmitted via RF circuit 610 to, for example, another mobile phone, or output to memory 620 for further processing.

[0168] Wi-Fi is a short-range wireless transmission technology. Through the Wi-Fi module 670, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 6 Wi-Fi module 670 is shown, but it is understood that it is not an essential component of a mobile phone and can be omitted as needed without changing the essence of the invention.

[0169] The processor 680 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It executes software programs and / or modules stored in the memory 620, and calls data stored in the memory 620 to perform various functions and process data, thereby providing overall monitoring of the phone. Optionally, the processor 680 may include one or more processing units; preferably, the processor 680 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 680.

[0170] The mobile phone also includes a power supply 690 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 680 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0171] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0172] In this embodiment, the processor 680 is configured to acquire personalized configuration information of a user, the personalized configuration information including filtering conditions and material template information; select filtered photos according to the filtering conditions; and generate sharing materials according to the filtered photos, the material template information, and a preset artificial intelligence (AI) model, wherein the quality of the photos in the sharing materials is greater than the preset photo quality.

[0173] The preset artificial intelligence (AI) model is a model obtained by training the sampled photos, or a model obtained by training the sampled photos and sampled material template information.

[0174] Optionally, the processor 680 is specifically used to input the selected photos into the preset AI model and output the target photos; and to generate sharing materials based on the target photos and the material template information.

[0175] Optionally, the processor 680 is further configured to acquire the sampled photograph; and to train a model on the sampled photograph to obtain the preset AI model.

[0176] Optionally, the processor 680 is specifically used to input the filtered photos and the material template information into the preset AI model and output the shared materials.

[0177] Optionally, the processor 680 is further configured to acquire the sampled photos and sampled material template information; and to perform model training based on the sampled photos and sampled material template information to obtain the preset AI model.

[0178] Optionally, the processor 680 is specifically configured to perform image quality evaluation on the sampled photos to obtain an image quality score; select the sampled photos according to the image quality score to obtain an actual selected photo; obtain the preset AI model when the actual selected photo is the same as the reference selected photo, or when the error between the actual selected photo and the reference selected photo is less than a preset error threshold, or when a preset number of learning iterations are reached.

[0179] Optionally, the processor 680 is specifically used to acquire image quality parameters of the sampled photographs, the image quality parameters including at least two of brightness, contrast, highlight / shadow ratio, saturation, white balance, color difference, and sharpness; and to calculate an image quality score for each sampled photograph based on the image quality parameters.

[0180] Optionally, the processor 680 is specifically used to calculate the score of each image quality parameter based on whether the value of each image quality parameter is within the corresponding range; and to calculate the image quality score of each sampled photo based on the score of each image quality parameter.

[0181] Optionally, the processor 680 is also configured to classify the sampled photos, evaluate the image quality of the classified sampled photos, and obtain an image quality score.

[0182] Optionally, target photos may include photos containing a favorite tag and / or a like tag;

[0183] The shared materials include at least one of the following: photos and videos.

[0184] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0185] It should be noted that the module division of the apparatus for generating and sharing materials shown in the embodiments of this application is illustrative and only represents a logical functional division. In actual implementation, there may be other division methods. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, exist as separate physical units, or have two or more units integrated into one unit. The integrated units can be implemented in hardware, as software functional units, or a combination of software and hardware.

[0186] It should be noted that, in the embodiments of this application, if the above-described methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0187] like Figure 7 The diagram illustrates the structure of a computer device according to an embodiment of this application. This computer device can be a server. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements the aforementioned methods.

[0188] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method provided in the above embodiments.

[0189] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the method provided in the above-described method embodiments.

[0190] Those skilled in the art will understand that the structure shown in the computer device is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0191] In one embodiment, the apparatus for generating and sharing materials provided in this application can be implemented as a computer program, which can be configured as follows: Figure 7The device operates on the computer device shown. The memory of the computer device can store the various program modules that make up the above-described apparatus. The computer program, composed of the various program modules, causes the processor to execute the steps of the methods in the various embodiments of this application described in this specification.

[0192] It should be noted that the descriptions of the storage medium and device embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium, storage medium, and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0193] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, they will not be repeated here.

[0194] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three kinds of relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0195] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0196] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or modules can be electrical, mechanical, or other forms.

[0197] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0198] In addition, each functional module in the various embodiments of this application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the integrated modules can be implemented in hardware or in the form of hardware plus software functional units.

[0199] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0200] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0201] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0202] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0203] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0204] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for generating shareable materials, characterized in that, The method includes: Obtain the user's personalized configuration information, which includes filtering conditions and material template information; Select photos to filter based on the filtering criteria; Based on the selected photos, the material template information, and the preset artificial intelligence (AI) model, shareable materials are generated, and the quality of the photos in the shareable materials is greater than the preset photo quality. The preset artificial intelligence (AI) model is a model obtained by training the sampled photos. The step of generating shareable materials based on the selected photos, the material template information, and the preset artificial intelligence (AI) model includes: The selected photos are input into the preset AI model, and the target photos are output. The sharing material is generated based on the target photo and the material template information; The preset AI model is trained by selecting actual photos from the sampled photos using image quality scores obtained from image quality assessment of the sampled photos.

2. The method according to claim 1, characterized in that, The method further includes: Obtain the sampled photos; The sampled photos are used to train a model to obtain the preset AI model.

3. The method according to claim 2, characterized in that, The step of training the model on the sampled photos to obtain the preset AI model includes: The sampled photos are subjected to image quality evaluation to obtain an image quality score; The sampled photos are selected based on the image quality score to obtain the actual selected photos; when the actual selected photos are the same as the reference selected photos, or when the error between the actual selected photos and the reference selected photos is less than a preset error threshold, or when a preset number of learning iterations are reached, the preset AI model is obtained.

4. The method according to claim 3, characterized in that, The step of evaluating the image quality of the sampled photographs to obtain an image quality score includes: The image quality parameters of the sampled photo are obtained, including at least two of the following: brightness, contrast, highlight / shadow ratio, saturation, white balance, color difference, and sharpness. Based on the image quality parameters, calculate the image quality score for each sampled photo.

5. The method according to claim 4, characterized in that, The step of calculating the image quality score for each sampled photograph based on the image quality parameters includes: Calculate a score for each image quality parameter based on whether its value is within the corresponding range; Calculate the image quality score for each sampled photo based on the score for each image quality parameter.

6. The method according to claim 3, characterized in that, The step of evaluating the image quality of the sampled photographs to obtain an image quality score includes: The sampled photos are classified, and the image quality of the classified sampled photos is evaluated to obtain an image quality score.

7. The method according to any one of claims 1-6, characterized in that, The target photos include those with a favorite tag and / or a like tag; The shared materials include at least one of the following: photos and videos.

8. An apparatus for generating shareable materials, characterized in that, include: The acquisition module is used to acquire the user's personalized configuration information, which includes filtering conditions and material template information. The selection module is used to select photos for filtering based on the filtering criteria. The generation module is used to generate sharing materials based on the selected photos, the material template information, and the preset artificial intelligence AI model, wherein the quality of the photos in the sharing materials is greater than the preset photo quality; The preset artificial intelligence (AI) model is a model obtained by training the sampled photos. The generation module is specifically used to input the selected photos into the preset AI model and output the target photos; and to generate sharing materials based on the target photos and the material template information. The preset AI model is trained by selecting actual photos from the sampled photos using image quality scores obtained from image quality assessment of the sampled photos.

9. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images