Shooting method and device based on machine learning model, equipment, medium and product

Abstract

The invention discloses a shooting method and device based on a machine learning model, equipment, a medium and a product, and relates to the technical field of image processing, and the method comprises the steps: obtaining a real-time scene image of a current shooting scene, carrying out the scene analysis of the real-time scene image according to a preset machine learning model, and obtaining the scene key information of the real-time scene image; matching the scene key information with a composition template in a preset composition template library to obtain a matched composition template, and outputting a composition suggestion containing a composition mode according to the matched composition template; if the real-time scene image comprises the figure image, determining a historical scene image matched with the real-time scene image in a preset scene image library, and outputting a posture suggestion containing a shooting posture according to the historical scene image; and determining shot image parameters adjusted by the user according to the composition suggestion and / or the posture suggestion, and outputting a shot image according to the adjusted shot image parameters. The method and the device can assist a user in shooting high-quality photos.

Description

Technical Field

[0001] The present application relates to a specific field of image processing technology, and in particular to shooting methods, devices, equipment, media and products based on machine learning models. Background Art

[0002] With the popularity of smartphones, smartphones have become an important tool for people to record and share their lives. And with the continuous improvement of smartphone camera hardware performance, such as higher pixels, stronger sensors and multi-lens systems, users can take higher-quality photos. However, since most users have not received professional photography training, the quality of photos taken by users with smartphones is poor. For example, the photos taken by users may have problems such as poor composition and inappropriate angles. Therefore, how to assist users in taking high-quality photos has become an urgent problem to be solved.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of this application is to provide a shooting method, device, equipment, medium and product based on a machine learning model, aiming to solve the technical problem of how to assist users in taking high-quality photos.

[0005] To achieve the above objectives, the present application proposes a shooting method based on a machine learning model, the method comprising:

[0006] Acquire a real-time scene image of the current shooting scene, and perform scene analysis on the real-time scene image according to a preset machine learning model to obtain scene key information of the real-time scene image;

[0007] Matching the scene key information with a composition template in a preset composition template library to obtain a matching composition template, and outputting a composition suggestion including a composition method according to the matching composition template;

[0008] If the real-time scene image includes a person image, determining a historical scene image that matches the real-time scene image in a preset scene image library, and outputting a posture suggestion including a shooting posture according to the historical scene image;

[0009] Determine the captured image parameters adjusted by the user according to the composition suggestion and / or the posture suggestion, and output the captured image according to the adjusted captured image parameters.

[0010] In one embodiment, the step of outputting a posture suggestion including a shooting posture according to the historical scene image comprises:

[0011] If there are at least two historical scene images, constructing a posture template according to the posture features of the historical figures in the historical scene images, wherein the posture template includes a shooting posture;

[0012] Determining a character posture feature of the character image in the real-time scene image, and determining a similarity between the character posture feature and the posture template;

[0013] Among the posture templates corresponding to the similarities greater than a preset similarity threshold, a posture template is selected as a matching posture template, and a posture suggestion including a shooting posture is generated according to the matching posture template, and the posture suggestion is output.

[0014] In one embodiment, the scene key information includes at least one of scene type, subject position, and light condition.

[0015] The step of matching the scene key information with a composition template in a preset composition template library to obtain a matching composition template comprises:

[0016] Matching a composition template in a preset composition template library with the scene type to obtain a matching first composition template;

[0017] Matching a composition template in a preset composition template library with the subject position to obtain a matching second composition template;

[0018] Matching a composition template in a preset composition template library with the light condition to obtain a matching third composition template;

[0019] According to a preset user preference, a composition template is selected from the first composition template, the second composition template and / or the third composition template as a matching composition template.

[0020] In one embodiment, the shooting method based on the machine learning model further includes:

[0021] Acquire feedback data from a user regarding the output composition suggestion and the posture suggestion;

[0022] The machine learning model is optimized according to the feedback data to obtain an optimized machine learning model.

[0023] In one embodiment, the shooting method based on the machine learning model further includes:

[0024] Acquire a training data set, wherein the training data set includes a plurality of training data, each training data includes a training scene image and an image label corresponding to the training scene image;

[0025] Inputting the training scene image into the machine learning model to be trained for prediction, and obtaining a first prediction result, wherein the first prediction result includes predicted scene key information;

[0026] Calculating a loss function value based on the image label and the predicted scene key information;

[0027] The machine learning model to be trained is optimized according to the loss function value until a preset training end condition is met to obtain a preset machine learning model.

[0028] In one embodiment, the machine learning model includes an adapter and a model ontology,

[0029] The step of optimizing the machine learning model to be trained according to the loss function value includes:

[0030] The model parameters of the model body are frozen, and the model parameters of the adapter are optimized according to the loss function value.

[0031] In addition, to achieve the above purpose, the present application also proposes a shooting device based on a machine learning model, comprising:

[0032] An acquisition module is used to acquire a real-time scene image of a current shooting scene, and perform scene analysis on the real-time scene image according to a preset machine learning model to obtain scene key information of the real-time scene image;

[0033] A composition suggestion module, used for matching the scene key information with a composition template in a preset composition template library to obtain a matching composition template, and outputting a composition suggestion including a composition method according to the matching composition template;

[0034] A posture suggestion module, configured to determine a historical scene image matching the real-time scene image in a preset scene image library if the real-time scene image includes a person image, and output a posture suggestion including a shooting posture according to the historical scene image;

[0035] The output module is used to determine the captured image parameters adjusted by the user according to the composition suggestion and / or the posture suggestion, and output the captured image according to the adjusted captured image parameters.

[0036] In addition, to achieve the above-mentioned objectives, the present application also proposes a shooting device based on a machine learning model, the device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the shooting method based on the machine learning model as described above.

[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes a medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the shooting method based on the machine learning model as described above are implemented.

[0038] In addition, to achieve the above-mentioned objectives, the present application also provides a product, which is a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps of the shooting method based on the machine learning model as described above.

[0039] In the present application, by acquiring the real-time scene image of the current shooting scene, and performing scene analysis on the real-time scene image according to the machine learning model, the scene key information of the real-time scene image is obtained, and the matching composition template matching the scene key information in the composition template library is determined, and the composition suggestion containing the composition method is output according to the matching composition template. And if the real-time scene image includes a person image, the historical scene image matching the real-time scene image in the scene image library is determined, and the posture suggestion containing the shooting posture is output according to the historical scene image, and the shooting image parameters adjusted by the user according to the composition suggestion and / or posture suggestion are determined, and the shooting image is output according to the adjusted shooting image parameters. Then, it can be realized to provide professional-level photography guidance for users to improve the user's shooting level and photography skills. And in the process of users using shooting equipment to shoot, users can obtain composition suggestions and posture suggestions, and can adjust the shooting image parameters by composition suggestions and / or posture suggestions, so as to make the shot images more professional and more beautiful. That is, the shooting equipment can use the machine learning model to assist users in adjusting the shooting image parameters according to the actual shooting scene, so that users can use the shooting equipment to shoot higher quality photos. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0042] Figure 1 A flowchart diagram of the first embodiment of the shooting method based on the machine learning model of the present application;

[0043] Figure 2A flowchart diagram of a second embodiment of a shooting method based on a machine learning model of the present application;

[0044] Figure 3 This is a schematic diagram of the overall architecture flow of the shooting method based on the machine learning model of this application;

[0045] Figure 4 This is a schematic diagram of the module architecture of a photographing device based on a machine learning model in this application;

[0046] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the shooting method based on the machine learning model in the embodiment of the present application.

[0047] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0049] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0050] Currently, when users use smartphones to take photos, they have not received professional photography training and are unable to take high-quality photos. For example, the photos taken may have problems such as poor composition, inappropriate angles, and improper light processing. Based on this, smartphones provide photo assistance technology to assist users in taking photos, but the current photo assistance technology is limited to simple filters and beauty functions. Although these functions improve the visual effects of photos to a certain extent, they cannot fundamentally solve the composition and posture selection problems faced by users when taking photos. In addition, some smartphone applications provide photo templates or manual guidance, but these methods are too rigid, lack intelligence and flexibility, and cannot make personalized recommendations based on actual scenarios and user needs. In other words, the current smartphone photography has the following defects:

[0051] Lack of intelligence and personalization: Most existing photo assistance functions are based on fixed templates or simple rules and cannot make intelligent recommendations based on actual shooting scenarios. These systems cannot dynamically analyze the user's shooting environment, lighting conditions, and subject to provide personalized composition and pose suggestions.

[0052] Inadequate user experience: Many existing photo-assistance applications require users to manually select templates or follow complex instructions, which is not only cumbersome but also easily interrupts the continuity of shooting, reducing the user experience.

[0053] Technical limitations: Some photo-assistance functions only provide basic post-processing effects such as filters and beauty enhancement, and are unable to provide real-time guidance and suggestions during the photo-taking process, resulting in users being unable to adjust and improve their composition and posture in a timely manner when taking photos.

[0054] Insufficient data utilization: Existing technologies do not fully utilize the composition and pose data of a large number of high-quality photos, and lack in-depth analysis and application of these data, resulting in the recommended composition and pose being not scientific and effective enough.

[0055] Therefore, in order to avoid the above defects, a shooting method based on a machine learning model is provided in this embodiment, which can analyze the shooting scene in real time, combine the composition and shooting posture of a large number of high-quality photos, and intelligently recommend the composition and posture suitable for the current environment to help users take more professional and beautiful photos. That is, in this embodiment, the shooting scene can be analyzed in real time through the machine learning model, and the suitable composition and shooting posture can be intelligently recommended, which improves the efficiency and quality of the user's photography. In addition, during the shooting process, the user can obtain guidance on composition and shooting posture in real time to help him adjust the shooting angle and posture, so as to take more professional and beautiful photos. It is also possible to collect user feedback data, continuously optimize and update the machine learning model, improve the accuracy of the recommendation and user satisfaction, and maintain the continuous progress of the system. In addition, in addition to recommending composition and shooting posture, this embodiment also provides other auxiliary functions, such as automatic exposure adjustment, color optimization and post-processing suggestions, to provide users with comprehensive shooting assistance.

[0056] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a shooting device based on a machine learning model, etc. The following takes a shooting device based on a machine learning model as an example to illustrate this embodiment and the following embodiments.

[0057] Based on this, the embodiment of the present application provides a shooting method based on a machine learning model, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the shooting method based on the machine learning model of the present application.

[0058] In this embodiment, the shooting method based on the machine learning model includes steps S10 to S40.

[0059] Step S10, obtaining a real-time scene image of the current shooting scene, and performing scene analysis on the real-time scene image according to a preset machine learning model to obtain scene key information of the real-time scene image;

[0060] It should be noted that the current shooting scene can be the scene where the cameraman is when shooting, such as indoor scenes, outdoor scenes, shopping mall scenes, etc. The real-time scene image can be the picture captured by the shooting device at the current moment, such as photos or video frames collected in real time. The machine learning model can be a convolutional neural network model or other large model that can assist the shooting device in taking pictures, which is not limited here. Scene analysis can be to extract the key features of the real-time scene image through the machine learning model, such as the subject, background, light conditions and other information in the real-time scene image, and combine this information together to form the scene key information.

[0061] Optionally, before the shooting device starts shooting, the device parameters may be initialized, such as adjusting the white balance, focus point, exposure setting, etc. of the shooting device, to ensure the best quality of the image processed by the shooting.

[0062] Optionally, after the shooting device performs device parameter initialization, the shooting device can acquire and capture real-time scene images of the current shooting scene in real time, and can input the real-time scene images into the machine learning model to perform scene analysis on the real-time scene images through the machine learning model to obtain scene key information of the real-time scene images.

[0063] Optionally, when performing scene analysis, it can be divided into three stages, namely image acquisition, scene recognition and feature extraction. In the image acquisition stage, the cameraman can use a shooting device (such as a smart phone) to obtain a real-time scene image of the current shooting scene. In the scene recognition stage, the real-time scene image is analyzed according to a preset machine learning model to identify key learning such as scene type, subject position and lighting conditions. In the feature extraction stage, a feature extraction network is used to extract high-level features (vectors) from the real-time scene image, and match them from the feature library of the machine learning model (such as a composition template library, a scene image library, etc.) to output corresponding shooting suggestions, such as composition suggestions, shooting posture suggestions, etc.

[0064] Step S20, matching the scene key information with a composition template in a preset composition template library to obtain a matching composition template, and outputting a composition suggestion including a composition method according to the matching composition template;

[0065] Optionally, the composition template library can be a pre-set database storing multiple composition templates. The composition template can be obtained by analyzing and extracting high-quality photos taken historically. For example, a large number of high-quality network images can be obtained on the Internet in advance, and the composition rules of the network images can be extracted to obtain corresponding composition templates, and the various composition templates can be aggregated into a database to obtain a composition template library. Optionally, some image rules can also be set to guide the communication of images. For example, the golden section method is used to divide the image into 9 equal parts, of which 4 equal parts are located in the upper half of the image and the other 5 equal parts are located in the lower half of the image. In addition, image rules can also include symmetry rules and center of gravity rules.

[0066] Optionally, after the real-time scene image is analyzed by a machine learning model to obtain scene key information corresponding to the real-time scene image, the scene key information can be matched with a composition template in a pre-set composition template library by the machine learning model to determine a matching composition template. The composition method contained in the matching composition template is output in the form of a composition suggestion. For example, a virtual frame, line or icon is superimposed on the viewfinder of the shooting device to guide the user to adjust the camera angle and position.

[0067] Optionally, the shooting device can use a machine learning model to output a composition suggestion including a composition method in the current scene based on a matching composition template that matches the real-time scene image, so that the user can adjust the shooting angle and shooting position according to the composition suggestion.

[0068] In a feasible embodiment, the scene key information includes at least one of scene type, subject position and light condition.

[0069] Optionally, scene types may include natural scenes (such as landscapes), man-made scenes (such as buildings), and human scenes (such as friends, family, etc.). The subject position may be the position of the subject in the image, and the subject may be a photographed subject, such as a human subject (friends, family, etc.), an animal subject (cats, dogs, etc.), a plant subject (flowers, trees, etc.). Light conditions may include the light used by the photographing device when taking the image, such as natural light, artificial light, and backlight. Optionally, scene key information may also include shooting angles, composition methods, and shooting postures. Shooting angles may include front, side, and overhead shots, etc. Composition methods may include the rule of thirds and symmetrical composition, etc. Shooting postures may include standing, sitting, and action, etc.

[0070] Optionally, in step S20, the scene key information is matched with a composition template in a preset composition template library to obtain a matching composition template, including steps a10-a40.

[0071] Step a10, matching a composition template in a preset composition template library with a scene type to obtain a matching first composition template;

[0072] Optionally, multiple composition templates in a composition template library may be determined, and the scene type is matched with each composition template in sequence to obtain a composition template matching the scene type, and the composition template is used as the first composition template.

[0073] Optionally, if the scene type is a natural scene, a composition template suitable for composing in a natural scene can be determined in the composition template library and used as the first matching composition template. For example, if the scene type is a natural scene and it is a scene containing a forest, a composition template obtained by feature extraction and conversion of an image containing a forest scene can be determined in the composition template library and used as the first composition template. Optionally, there can be multiple first composition templates.

[0074] Optionally, if the scene type is a man-made scene, a composition template suitable for composing in a man-made scene can be determined in the composition template library and used as the first matching composition template. For example, if the scene type is a man-made scene and it is a scene containing a historical building, a composition template obtained by feature extraction and conversion of an image containing a historical building scene can be determined in the composition template library and used as the first composition template.

[0075] Optionally, if the scene type is a human scene, a composition template suitable for composing in a human scene in the composition template library may be determined and used as the first matching composition template. For example, if the scene type is a human scene and the scene includes a child, a composition template obtained by performing feature extraction and conversion on an image of a scene including a child may be determined in the composition template library and used as the first composition template.

[0076] Step a20, matching a composition template in a preset composition template library with the subject position to obtain a matching second composition template;

[0077] Optionally, the subject position may be matched with each composition template in sequence to obtain a matching composition template, and the matching composition template may be used as a second matching composition template.

[0078] Optionally, when performing subject position matching, the subject type in the real-time scene image can be determined first, and then the subject position can be determined. Then, a composition template obtained by extracting features from an image containing an image that is consistent with the subject type and the subject position is determined in the composition template library, and used as the first composition template. For example, if the subject type of the real-time scene image is a child, then it is necessary to determine the first position of the child in the image. A second composition template that matches it can be determined in the composition template library. The second composition template can be a composition template obtained by extracting composition mode features from a network image obtained on the Internet that contains a child, and the child is located in the first position.

[0079] Optionally, there may be multiple second composition templates.

[0080] Step a30, matching a composition template in a preset composition template library with the light condition to obtain a matching third composition template;

[0081] Optionally, the light condition may be matched with each composition template in turn to obtain a matching composition template, and the matching composition template may be used as the third matching composition template. For example, when the light condition is natural light, a composition template suitable for natural light may be selected as the third composition template.

[0082] Step a40: selecting a composition template from the first composition template, the second composition template and / or the third composition template as a matching composition template according to a preset user preference.

[0083] The user preference analysis can be performed based on the historical shooting records of the shooting device to obtain the user preference. Optionally, the user preference analysis can be used to understand what kind of photos, themes, colors and styles the user likes, thereby providing a reference for subsequent composition suggestions.

[0084] Optionally, after acquiring the first composition template, the second composition template and / or the third composition template, a selection may be made according to user preference, and a composition template closest to the user preference may be selected as the matching composition template.

[0085] Optionally, multiple first composition templates, multiple second composition templates and multiple third composition templates may be compared in sequence. If a first composition template is identical to both the second composition template and the third composition template, the first composition template may be used as a matching composition template.

[0086] In this embodiment, the scene key information includes at least one of the scene type, subject position and light condition. A first composition template matching the scene type, a second composition template matching the subject position and a third composition template matching the light condition in the composition template library are determined, and one of the first composition template, the second composition template and the third composition template is selected as a matching composition template according to user preference, and then a composition suggestion including a composition method is output according to the matching composition template, so that the composition suggestion output by the shooting device can fit the current scene while meeting the user's shooting needs to the greatest extent, so that the final captured image of the shooting device is more professional and high-quality.

[0087] Step S30, if the real-time scene image includes a person image, determining a historical scene image that matches the real-time scene image in a preset scene image library, and outputting a posture suggestion including a shooting posture according to the historical scene image;

[0088] Optionally, when the shooting device performs scene analysis on the real-time scene image through a machine learning model to obtain scene key information, and the scene type in the scene key information is a human scene, it can be determined that the real-time scene image includes a human image.

[0089] Optionally, the scene image library may be a pre-set database storing a plurality of scene images. The scene images may be collected from social media, photo libraries, and professional photography websites, such as scene images containing people and with buildings as the background.

[0090] Optionally, the real-time scene image can be matched with each scene image in the scene image library through a machine learning model, and the matched scene image can be used as the historical scene image. Optionally, when matching, the image similarity between the real-time scene image and each scene image can be calculated, and the scene image with the highest image similarity can be selected as the historical scene image. The scene image with an image similarity greater than a similarity threshold can also be used as the historical scene image.

[0091] Optionally, after determining the historical scene image, the shooting posture of the person in the historical scene image can be determined, such as standing, sitting, and action, etc. A posture suggestion is constructed based on the shooting posture, and the posture suggestion is output through a machine learning model, so that the user can guide the person subject to be photographed (such as the user's friend) to adjust the posture according to the output posture suggestion.

[0092] Optionally, when the machine learning model outputs posture suggestions, the camera device can guide the user to strike a recommended posture according to the posture suggestions by means of demonstration animation or voice prompts.

[0093] In a feasible embodiment, in step S30, the step of outputting a posture suggestion including a shooting posture according to the historical scene image includes steps b10-b30.

[0094] Step b10, if there are at least two historical scene images, constructing a posture template according to the posture features of the historical figures in the historical scene images, wherein the posture template includes the shooting posture;

[0095] Optionally, if there are at least two historical scene images. The posture features of historical figures in the historical scene images can be extracted, that is, only the posture features of the figures are retained in the historical scene images, and other elements are deleted. The posture features of the figures in the historical scene images are extracted to obtain the posture features of the historical figures, and a posture template is constructed based on the posture features of the historical figures. The posture features of the historical figures can be the features of the shooting postures of the figures in the historical scene images, such as the features of standing. Optionally, the posture template at least contains all the posture features of the historical figures in the historical scene images.

[0096] Step b20, determining the character posture features of the character image in the real-time scene image, and determining the similarity between the character posture features and the posture template;

[0097] Optionally, the character posture feature of the character image in the real-time scene image may be determined. The character posture feature may be the feature of the shooting posture of the character in the real-time scene image, such as the feature of standing.

[0098] Optionally, the similarity between the posture feature of the person and the posture feature of the historical person in each posture template may be calculated (hereinafter, the similarity corresponding to the posture template is used for explanation).

[0099] Step b30, among the posture templates corresponding to the similarities greater than a preset similarity threshold, select a posture template as a matching posture template, generate a posture suggestion including the shooting posture according to the matching posture template, and output the posture suggestion.

[0100] Optionally, the similarity corresponding to each posture template may be compared with a preset similarity threshold to determine posture templates corresponding to similarities greater than the preset similarity threshold, and a posture template is randomly selected from these posture templates as a matching posture template.

[0101] Determine the posture features of historical figures contained in the matching posture template, and determine the shooting posture based on the posture features of the historical figures, build a posture suggestion including the shooting posture, and output it through a machine learning model, such as outputting it to the display screen of the shooting device and displaying it through a demonstration animation.

[0102] In this embodiment, when it is determined that there are at least two historical scene images, the posture template corresponding to each historical scene image is determined, and the character posture features of the character image in the real-time scene image are determined, and the similarity between the character posture features and the posture template is determined, and the posture templates corresponding to the similarity greater than the preset similarity threshold are summarized, and one is selected as a matching posture template, and then a posture suggestion including the shooting posture is generated according to the matching posture template, and the posture suggestion is output, so that the user can adjust the shooting image parameters according to the output posture suggestion, so that the captured images taken by the shooting device are more professional and more beautiful.

[0103] Step S40, determining the captured image parameters adjusted by the user according to the composition suggestion and / or the posture suggestion, and outputting the captured image according to the adjusted captured image parameters.

[0104] Optionally, the captured image parameters adjusted by the user according to the composition suggestion and / or the posture suggestion may be determined, such as the shooting angle, exposure time, etc. When it is determined that the user has adjusted the captured image parameters and a shooting instruction input by the user is received, the shooting device shoots the image and outputs the captured image.

[0105] Furthermore, in a feasible embodiment, the shooting method based on the machine learning model also includes steps c10-c20.

[0106] Step c10, obtaining feedback data from the user regarding the output composition suggestions and / or posture suggestions;

[0107] Step c20, optimizing the machine learning model based on the feedback data to obtain an optimized machine learning model.

[0108] Optionally, the feedback data may include satisfaction scores or suggested adjustment data, etc.

[0109] Optionally, after the photographing device outputs the composition suggestion and / or the posture suggestion, if it receives feedback data from the user, it can optimize and update the training data set used to train the machine learning model according to the feedback data, and perform model training on the machine learning model according to the optimized and updated training data set until the preset training end condition is met to obtain the optimized machine learning model. The optimized machine learning model can then continue to assist the photographing device in capturing images.

[0110] Optionally, the composition templates in the composition template library may be adjusted according to the feedback data, such as adding a new composition template, deleting a poor composition template specified by the feedback data, etc. The scene images in the scene image library may also be adjusted according to the feedback data.

[0111] In addition, the shooting device in this embodiment also has other auxiliary functions, such as automatic exposure adjustment, color optimization and post-processing suggestions, to further improve the shooting effect. Provide detailed shooting posture guidance, through demonstration animation or voice prompts, to help users pose more natural and beautiful postures.

[0112] In this embodiment, by acquiring the real-time scene image of the current shooting scene, and performing scene analysis on the real-time scene image according to the machine learning model, the scene key information of the real-time scene image is obtained, and the matching composition template matching the scene key information in the composition template library is determined, and the composition suggestion containing the composition method is output according to the matching composition template. And if the real-time scene image includes a person image, the historical scene image matching the real-time scene image in the scene image library is determined, and the posture suggestion containing the shooting posture is output according to the historical scene image, and the shooting image parameters adjusted by the user according to the composition suggestion and / or posture suggestion are determined, and the shooting image is output according to the adjusted shooting image parameters. Then, it can be realized to provide professional-level photography guidance for users to improve the shooting level and photography skills of users. And in the process of users using shooting equipment to shoot, users can obtain composition suggestions and posture suggestions, and can adjust the shooting image parameters according to the composition suggestions and / or posture suggestions, so as to make the shot images more professional and more beautiful. That is, the shooting equipment can use the machine learning model to assist users in adjusting the shooting image parameters according to the actual shooting scene, so that users can use the shooting equipment to shoot higher-quality photos.

[0113] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above introduction, and will not be described in detail later. Figure 2 , the shooting method based on the machine learning model also includes steps d10-d40.

[0114] Step d10, obtaining a training data set, wherein the training data set includes a plurality of training data, each training data includes a training scene image and an image label corresponding to the training scene image;

[0115] Optionally, before shooting, the shooting device needs to first train the machine learning model, and then assist the shooting device in shooting based on the trained machine learning model.

[0116] Optionally, a large number of high-quality images can be collected in advance from social media, image libraries, and professional photography websites and used as training scene images, where the training scene images contain various shooting scenes, composition methods, and shooting postures.

[0117] Optionally, each training scene image may be labeled to determine an image label corresponding to the training scene image. Optionally, when performing the labeling process, the scene key information of each training scene image may be determined and used as an image label. Optionally, the image label includes but is not limited to scene type (such as landscape, person, building, etc.), shooting angle (such as front, side, overhead, etc.), composition method (such as rule of thirds, symmetrical composition, etc.), lighting conditions (such as natural light, artificial light, backlight, etc.) and shooting posture (such as standing, sitting, action, etc.).

[0118] Step d20, inputting the training scene image into the machine learning model to be trained for prediction, and obtaining a first prediction result, wherein the first prediction result includes predicted scene key information;

[0119] Optionally, the training scene image is preprocessed to convert the format of the training scene image into a format suitable for the machine learning model. The training scene image is then input into the machine learning model to be trained to perform model training, and the machine learning model outputs a first prediction result, which at least includes scene key information for predicting the training scene image.

[0120] Optionally, the machine learning model includes a feature extraction network and an image classification module. When training the model, the feature extraction network can be used to extract features from the training scene image, and the extracted features can be mapped to corresponding label information (i.e., scene key information) through the image classification module and output. For example, scene type, shooting angle, communication method, lighting conditions, shooting posture, etc.

[0121] Optionally, when the machine learning model performs model training on training scene images, its training goal can be to enable the machine learning model to accurately identify different shooting scenes from the input images and associate them with corresponding label information.

[0122] Step d30, calculating the loss function value based on the image label and the predicted scene key information;

[0123] Optionally, a loss function corresponding to the machine learning model may be determined, and the image label and the predicted scene key information may be input into the loss function for calculation to obtain a loss function value.

[0124] Step d40, optimize the machine learning model to be trained according to the loss function value until the preset training end condition is met to obtain the preset machine learning model.

[0125] The preset training end condition may be a condition that needs to be met for the end of the pre-training, and may be set as needed, and is not limited in this embodiment. For example, in a specific implementation, it may be set to the convergence of the loss function of the machine learning model, or to the number of training rounds reaching a set number of rounds, or to the duration of training reaching a set duration, etc.

[0126] Optionally, after determining the loss function value, the machine learning model can be reversely updated and optimized according to the loss function value, such as updating the model parameters of the machine learning model until the preset training end conditions are met to obtain a trained machine learning model, and use the trained machine learning model as the preset machine learning model.

[0127] In this embodiment, the training scene images in the training data set are input into the machine learning model for prediction to obtain a first prediction result, the loss function value is calculated based on the image label and the first prediction result, and the machine learning model is optimized based on the loss function value until the preset training end conditions are met to obtain a preset machine learning model. In turn, the shooting device can output composition suggestions and posture suggestions based on the machine learning model during subsequent shooting to assist users in taking high-quality images.

[0128] Further, in a feasible embodiment, the machine learning model includes an adapter and a model ontology,

[0129] Optionally, in step d40, the step of optimizing the machine learning model to be trained according to the loss function value includes step e10.

[0130] Step e10, freeze the model parameters of the model body, and optimize the model parameters of the adapter according to the loss function value.

[0131] Optionally, the machine learning model includes an adapter and a model body, and the adapter may include an adapter located above the model body and an adapter located below the model body.

[0132] Optionally, when training the machine learning model, the model parameters of the model body can be frozen so that the model parameters of the model body remain unchanged during the training process. The model parameters in the adapter are reversely optimized according to the loss function value.

[0133] In this embodiment, by freezing the model parameters of the model body when training the machine learning model, and optimizing the model parameters of the adapter according to the loss function value, the model training efficiency can be improved.

[0134] In addition, to assist in understanding the overall process of the shooting method based on the machine learning model in this embodiment, an example is given below.

[0135] For example, Figure 3 As shown, in the model training stage, the shooting device first collects data, including collecting a large number of high-quality photos (i.e., training scene images) from social media, photo libraries, and professional photography websites, and labels the collected photos to obtain image labels corresponding to the training scene images. Then, the machine learning model is trained based on the training scene images and the corresponding image labels to obtain a trained machine learning model. In the model application stage, input data, including real-time scene images, is obtained and input into the machine learning model. The real-time scene images are feature extracted through the machine learning model, and real-time scene analysis is performed. When performing real-time scene analysis, scene recognition can be performed to obtain scene key information, match the scene key information with the composition template in the preset composition template library, and make composition recommendations based on the matched composition template, such as outputting composition suggestions including composition methods. Users can adjust the shooting image parameters according to the composition suggestions, and after the adjustment, perform user photo operations, and the shooting device can output the shot image. In addition, if there is a person image in the real-time scene image, the historical scene image matching the real-time scene image can be determined in the real-time scene library, and the feature of the person image in the real-time scene image can be extracted, and the person image can be matched with the posture template corresponding to the historical scene image to determine the matching posture template, and the posture recommendation is performed based on the matching posture template, and the posture suggestion containing the shooting posture is output. The user can adjust the shooting image parameters according to the posture suggestion, and after the adjustment, the user takes a photo, and the shooting device can output the shot image. Optionally, user feedback and optimization can also be performed, that is, the machine learning model is optimized according to the feedback data of the user feedback to obtain an optimized machine learning model.

[0136] Further, refer to Figure 4 The present application provides a shooting device based on a machine learning model, and the shooting device based on a machine learning model includes:

[0137] The acquisition module A10 is used to acquire a real-time scene image of the current shooting scene, and perform scene analysis on the real-time scene image according to a preset machine learning model to obtain scene key information of the real-time scene image;

[0138] The composition suggestion module A20 is used to match the scene key information with the composition template in the preset composition template library to obtain the matching composition template, and output the composition suggestion including the composition method according to the matching composition template;

[0139] A posture suggestion module A30, for determining a historical scene image matching the real-time scene image in a preset scene image library if the real-time scene image includes a person image, and outputting a posture suggestion including a shooting posture according to the historical scene image;

[0140] The output module A40 is used to determine the captured image parameters adjusted by the user according to the composition suggestion and / or the posture suggestion, and output the captured image according to the adjusted captured image parameters.

[0141] Optionally, the posture suggestion module A30 is used to:

[0142] If there are at least two historical scene images, constructing a posture template according to the posture features of the historical figures in the historical scene images, wherein the posture template includes the shooting posture;

[0143] Determine the character posture features of the character image in the real-time scene image, and determine the similarity between the character posture features and the posture template;

[0144] Among the posture templates corresponding to the similarities greater than a preset similarity threshold, a posture template is selected as a matching posture template, and a posture suggestion including a shooting posture is generated according to the matching posture template, and the posture suggestion is output.

[0145] Optionally, the scene key information includes at least one of the scene type, subject position and light condition, and the composition suggestion module A20 is used to:

[0146] Matching a composition template in a preset composition template library with the scene type to obtain a matching first composition template;

[0147] Matching a composition template in a preset composition template library with the subject position to obtain a matching second composition template;

[0148] Matching a composition template in a preset composition template library with the light condition to obtain a matching third composition template;

[0149] According to a preset user preference, a composition template is selected from the first composition template, the second composition template and / or the third composition template as a matching composition template.

[0150] Optionally, the output module A40 is used to:

[0151] Acquire feedback data from users regarding the output composition suggestions and posture suggestions;

[0152] The machine learning model is optimized according to the feedback data to obtain an optimized machine learning model.

[0153] Optionally, the shooting device based on the machine learning model includes a training module (not shown in the figure).

[0154] Training modules for:

[0155] Obtain a training data set, wherein the training data set includes multiple training data, each training data includes a training scene image and an image label corresponding to the training scene image;

[0156] Inputting the training scene image into the machine learning model to be trained for prediction, and obtaining a first prediction result, wherein the first prediction result includes predicted scene key information;

[0157] The loss function value is calculated based on the image label and the predicted scene key information;

[0158] The machine learning model to be trained is optimized according to the loss function value until the preset training end condition is met to obtain the preset machine learning model.

[0159] Optionally, the machine learning model includes an adapter and a model ontology, and the training module is used to:

[0160] Freeze the model parameters of the model body and optimize the model parameters of the adapter according to the loss function value.

[0161] The shooting device based on the machine learning model provided in this application can assist users in taking high-quality photos. Compared with the prior art, the beneficial effects of the shooting device based on the machine learning model provided in this application are the same as the beneficial effects of the shooting method based on the machine learning model provided in the above embodiment, which will not be repeated here.

[0162] The present application provides a shooting device based on a machine learning model, and the shooting device based on a machine learning model includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the shooting method based on the machine learning model in the above-mentioned embodiment one.

[0163] Reference below Figure 5, which shows a schematic diagram of the structure of a shooting device based on a machine learning model suitable for implementing an embodiment of the present application. The shooting device based on a machine learning model in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The shooting device based on a machine learning model shown in the figure is only an example and should not bring any limitations to the functions and scope of use of the embodiments of the present application.

[0164] The shooting device based on the machine learning model may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the shooting device based on the machine learning model to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a shooting device based on the machine learning model with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0165] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0166] The shooting device based on the machine learning model provided in the present application adopts the shooting method based on the machine learning model in the above embodiment, which can assist the user in taking high-quality photos. Compared with the prior art, the beneficial effects of the shooting device based on the machine learning model provided in the present application are the same as the beneficial effects of the shooting method based on the machine learning model provided in the above embodiment, and the other technical features of the shooting device based on the machine learning model are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0167] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0168] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0169] The present application provides a medium, which is a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, and the computer-readable program instructions are used to execute the shooting method based on the machine learning model in the above-mentioned embodiment.

[0170] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0171] The above-mentioned computer-readable storage medium may be included in the shooting device based on the machine learning model; or it may exist independently without being assembled into the shooting device based on the machine learning model.

[0172] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by a shooting device based on a machine learning model, the shooting device based on the machine learning model can execute the step process of the above-mentioned shooting method based on the machine learning model.

[0173] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0174] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0175] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0176] The medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned shooting method based on machine learning models, and can assist users in taking high-quality photos. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the shooting method based on machine learning models provided in the above-mentioned embodiments, and will not be elaborated here.

[0177] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the shooting method based on the machine learning model as described above.

[0178] The computer program product provided in this application can assist users in taking high-quality photos. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the shooting method based on the machine learning model provided in the above embodiment, which will not be repeated here.

[0179] The above are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A shooting method based on a machine learning model, characterized in that: The method comprises: Acquire a real-time scene image of the current shooting scene, and perform scene analysis on the real-time scene image according to a preset machine learning model to obtain scene key information of the real-time scene image; Matching the scene key information with a composition template in a preset composition template library to obtain a matching composition template, and outputting a composition suggestion including a composition method according to the matching composition template; If the real-time scene image includes a person image, determining a historical scene image that matches the real-time scene image in a preset scene image library, and outputting a posture suggestion including a shooting posture according to the historical scene image; Determine the captured image parameters adjusted by the user according to the composition suggestion and / or the posture suggestion, and output the captured image according to the adjusted captured image parameters.

2. The shooting method based on the machine learning model according to claim 1, characterized in that: The step of outputting a posture suggestion including a shooting posture according to the historical scene image comprises: If there are at least two historical scene images, constructing a posture template according to the posture features of the historical figures in the historical scene images, wherein the posture template includes a shooting posture; Determining a character posture feature of the character image in the real-time scene image, and determining a similarity between the character posture feature and the posture template; Among the posture templates corresponding to the similarities greater than a preset similarity threshold, a posture template is selected as a matching posture template, and a posture suggestion including a shooting posture is generated according to the matching posture template, and the posture suggestion is output.

3. The shooting method based on the machine learning model according to claim 1, characterized in that: The scene key information includes at least one of scene type, subject position and light condition, The step of matching the scene key information with a composition template in a preset composition template library to obtain a matching composition template comprises: Matching a composition template in a preset composition template library with the scene type to obtain a matching first composition template; Matching a composition template in a preset composition template library with the subject position to obtain a matching second composition template; Matching a composition template in a preset composition template library with the light condition to obtain a matching third composition template; According to a preset user preference, a composition template is selected from the first composition template, the second composition template and / or the third composition template as a matching composition template.

4. The shooting method based on the machine learning model according to claim 1, characterized in that: The method further comprises: Acquire feedback data from a user regarding the output composition suggestion and the posture suggestion; The machine learning model is optimized according to the feedback data to obtain an optimized machine learning model.

5. The shooting method based on a machine learning model according to any one of claims 1 to 4, characterized in that: The method further comprises: Acquire a training data set, wherein the training data set includes a plurality of training data, each training data includes a training scene image and an image label corresponding to the training scene image; Inputting the training scene image into the machine learning model to be trained for prediction, and obtaining a first prediction result, wherein the first prediction result includes predicted scene key information; Calculating a loss function value based on the image label and the predicted scene key information; The machine learning model to be trained is optimized according to the loss function value until a preset training end condition is met to obtain a preset machine learning model.

6. The shooting method based on the machine learning model according to claim 5, characterized in that: The machine learning model includes an adapter and a model ontology, The step of optimizing the machine learning model to be trained according to the loss function value includes: The model parameters of the model body are frozen, and the model parameters of the adapter are optimized according to the loss function value.

7. A shooting device based on a machine learning model, characterized in that: The shooting device based on the machine learning model includes: An acquisition module is used to acquire a real-time scene image of a current shooting scene, and perform scene analysis on the real-time scene image according to a preset machine learning model to obtain scene key information of the real-time scene image; A composition suggestion module, used for matching the scene key information with a composition template in a preset composition template library to obtain a matching composition template, and outputting a composition suggestion including a composition method according to the matching composition template; A posture suggestion module, configured to determine a historical scene image matching the real-time scene image in a preset scene image library if the real-time scene image includes a person image, and output a posture suggestion including a shooting posture according to the historical scene image; The output module is used to determine the captured image parameters adjusted by the user according to the composition suggestion and / or the posture suggestion, and output the captured image according to the adjusted captured image parameters.

8. A shooting device based on a machine learning model, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the shooting method based on a machine learning model as described in any one of claims 1 to 6.

9. A medium, characterized in that The medium is a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the shooting method based on a machine learning model as described in any one of claims 1 to 6 are implemented.

10. A product, characterized in that The product is a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the shooting method based on the machine learning model as described in any one of claims 1 to 6 are implemented.

Images