Photographing method, photographing apparatus, electronic device, and computer-readable storage medium

Abstract

The present application provides a photographing method, a photographing device, an electronic device and a computer readable storage medium, the method comprises the steps of: acquiring a real-time acquisition image; performing scene recognition on the real-time acquisition image to determine the current photographing scene corresponding to the real-time acquisition image; matching a target expert model corresponding to the current photographing scene; obtaining a target photographing parameter corresponding to the real-time acquisition image through the target expert model; and performing a photographing operation based on the target photographing parameter. By matching the target expert model corresponding to the current photographing scene, the target photographing parameter suitable for the current photographing scene can be output through the expert model most suitable for the scene, so that the user does not need to manually select the photographing parameter, and the convenience and accuracy of the photographing parameter setting are improved.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to a photographic method, photographic apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] With the rapid development of mobile terminal camera technology and artificial intelligence algorithms, intelligent photography systems typically support a variety of preset shooting functions such as portrait mode, night scene mode, and HDR mode, thereby enabling the quick setting of preset camera parameters; however, in actual shooting, users need to select based on their own shooting needs, which is relatively cumbersome. Summary of the Invention

[0003] The main objective of this invention is to provide several embodiments of a photographic method, photographic device, electronic device, and computer-readable storage medium, wherein at least one embodiment can solve the problem of cumbersome preset camera parameter settings in the prior art.

[0004] To achieve the above objectives, the present invention provides a photographic method, the method comprising the following steps: Acquire real-time images; Scene recognition is performed on real-time acquired images to determine the current photographic scene corresponding to the real-time acquired images; Match the target expert model with the current photography scene; The target photography parameters corresponding to the real-time acquired images are obtained through a target expert model; Perform photographic operations based on the target photographic parameters.

[0005] Optionally, scene recognition of the real-time acquired images to determine the current photographic scene corresponding to the real-time acquired images includes: Semantic analysis is performed on real-time acquired images to obtain corresponding subject features and environmental features; The scene semantic vector corresponding to the current photography scene is obtained by fusing the subject features with the environmental features.

[0006] Optionally, the target expert model matching the current photography scene includes: Obtain the scene semantic vector corresponding to the current photography scene; Determine the scene matching degree between multiple preset expert models and scene semantic vectors; The preset number of expert models with the highest scene matching degree are selected as the target expert models.

[0007] Optionally, the target photography parameters corresponding to the real-time acquired images obtained through the target expert model include: Obtain the model photography parameters output by multiple target expert models for real-time acquired images; The fusion weights of each target expert model are determined based on the current photography scene. The scene matching degree between the current photography scene and the target expert model is positively correlated with the fusion weights of the target expert models. The target photographic parameters are obtained by weighted fusion of the model photographic parameters according to the fusion weights.

[0008] Optionally, the fusion weights of each target expert model are determined based on the current photography scene, including: Obtain user parameter instructions and determine the user requirement characteristics corresponding to the user parameter instructions; Determine the degree of matching between each target expert model and the instructions corresponding to user demand characteristics; The fusion weights of the target expert model are determined by combining the scene matching degree and the instruction matching degree; among them, the instruction matching degree is positively correlated with the fusion weights.

[0009] Optionally, the fusion weights of each target expert model are determined based on the current photography scene, including: Obtain user photography style characteristics; Determine the style matching degree between each target expert model and the user's photography style features; The fusion weights of the target expert model are determined by combining scene matching degree and style matching degree; among them, style matching degree is positively correlated with fusion weight.

[0010] Optionally, the fusion weights of each target expert model are determined based on the current photography scene, including: Determine the scene credibility corresponding to the current photography scene, the command credibility corresponding to the user parameter commands, and the style credibility corresponding to the user's photography style characteristics; Determine the scene weight, instruction weight, and style weight corresponding to scene credibility, instruction credibility, and style credibility; Determine the scene matching degree, instruction matching degree, and style matching degree corresponding to each target expert model; The fusion weights of the target expert model are obtained by weighting and summing the scene matching degree, instruction matching degree, and style matching degree using scene weight, instruction weight, and style weight.

[0011] To achieve the above objectives, the present invention also provides a photographic apparatus, the photographic apparatus comprising: The image acquisition module is used to acquire real-time images; The scene recognition module is used to perform scene recognition on real-time acquired images to determine the current photographic scene corresponding to the real-time acquired images; The model matching module is used to match the target expert model corresponding to the current photography scene; The parameter determination module is used to obtain the target photography parameters corresponding to the real-time acquired images through the target expert model; The operation execution module is used to perform photographic operations based on the target photographic parameters.

[0012] To achieve the above objectives, the present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the above-described photographic method.

[0013] To achieve the above objectives, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described photographic method.

[0014] This invention proposes a photography method, photography device, electronic device, and computer-readable storage medium to acquire real-time images; perform scene recognition on the real-time acquired images to determine the current photography scene corresponding to the real-time acquired images; match a target expert model corresponding to the current photography scene; obtain target photography parameters corresponding to the real-time acquired images through the target expert model; and perform photography operations based on the target photography parameters. By matching the current photography scene with the corresponding target expert model, the target photography parameters suitable for the current photography scene can be output through the expert model best suited to the scene, thus eliminating the need for users to manually select photography parameters and improving the convenience and accuracy of photography parameter settings. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the first embodiment of the photographic method of the present invention; Figure 2 This is a detailed flowchart of the photographic method of the present invention; Figure 3 This is a schematic diagram of the module structure of the electronic device of the present invention. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0019] This invention provides a photographic method applicable to electronic devices, such as cameras, mobile phones, tablets, wearable devices, and other devices with image acquisition capabilities; see reference. Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the photographic method of the present invention, which includes the following steps: Step S10: Acquire real-time images; Real-time image acquisition refers to the real-time output images from the camera module sensor of the electronic device; specifically, real-time image acquisition refers to preview frame images; the specific parameters of real-time image acquisition can be set according to actual needs, such as the camera module continuously outputting real-time image acquisition at 30fps to 60fps; real-time image acquisition in RAW format is continuously obtained through the camera sensor. The system copies the preview frames to the analysis pipeline as real-time image acquisition before ISP processing.

[0020] Step S20: Perform scene recognition on the real-time acquired images to determine the current photography scene corresponding to the real-time acquired images; The current photographic scene indicates the type of scene in which the photograph takes place; the current photographic scene can be characterized by a variety of elements, such as, but not limited to, the subject in the picture, the state of the subject, the environment, the time, the light, and the location.

[0021] It is understandable that different scenarios have different photographic requirements. For example, portrait photography requires optimization of facial features; landscape photography requires optimization of image clarity; and different lighting conditions require optimization of image brightness. The above optimizations are just examples. In practical applications, specific scene optimization settings can be made based on actual needs.

[0022] The specific methods for scene recognition can be set according to actual needs, such as recognizing scenes by setting up scene detection models; the specific type and settings of scene detection models can be set according to actual needs.

[0023] Step S30: Match the target expert model corresponding to the current photography scene; Expert models are independently trained models that are specifically optimized for a single, segmented photographic scene; the specific type of model can be set based on actual needs, such as an image processing neural network model.

[0024] The target expert model is an expert model that matches the current photography scene.

[0025] Understandably, expert models are optimized for specific photography scenarios. Therefore, a target expert model is obtained by matching the current photography scenario, so that the target expert model can be optimized for the current photography scenario and output photography parameters that meet the photography requirements of the current photography scenario.

[0026] Step S40: Obtain the target photography parameters corresponding to the real-time acquired images through the target expert model; The photography parameters are a complete set of control parameters used in the ISP (Image Signal Processor) imaging pipeline; the specific types of photography parameters can be set according to actual needs, such as photography parameters including but not limited to Exposure, ISO, white balance, contrast, saturation, gamma correction, noise reduction, and sharpening.

[0027] The target photography parameters are the photography parameters that the target expert model outputs after detecting real-time acquired images and optimizing them for the current photography scene.

[0028] Step S50: Perform photography operations based on target photography parameters.

[0029] After obtaining the target photography parameters, the photography parameters of the camera module are set to the target photography parameters. It can be understood that the real-time acquired image is a preview frame image captured by the camera module in real time. Therefore, it can be obtained without user operation. By analyzing the real-time acquired image, the target photography parameters that match the real-time photography situation can be updated in real time. This allows the image to be captured based on the obtained target photography parameters when the user performs the photography operation. This ensures that the photography parameters used during imaging match the current photography scene.

[0030] This embodiment matches the current photography scene with a corresponding expert model, enabling the output of target photography parameters suitable for the current photography scene through the expert model that best matches the scene. This eliminates the need for users to manually select photography parameters, thus improving the convenience and accuracy of photography parameter settings.

[0031] Further details will follow. Figure 2In the second embodiment of the photographic method of the present invention based on the first embodiment, step S20 includes the following steps: Step S21: Perform semantic analysis on the real-time acquired images to obtain the corresponding subject features and environmental features; Step S22: The subject features and environmental features are fused to obtain the scene semantic vector corresponding to the current photography scene.

[0032] The main features are the characteristics representing the subject of the scene and its state, extracted from real-time acquired images.

[0033] The main subject indicated by the main features includes, but is not limited to, people, animals, scenery, buildings, food, plants, starry sky, moving objects, etc. The subject's status includes, but is not limited to, the number of subjects, their positions, their proportion, and their movement. For example, the main features can be indicated as a whole, such as a single portrait, a double portrait, a person's state, a group photo, a fast-moving animal, a distant landscape, or a close-up of food. When it comes to portrait processing, it is necessary to combine the person's state to control the photography parameters. Therefore, the person's state can be identified, which can include facial state, body state, etc., such as smiling, closing eyes, jumping, walking, standing still, and running.

[0034] Environmental features are those extracted from real-time acquired images that characterize the state of the environment.

[0035] Environmental characteristics such as lighting characteristics, weather characteristics, time characteristics, and location characteristics; Lighting characteristics such as front lighting, backlighting, side lighting, low light, high dynamic range, and point light source scenes; Weather characteristics such as sunny, cloudy, rainy, snowy, and foggy; Time-related features such as current time, seasonal information, and holiday information; Location characteristics include GPS geolocation and scene location type; location type includes seaside, mountainous, urban, grassland, coffee shop, shopping mall, and indoor.

[0036] Environmental features can indicate the overall photographic environment, such as a beach at dusk, a city at night, or warm indoor lighting.

[0037] Semantic analysis methods can be configured based on actual needs, such as using an attention mechanism model to perform semantic analysis.

[0038] Subject features and environmental features can be set in the form of feature vectors; the subject features and environmental features are fused to obtain the scene semantic vector; the specific fusion method can be set according to actual needs, such as using the Transformer attention mechanism for multimodal feature fusion.

[0039] It is understandable that scene semantic vectors indicate the characteristics of the current photographic scene, and therefore, scene semantic vectors can represent the current photographic scene.

[0040] In this embodiment, semantic analysis is used to split the features into two dimensions: subject and environment, covering multimodal feature information in the current photography scene and accurately determining the current photography scene.

[0041] Furthermore, in the third embodiment of the photographic method of the present invention based on the first embodiment, step S30 includes the following steps: Step S31: Obtain the scene semantic vector corresponding to the current photography scene; Step S32: Determine the scene matching degree between multiple preset expert models and scene semantic vectors; Step S33: Select the preset number of expert models with the highest scene matching degree as the target expert models.

[0042] The preset expert model is a pre-set image processing model that is independently trained for a single shooting scene to optimize the corresponding shooting scene.

[0043] The specific photography scene type targeted by the preset expert model can be set according to actual needs, such as portrait expert model, landscape expert model, animal expert model, night scene expert model, food expert model, starry sky expert model, cinematic expert model, macro expert model, and sports photography expert model; for example, the portrait expert model focuses on optimizing the skin tone, facial details, bokeh effect, and face exposure of the person in the picture; the landscape expert model focuses on optimizing HDR dynamic range, sky layering, color enhancement, and distant scene sharpening.

[0044] Scene matching degree is used to quantify the degree of fit between the preset expert model and the current photographic scene represented by the scene semantic vector.

[0045] The scene matching degree can be calculated based on the scene semantic vector by setting the expert model routing scheduling unit Router; the Router can be set according to actual needs.

[0046] The higher the scene matching degree, the better the preset expert model matches the current photography scene corresponding to the scene semantic vector. Therefore, it is more likely to be selected as the target expert model.

[0047] The target expert model is a set of pre-defined expert models that rank highly in terms of scene matching; the target expert model is a pre-defined expert model that is ultimately applied to the target photography parameter output in the current photography scene.

[0048] The specific number of target expert models can be set based on actual needs. Understandably, there may be multiple photography parameter adjustments required in a photography scene, such as when there are people, landscapes, and night scenes in the current photography scene. In this case, multiple target expert models can be called at the same time to comprehensively determine the target photography model, thereby improving the adaptability to scenes with complex photography parameter requirements.

[0049] The specific value of the preset quantity can be set based on actual needs, such as 2 to 5.

[0050] In practical applications, in order to eliminate preset expert models that differ significantly from the current photography scene, a scene matching threshold can be set, and the target expert model can be selected only from preset expert models whose scene matching degree is greater than the scene matching threshold. The specific value of the scene matching threshold can be set based on actual needs.

[0051] If the scene matching threshold is set to 0.7 and the preset number is 2; in a scene of two people portraits at a beach at dusk, the scene matching degree corresponding to the portrait expert model is 0.93, the scene matching degree corresponding to the landscape expert model is 0.82, the scene matching degree corresponding to the night scene expert model is 0.31, and the scene matching degree corresponding to the animal expert model is 0.04; then the selected target expert models are the portrait expert model and the landscape expert model.

[0052] This embodiment calculates scene matching degree based on scene semantic vectors, accurately quantifies the degree of fit between each preset expert model and the current photography scene, and improves the matching accuracy of the target expert model selection.

[0053] Furthermore, in the fourth embodiment of the photographic method of the present invention based on the first embodiment, step S40 includes the following steps: Step S41: Obtain the model photography parameters output by multiple target expert models for real-time acquired images; Step S42: Determine the fusion weights of each target expert model based on the current photography scene, wherein the scene matching degree between the current photography scene and the target expert model is positively correlated with the fusion weights of the target expert models. Step S43: The model photography parameters are weighted and fused according to the fusion weights to obtain the target photography parameters.

[0054] The model photography parameters are a set of ISP control parameters for the target expert model outputting real-time acquired images; the parameter types included in the model photography parameters are consistent with those in the target photography parameters.

[0055] It should be noted that if the number of target expert models is 1, then the model photography parameters output by the target expert model are used as the target photography parameters.

[0056] If there are multiple target expert models, the model photography parameters will be different because each expert model outputs model photography parameters based on its own target scene. In order to integrate the optimization suggestions of all target expert models, the model photography parameters are fused to obtain the final target photography parameters.

[0057] The fusion weights are the normalized coefficients of the target expert models when fusion is performed.

[0058] It is understandable that the higher the matching degree between the target expert model and the current photography scene, the better the optimization effect of the target expert model for the current photography scene. Therefore, the positive impact of its output model photography parameters on the target photography parameters is greater. Therefore, in this embodiment, the scene matching degree is set to be positively correlated with the fusion weight corresponding to the target expert model.

[0059] This embodiment sets a positive correlation between scene matching degree and fusion weight, so that the target expert model with higher adaptability has a larger parameter contribution ratio, thus avoiding the impact of low-adaptability models on target photography parameters.

[0060] Further, step S42 includes the following steps: Step S421: Obtain user parameter instructions and determine the user demand characteristics corresponding to the user parameter instructions; Step S422: Determine the matching degree between each target expert model and the instructions corresponding to the user demand features; Step S423: Determine the fusion weight of the target expert model by combining the scene matching degree and the instruction matching degree; wherein, the instruction matching degree is positively correlated with the fusion weight.

[0061] User parameter commands are instructions entered by the user to adjust photography parameters; the specific input method for user parameter commands can be set according to actual needs, such as voice, text, button, etc.

[0062] It should be noted that the user parameter commands are input in natural language, such as "warmer", "more cinematic", "clearer sky", "more natural skin tone", "not too bright"; that is, in this embodiment, the user is not limited to inputting simple keywords corresponding to actual photography parameters, but rather describes the abstract adjustment needs of the image based on actual needs and understanding.

[0063] In order to map user parameter commands to specific photography parameters, this embodiment identifies the user parameter commands to obtain the corresponding user demand features.

[0064] User demand characteristics indicate the adjustment of photography parameters corresponding to user parameter commands, such as: The user parameter command was "warmer," and the identified user demand feature was "color temperature increase." The user parameter command is "cinematic feel," and the identified user demand feature is "tone curve adjustment." The user parameter command is transparent, and the identified user demand feature is local contrast enhancement; The user parameter command was set to "soft," and the identified user demand feature was "reduced sharpness." The user parameter instruction was to reduce highlights, and the identified user demand feature was HDR highlight compression. The user's parameter command was to blur the background more and warm the overall color tone more. The identified user needs were adjustments to virtual depth of field parameters, white balance parameters, color temperature parameters, background blur intensity, and color style parameters.

[0065] The above user parameter commands and user requirement characteristics are only illustrative examples. In actual applications, they can be set according to actual needs.

[0066] Understandably, user demand characteristics directly indicate adjustments to photographic parameters. The specific method for identifying user demand characteristics can be set based on actual needs, such as using a large language model to identify user parameter commands to obtain the corresponding user demand characteristics.

[0067] Instruction matching degree indicates the degree of fit between the optimization style of the target expert model and the user's needs features. The determination of instruction matching degree can be based on actual needs, such as calculating the similarity between the user's needs feature vector and the built-in style features of each target expert model to obtain the corresponding instruction matching degree.

[0068] It is understandable that user demand characteristics indicate the user's own photography parameter setting requirements. Therefore, the more a target expert model matches the user demand characteristics, the more it can adjust the photography parameters in the direction required by the user. Therefore, in this embodiment, the instruction matching degree is set to be positively correlated with the fusion weight, so as to ensure that the imaging adapts to the current photography scene while fully responding to the personalized adjustment requirements put forward by the user.

[0069] Further, step S42 includes the following steps: Step S424: Obtain user photography style characteristics; Step S425: Determine the style matching degree between each target expert model and the user's photography style features; Step S426: Determine the fusion weight of the target expert model by combining scene matching degree and style matching degree; wherein, style matching degree is positively correlated with fusion weight.

[0070] User photography style features are structured feature vectors that reflect user photography habits and preferences, obtained by learning user photography behavior.

[0071] User photography style features can be modeled based on users' long-term photography behavior, such as collecting data on users' image saving, deletion, editing, sharing, and parameter adjustment behaviors during the photography process, and constructing user photography style features based on the collected behavioral data. User photography style features include, but are not limited to, color preferences, HDR preferences, saturation preferences, bokeh preferences, and cinematic preferences.

[0072] User photography style features are continuously updated as users use the service, so that the user photography style features can increasingly match the user's actual photography preferences; specifically, a low learning rate incremental update method can be adopted to avoid fluctuations in user photography style features.

[0073] Furthermore, users can share their own photography style characteristics with other users, such as through a sharing code. Other users can then optimize the style based on the user's photography style characteristics to create a new style.

[0074] Style matching degree indicates the degree of fit between the optimized style of the target expert model and the user's photography style. The style matching degree can be determined based on actual needs, such as calculating the similarity between the user's photography style features and the built-in style features of each target expert model to obtain the corresponding style matching degree.

[0075] It is understandable that user photography style characteristics indicate the user's photography style and preferences. Therefore, the more a target expert model matches the user's photography style characteristics, the more it can adjust the photography parameters in the direction of the user's preferences. Therefore, in this embodiment, the style matching degree is set to be positively correlated with the fusion weight, so as to ensure that the image adapts to the current photography scene while fully matching the user's photography habits.

[0076] Further, step S42 includes the following steps: Step S427: Determine the scene credibility corresponding to the current photography scene, the instruction credibility corresponding to the user parameter instruction, and the style credibility corresponding to the user photography style characteristics. Step S428: Determine the scene weight, instruction weight, and style weight corresponding to scene credibility, instruction credibility, and style credibility; Step S429: Determine the scene matching degree, instruction matching degree, and style matching degree corresponding to each target expert model; Step S4210: The scene matching degree, instruction matching degree, and style matching degree are weighted and summed using scene weight, instruction weight, and style weight to obtain the fusion weight of the target expert model.

[0077] Scene credibility refers to the completeness, clarity, and effectiveness of the current photographic scene recognition result represented by the scene semantic vector; scene credibility can be the confidence level of the model output that obtains scene credibility.

[0078] Command credibility is the degree to which user parameter commands are clearly recognized and semantically valid; when there is no voice or other user parameter command input, the command credibility is 0; the more ambiguous the user parameter command, such as the greater the voice noise, the lower the command credibility; the command credibility can be specifically calculated by the signal-to-noise ratio of the user parameter command.

[0079] Style credibility refers to the reliability of a user's photographic style features. Style credibility can be determined by the state of the behavioral data that yields the user's photographic style features, such as the quantity of behavioral data that yields the user's photographic style features. The larger the quantity, the more sufficient the data is considered, and the more reliable the preference fit is. Therefore, it corresponds to a higher style credibility.

[0080] Normalize the scene credibility, instruction credibility, and style credibility to obtain the corresponding scene weight, instruction weight, and style weight; for example:

[0081]

[0082]

[0083] Where Cs is the scene credibility; α is the scene weight; Cl is the instruction credibility; β is the instruction weight; Cu is the style credibility; γ is the style weight; α + β + γ = 1.

[0084] Scene weight, instruction weight, and style weight reflect the reliability of the indication in terms of scene, instruction, and style. The higher the weight, the more reliable the photographic parameter settings required in that direction.

[0085] After obtaining the scene weight, instruction weight, and style weight, the scene matching degree, instruction matching degree, and style matching degree are then weighted and summed:

[0086] Among them, Score i SceneScore is the fusion weight of the expert weights for the i-th target; i The scene matching degree is the weight of the expert for the i-th target; SemanticScore i UserScore is the instruction matching degree of the expert weight for the i-th target; i Let be the style matching degree of the expert weight for the i-th target.

[0087] After obtaining the fusion weights, the model photography parameters output by each target expert model can be fused based on the fusion weights to obtain the target photography parameters; for example, let the model photography parameters Pi of the i-th expert model be:

[0088] According to the fusion weight Score i The model photography parameters output by each target expert model are weighted and fused, such as: Final exposure parameters:

[0089] Final ISO parameters:

[0090] Final white balance parameters:

[0091] All other photographic parameters were fused using the same method to generate complete target photographic parameters:

[0092] The ISP image processing pipeline performs image processing such as exposure control, white balance adjustment, color mapping, sharpening, and noise reduction based on the final generated target photography parameters, and outputs the final photography result.

[0093] The following section describes the target photography parameter generation process for this application, using a specific scenario as an example: 1. The current shooting scene is a seaside landscape shot at dusk: 1. The environmental semantic analysis module performs scene recognition on real-time acquired images to obtain scene semantic vectors; Scene semantic vectors include: Scene: Seaside scenery; Time: 18:30 (sunset); Lighting: Low intensity; Weather: Sunny; Number of characters: 1; Phone orientation: Landscape.

[0094] The scene matching degree corresponding to the target expert model is: Landscape expert model -0.95; Portrait expert model -0.58; Night scene expert model -0.72.

[0095] 2. User parameter commands; User inputs via voice: "Make the shot look more cinematic, and warmer in color." The natural language photography optimization module analyzed and obtained the following user demand characteristics: Cinematic feel, warm color tone, low to medium exposure, soft contrast.

[0096] The instruction matching degree corresponding to the target expert model is: Landscape expert model -0.88; Portrait expert model -0.46; Night scene expert model -0.79.

[0097] 3. User's photography style; By reading the user's long-term photography model and analyzing the most recent 1000 photos, the user's photographic style characteristics are determined as follows: 82% of the photos used a warm color scheme; 76% of the photos used a cinematic style; and 68% of the photos had their exposure compensation reduced.

[0098] The style matching degree corresponding to the target expert model is: Landscape expert model -0.84; Portrait expert model -0.63; Night scene expert model -0.61.

[0099] 4. Weighting; The system calculates the confidence levels for each direction as follows: Scene credibility Cs = 0.95; instruction credibility Cl = 0.88; style credibility Cu = 0.84.

[0100] The calculated weights for each direction are: Scene weight α = 0.355; instruction weight β = 0.329; style weight γ = 0.316.

[0101] The fusion weights corresponding to the landscape expert model are: Score = 0.355×0.95 + 0.329×0.88 + 0.316×0.84 = 0.893 The fusion weights corresponding to the portrait expert model are: Score = 0.355×0.58 + 0.329×0.46 + 0.316×0.63 = 0.557 The fusion weights corresponding to the night scene expert model are: Score = 0.355×0.72 + 0.329×0.79 + 0.316×0.61 = 0.708 The portrait expert model was removed because its weight was too low. The model photography parameters corresponding to the Landscape Expert model are: EV -0.55; ISO -120; White Balance -5600K; Contrast -+15; Saturation -+8; Sharpen -18; The model photography parameters corresponding to the Night Scene Expert Model are: EV-0.60; ISO-200; White Balance-4800K; Contrast-+12; Saturation-+5; Sharpen-10; 5. Calculate the target imaging parameters; Exposure: EV = 0.558 × 0.55 + 0.442 × 0.60 = 0.572 ISO: ISO = 0.558 × 120 + 0.442 × 200 = 155 White balance: WB = 0.558 × 5600 + 0.442 × 4800 = 5246K 0.558 and 0.442 are the normalized values ​​of the fusion weights 0.893 and 0.708 for the landscape expert model and the night scene expert model, respectively. The remaining parameters are fused and calculated in the same way to form a complete set of ISP image processing parameters.

[0102] The image processing pipeline performs image processing such as exposure control, white balance adjustment, color mapping, sharpening, and noise reduction based on the final ISP parameters, and outputs the final photographic result.

[0103] II. The current photography scenario is a user taking portrait photos in a coffee shop: 1. The environmental semantic analysis module performs scene recognition on real-time acquired images to obtain scene semantic vectors; Scene semantic vectors include: Scene: Indoor coffee shop; Lighting: Warm artificial light; Time: 3:00 PM; Number of characters: 1; Distance between characters: Approximately 1.5 meters; Phone orientation: Portrait mode.

[0104] The scene matching degree corresponding to the target expert model is: Landscape expert model -0.28; Portrait expert model -0.96; Night scene expert model -0.41.

[0105] 2. User parameter commands; User inputs via voice: "Please make my skin look more natural, my face look slimmer, and my photos look more sophisticated." The natural language photography optimization module analyzed and obtained the following user demand characteristics: Natural Skin, Slim Face, Premium Portrait; The instruction matching degree corresponding to the target expert model is: Landscape expert model -0.18; Portrait expert model -0.94; Night scene expert model -0.35.

[0106] 3. User's photography style; By reading the user's long-term photography model and analyzing the most recent 1000 photos, the user's photographic style characteristics are determined as follows: 85% of photos have background blur enabled; 81% prefer warm skin tones; 76% prefer natural skin smoothing; and 69% prefer centered portraits.

[0107] The style matching degree corresponding to the target expert model is: Landscape expert model -0.22; Portrait expert model -0.91; Night scene expert model -0.40.

[0108] 4. Weighting; The system calculates the confidence levels for each direction as follows: Scene credibility Cs = 0.96; instruction credibility Cl = 0.94; style credibility Cu = 0.91.

[0109] The calculated weights for each direction are: Scene weight α = 0.341; instruction weight β = 0.334; style weight γ = 0.325.

[0110] The fusion weights corresponding to the portrait expert model are: Score = 0.341×0.96 + 0.334×0.94 + 0.325×0.91= 0.937 The portrait expert model has the highest weight, while other expert models with lower weights are eliminated. The model photography parameters corresponding to the Landscape Expert model are: EV -0.55; ISO -120; White Balance -5600K; Contrast -+15; Saturation -+8; Sharpen -18; The model photography parameters corresponding to the portrait expert model are: EV -0.48; ISO -100; White Balance -5100K; Contrast -+6; Saturation -+4; Skin Tone -+18; Sharpen -12; 5. Calculate the target imaging parameters; The target photography parameters are the model photography parameters output by the portrait expert model.

[0111] The ISP pipeline completes skin tone optimization, face enhancement, background blurring, and detail enhancement, outputting the final portrait photography result.

[0112] 3. The current shooting scene is an urban night scene photographing buildings: 1. The environmental semantic analysis module performs scene recognition on real-time acquired images to obtain scene semantic vectors; Scene semantic vectors include: Scene: City night view; Lighting: Low intensity; Time: 21:10; Subject: Buildings; Phone position: Handheld.

[0113] The scene matching degree corresponding to the target expert model is: Landscape expert model -0.74; Portrait expert model -0.15; Night scene expert model -0.97.

[0114] 2. User parameter commands; User inputs via voice: "Make the shot brighter for me, but don't overexpose it; preserve the neon colors." The natural language photography optimization module analyzed and obtained the following user demand characteristics: Night scene brightening, highlight suppression, neon colors.

[0115] The instruction matching degree corresponding to the target expert model is: Landscape expert model -0.72; Portrait expert model -0.11; Night scene expert model -0.95.

[0116] 3. User's photography style; By reading the user's long-term photography model and analyzing the most recent 1000 photos, the user's photographic style characteristics are determined as follows: 79% prefer night mode; 83% prefer high dynamic range; 72% prefer reduced noise; 77% prefer to preserve the color of the lights.

[0117] The style matching degree corresponding to the target expert model is: Landscape expert model -0.68; Portrait expert model -0.21; Night scene expert model -0.93.

[0118] 4. Weighting; The system calculates the confidence levels for each direction as follows: Scene credibility Cs = 0.97; instruction credibility Cl = 0.95; style credibility Cu = 0.93.

[0119] The calculated weights for each direction are: Scene weight α = 0.340; instruction weight β = 0.333; style weight γ = 0.327.

[0120] The fusion weights corresponding to the landscape expert model are: Score = 0.340×0.74 + 0.333×0.72 + 0.327×0.68 = 0.714 The fusion weights corresponding to the night scene expert model are: Score = 0.340×0.97 + 0.333×0.95 + 0.327×0.93 = 0.950 The portrait expert model was eliminated due to its low weight, and the night scene expert model was selected as the main model. At the same time, the landscape expert model was introduced to assist in color optimization. The model photography parameters for the Landscape Expert model are: EV-0.76; ISO-280; White Balance-4700K; Contrast-+14; Saturation-+8; Noise Reduction-18; Sharpen-12. The model photography parameters for the Night Scene Expert model are: EV-0.82; ISO-420; White Balance-4300K; Contrast-+10; Saturation-+6; Noise Reduction-24; Sharpen-8. 5. Calculate the target imaging parameters; EV = 0.79; ISO = 358; White Balance = 4470K; Contrast = +12; Saturation = +7; Noise Reduction = 21; Sharpen = 10; The ISP image processing pipeline completes multi-frame noise reduction, high dynamic range fusion, light protection and color restoration processing based on the final parameters, and outputs the final night scene photography result.

[0121] IV. The current photography scenario is taking a group photo during a friend's gathering: 1. The environmental semantic analysis module performs scene recognition on real-time acquired images to obtain scene semantic vectors; Scene semantic vectors include: Scene: Indoor gathering; Lighting: Warm artificial lighting; Number of people: 6; People distribution: Two rows; Shooting distance: Approximately 3 meters; Phone orientation: Landscape; Ambient brightness: Low to medium.

[0122] The scene matching degree corresponding to the target expert model is: Group photo expert model -0.97; landscape expert model -0.15; portrait expert model -0.89; night scene expert model -0.62.

[0123] 2. User parameter commands; User inputs via voice: "Make the people in the background clearer too, so the overall look is more cinematic and the skin tone is more natural." The natural language photography optimization module analyzed and obtained the following user demand characteristics: Improved overall depth of field; ensured sharp focus for multiple subjects; enhanced cinematic style; optimized natural skin tones; preserved ambient light.

[0124] The instruction matching degree corresponding to the target expert model is: Group Photo Expert Model -0.95; Landscape Expert Model -0.21; Portrait Expert Model -0.91; Night Scene Expert Model -0.66.

[0125] 3. User's photography style; By reading the user's long-term photography model and analyzing the most recent 1000 photos, the user's photographic style characteristics are determined as follows: 81% prefer a cinematic style; 86% prefer natural skin tones; 73% prefer all photos of multiple people to be clear; 78% prefer warm color tones.

[0126] The style matching degree corresponding to the target expert model is: Group Photo Expert Model -0.92; Landscape Expert Model -0.24; Portrait Expert Model -0.89; Night Scene Expert Model -0.58.

[0127] 4. Weighting; The system calculates the confidence levels for each direction as follows: Scene credibility Cs = 0.97; instruction credibility Cl = 0.95; style credibility Cu = 0.92.

[0128] The calculated weights for each direction are: Scene weight α = 0.342; instruction weight β = 0.335; style weight γ = 0.323.

[0129] The fusion weights corresponding to the expert model for group photos are: Score = 0.342 × 0.97 + 0.335 × 0.95 + 0.323 × 0.92 = 0.947 The fusion weights corresponding to the portrait expert model are: Score = 0.342 × 0.89 + 0.335 × 0.91 + 0.323 × 0.89 = 0.896 The fusion weights corresponding to the night scene expert model are: Score = 0.342 × 0.62 + 0.335 × 0.66 + 0.323 × 0.58 = 0.620 Select a group photo expert model as the main model, and add a portrait expert model as an auxiliary model.

[0130] The model photography parameters corresponding to the expert model for group photos are: The standard settings for the Portrait Expert model are: EV-0.58; ISO-220; White Balance-4900K; Contrast-+8; Saturation-+5; Aperture (virtual depth of field)-F4.5; Face Exposure-+12; Sharpen-14. The corresponding model photography parameters are: EV-0.54; ISO-180; White Balance-5100K; Contrast-+6; Saturation-+4; Skin Tone-+16; Face Exposure-+15; Sharpen-12.

[0131] 5. Calculate the target imaging parameters; Exposure: EV = 0.514 × 0.58 + 0.486 × 0.54 = 0.561 ISO: 0.514 × 220 + 0.486 × 180 = 201 White balance: 0.514 × 4900 + 0.486 × 5100 = 4997K The remaining parameters are fused and calculated in the same way to form a complete set of ISP image processing parameters.

[0132] The image processing pipeline performs image processing such as exposure control, white balance adjustment, color mapping, sharpening, and noise reduction based on the final ISP parameters, and outputs the final photographic result.

[0133] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0134] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0135] To achieve the above objectives, the present invention also provides a photographic apparatus, the photographic apparatus comprising: The image acquisition module is used to acquire real-time images; The scene recognition module is used to perform scene recognition on real-time acquired images to determine the current photographic scene corresponding to the real-time acquired images; The model matching module is used to match the target expert model corresponding to the current photography scene; The parameter determination module is used to obtain the target photography parameters corresponding to the real-time acquired images through the target expert model; The operation execution module is used to perform photographic operations based on the target photographic parameters.

[0136] Furthermore, the scene recognition module includes: The first analysis unit is used to perform semantic analysis on real-time acquired images to obtain corresponding subject features and environmental features; The first fusion unit is used to fuse the subject features with the environmental features to obtain the scene semantic vector corresponding to the current photography scene.

[0137] Furthermore, the model matching module includes: The first acquisition unit is used to acquire the scene semantic vector corresponding to the current photography scene; The first determining unit is used to determine the scene matching degree between multiple preset expert models and scene semantic vectors; The first execution unit is used to select the preset number of expert models with the highest scene matching degree as the target expert models.

[0138] Furthermore, the parameter determination module includes: The second acquisition unit is used to acquire the model photography parameters output by multiple target expert models for real-time acquired images; The second determining unit is used to determine the fusion weights of each target expert model based on the current photography scene, wherein the scene matching degree between the current photography scene and the target expert model is positively correlated with the fusion weights corresponding to the target expert models. The second fusion unit is used to perform weighted fusion of the model's photographic parameters according to the fusion weights to obtain the target's photographic parameters.

[0139] Furthermore, the second determining unit includes: The first acquisition subunit is used to acquire user parameter instructions and determine the user demand characteristics corresponding to the user parameter instructions. The first determining subunit is used to determine the degree of matching between each target expert model and the instructions corresponding to the user demand features; The second determining subunit is used to determine the fusion weight of the target expert model by combining the scene matching degree and the instruction matching degree; wherein, the instruction matching degree is positively correlated with the fusion weight.

[0140] Furthermore, the second determining unit includes: The second acquisition subunit is used to acquire user photography style features; The third determining subunit is used to determine the style matching degree between each target expert model and the user's photography style features; The fourth determination subunit is used to determine the fusion weight of the target expert model by combining scene matching degree and style matching degree; among which, style matching degree is positively correlated with fusion weight.

[0141] Furthermore, the second determining unit includes: The fifth determination subunit is used to determine the scene credibility corresponding to the current photography scene, the command credibility corresponding to the user parameter command, and the style credibility corresponding to the user photography style characteristics. The sixth determining subunit is used to determine the scene weight, instruction weight, and style weight corresponding to scene credibility, instruction credibility, and style credibility; The seventh determination subunit is used to determine the scene matching degree, instruction matching degree, and style matching degree corresponding to each target expert model; The first computational subunit is used to perform a weighted summation of the scene matching degree, instruction matching degree, and style matching degree using scene weight, instruction weight, and style weight to obtain the fusion weight of the target expert model.

[0142] Reference Figure 3 In terms of hardware structure, the electronic device may include components such as a communication module 10, a memory 20, and a processor 30. In the electronic device, the processor 30 is connected to both the memory 20 and the communication module 10. The memory 20 stores a computer program, which is executed by the processor 30. When the computer program is executed, it implements the steps of the above-described method embodiments.

[0143] The communication module 10 can connect to external communication devices via a network. The communication module 10 can receive requests from external communication devices and can also send requests, instructions, and information to external communication devices. External communication devices can be other electronic devices, servers, or IoT devices, such as televisions, etc.

[0144] The memory 20 can be used to store software programs and various data. The memory 20 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as acquiring real-time images), etc.; the data storage area may include a database, and may store data or information created based on system usage. Furthermore, the memory 20 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0145] The processor 30 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 20, and by calling data stored in the memory 20, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. The processor 30 may include one or more processing units; optionally, the processor 30 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 30.

[0146] although Figure 3 Not shown, but the above electronic device may also include a circuit control module for connecting to a power supply to ensure the normal operation of other components. Those skilled in the art will understand that... Figure 3 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0147] The present invention also proposes a computer-readable storage medium having a computer program stored thereon. The computer-readable storage medium may be... Figure 3 The memory 20 in the electronic device may be at least one of ROM (Read-Only Memory) / RAM (Random Access Memory), magnetic disk, optical disk, etc. The computer-readable storage medium includes several instructions to cause a terminal device with a processor (which may be a television, automobile, mobile phone, computer, server, terminal, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0148] In this invention, the terms "first," "second," "third," "fourth," and "fifth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0149] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0150] Although embodiments of the present invention have been shown and described above, the scope of protection of the present invention is not limited thereto. It is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, and substitutions to the above embodiments within the scope of the present invention, and such changes, modifications, and substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A photographic method, characterized in that, The photographic method includes: Acquire real-time images; Scene recognition is performed on the real-time acquired images to determine the current photographic scene corresponding to the real-time acquired images; Match the target expert model corresponding to the current photography scene; The target photography parameters corresponding to the real-time acquired image are obtained through the target expert model. The photography operation is performed based on the target photography parameters.

2. The photographic method as described in claim 1, characterized in that, The step of performing scene recognition on the real-time acquired image to determine the current photography scene corresponding to the real-time acquired image includes: Semantic analysis is performed on the real-time acquired images to obtain the corresponding subject features and environmental features; The subject features and the environmental features are fused to obtain the scene semantic vector corresponding to the current photography scene.

3. The photographic method as described in claim 1, characterized in that, The target expert model that matches the current photography scene includes: Obtain the scene semantic vector corresponding to the current photography scene; Determine the scene matching degree between multiple preset expert models and the scene semantic vector; The preset number of expert models with the highest scene matching degree are selected as the target expert models.

4. The photographic method as described in claim 1, characterized in that, The step of obtaining the target photography parameters corresponding to the real-time acquired image through the target expert model includes: Obtain the model photography parameters output by multiple target expert models for the real-time acquired images; The fusion weights of each of the target expert models are determined based on the current photography scene, wherein the scene matching degree between the current photography scene and the target expert model is positively correlated with the fusion weights corresponding to the target expert models; The target photography parameters are obtained by weighted fusion of the model photography parameters according to the fusion weights.

5. The photographic method as described in claim 4, characterized in that, The step of determining the fusion weights of each of the target expert models based on the current photography scene includes: Obtain user parameter instructions and determine the user demand characteristics corresponding to the user parameter instructions; Determine the degree of matching between each target expert model and the instructions corresponding to the user demand features; The fusion weights of the target expert model are determined by combining the scene matching degree and the instruction matching degree; wherein the instruction matching degree is positively correlated with the fusion weights.

6. The photographic method as described in claim 4, characterized in that, The step of determining the fusion weights of each of the target expert models based on the current photography scene includes: Obtain user photography style characteristics; Determine the style matching degree between each of the target expert models and the user's photography style features; The fusion weights of the target expert model are determined by combining the scene matching degree and the style matching degree; wherein the style matching degree is positively correlated with the fusion weights.

7. The photographic method according to any one of claims 4 to 6, characterized in that, The step of determining the fusion weights of each of the target expert models based on the current photography scene includes: Determine the scene credibility corresponding to the current photography scene, the instruction credibility corresponding to the user parameter command, and the style credibility corresponding to the user's photography style features; Determine the scene weight, instruction weight, and style weight corresponding to the scene credibility, instruction credibility, and style credibility; Determine the scene matching degree, instruction matching degree, and style matching degree corresponding to each of the target expert models; The fusion weight of the target expert model is obtained by weighting and summing the scene matching degree, the instruction matching degree, and the style matching degree using the scene weight, instruction weight, and style weight.

8. A photographic apparatus, characterized in that, The photographic device includes: The image acquisition module is used to acquire real-time images; The scene recognition module is used to perform scene recognition on the real-time acquired image to determine the current photography scene corresponding to the real-time acquired image; The model matching module is used to match the target expert model corresponding to the current photography scene; The parameter determination module is used to obtain the target photography parameters corresponding to the real-time acquired image through the target expert model. The operation execution module is used to perform photographic operations based on the target photographic parameters.

9. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the photographic method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the photographic method as described in any one of claims 1 to 7.

Images