Shooting scene determination method and device, model training method and device, and electronic device

Abstract

This application discloses a method for determining shooting scenes, a model training method, a device, and an electronic device, relating to the field of artificial intelligence technology. The specific technical solution is as follows: upon acquiring a captured image, a first decision model is used to determine the confidence score of the captured image for each second shooting scene; a second decision model is used to determine whether the shooting scene corresponding to the captured image is a first shooting scene based on the confidence score of the captured image for each second shooting scene; wherein, the first shooting scene is a newly added shooting scene; the second decision model is constructed based on a first model configuration file corresponding to the first shooting scene issued by a server.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to a method for determining shooting scenes, a model training method, a device, and electronic equipment. Background Technology

[0002] With the development of photography technology, AI scene recognition has become a key technology for improving the shooting experience.

[0003] In related technologies, AI scene recognition solutions typically employ deep learning models. This involves pre-defining dozens of common scenes before shipment, training them with massive amounts of labeled data, and then embedding them in the device's firmware. During shooting, the preview image is processed in real-time based on the on-device deep learning model. However, when expanding to new recognition categories, it's necessary to re-collect massive amounts of samples, perform full fine-tuning of the deep learning model, and push the updated full model to the electronic device. This results in long model deployment cycles and high data costs. Summary of the Invention

[0004] The purpose of this application is to provide a method for determining shooting scenes, a model training method, an apparatus, and an electronic device that can shorten the model deployment cycle and reduce the data cost of model deployment.

[0005] In a first aspect, embodiments of this application provide a method for determining a shooting scene. The method includes: when an image is acquired, using a first decision model to determine the confidence score of the acquired image for each second shooting scene; using a second decision model, determining whether the shooting scene corresponding to the acquired image is a first shooting scene based on the confidence score of the acquired image for each second shooting scene; wherein the first shooting scene is a newly added shooting scene; and the second decision model is constructed based on a first model configuration file corresponding to the first shooting scene issued by a server.

[0006] Secondly, embodiments of this application provide a model training method, which includes: inputting training sample images corresponding to a first shooting scene into a first decision model for scene recognition processing to obtain a confidence score for each training sample image for each second shooting scene; determining a target shooting scene related to the first shooting scene from each second shooting scene; for each training sample image, inputting the confidence score of the training sample image for each target shooting scene and the shooting scene label of the first shooting scene into the second decision model, and training the second decision model based on the shooting scene recognition result of the second decision model and the shooting scene label of the first shooting scene.

[0007] Thirdly, embodiments of this application provide a shooting scene determination device, which includes: a processing module, configured to: upon acquiring a captured image, determine the confidence score of the captured image for each second shooting scene using a first decision model; and determine, using a second decision model, whether the shooting scene corresponding to the captured image is a first shooting scene based on the confidence score of the captured image for each second shooting scene; wherein the first shooting scene is a newly added shooting scene; and the second decision model is constructed based on a first model configuration file corresponding to the first shooting scene issued by a server.

[0008] Fourthly, embodiments of this application provide a model training apparatus, which includes: a processing module, configured to: input training sample images corresponding to a first shooting scene into a first decision model for scene recognition processing to obtain a confidence score for each training sample image for each second shooting scene; determine a target shooting scene related to the first shooting scene from each second shooting scene; for each training sample image, input the confidence score of the training sample image for each target shooting scene and the shooting scene label of the first shooting scene into the second decision model, and train the second decision model based on the shooting scene recognition result of the second decision model and the shooting scene label of the first shooting scene.

[0009] Fifthly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions, when executed by the processor, implementing the steps of the method described in the first aspect.

[0010] In a sixth aspect, embodiments of this application provide a server including a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions, when executed by the processor, implementing the steps of the method described in the second aspect.

[0011] In a seventh aspect, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect or the steps of the method described in the second aspect.

[0012] Eighthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect or the method as described in the second aspect.

[0013] Ninthly, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method as described in the first aspect or the method as described in the second aspect.

[0014] In this embodiment, upon acquiring a captured image, a first decision model is used to determine the confidence score of the captured image for each second shooting scene; a second decision model is used to determine whether the shooting scene corresponding to the captured image is the first shooting scene based on the confidence score of the captured image for each second shooting scene; wherein, the first shooting scene is a newly added shooting scene; the second decision model is constructed based on the first model configuration file corresponding to the first shooting scene issued by the server. Through this solution, the electronic device can construct a second decision model based on the first model configuration file for a specific shooting scene, and use this second decision model to determine whether the shooting scene corresponding to the captured image is the first shooting scene based on the confidence score of the captured image for each second shooting scene. This enables the electronic device to recognize newly added shooting scene categories, thereby improving its ability to recognize newly added shooting scenes. Furthermore, while expanding the electronic device's ability to recognize newly added shooting scenes, it shortens the model deployment cycle and reduces the data cost of model deployment. Attached Figure Description

[0015] Figure 1 One of the flowcharts illustrating the shooting scene determination method provided in the embodiments of this application;

[0016] Figure 2 A second schematic flowchart illustrating the shooting scene determination method provided in this application embodiment;

[0017] Figure 3 One of the flowcharts of the model training method provided in the embodiments of this application;

[0018] Figure 4 A second schematic flowchart illustrating the model training method provided in this application embodiment;

[0019] Figure 5 A flowchart illustrating the interaction method provided in an embodiment of this application;

[0020] Figure 6 This is a schematic diagram of the shooting scene determination device provided in the embodiments of this application;

[0021] Figure 7 This is a schematic diagram of the structure of the model training device provided in the embodiments of this application;

[0022] Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0023] Figure 9 A schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application;

[0024] Figure 10 This is a schematic diagram of the hardware structure of the server provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0026] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0027] The terms "at least one," "at least one of," etc., used in the specification and claims of this application refer to any one, any two, or a combination of two or more of the included items. For example, at least one of a, b, and c can mean: "a," "b," "c," "a and b," "a and c," "b and c," and "a, b, and c," where a, b, and c can be single or multiple. Similarly, "at least two" refers to two or more items, and its meaning is similar to that of "at least one."

[0028] With the rapid development of mobile internet and computational photography technology, the camera function of smartphones and other mobile devices has become a key indicator of device performance. To lower the barrier to photography for ordinary users, AI scene recognition technology has emerged. This technology analyzes the real-time preview stream captured by the camera, automatically identifies the current subject or scene (such as portraits, night scenes, food, blue skies, etc.), and intelligently adjusts ISP (Image Signal Processing) parameters such as exposure, white balance, and color saturation accordingly, thereby achieving the effect of "taking great photos with ease."

[0029] Current AI scene recognition technologies primarily rely on deep learning models such as deep convolutional neural networks (CNNs) or visual transformers (ViTs). The typical development process involves manufacturers pre-defining dozens of common shooting scenes, collecting massive amounts of labeled data, training the model on a cloud server, and finally deploying the quantized and compressed model to the mobile device's NPU or DSP. When the user frames the shot, the on-device model outputs the scene classification probability in real time, triggering corresponding image enhancement strategies.

[0030] Although existing AI scene recognition technology is relatively mature, it is gradually revealing a series of insurmountable pain points when faced with increasingly diverse and personalized user shooting needs:

[0031] (1) High cost and long cycle for scene expansion: In the traditional fully supervised learning model, every time a new sub-scene is added (such as the recently popular "tea brewing around a fire" or a specific "autumn" scene), tens of thousands of sample images need to be collected again and manually labeled. Subsequently, the huge edge model also needs to be retrained or fine-tuned. This not only consumes huge computing power and human resources, but also makes the development cycle of new functions last for several months, making it difficult to keep up with the hot topics on the Internet.

[0032] (2) Delayed firmware upgrades and lack of flexibility: Existing solutions often "hard-write" the recognition model into the system firmware or large app installation packages. Once the model is updated, users must upgrade the system via OTA or download an application update package of hundreds of megabytes to obtain the new features. This "one-size-fits-all" update method is extremely cumbersome and cannot achieve rapid distribution and hot updates of recognition capabilities for specific scenarios.

[0033] (3) The contradiction between model generalization and edge power consumption: In order to identify more long-tail scenarios, the model is often getting bigger and bigger, which leads to increased edge inference latency and serious overheating of the mobile phone. However, in order to reduce power consumption and shrink the model, the recognition accuracy of complex and similar scenarios (such as distinguishing between "cat" and "tiger") will drop significantly.

[0034] To address the aforementioned issues, this application provides a method for determining shooting scenes, breaking away from the traditional mindset of "end-to-end full training." It innovatively proposes a decoupled architecture of "general semantic feature extraction + lightweight dynamic decision-making," achieving low-cost, second-level adaptation and deployment of new scenes. In this application embodiment, upon acquiring a captured image, a first decision model is used to determine the confidence score of the captured image for each second shooting scene; a second decision model is used to determine whether the shooting scene corresponding to the captured image is the first shooting scene based on the confidence score of the captured image for each second shooting scene; wherein, the first shooting scene is a newly added shooting scene; the second decision model is constructed based on the first model configuration file corresponding to the first shooting scene issued by the server. This solution enables electronic devices to construct a second decision model based on a first model configuration file for a specific shooting scenario. The second decision model then uses the confidence scores of the acquired images for each second shooting scenario to determine whether the shooting scenario corresponding to the acquired image is the first shooting scenario. This allows the electronic device to recognize new shooting scenario categories, thereby improving its ability to recognize new shooting scenarios. Furthermore, while expanding the electronic device's ability to recognize new shooting scenarios, it also shortens the model deployment cycle and reduces the data cost of model deployment.

[0035] The method for determining the shooting scene provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0036] Figure 1 This is a flowchart illustrating the shooting scene determination method provided in the embodiments of this application, as shown below. Figure 1 As shown, the method for determining the shooting scene may include the following steps 201 and 202:

[0037] Step 201: After acquiring the captured image, the electronic device uses the first decision model to determine the confidence score of the captured image for each second shooting scene.

[0038] In some embodiments of this application, the acquired images may be images captured by a camera, such as images captured in real time by a camera or image frames from a video.

[0039] In some embodiments of this application, the acquired images may be images captured by a camera but not actually taken. For example, images captured by a camera and displayed in a shooting preview interface.

[0040] In some embodiments of this application, the electronic device inputs the acquired image into a first decision model for scene recognition processing. After calculation, the first decision model outputs a confidence score for each second-shot scene corresponding to the acquired image.

[0041] In some embodiments of this application, the second shooting scene described above may include at least one of the following: landscape, night scene, portrait, text, animal, plant, building, sky, etc.

[0042] It should be noted that the second shooting scene can also be other scenes, and this application embodiment does not limit this.

[0043] For example, when the electronic device displays a shooting preview interface, the captured image in real time by the camera is input into the first decision model for scene recognition processing. After calculation, the first decision model outputs scene information (i.e., scene labels, such as "portrait", "fireworks", "night scene") of multiple second shooting scenes corresponding to the captured image, as well as the confidence score of each shooting scene (for example, the score of the portrait scene is 0.8, and the score of the fireworks scene is 0.7).

[0044] Step 202: The electronic device uses the second decision model to determine whether the shooting scene corresponding to the acquired image is the first shooting scene based on the confidence score of the acquired image for each second shooting scene.

[0045] The first shooting scene mentioned above is a newly added shooting scene; the second decision model mentioned above is constructed based on the first model configuration file corresponding to the first shooting scene issued by the server.

[0046] In some embodiments of this application, the first shooting scene described above can be a specific shooting scene that the electronic device needs to recognize.

[0047] In some embodiments of this application, the first shooting scene can be a fireworks scene (such as a fireworks portrait scene), a sunrise scene, a sunset scene, an aurora scene, a pet scene, a scene of brewing tea around a fire, etc.

[0048] It should be noted that the first shooting scene can also be other scenes involved in the shooting, and this application embodiment does not limit this.

[0049] In some embodiments of this application, the electronic device can input all confidence scores of each second shooting scene into a second decision model for processing to determine whether the shooting scene corresponding to the acquired image is the first shooting scene.

[0050] In some embodiments of this application, the electronic device can input a portion of the confidence scores of each second shooting scene into a second decision model for processing to determine whether the shooting scene corresponding to the acquired image is the first shooting scene.

[0051] In some embodiments of this application, the electronic device determines at least one first confidence score from the confidence scores of each second shooting scene, and uses a second decision model to determine whether the shooting scene corresponding to the captured image is the first shooting scene based on the at least one confidence score.

[0052] In some embodiments of this application, the aforementioned first confidence score is a confidence score of at least one shooting scene associated with the first shooting scene (e.g., a "fireworks portrait scene"), such as the score of a "portrait" scene, the score of a "fireworks" scene, and the score of a "night scene".

[0053] In some embodiments of this application, the electronic device inputs the aforementioned first confidence score into a second decision model for classification decision processing. Based on its internal decision logic (e.g., a binary classifier), the second decision model outputs identification information, which is used to explicitly indicate whether the captured image belongs to the first shooting scene. For example, outputting the identification information "yes" indicates that the captured image is an image of a fireworks portrait scene.

[0054] The following specific examples illustrate the shooting scene determination method provided in the embodiments of this application.

[0055] For example, when a user opens the camera app and uses the camera to photograph a person holding a sparkler, the preview interface displays the image in real time. The electronic device inputs each frame of the preview image into a first decision model, which outputs a series of confidence scores such as "Portrait: 0.9", "Fireworks: 0.85", "Night Scene: 0.6", and "Beach: 0.01". Subsequently, the electronic device extracts three confidence scores—"Portrait", "Fireworks", and "Night Scene"—that are strongly correlated with "Fireworks Portrait" and inputs them into a second decision model specifically trained for the "Fireworks Portrait Scene". After calculation, the second decision model outputs the recognition information "It is a Fireworks Portrait Scene".

[0056] The shooting scene determination method provided in this application, upon acquiring a captured image, utilizes a first decision model to determine the confidence score of the captured image for each second shooting scene; and utilizes a second decision model to determine whether the shooting scene corresponding to the captured image is a first shooting scene based on the confidence score of the captured image for each second shooting scene. The first shooting scene is a newly added shooting scene. The second decision model is constructed based on a first model configuration file corresponding to the first shooting scene issued by a server. Through this solution, an electronic device can construct a second decision model based on a first model configuration file for a specific shooting scene, and use this second decision model to determine whether the shooting scene corresponding to the captured image is a first shooting scene based on the confidence score of the captured image for each second shooting scene. This enables the electronic device to recognize newly added shooting scene categories, thereby improving its ability to recognize newly added shooting scenes. Furthermore, while expanding the electronic device's ability to recognize newly added shooting scenes, it shortens the model deployment cycle and reduces the data cost of model deployment.

[0057] In some embodiments of this application, the above-described method for determining the shooting scene may further include steps A1 and A2:

[0058] Step A1: The electronic device acquires and loads the first model configuration file corresponding to the first shooting scene.

[0059] Step A2: The electronic device constructs a second decision model based on the first model configuration file.

[0060] The aforementioned first model configuration file is used to configure the electronic device's ability to recognize the first shooting scene. The electronic device is equipped with a first decision model, which is used to output the confidence scores of the acquired images in multiple shooting scenes.

[0061] In some embodiments of this application, the aforementioned first model configuration file is a lightweight, dynamically loadable data file used to configure the electronic device's ability to recognize a specific first shooting scene.

[0062] In some embodiments of this application, the electronic device receives a first model configuration file sent by the server and loads the first model configuration file into the memory of the electronic device.

[0063] In some embodiments of this application, the electronic device can receive a first model configuration file sent by a server (cloud server) via a network (such as Wi-Fi, mobile data network).

[0064] In some embodiments of this application, the first model configuration file described above can be transmitted in a compressed format to ensure transmission efficiency.

[0065] In some embodiments of this application, after successfully receiving and verifying the integrity of the first model configuration file, the electronic device loads the first model configuration file into its memory. The loading process may include steps such as decompression, parsing file content, and constructing corresponding data structures in memory according to file description, so as to facilitate the subsequent construction of the second decision model.

[0066] For example, when a user clicks the "Participate in experiencing the new fireworks portrait scene" control on their electronic device, the device sends a request to the server. Upon receiving the request, the server pushes the model configuration file of the second decision model built for the fireworks portrait scene to the user's electronic device. After receiving the model configuration file, the electronic device loads it into the memory corresponding to the camera application in the background.

[0067] In this embodiment, a collaborative mode of server-side distribution and terminal loading enables remote and dynamic deployment of scene recognition capabilities. This avoids the cumbersome operation of users manually downloading and installing packages, greatly improving the convenience of feature updates. Furthermore, since only lightweight configuration files for new scenes are transmitted and loaded, rather than the entire decision model file, the update process is efficient and resource-efficient.

[0068] In some embodiments of this application, a basic first decision model is deployed within the electronic device. This model is a deep learning model with a large number of parameters and a complex structure. Its function is to perform calculations on the input image and output the confidence score of the image in multiple preset shooting scenarios (e.g., covering general scenarios such as landscape, night scene, portrait, and text).

[0069] In some embodiments of this application, the first decision model described above is a parameter-frozen deep learning model.

[0070] In some embodiments of this application, "parameter freezing" means that during the training and deployment of the model, the parameters such as the neuron connection weights and biases inside the model no longer change, thereby providing stable and high-quality outputs for task processing (such as confidence scores for each scenario).

[0071] In this embodiment, the first decision model is designed as a parameter-frozen model, achieving decoupling between basic feature extraction capabilities and specific scene recognition capabilities. The frozen parameters ensure the stability and consistency of the basic model's output, enabling the second decision model trained based on its output to have better generalization capabilities. Simultaneously, it simplifies the complexity of model updates, thereby significantly improving R&D efficiency and deployment flexibility.

[0072] In some embodiments of this application, the first decision model may include a visual encoder.

[0073] For ease of understanding, this first decision-making model can be called the first-stage model.

[0074] In some embodiments of this application, a first decision model with frozen parameters is fixed and deployed at the time the electronic device leaves the factory or through firmware.

[0075] In some embodiments of this application, the first decision model is pre-trained on a large-scale dataset and has strong generalization ability, enabling it to map input images to confidence scores of thousands of basic semantic labels (such as "sky", "face", "firelight").

[0076] In some embodiments of this application, the model parameters of the first decision model described above can remain unchanged throughout the entire lifecycle of the electronic device, that is, the model parameters are frozen and do not participate in subsequent incremental updates.

[0077] In some embodiments of this application, the second decision model described above is used to determine whether the acquired image belongs to the first shooting scene based on the confidence score output by the first decision model.

[0078] In some embodiments of this application, after successfully loading the first model configuration file, the electronic device can construct a second decision model based on the first model configuration file.

[0079] In some embodiments of this application, the second decision model is a lightweight, specialized model whose input is not the original image, but rather a partial confidence score output by the first decision model that is related to the first shooting scene. This second decision model is used to perform secondary classification decisions on the input confidence score, thereby accurately determining whether the currently acquired image belongs to the first shooting scene that the user may be interested in.

[0080] In some embodiments of this application, when a new scene (such as fireworks and human figures) needs to be identified, the server (such as a cloud server) receives an instruction to acquire a small number of sample images (e.g., 100-500 images) of the new scene. These images are then input into a generic visual encoder with frozen parameters, identical to the one used on the device side—the first decision model—to obtain the confidence score distribution of each image across thousands of basic labels. Next, using this confidence score distribution as input features and whether the scene is new as the label, a very lightweight second-order decision model is trained—the second decision model. This second decision model learns how to infer high-level scene concepts from combinations of basic semantic labels (score distributions). Finally, the server packages the trained second decision model into a miniature configuration file and sends it to the electronic device.

[0081] In some embodiments of this application, the second decision model described above can be a decision tree model, a random forest model, or a linear classifier model.

[0082] For example, in a specific application scenario, suppose the first shooting scene is a "fireworks portrait scene". An electronic device pre-deploys a first decision model capable of recognizing 100 general scenes. After the server trains a lightweight second decision model for the "fireworks portrait scene", it encapsulates it into a specific model configuration file (such as an XML or JSON file) and sends it to the electronic device. The electronic device obtains this configuration file, loads it into memory, and dynamically constructs a second decision model specifically for recognizing the "fireworks portrait scene" in memory based on the parameters and structure information in the configuration file. This second decision model can use multiple confidence scores (such as 15) highly correlated with fireworks, portraits, and night scenes from the large number of shooting scene confidence scores (such as 100) output by the first decision model to determine whether the current image belongs to the fireworks portrait scene. In this embodiment, by obtaining and loading the model configuration file to construct the second decision model, the ability to flexibly add or enhance the electronic device's recognition of specific shooting scenes is achieved without updating the large first decision model. This avoids the excessive bandwidth consumption and storage space requirements caused by frequent updates to the basic large model, making the expansion of scene recognition functionality more efficient and reducing development and maintenance costs. Furthermore, since the second decision model makes decisions based on the output of the first decision model, it can inherit the feature extraction capabilities of the first decision model, thus ensuring the accuracy of newly added scene recognition.

[0083] In this embodiment, the electronic device acquires and loads a first model configuration file corresponding to a first shooting scene. This first model configuration file configures the electronic device's ability to recognize the first shooting scene. The electronic device deploys a first decision model, which outputs confidence scores for the acquired image across multiple shooting scenes. Based on the first model configuration file, a second decision model is constructed. This second decision model determines whether the acquired image belongs to the first shooting scene based on the confidence scores output by the first decision model. Through this solution, the electronic device can acquire and load a first model configuration file for a specific shooting scene and construct a corresponding decision model. Thus, by constructing a lightweight second decision model, the electronic device achieves recognition of newly added shooting scene categories. This expands the electronic device's ability to recognize new shooting scenes while shortening the model deployment cycle and reducing the data cost of model deployment.

[0084] In some embodiments of this application, the process of determining the confidence score of the acquired image for each second shooting scene using the first decision model in step 201 above may include the following step B1:

[0085] Step B1: The electronic device inputs the acquired image into the first decision model for scene recognition processing to obtain the confidence score of the acquired image for each second shooting scene.

[0086] For example, in conjunction with step B1 above, the process in step 202 where the electronic device uses the second decision model to determine whether the shooting scene corresponding to the acquired image is the first shooting scene based on the confidence score of the acquired image for each second shooting scene may include the following step B2:

[0087] Step B2: The electronic device inputs the confidence scores of the acquired images for each second shooting scene into the second decision model for classification decision processing to obtain recognition information.

[0088] The aforementioned identification information is used to indicate whether the shooting scene corresponding to the acquired image is the first shooting scene.

[0089] In some embodiments of this application, when the electronic device displays a shooting preview interface, the image captured in real time by the camera is input into a first decision model for scene recognition processing. After calculation, the first decision model outputs scene information (i.e., scene labels, such as "portrait", "fireworks", "night scene") of multiple shooting scenes corresponding to the image, as well as a confidence score for each shooting scene (e.g., the score for the portrait scene is 0.8, and the score for the fireworks scene is 0.7).

[0090] In some embodiments of this application, when the electronic device displays a shooting preview interface, the image captured by the camera is input into a first decision model for scene recognition processing. After calculation, the first decision model outputs scene information of multiple shooting scenes corresponding to the image and a confidence score for each shooting scene.

[0091] In some embodiments of this application, the aforementioned first confidence score is a confidence score of at least one shooting scene associated with the first shooting scene (e.g., a "fireworks portrait scene"), such as the score of a "portrait" scene, the score of a "fireworks" scene, and the score of a "night scene". Subsequently, the electronic device inputs the aforementioned first confidence score into a second decision model for classification decision processing. Based on its internal decision logic (e.g., a binary classifier), the second decision model outputs identification information that explicitly indicates whether the captured image belongs to the first shooting scene. For example, outputting the identification information "yes" indicates that the captured image is an image from the fireworks portrait scene.

[0092] In some embodiments of this application, the method for determining the shooting scene provided in this application may further include the following step D1:

[0093] Step D1: When the shooting scene corresponding to the captured image is the first shooting scene, the electronic device executes the shooting control operation associated with the first shooting scene;

[0094] The shooting control operation includes at least one of the following:

[0095] Adjust image signal processing parameters, apply the filter corresponding to the first shooting scene, trigger the augmented reality effect corresponding to the first shooting scene, and start the shooting function mode corresponding to the first shooting scene.

[0096] In some embodiments of this application, when the recognition result indicates that the captured image belongs to a first shooting scene, the electronic device will automatically execute shooting control operations corresponding to or associated with the first shooting scene to achieve intelligent shooting assistance. For example, when the user is framing the shot in real time, the electronic device's first decision model operates normally, outputting the basic semantic label confidence score for the current frame. Then, the above score is fed into a newly loaded second decision model for inference, and the second decision model outputs the final new scene recognition result, such as the confidence score for "fireworks portrait". Finally, based on the recognition result, the electronic device automatically invokes a preset shooting control strategy (such as adjusting exposure, white balance, or triggering a specific filter).

[0097] In some embodiments of this application, the above-described shooting control operation may include at least one of the following:

[0098] Adjust image signal processing parameters, apply specific filters, trigger augmented reality effects, and activate specific shooting modes.

[0099] It should be noted that the shooting control operation includes not only adjusting ISP parameters such as exposure and white balance, but also recommending relevant filters (LUT), automatically triggering AR effects, intelligent album categorization, and even activating specific system modes (such as automatically switching to scanning mode when a document is recognized, or automatically jumping to a link when a QR code is recognized).

[0100] For example, adjusting the parameters of the image signal processor (ISP) can include: dynamically adjusting contrast, saturation, noise reduction intensity, etc.; applying a specific filter can be applying a specific filter to the preview or final image, such as a retro color or autumn color filter to enhance the atmosphere; triggering augmented reality effects can be triggering the overlay of specific effects in an augmented reality (AR) scene, such as overlaying dynamic starlight or text blessing effects on the screen when a fireworks scene is detected; activating a specific shooting function mode can be automatically switching to night mode, portrait mode, or high-pixel mode, etc.

[0101] For example, when the electronic device recognizes that the current scene is a "fireworks portrait scene," the shooting control operation it performs can be a combination of operations. First, adjust the ISP parameters, such as increasing the brightness of the shadow areas to make the face clearer, while appropriately reducing the gain of the highlight areas to prevent the fireworks from being overexposed. Second, automatically activate specific shooting function modes, such as enabling multi-frame noise reduction and high dynamic range functions to cope with high contrast scenes.

[0102] In this embodiment, by associating rich and diverse shooting control operations with specific shooting scenes, the electronic device can intelligently call the most suitable hardware parameters and software algorithms based on the semantic information of the scene. Thus, through refined scene adaptation capabilities, the shooting quality is greatly improved.

[0103] For example, combining the above example, when a user opens the camera app and uses the camera to photograph a person holding a sparkler, the preview interface displays the image in real time. The electronic device inputs each frame of the preview image into a first decision model, which outputs a series of confidence scores such as "Portrait: 0.9", "Fireworks: 0.85", "Night Scene: 0.6", and "Beach: 0.01". Subsequently, the electronic device extracts three confidence scores—"Portrait", "Fireworks", and "Night Scene"—that are strongly correlated with "Fireworks Portrait" and inputs them into a second decision model specifically trained for the "Fireworks Portrait Scene". The second decision model calculates and outputs the recognition information "It is a Fireworks Portrait Scene". Based on this recognition information, the electronic device triggers shooting control operations associated with the Fireworks Portrait Scene, such as automatically adjusting the ISO and shutter speed to balance the exposure of the highlights of the fireworks and the shadows of the portrait, and enabling a specific multi-frame fusion algorithm to ensure that the person's face is clear, the fireworks details are rich, and the image is not overexposed in the final photo.

[0104] In this embodiment, by using a second decision model to perform a secondary decision on the output of the first decision model, a large amount of noise interference from irrelevant scenes can be filtered out, greatly improving the accuracy and robustness of specific scene recognition. Furthermore, after accurately identifying the scene, associated shooting control operations are automatically executed, achieving an intelligent shooting experience and reducing the difficulty for users to shoot in complex scenes.

[0105] In some embodiments of this application, step B2 may include the following steps C1 and C2:

[0106] Step 204a: Perform classification decision calculation on the confidence scores of the acquired images for each second shooting scene using the second decision model to obtain the second confidence score.

[0107] The aforementioned second confidence score represents the confidence level that the acquired image belongs to the first shooting scene;

[0108] Step 204b: If the electronic device obtains the first identification information when the second confidence score is greater than or equal to the confidence threshold.

[0109] The first identification information indicates that the shooting scene corresponding to the acquired image is the first shooting scene.

[0110] In some embodiments of this application, the aforementioned second confidence score is a normalized numerical value used to quantitatively characterize the degree of confidence that the currently acquired image belongs to the first shooting scene. The higher the value, the greater the probability that it belongs to the first shooting scene.

[0111] In some embodiments of this application, after obtaining a second confidence score, the electronic device compares it with a preset confidence threshold. It should be noted that this confidence threshold is determined during the model training or debugging phase and is used to balance the accuracy and recall of scene recognition. If the second confidence score is greater than or equal to the confidence threshold, the second decision model outputs first recognition information, which indicates that the acquired image belongs to the first shooting scene. Conversely, if it is below the threshold, information indicating that the image does not belong to the scene is output.

[0112] For example, following the above example, suppose the second decision model is a logistic regression model trained for a "fireworks portrait scene". When the confidence score of the second decision model is [portrait score 0.9, fireworks score 0.85, night scene score 0.6], the second decision model calculates a second confidence score of 0.95 through a logistic function. Assuming the preset threshold is 0.7, since 0.95 ≥ 0.7, the second decision model outputs "True" or "1" recognition information, indicating that the currently acquired image is a "fireworks portrait scene". Alternatively, if the input is a confidence score of [portrait 0.9, fireworks 0.1, night scene 0.2], and the second decision model calculates a second confidence score of 0.1, which is lower than the threshold of 0.7, then it outputs "False" or "0" recognition information, indicating that the currently acquired image does not belong to the "fireworks portrait scene".

[0113] In this embodiment of the application, a second decision model performs classification decision operation based on at least one first confidence score to obtain a second confidence score, and compares the second confidence score with a confidence threshold. In this way, by setting clear, objective and configurable judgment criteria for scene recognition, scene recognition can be performed accurately and flexibly.

[0114] In some embodiments of this application, the shooting scene determination method provided in this application may further include the following steps E1 and E2:

[0115] Step E1: The electronic device acquires a second model profile for the first shooting scene.

[0116] The aforementioned second model configuration file is used to adjust the recognition accuracy of the first shooting scene, or to configure the electronic device's ability to recognize sub-scenes of the first shooting scene.

[0117] Step E2: The electronic device updates the second decision model based on the second model configuration file described above.

[0118] In some embodiments of this application, the aforementioned second model configuration file corresponds to a new third shooting scene (e.g., a "Mid-Autumn Festival moon-viewing scene"); or the second model configuration file is a further refinement of the original first shooting scene, for example, the first shooting scene is a fireworks scene, and the third shooting scene is a fireworks portrait scene or a large-scale fireworks scene; or the second model configuration file is a further enhancement of the recognition capability (e.g., recognition accuracy) of the original first shooting scene, for example, improving the recognition accuracy of the first shooting scene.

[0119] In some embodiments of this application, after obtaining the second model configuration file, the electronic device dynamically updates the second decision model based on the second model configuration file to obtain an updated second decision model. Specifically, the updated second decision model can handle new recognition tasks, perform more refined sub-scene type recognition, or perform recognition of the first shooting scene with higher accuracy.

[0120] For example, suppose an electronic device, after obtaining a model configuration file for a "fireworks portrait scene," loads and constructs a second decision model to recognize the fireworks portrait scene. After using the second decision model for a period of time (e.g., one month), the server-side second decision model is retrained based on user feedback data, training a second decision model V2 with higher recognition accuracy and coverage of more subdivided scenes (e.g., "large fireworks scene" and "handheld small fireworks scene"). Subsequently, the server encapsulates the updated second decision model V2 into a model configuration file (i.e., the second model configuration file) and sends it to the electronic device. After receiving the model configuration file, the electronic device uses it to update the second decision model, obtaining the updated second decision model, enabling the electronic device to recognize "large fireworks scene" and "small fireworks scene" with greater accuracy.

[0121] It should be noted that steps E1 and E2 can be performed after or before step 202. This application does not limit the timing of the execution of steps E1 and E2.

[0122] In this embodiment, by sending a new model configuration file for the first shooting scene to the electronic device, the electronic device can update the second decision model accordingly. In this way, by simply sending a lightweight new configuration file, the iterative upgrade of the device's AI recognition capabilities can be achieved without waiting for a major system version update, thereby improving the efficiency of model update.

[0123] In some embodiments of this application, the electronic device may receive a third model configuration file sent by a server, which is used to configure the electronic device's ability to recognize a fourth shooting scene.

[0124] In some embodiments of this application, the electronic device may construct a third decision model based on the third model configuration file. The third decision model is used to determine whether the acquired image belongs to the fourth shooting scene based on the confidence score output by the first decision model.

[0125] For example, suppose an electronic device, after obtaining a model configuration file for a "fireworks portrait scene," loads and constructs a second decision model to recognize the fireworks portrait scene. After using the second decision model for a period of time (e.g., one month), the server adds a new shooting scene (e.g., an aurora scene) and obtains training sample images to train a third decision model. Subsequently, the server encapsulates the third decision model into a model configuration file (i.e., the third model configuration file) and sends this model configuration file to the electronic device. After receiving the model configuration file, the electronic device constructs the third decision model locally, enabling the electronic device to have the new ability to recognize "autumn" scenes.

[0126] In this embodiment of the application, by sending a new model configuration file for other newly added shooting scenarios to the electronic device, the electronic device can build a corresponding decision model based on it. In this way, by simply sending a lightweight configuration file for other newly added scenarios, the AI ​​recognition capability of the device can be expanded without waiting for a major system version update, thereby improving the efficiency of model update.

[0127] In some embodiments of this application, the shooting scene determination method provided in this application may further include the following steps F1 to F3:

[0128] Step F1: The electronic device inputs the training sample images corresponding to the first shooting scene into the first decision model for scene recognition processing, and obtains the confidence score of each training sample image for each second shooting scene.

[0129] In some embodiments of this application, the electronic device inputs training sample images corresponding to a first shooting scene into a pre-trained, parameter-frozen first decision model for scene recognition processing. Through forward propagation, scene information and confidence scores of multiple shooting scenes corresponding to each training sample image are obtained. Thus, by utilizing the feature extraction capability of the first decision model, the original image data is transformed into high-dimensional, information-rich feature vectors (i.e., confidence scores of each scene), providing a data foundation for the subsequent scene recognition of the second decision model.

[0130] Step F2: The electronic device determines the target shooting scene related to the first shooting scene from each of the second shooting scenes.

[0131] In some embodiments of this application, the electronic device determines a target shooting scene that is semantically or visually related to a first shooting scene from multiple shooting scenes in each training sample image. For example, for a "fireworks portrait scene," the associated scene could be "portrait," "fireworks," "night scene," "party," etc.

[0132] Step F3: For each training sample image, the electronic device inputs the confidence score of the training sample image for each target shooting scene, as well as the shooting scene label of the first shooting scene, into the second decision model. Based on the shooting scene recognition result of the second decision model and the shooting scene label of the first shooting scene, the second decision model is trained.

[0133] In some embodiments of this application, the electronic device inputs the confidence score of the selected target shooting scene and the shooting scene label of the sample image into the second decision model for training. Through backpropagation and optimization algorithms, the parameters of the second decision model are continuously adjusted, ultimately resulting in a well-trained second decision model capable of accurately identifying the first shooting scene.

[0134] In this embodiment, by using the output of the first decision model as the input feature for training the new model, the amount of training data is greatly reduced, and the complexity of model training is simplified. Through scene association filtering, the discrimination accuracy of the final model is improved, thereby obtaining a second decision model capable of accurately identifying the first shooting scene.

[0135] The following specific embodiments illustrate the shooting scene determination method provided in this application.

[0136] For example, such as Figure 2 As shown, the method for determining the shooting scene may include the following steps:

[0137] Step 11: The electronic device performs the first stage of semantic perception and obtains the recognition result.

[0138] For example, the electronic device inputs the preview image captured by the camera into the first decision model and outputs the confidence score of each shooting scene, such as outdoor, night sky, moon, portrait, stage, fireworks and other scenes.

[0139] For example, the first decision model described above is embedded in the system firmware of an electronic device (such as a smartphone).

[0140] For example, when a user opens the camera to frame a shot, the electronic device first calls a first decision model to extract multiple scene labels for the current frame, such as night sky, portrait, and fireworks; then, it immediately inputs these labels into a second decision model. The second decision model calculates the confidence level that the current image belongs to a newly added scene based on preset logic.

[0141] Step 12: The electronic device makes a two-stage rapid decision based on the recognition results.

[0142] For example, the confidence scores of each shooting scene output by the first decision model are input into the second decision model. When the confidence score output by the second-stage model exceeds a preset threshold, the current scene is confirmed as the target new scene.

[0143] Step 13: The electronic device adaptively matches the shooting parameters based on the recognition results.

[0144] For example, when the current scene is determined to be a new target scene, the electronic device immediately invokes the ISP (Image Signal Processing) control strategy bound to it, such as adjusting the exposure time, white balance gain, or color saturation, to complete the shooting optimization for that scene.

[0145] In the embodiments of this application, this application enables existing terminal devices to recognize entirely new scenarios with only extremely low data and computing power costs, greatly improving user experience and system flexibility.

[0146] Figure 3 The model training method provided in the embodiments of this application, such as Figure 3 As shown, the model training method may include the following steps 301 to 303:

[0147] Step 301: The server inputs the training sample images corresponding to the first shooting scene into the first decision model for scene recognition processing, and obtains the confidence score of each training sample image for each second shooting scene.

[0148] In some embodiments of this application, the server inputs the training sample images corresponding to the first shooting scene into a pre-trained, parameter-frozen first decision model for scene recognition processing. Through forward propagation, scene information and confidence scores of multiple shooting scenes corresponding to each training sample image are obtained. Thus, by utilizing the feature extraction capability of the first decision model, the original image data is transformed into high-dimensional, information-rich feature vectors (i.e., confidence scores of each scene), providing a data foundation for the subsequent scene recognition of the second decision model.

[0149] Step 302: The server determines the target shooting scene related to the first shooting scene from each of the second shooting scenes.

[0150] In some embodiments of this application, the server determines at least one target shooting scene that is semantically or visually associated with the first shooting scene from multiple shooting scenes of each training sample image. For example, for a "fireworks portrait scene", the associated scene could be "portrait", "fireworks", "night scene", "party", etc.

[0151] Step 303: For each training sample image, the server inputs the confidence score of the training sample image for each target shooting scene and the shooting scene label of the first shooting scene into the second decision model, and trains the second decision model based on the shooting scene recognition result of the second decision model and the shooting scene label of the first shooting scene.

[0152] The aforementioned shooting scene label is used to indicate that the training sample image belongs to the first shooting scene.

[0153] In some embodiments of this application, the second decision model described above is used to determine whether the acquired image belongs to the first shooting scene based on the confidence score output by the first decision model.

[0154] In some embodiments of this application, the server determines at least one shooting scene semantically or visually associated with the first shooting scene from multiple shooting scenes in each training sample image. For example, for a "fireworks portrait scene," the associated scene could be "portrait," "fireworks," "night scene," "party," etc. Then, the confidence scores of the aforementioned selected associated scenes, along with the label information of the sample image, are input into a second decision model for training. Through backpropagation and optimization algorithms, the parameters of the second decision model are continuously adjusted, ultimately resulting in a well-trained second decision model capable of accurately identifying the first shooting scene.

[0155] In some embodiments of this application, the above-mentioned label information indicates whether the image belongs to a first shooting scene (such as a "fireworks portrait scene"), for example, a positive sample label is 1 and a negative sample label is 0.

[0156] In some embodiments of this application, the second decision model described above can be a logistic regression model to be trained or a support vector machine (SVM).

[0157] In this embodiment, by using the output of the first decision model as the input feature for training the new model, the amount of training data is greatly reduced, and the complexity of model training is simplified. Through scene association filtering, the discrimination accuracy of the final model is improved, thereby obtaining a second decision model capable of accurately identifying the first shooting scene.

[0158] In some embodiments of this application, the model training method provided in this application may further include steps 304 and 305:

[0159] Step 303: The server encapsulates the second decision model into a configuration file for the first model.

[0160] Step 304: The server sends the first model configuration file to the electronic device.

[0161] The aforementioned first model configuration file is used to configure the electronic device's ability to recognize the first shooting scene.

[0162] In some embodiments of this application, the server first encapsulates the trained second decision model (including information such as model structure, weight parameters, and decision thresholds) according to a predetermined protocol and format to generate a standardized model configuration file. Then, the model configuration file is sent to an electronic device via a cloud server or other distribution channels.

[0163] In some embodiments of this application, after receiving the first model file, the electronic device loads and builds the model to gain the ability to recognize new scenes.

[0164] For example, the cloud server completes the training of the second decision model for the "fireworks portrait scene". After training, the model is exported and packaged into a standard configuration file containing information such as the model version number, model type (e.g., "support vector machine"), coefficients of each support vector, and kernel function parameters. This configuration file is then pushed to an Over-The-Air (OTA) server. Electronic devices can automatically download this configuration file from the OTA server, thereby adding the ability to recognize the "fireworks portrait scene".

[0165] It should be noted that steps 304 and 305 can be performed after step 303 or before step 303. This application does not limit the timing of the execution of steps 304 and 305.

[0166] In this embodiment of the application, by packaging the trained model into a standardized configuration file and distributing it to the electronic device, the model configuration file for the new scenario can reach the electronic device at the lowest cost and the fastest speed through the collaboration between the electronic device and the server.

[0167] In some embodiments of this application, the server may send a second model configuration file for a first shooting scene to an electronic device, so that the electronic device updates the second decision model based on the second model configuration file.

[0168] The second model configuration file is used to adjust the recognition accuracy of the first shooting scene; or, the second configuration file is used to configure the electronic device's ability to recognize sub-scenes of the first shooting scene.

[0169] In some embodiments of this application, the server may send a third model configuration file to the electronic device, which is used to configure the electronic device's ability to recognize a fourth shooting scene.

[0170] In some embodiments of this application, the electronic device may construct a third decision model based on the third model configuration file. The third decision model is used to determine whether the acquired image belongs to the fourth shooting scene based on the confidence score output by the first decision model.

[0171] The following specific embodiments illustrate the model training method provided in this application.

[0172] For example, such as Figure 4 As shown, the model training method may include the following steps:

[0173] Step 21: The server builds the first-stage semantic awareness model for coarse scene recognition.

[0174] For example, a deep convolutional neural network model pre-trained on a large dataset is deployed on the server. This first-stage model serves as a feature extraction base, and its parameters remain frozen during subsequent expansion. The model output contains generalized semantic information ranging from "texture" and "color" to "basic objects (such as sky, grass, and faces)," with a coarse label capacity of at least several thousand units.

[0175] Step 22: The server obtains the extended definition request for the newly added scene (i.e., the first shooting scene) and a small sample dataset.

[0176] For example, when the system needs to expand to new recognition categories (such as adding the popular "fireworks portrait mode"), the server receives the expansion instruction and only needs to acquire a small number of corresponding sample images (on the order of 100 to 500), without having to re-collect the full amount of data covering all scenes.

[0177] Step 23: The server performs coarse label identification on the samples based on the first-stage model (i.e., the first decision model).

[0178] For example, the server uses a general base model structure, namely the first decision model, to perform recognition and prediction on the acquired small sample images, mapping the original image pixels to thousands of corresponding label confidence scores. This process transforms unstructured image data into structured label confidence score distribution data.

[0179] Step 24: The server builds a second-stage lightweight decision model (i.e., the second decision model) for the new scenario.

[0180] For example, the server selects tags related to the newly added scene as the category of interest. Taking the newly added recognition of "fireworks portrait" as an example, it selects the coarse recognition confidence scores of related tags such as night, sky, light, fireworks, and portrait, and uses them as input data for a two-stage decision model to train a lightweight decision model. The decision result is the final recognition target, such as "fireworks portrait".

[0181] For example, the decision model is preferably based on architectures such as decision trees or random forests, because these architectures can fit the input score distribution very well with extremely low computational overhead, resulting in excellent decision-making efficiency in this application mode. Furthermore, models based on this architecture are small in size and can be distributed or updated via the network.

[0182] Step 25: The server generates a scene extension configuration file and dynamically distributes it to electronic devices.

[0183] For example, the server encapsulates the trained second-stage model (i.e., the second decision model) into a miniature configuration file (typically less than 100KB in size) and pushes it to the terminal device via a cloud OTA interface. The terminal device hot-loads this configuration file without updating the app or firmware and builds a decision logic layer for the new scenario in memory.

[0184] Figure 5 A schematic diagram of the interaction method provided in the embodiments of this application, such as Figure 5 As shown, the interaction method may include the following steps:

[0185] Step 401: The server inputs the training sample images corresponding to the first shooting scene into the first decision model for scene recognition processing, and obtains the confidence score of each training sample image for each second shooting scene.

[0186] Step 402: The server determines the target shooting scene related to the first shooting scene from each of the second shooting scenes.

[0187] Step 403: For each training sample image, the server inputs the confidence score of the training sample image for each target shooting scene and the shooting scene label of the first shooting scene into the second decision model. The second decision model is trained based on the shooting scene recognition result of the second decision model and the shooting scene label of the first shooting scene.

[0188] Step 404: The server encapsulates the second decision model into a first model configuration file and sends it to the electronic device.

[0189] Step 405: The electronic device receives the first model configuration file sent by the server and loads it into memory.

[0190] Step 406: The electronic device constructs a second decision model based on the first model configuration file.

[0191] Step 407: After acquiring the captured image, the electronic device uses the first decision model to determine the confidence score of the captured image for each second shooting scene.

[0192] Step 408: The electronic device uses the second decision model to determine whether the shooting scene corresponding to the acquired image is the first shooting scene based on the confidence score of the acquired image for each second shooting scene.

[0193] Step 409: When the electronic device determines that the shooting scene for the acquired image is the first shooting scene, it executes the shooting control operation related to the first shooting scene.

[0194] In some embodiments of this application, the following steps 410 and 411 may also be included:

[0195] Step 410: The server sends the second model configuration file for the first shooting scene to the electronic device.

[0196] Step 411: The electronic device obtains and loads the second model configuration file, and updates the second decision model based on the second model configuration file.

[0197] It should be noted that the explanation of this embodiment can be found in the description of the above embodiments, and will not be repeated here to avoid repetition.

[0198] It should be noted that although the embodiments of this application take image recognition (vision) as an example, the architecture of "general feature extraction + lightweight logic decision" is also applicable to audio scene recognition (such as recognizing new environmental noise, instrument sounds) or video motion capture.

[0199] This application breaks away from the traditional mindset of "end-to-end full training" and innovatively proposes a decoupled architecture of "general semantic feature extraction + lightweight dynamic decision-making," achieving low-cost, second-level adaptation and deployment for new scenes. Addressing the issue of cumbersome AI model updates when identifying new shooting scenes, this application designs a "two-stage extremely low-parameter update" architecture: In the first stage, a parameter-frozen general visual encoder is deployed on the edge as a base, remaining unchanged during application, breaking the traditional paradigm of globally updating model weights. When a new scene is added, based on the confidence score distribution of the first-stage encoder across thousands of general categories, the category scores related to the new scene are selected as the input to the second-stage model, forming its unique distribution, and an extremely lightweight configuration file is then distributed accordingly.

[0200] On the one hand, based on a frozen base, this application utilizes a small-sample agile adaptation. Unlike existing technologies that require full-data retraining, this application deploys a general-purpose visual encoder with frozen parameters on the terminal, used only for coarse scene recognition. The recognition result is a confidence score on a massive number of coarse scene labels, but this score does not directly participate in the final decision. When a new recognition category needs to be added, only a very small number of samples (e.g., 200 images) need to be input, and a very lightweight logical decision layer (e.g., a decision tree or linear classification head) can be quickly built or updated through training. This design transforms the training of new scenes from "building a large model" to "computing geometric boundaries," achieving true second-level adaptation. On the other hand, a "feature hot update" mechanism in the form of a configuration file is used: this application encapsulates the recognition logic of new scenes into an independent, miniaturized configuration file (usually only a few KB). Terminal devices do not need to undergo OTA system upgrades or app updates; they only need to download the miniature configuration file online and load it into the decision layer to immediately gain the ability to recognize new scenes. This allows manufacturers to closely follow holidays and trending events (such as the "Spring Festival fireworks mode") and push customized shooting AI functions to users in real time. On the other hand, it achieves a balance between high precision and low power consumption: this solution fully utilizes the powerful feature extraction capabilities of a general-purpose large model, ensuring the accuracy of generalized understanding of unknown objects; simultaneously, it decomposes frequently changing classification logic into simple shallow calculations, greatly reducing the real-time inference computing power consumption of terminal devices. Through this "separation of static and dynamic elements" design, it solves the problem of difficult recognition of long-tail scenes and avoids device overheating and lag caused by model expansion.

[0201] It should be noted that each of the above method embodiments, or various possible implementations of each method embodiment, can be executed individually or in combination of any two or more. The specific implementation can be determined according to actual usage requirements, and this application embodiment does not impose any restrictions on this.

[0202] The shooting scene determination method provided in this application can be executed by a shooting scene determination device. This application uses the shooting scene determination device executing the shooting scene determination method as an example to illustrate the shooting scene determination device provided in this application.

[0203] Figure 6 This is a schematic diagram of a shooting scene determination device provided in an embodiment of this application. Figure 6 As shown, the shooting scene determination device 500 includes: a processing module 501, used to: determine the confidence score of the captured image for each second shooting scene using a first decision model when the captured image is acquired;

[0204] Using the second decision model, based on the confidence scores of the acquired images for each second shooting scene, it is determined whether the shooting scene corresponding to the acquired image is the first shooting scene; where the first shooting scene is a newly added shooting scene; the second decision model is constructed based on the first model configuration file corresponding to the first shooting scene issued by the server.

[0205] In some embodiments of this application, the processing module is further configured to: obtain and load a first model configuration file corresponding to a first shooting scene; and construct a second decision model based on the first model configuration file.

[0206] In some embodiments of this application, the processing module is specifically used to: input the acquired image into a first decision model for scene recognition processing, and obtain the confidence score of the acquired image for each second shooting scene;

[0207] The confidence scores of the acquired images for each second shooting scene are input into the second decision model for classification and decision processing to obtain recognition information; the recognition information is used to indicate whether the shooting scene corresponding to the acquired image is the first shooting scene.

[0208] In some embodiments of this application, the processing module is specifically used to: perform classification decision calculation on the confidence scores of the acquired image for each second shooting scene using a second decision model to obtain a second confidence score; and obtain first identification information if the second confidence score is greater than or equal to a confidence threshold; the first identification information indicates that the shooting scene corresponding to the acquired image is a first shooting scene.

[0209] In some embodiments of this application, the processing module is further configured to: when the shooting scene corresponding to the acquired image is a first shooting scene, perform a shooting control operation associated with the first shooting scene; wherein the shooting control operation includes at least one of the following: adjusting image signal processing parameters, applying a filter corresponding to the first shooting scene, triggering an augmented reality effect corresponding to the first shooting scene, and activating a shooting function mode corresponding to the first shooting scene.

[0210] In some embodiments of this application, the processing module is further configured to: obtain a second model configuration file for the first shooting scene; the second model configuration file is used to adjust the recognition accuracy of the first shooting scene, or to configure the electronic device to recognize sub-scenes of the first shooting scene; and update the second decision model based on the second model configuration file.

[0211] In some embodiments of this application, the first decision model is a parameter-frozen deep learning model.

[0212] The shooting scene determination device provided in this application embodiment, upon acquiring a captured image, utilizes a first decision model to determine the confidence score of the captured image for each second shooting scene; and utilizes a second decision model to determine whether the shooting scene corresponding to the captured image is a first shooting scene based on the confidence score of the captured image for each second shooting scene. The first shooting scene is a newly added shooting scene. The second decision model is constructed based on a first model configuration file corresponding to the first shooting scene issued by a server. Through this solution, an electronic device can construct a second decision model based on a first model configuration file for a specific shooting scene, and use this second decision model to determine whether the shooting scene corresponding to the captured image is a first shooting scene based on the confidence score of the captured image for each second shooting scene. This enables the electronic device to recognize newly added shooting scene categories, thereby improving its ability to recognize newly added shooting scenes. Furthermore, while expanding the electronic device's ability to recognize newly added shooting scenes, it shortens the model deployment cycle and reduces the data cost of model deployment.

[0213] The scene determination device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0214] The model training device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0215] The model training apparatus provided in this application embodiment can realize all the processes implemented in the model training method embodiment, and will not be described again here to avoid repetition.

[0216] The model training method provided in this application can be executed by a model training device. This application uses a model training device to perform model training as an example to illustrate the model training apparatus provided in this application.

[0217] Figure 7 This is a schematic diagram of a model training device provided in an embodiment of this application. Figure 7As shown, the model training device 600 includes a processing module 601, configured to: input training sample images corresponding to a first shooting scene into a first decision model for scene recognition processing, and obtain a confidence score for each training sample image for each second shooting scene; determine a target shooting scene related to the first shooting scene from each second shooting scene; for each training sample image, input the confidence score of the training sample image for each target shooting scene, and the shooting scene label of the first shooting scene into a second decision model, and train the second decision model based on the shooting scene recognition result of the second decision model and the shooting scene label of the first shooting scene.

[0218] In some embodiments of this application, the processing module is further configured to encapsulate the second decision model into a first model configuration file; the sending module is configured to send the first model configuration file to the electronic device; wherein the first model configuration file is used to configure the electronic device's ability to recognize the first shooting scene.

[0219] The model training device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0220] The model training device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0221] The model training apparatus provided in this application embodiment can realize all the processes implemented in the model training method embodiment, and will not be described again here to avoid repetition.

[0222] Optionally, such as Figure 8 As shown, this application embodiment also provides an electronic device 800, including a processor 801 and a memory 802. The memory 802 stores a program or instructions that can run on the processor 801. When the program or instructions are executed by the processor 801, they implement the various steps of the above-described shooting scene determination method embodiment or the various steps of the above-described model training method embodiment, and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0223] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0224] Figure 9 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0225] The electronic device 100 includes, but is not limited to, components such as: radio frequency unit 101, network module 102, audio output unit 103, input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, and processor 110.

[0226] Those skilled in the art will understand that the electronic device 100 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 110 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 9 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0227] The processor 110 is configured to: upon acquiring an image, use a first decision model to determine the confidence score of the acquired image for each second shooting scene; and use a second decision model to determine whether the shooting scene corresponding to the acquired image is the first shooting scene based on the confidence score of the acquired image for each second shooting scene; wherein the first shooting scene is a newly added shooting scene; and the second decision model is constructed based on the first model configuration file corresponding to the first shooting scene issued by the server.

[0228] In some embodiments of this application, the processor 110 is further configured to: acquire and load a first model configuration file corresponding to a first shooting scene; and construct a second decision model based on the first model configuration file.

[0229] In some embodiments of this application, the processor 110 is specifically used to: input the acquired image into a first decision model for scene recognition processing, and obtain the confidence score of the acquired image for each second shooting scene;

[0230] The confidence scores of the acquired images for each second shooting scene are input into the second decision model for classification and decision processing to obtain recognition information; the recognition information is used to indicate whether the shooting scene corresponding to the acquired image is the first shooting scene.

[0231] In some embodiments of this application, the processor 110 is specifically configured to: perform classification decision calculation on the confidence scores of the acquired image for each second shooting scene using a second decision model to obtain a second confidence score; and obtain first identification information if the second confidence score is greater than or equal to a confidence threshold; the first identification information indicates that the shooting scene corresponding to the acquired image is a first shooting scene.

[0232] In some embodiments of this application, the processor 110 is further configured to: when the shooting scene corresponding to the acquired image is a first shooting scene, execute a shooting control operation associated with the first shooting scene; wherein the shooting control operation includes at least one of the following: adjusting image signal processing parameters, applying a filter corresponding to the first shooting scene, triggering an augmented reality effect corresponding to the first shooting scene, and activating a shooting function mode corresponding to the first shooting scene.

[0233] In some embodiments of this application, the processor 110 is further configured to: obtain a second model configuration file for the first shooting scene; the second model configuration file is used to adjust the recognition accuracy of the first shooting scene, or configure the electronic device to recognize sub-scenes of the first shooting scene; and update the second decision model based on the second model configuration file.

[0234] In some embodiments of this application, the first decision model is a parameter-frozen deep learning model.

[0235] The electronic device provided in this application, upon acquiring a captured image, utilizes a first decision model to determine the confidence score of the captured image for each second shooting scene; and utilizes a second decision model to determine whether the shooting scene corresponding to the captured image is a first shooting scene based on the confidence score of the captured image for each second shooting scene; wherein the first shooting scene is a newly added shooting scene; and the second decision model is constructed based on a first model configuration file corresponding to the first shooting scene issued by a server. Through this solution, the electronic device can construct a second decision model based on a first model configuration file for a specific shooting scene, and use this second decision model to determine whether the shooting scene corresponding to the captured image is a first shooting scene based on the confidence score of the captured image for each second shooting scene. This enables the electronic device to recognize newly added shooting scene categories, thereby improving its ability to recognize newly added shooting scenes. Furthermore, while expanding the electronic device's ability to recognize newly added shooting scenes, it shortens the model deployment cycle and reduces the data cost of model deployment.

[0236] It should be understood that, in this embodiment, the input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 107 includes at least one of a touch panel 1071 and other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0237] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 109 may include volatile memory or non-volatile memory, or it may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 109 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0238] Processor 110 may include one or more processing units; optionally, processor 110 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 110.

[0239] Figure 10 This is a schematic diagram of the hardware structure of a server provided in an embodiment of this application. Figure 10 As shown, the server 900 may include: one or more processors 901, memory 902, communication interface 903, and bus 904.

[0240] In this embodiment, one or more processors 901, memory 902, and communication interface 903 are interconnected via bus 904. Bus 904 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Bus 904 can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 9 The bus is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. Additionally, the server 900 may include some functional modules not shown, which will not be elaborated upon here.

[0241] In some embodiments of this application, the processor 901 is configured to input training sample images corresponding to a first shooting scene into a first decision model for scene recognition processing, to obtain a confidence score for each training sample image for each second shooting scene; to determine a target shooting scene related to the first shooting scene from each second shooting scene; and for each training sample image, to input the confidence score of the training sample image for each target shooting scene and the shooting scene label of the first shooting scene into a second decision model, and to train the second decision model based on the shooting scene recognition result of the second decision model and the shooting scene label of the first shooting scene.

[0242] In some embodiments of this application, the processor 901 is further configured to encapsulate the second decision model into a first model configuration file; the communication interface 903 is configured to send the first model configuration file to the electronic device; wherein the first model configuration file is configured to configure the electronic device's ability to recognize the first shooting scene.

[0243] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described shooting scene determination method embodiment or the various processes of the above-described model training method embodiment, and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0244] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0245] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described shooting scene determination method embodiment or the various processes of the above-described model training method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0246] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0247] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described shooting scene determination method embodiment or the various processes of the above-described model training method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0248] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0249] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0250] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method of determining a shot scene, characterized by, The method includes: Once the captured images are obtained, the confidence scores of the captured images for each second shooting scene are determined using the first decision model; Using a second decision model, based on the confidence scores of the acquired images for each second shooting scene, it is determined whether the shooting scene corresponding to the acquired image is the first shooting scene; The first shooting scene is a newly added shooting scene; the second decision model is constructed based on the first model configuration file corresponding to the first shooting scene issued by the server.

2. The method of claim 1, wherein, The method includes: Obtain and load the first model configuration file corresponding to the first shooting scene; Based on the first model configuration file, the second decision model is constructed.

3. The method according to claim 1, characterized in that, The step of determining the confidence score of the acquired image for each second shooting scene using the first decision model includes: The acquired images are input into the first decision model for scene recognition processing to obtain the confidence scores of the acquired images for each second shooting scene; The step of using a second decision model to determine whether the shooting scene corresponding to the acquired image is the first shooting scene based on the confidence score of the acquired image for each second shooting scene includes: The confidence scores of the acquired images for each second shooting scene are input into the second decision model for classification decision processing to obtain identification information; the identification information is used to indicate whether the shooting scene corresponding to the acquired image is the first shooting scene.

4. The method according to claim 3, characterized in that, The step of inputting the confidence scores of the acquired images for each second shooting scene into the second decision model for classification decision processing to obtain recognition information includes: The second decision model is used to perform classification decision calculations on the confidence scores of the acquired images for each second shooting scene to obtain the second confidence score; If the second confidence score is greater than or equal to the confidence threshold, first identification information is obtained; the first identification information indicates that the shooting scene corresponding to the acquired image is the first shooting scene.

5. The method according to claim 1, characterized in that, The method further includes: If the shooting scene corresponding to the acquired image is the first shooting scene, then a shooting control operation associated with the first shooting scene is executed. The shooting control operation includes at least one of the following: Adjust image signal processing parameters, apply the filter corresponding to the first shooting scene, trigger the augmented reality effect corresponding to the first shooting scene, and start the shooting function mode corresponding to the first shooting scene.

6. The method according to claim 1, characterized in that, The method further includes: Obtain a second model configuration file for the first shooting scene; the second model configuration file is used to adjust the recognition accuracy of the first shooting scene, or to configure the electronic device's recognition capability for sub-scenes of the first shooting scene; The second decision model is updated based on the second model configuration file.

7. The method according to claim 1, characterized in that, The first decision model is a deep learning model with frozen parameters.

8. A model training method, characterized in that, The method includes: The training sample images corresponding to the first shooting scene are input into the first decision model for scene recognition processing to obtain the confidence score of each training sample image for each second shooting scene. Identify target shooting scenes related to the first shooting scene from each of the second shooting scenes; For each training sample image, the confidence score of the training sample image for each target shooting scene and the shooting scene label of the first shooting scene are input into the second decision model. The second decision model is trained based on the shooting scene recognition result of the second decision model and the shooting scene label of the first shooting scene.

9. The method according to claim 8, characterized in that, The method further includes: The second decision model is encapsulated into a first model configuration file; Send the first model configuration file to the electronic device; The first model configuration file is used to configure the electronic device's ability to recognize the first shooting scene.

10. A shooting scene determination device, characterized in that, The device includes a processing module; the processing module is used for: Once the captured images are obtained, the confidence scores of the captured images for each second shooting scene are determined using the first decision model; Using a second decision model, based on the confidence scores of the acquired images for each second shooting scene, it is determined whether the shooting scene corresponding to the acquired image is the first shooting scene; The first shooting scene is a newly added shooting scene; the second decision model is constructed based on the first model configuration file corresponding to the first shooting scene issued by the server.

11. The apparatus according to claim 10, characterized in that, The processing module is further configured to: Obtain and load the first model configuration file corresponding to the first shooting scene; Based on the first model configuration file, the second decision model is constructed.

12. The apparatus according to claim 10, characterized in that, The processing module is specifically used for: The acquired images are input into the first decision model for scene recognition processing to obtain the confidence scores of the acquired images for each second shooting scene; The confidence scores of the acquired images for each second shooting scene are input into the second decision model for classification decision processing to obtain identification information; the identification information is used to indicate whether the shooting scene corresponding to the acquired image is the first shooting scene.

13. A model training device, characterized in that, The device includes a processing module; the processing module is used for: The training sample images corresponding to the first shooting scene are input into the first decision model for scene recognition processing to obtain the confidence score of each training sample image for each second shooting scene. Identify target shooting scenes related to the first shooting scene from each of the second shooting scenes; For each training sample image, the confidence score of the training sample image for each target shooting scene and the shooting scene label of the first shooting scene are input into the second decision model. The second decision model is trained based on the shooting scene recognition result of the second decision model and the shooting scene label of the first shooting scene.

14. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the shooting scene determination method as described in any one of claims 1-7.

15. A server, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the model training method as described in claim 8 or 9.

Images