Image data processing method, electronic device, readable medium and program product

Abstract

The application provides a kind of image data processing method, electronic equipment, readable medium and program product, the image data processing method includes: in response to the starting operation of camera application, starting camera to collect image data and display camera preview interface, camera preview interface displays first image;Scene category is obtained based on image data;When scene category belongs to night scene, shooting mode and shooting style are obtained based on user data and system information;In camera preview interface, the second image that meets the requirement of best shooting mode and shooting style is displayed.Because user data includes the travel preference of user, system information includes the weather information of user current location, position information and current time, therefore, the shooting mode and shooting style obtained can provide specific, and more suitable for current time, current location shooting mode and style for different users, so that the effect of shooting image has personalization, also improves man-machine interaction efficiency.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image data processing method, electronic device, computer program product, and computer-readable storage medium. Background Technology

[0002] With the widespread use of mobile phones and other electronic devices, users' demand for photography functions is growing daily. Whether it's recording daily life, sharing social media updates, or engaging in professional creative work, high-quality image capture has become one of the core needs of users. In night scene shooting scenarios, whether it's urban nightscapes, low-light portraits, or astrophotography, users expect devices to overcome light limitations and deliver clear, bright, detailed, and well-controlled images.

[0003] However, the current night scene shooting scenarios use the system's default night scene processing mode, which lacks personalized design and has low operation and interaction efficiency. Summary of the Invention

[0004] This application provides an image data processing method, electronic device, computer program product, and computer-readable storage medium, which can realize personalized processing of night scene images and improve human-computer interaction efficiency.

[0005] To achieve the above objectives, this application provides the following technical solution:

[0006] In a first aspect, this application provides an image data processing method applied to an electronic device including a camera. The method includes: responding to a camera application startup operation, starting the camera to acquire image data and displaying a camera preview interface; wherein the camera preview interface displays a first image, the first image corresponding to the image data acquired by the camera; based on the image data acquired by the camera, obtaining a scene category corresponding to the image data; if the scene category is a night scene, obtaining a shooting mode and shooting style for the night scene based on user data and system information; the user data includes the user's travel preferences, and the system information includes weather information, location information, and current time of the user's current location; and displaying a second image on the camera preview interface, the second image meeting the requirements of the shooting mode and shooting style.

[0007] In the above implementation scheme, the shooting mode and style for night scene photography are obtained based on user data and system information. A second image that meets the requirements of the shooting mode and style is displayed on the camera preview interface. Since user data includes user travel preferences, reflecting their travel preferences, and system information includes weather information, location information, and current time, obtaining the shooting mode and style for night scene photography based on user data and system information can provide different users with specific shooting modes and styles more suitable for their current time and location, resulting in more refined image quality and achieving personalized night scene photography and stylized effects. Furthermore, after the user launches the camera application and the electronic device enters the camera preview interface, the shooting mode and style for the night scene can be obtained based on user data and system information, eliminating the need for complex manual operations. This preserves the professional-grade depth of night scene photography while freeing up the user's hands and improving human-computer interaction efficiency.

[0008] In some possible implementations, the shooting mode and shooting style of the night scene are obtained based on user data and system information, including: obtaining the shooting mode of the night scene based on user data and system information; and obtaining the shooting style of the night scene based on user data and system information when the shooting mode of the night scene includes multiple shooting styles.

[0009] In some possible implementations, before displaying the second image on the camera preview interface, the process further includes: processing the image data acquired by the camera to obtain the image data of the second image; the camera acquiring image data based on the shooting mode and shooting style of the night scene; and / or, acquiring the image data of the second image sent by the cloud-side device.

[0010] In this implementation, the cloud-side device has stronger computing power, larger storage space, and can support more complex image processing algorithms. The cloud-side device can obtain image data of the second image and obtain an image with better image quality.

[0011] In some possible implementations, before displaying the second image on the camera preview interface, the method further includes: displaying a floating window on the camera preview interface to indicate the shooting mode and style for night scene shooting; displaying the second image on the camera preview interface includes: displaying the second image in response to a user's enable operation, whereby the enable operation confirms that the image is captured using the night scene shooting mode and style. In this implementation, displaying a floating window on the camera preview interface to indicate the shooting mode and style for night scene shooting, and displaying the second image on the camera preview interface when enabled by the user, can also enhance the user's interactive experience.

[0012] Some possible implementations include: displaying capsules in the camera preview interface, the capsules containing multiple night scene scenes and / or multiple shooting styles for night scene scenes; in response to the user selecting a night scene scene or shooting style, displaying a second image in the camera preview interface, and a third image that meets the requirements of the night scene scene or shooting style selected by the user. This implementation allows for adjusting the night scene scene or shooting style of the image according to user needs, which can improve the user's interactive experience and enhance user engagement.

[0013] In some possible implementations, the shooting mode of the night scene is obtained based on user data and system information, including: obtaining the fusion features of user data and system information; calling the first classification model based on the fusion features to obtain the shooting mode of the night scene; wherein, the first classification model is trained using the user profile of the training sample of the first classification model, and the user profile of the training sample of the first classification model is constructed based on the point information of the image, the distribution of historical shooting time periods, the preference for commonly used shooting modes, and the geographic location heat map.

[0014] In this implementation, the geographic location heatmap is used to indicate the distribution density and hotspot areas of the photo shooting locations. The first classification model is trained based on the user profiles of the training samples of the first classification model, which enables the first classification model to learn the common shooting patterns of users at common locations and times of image shooting. This ensures that the shooting patterns obtained by the first classification model are more in line with the choices of a large number of users and guarantees the image shooting quality.

[0015] In some possible implementations, the shooting style of night scene is obtained based on user data and system information, including: obtaining the fusion features of user data and system information; calling a second classification model based on the fusion features to obtain the shooting style of night scene; wherein, the second classification model is trained using the user profile of the training samples of the second classification model, and the user profile of the training samples of the second classification model is constructed based on the image's dot information, analyzing the distribution of historical shooting time periods, commonly used filter type preferences, and geographical location heatmaps.

[0016] In this implementation, the second classification model is trained based on the user profiles of the training samples of the second classification model. This allows the first classification model to learn the commonly used filters of users at common locations and times when images are taken, ensuring that the shooting style obtained by the second classification model is more in line with the choices of a large number of users, and further ensuring the image shooting quality.

[0017] In some possible implementations, the method further includes: saving a second image in response to an image capture operation; displaying an image browsing interface for the second image in response to an image browsing operation for the second image, the image browsing interface including an edit button; displaying an image editing interface in response to a click on the edit button, the image editing interface displaying a filter list, the filter list including multiple filters, each filter corresponding to a shooting style, and styles including multiple shooting styles for night scenes; and displaying an image that meets the requirements of the user-selected filter in the image editing interface based on the filter selected by the user. This implementation allows users to change the filters on the final image, enhancing the user's interactive experience.

[0018] In some possible implementations, before displaying an image that meets the requirements of the user-selected filter in the image editing interface, based on the user-selected filter and the image data of the second image, the method further includes: processing the image data of the second image to obtain image data of the image that meets the requirements of the user-selected filter; and / or receiving the image data of the image that meets the requirements of the user-selected filter sent by the cloud-side device.

[0019] In a second aspect, this application provides an electronic device, including: one or more processors, a memory, a camera, and a display screen; the memory, camera, and display screen are coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the electronic device performs the image data processing method as described in any one of the first aspects.

[0020] Thirdly, this application provides a computer-readable storage medium for storing a computer program, which, when executed, is specifically used to implement the image data processing method as described in any one of the first aspects.

[0021] Fourthly, this application provides a computer program product that, when run on a computer, causes the computer to perform the image data processing method as described in any one of the first aspects. Attached Figure Description

[0022] Figure 1 A schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application;

[0023] Figure 2 A schematic diagram of the software structure of the electronic device provided in the embodiments of this application;

[0024] Figure 3 A schematic diagram of the runtime sequence of the image data processing method provided in the embodiments of this application;

[0025] Figure 4A flowchart illustrating the image data processing method provided in the embodiments of this application;

[0026] Figure 5 A schematic diagram illustrating the optimal shooting mode and shooting style for night scene using the camera Hal provided in this application embodiment, as well as the process of processing image data;

[0027] Figure 6 A flowchart illustrating another image data processing method provided in this application embodiment;

[0028] Figure 7 A flowchart illustrating another image data processing method provided in this application embodiment;

[0029] Figure 8 This is a flowchart illustrating another image data processing method provided in an embodiment of this application. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. The terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to be a limitation of this application. As used in the specification and appended claims of this application, the singular expressions "a," "an," "the," "the," "the," and "this" are intended to also include expressions such as "one or more," unless the context clearly indicates otherwise. It should also be understood that in the embodiments of this application, "one or more" refers to one, two, or more; "and / or" describes the relationship between related objects, indicating that three relationships may exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0031] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0032] The "multiple" mentioned in the embodiments of this application refers to two or more. It should be noted that in the description of the embodiments of this application, terms such as "first" and "second" are used only for the purpose of distinguishing descriptions and should not be construed as indicating or implying relative importance, nor should they be construed as indicating or implying order.

[0033] With the widespread use of smartphones and other electronic devices, users' demand for camera functions is growing daily. Whether recording daily life, sharing social media updates, or engaging in professional creative work, high-quality image capture has become one of the core needs of users. Users are pursuing higher image quality, faster focusing speed, and stronger low-light performance. In night scene shooting scenarios, whether it's urban nightscapes, low-light portraits, or astrophotography, users expect their devices to overcome light limitations and deliver clear, bright, detailed, and noise-controlled images.

[0034] However, the following pain points still exist in current night scene shooting scenarios:

[0035] Insufficient scene recognition accuracy: The ability to subdivide and recognize complex night scene scenes is limited, especially in accurately distinguishing night scene types with significant differences, such as supermoons, fireworks displays, and urban light landscapes.

[0036] Limited image style options: The system's default night scene processing mode lacks personalized options, preventing users from selecting specific visual styles based on their creative needs, such as industrial black and gold effects, romantic soft light styles, or cinematic CineStill 800t film textures.

[0037] Inefficient operation and interaction: Key shooting functions are often hidden in multi-level menus, and users need to go through tedious manual switching to complete the mode selection. This complicated operation process significantly reduces the quality of the real-time shooting experience.

[0038] These pain points make it difficult for users to quickly obtain ideal imaging results in actual night scene shooting.

[0039] To address this, this application provides an image data processing method. When an electronic device takes photos of a night scene, it performs scene-based and style-based recommendations to provide users with recommendations for the best night scene shooting mode and shooting style under the current time, location, weather, and shooting scene.

[0040] Furthermore, electronic devices can utilize image data processing methods to obtain images that meet the optimal shooting mode and style for night scenes, and support users in modifying the effects and styles of the final images.

[0041] The image data processing method provided in this application embodiment can be applied to electronic devices with shooting functions such as mobile phones, tablets, personal digital assistants (PDAs), desktop, laptop, and notebook computers, ultra-mobile personal computers (UMPCs), handheld computers, netbooks, wearable devices, and digital cameras.

[0042] Taking mobile phones as an example, Figure 1 An example of the composition of an electronic device provided in this application embodiment.

[0043] like Figure 1 As shown, the electronic device 100 may include a processor 110, an internal memory 120, a camera 130, a display screen 140, a mobile communication module 150, a wireless communication module 160, and a sensor module 170, etc.

[0044] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0045] The processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a smart sensor hub, and / or a neural network processing unit (NPU).

[0046] Internal memory 120 can be used to store computer executable program code, including instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 120. Furthermore, internal memory 120 may include high-speed random access memory and may also include non-volatile memory. Processor 110 executes various functional applications and data processing of electronic device 100 by running instructions stored in internal memory 120 and / or instructions stored in memory disposed within the processor.

[0047] In some embodiments, the internal memory 120 stores instructions for performing image data processing methods. The processor 110 can execute the instructions stored in the internal memory 120 to perform scene-based and style-based recommendations based on image content, system content, and other information when taking night scene photos. It can also obtain image data that meets the recommended scene and style when the user enables the function to capture the image.

[0048] Electronic devices can perform some shooting functions through ISP, camera 130, GPU, display 140 and application processor.

[0049] The ISP is used to process the data fed back by the camera 130.

[0050] Camera 130 is used to capture still images or videos. An object passes through the lens to generate an optical image that is projected onto a photosensitive element. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP (Internet Service Provider) for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP (Digital Signal Processor) for processing. The DSP converts the digital image signal into image signals in standard formats such as RGB and YUV. In some embodiments, the electronic device may include one or N cameras 130, where N is a positive integer greater than 1.

[0051] Electronic devices implement display functions through GPUs, display screens 140, and application processors. A GPU is a microprocessor for image processing, connected to the display screen 140 and the application processor. The GPU performs mathematical and geometric calculations for image rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0052] The display screen 140 is used to display images, video interfaces, etc. The display screen 140 includes a display panel. In some embodiments, the electronic device may include one or N display screens 140, where N is a positive integer greater than 1.

[0053] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0054] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals.

[0055] Mobile communication module 150 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for use on electronic device 100. Wireless communication module 160 can provide wireless communication solutions, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies, for use on electronic device 100.

[0056] Pressure sensor 170A is used to sense pressure signals and convert them into electrical signals. When force is applied to pressure sensor 170A, the capacitance between the electrodes changes. The electronic device determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 140, the electronic device detects the intensity of the touch operation based on pressure sensor 170A. The electronic device can also calculate the touch location based on the detection signal from pressure sensor 170A.

[0057] Touch sensor 170B, also known as a "touch device," can be located on display screen 140. The touch sensor 170B and display screen 140 together form a touchscreen, also known as a "touchscreen." Touch sensor 170B is used to detect touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 140.

[0058] An ambient light sensor 180C is used to sense the ambient light intensity. Electronic device 100 can determine the ambient light intensity in which the electronic device is located based on the sensed ambient light intensity.

[0059] The gyroscope sensor 180D can be used to determine the motion attitude of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 about three axes (i.e., the x, y, and z axes) can be determined by the gyroscope sensor 180D.

[0060] Electronic devices Figure 1 The hardware components on display also run an operating system. For example... operating system, operating system, Operating system, etc. Applications such as camera apps can be installed and run on the operating system.

[0061] Figure 2 This is a schematic diagram of the software structure of an electronic device provided in an embodiment of this application.

[0062] A layered architecture divides the operating system of an electronic device into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the operating system of the electronic device is the Android system. The Android system can be divided into five layers, from top to bottom: the application (APP) layer, the application framework layer (FWK), the system libraries, the hardware abstraction layer (HAL), and the kernel layer.

[0063] The application layer can include a series of application packages. For example... Figure 2 As shown, the application package can include applications such as camera and gallery.

[0064] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0065] like Figure 2 As shown, the application framework layer may include window management services, content providers, view systems, resource managers, camera services, and data migration services (DMS), etc.

[0066] The window management service manages window applications. It allows for adding, deleting, showing, and hiding windows. The content provider stores and retrieves data, making it accessible to applications. This data can include video, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc. The view system includes visual controls, such as controls for displaying text and images. The view system can be used to build applications. The resource manager provides applications with various resources, such as localized strings, icons, images, layout files, video files, etc.

[0067] In some embodiments, the camera service may also be referred to as the camera framework, which is used to receive video recording requests, image capture requests, etc. from the camera application, while maintaining the business logic of the internal flow of video recording requests, image capture requests, etc., and sending the final result of the request to the camera application.

[0068] In some embodiments, the DMS is used to interact with the cloud side, can transmit local data of electronic devices to the cloud side, can receive data sent by the cloud side, and can send data from the cloud side to upper and lower layer modes.

[0069] The Android Runtime includes core libraries and a virtual machine. The Android Runtime is responsible for scheduling and managing the Android system. In some embodiments of this application, application cold starts run within the Android Runtime, which obtains the application's optimized file status parameters. The Android Runtime can then use these parameters to determine whether the optimized file has become outdated due to system upgrades and returns the result to the application management module.

[0070] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0071] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0072] System libraries can include multiple functional modules. For example, a surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), and 2D graphics engines (e.g., SGL).

[0073] The Surface Manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The Media Library supports playback and recording of various common audio and video formats, as well as still image files. The Media Library supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D Graphics Processing Library implements 3D graphics drawing, image rendering, compositing, and layer processing. The 2D Graphics Engine is the drawing engine for 2D graphics.

[0074] The hardware abstraction layer is a software located between the operating system kernel and the hardware circuitry. It is typically used to abstract the hardware to enable interaction between the operating system and the hardware circuitry at the logic layer.

[0075] In some embodiments, the hardware abstraction layer includes a camera Hal, which is primarily responsible for constructing, linking, and controlling the image chain. For example, the camera Hal can also be configured with a recommendation algorithm, which the electronic device can use to perform contextualized and stylistic recommendations.

[0076] The kernel layer is the layer between hardware and software. It receives commands from the control layer and translates these commands into actual device operation instructions. These instructions include power-on / off control, device mode register instruction generation, and delivery. The kernel layer includes at least image sensor drivers.

[0077] Within the aforementioned five-layer architecture, the electronic device also includes a hardware layer, which may include the aforementioned electronic device hardware components. For example, Figure 2 The camera, ambient light sensor, and gyroscope sensor were showcased.

[0078] It should be noted that, although the embodiments of this application are based on... The system is used as an example for explanation, but its basic principles also apply to systems based on... Electronic devices with operating systems, etc.

[0079] The electronic devices mentioned earlier provide contextual and stylistic recommendations when taking night scene photos, offering users the best shooting mode and style for the current time, location, weather, and shooting scene. This can be referred to as the contextual and stylistic recommendation function, which can be manually turned on or off by the user.

[0080] In some embodiments, the electronic device may be configured with an entry point for contextualized and stylized recommendation functions.

[0081] For example, the camera app interface may display a button that serves as a toggle for the contextualized and stylized recommendation function. Users can use this button to turn the contextualized and stylized recommendation function on or off. Optionally, this button can be located on the camera app's camera preview interface or on the camera app's settings interface.

[0082] In other embodiments, the gallery application of the electronic device may also be configured with an entry point for modifying image styles.

[0083] For example, the image preview interface of a gallery application may include an edit button. When triggered by a user, the electronic device can display an image editing interface. This editing interface may include an image and filter buttons. When triggered by a user, the electronic device can also display a filter list within the image editing interface. This filter list may include various filters, each corresponding to a specific style. Optionally, the filter list may include various traditional filters, such as grayscale and black-and-white filters, as well as various night scene style filters, such as cyberpunk, black-and-gold, and soft-focus filters for urban landscapes; retro styles reminiscent of CineStill 800T for scenes in Chungking Express or alleyways; and blues styles and sunrise pastel tones for blues scenes.

[0084] In some embodiments, the filter list may hide some filters, and users can swipe left or right in the filter list to reveal the hidden filters.

[0085] When a user selects a filter from the filter list, the image displayed in the editing interface can be processed by the electronic device using the selected filter and then displayed.

[0086] After a user enables the contextualized and stylized recommendation function, the electronic device can execute the image data processing method provided in the embodiments of this application in the shooting image scene to recommend the best night scene shooting mode and shooting style for the user under the current time, location, weather and shooting scene.

[0087] Figure 3 This diagram illustrates the runtime sequence of the image data processing method provided in an embodiment of this application.

[0088] In some application scenarios, combined with Figure 3 As shown, when a user launches the camera application to activate the camera, the image data captured by the camera is reported to the camera Hal via the image sensor driver. The camera application can also acquire image data to display the camera preview interface. After the camera application is launched, it can also acquire system information and user data stored in the system, and send the user data and system information to the camera Hal through the camera service.

[0089] For example, system information may include: weather information, location information, city information, and time information indicating the current time of the user's current location; wherein, city information may include multi-dimensional information such as basic geographic information, economic and consumption characteristics, climate and environment, culture and lifestyle; and time information may include system clock and holiday information.

[0090] User data may include: the user's current travel information, basic user data, and user preference information recorded by the system.

[0091] For example, basic user data may include multi-dimensional information such as user age, gender, and occupation. User preference information may include user travel preference information, used to indicate the user's preferred travel purpose. Optionally, the user's basic data and the user's travel destination preference information may be derived from the user's travel data. As another example, user preference information may also include user preferences when taking images. Optionally, this may include records of the user's historical behavior, which may be the user's feedback on the electronic device's historically recommended best night scene shooting modes and shooting styles, such as the user rejecting the electronic device's recommended best night scene shooting modes or shooting styles, or the user using the electronic device's recommended best night scene shooting modes or shooting styles; optionally, user preference information may also include the user's commonly used image shooting styles and shooting modes, etc.

[0092] The camera's Hal sensor can also acquire ambient light intensity detected by the ambient light sensor, and determine whether the user is taking pictures at night based on the ambient light intensity, that is, whether the image taken by the camera application belongs to a night scene.

[0093] In some embodiments, the ambient light intensity detected by the ambient light sensor can be reported to the camera application through modules in the kernel layer and application framework layer, and the camera application can send it to the camera Hal through the camera service.

[0094] When the camera Hal determines that an image captured by the camera app belongs to a night scene, it can recommend the best night scene shooting mode and style based on ambient light intensity, image data, user data, and system information, taking into account the current time, location, weather, and shooting scenario. This recommendation is then reported to the camera app via the camera service. The camera app can then display a reminder message to the user based on the recommended best night scene shooting mode and style, suggesting that the user utilize these modes for better image quality. When the user confirms the use of the best night scene shooting mode and style, the camera app can also pass instructions to lower-level modules to control the electronic device to capture images using that mode and style. Furthermore, the camera app can record the user's behavior—whether they use the best night scene shooting mode and style or not—as user preference information.

[0095] For example, night scene shooting modes, also known as night scene modes, may include: Super Moon mode, Stage mode, Fireworks mode, Night scene long exposure mode, Light Painting mode, Night scene telephoto AI ultra-clear mode, City distant scene mode, Street scene mode, and Blues scene mode, etc.

[0096] For example, the shooting style, or night scene style, can determine the overall atmosphere and emotional message of the image. Optionally, night scene styles may include: cyberpunk, black and gold, and soft-focus effects in urban landscapes; Chungking Express and CineStill 800T-inspired retro styles in alleyways; and blues styles and pale sunrise tones in blues scenes.

[0097] The specific implementation of how the camera Hal obtains recommendations for the best shooting mode and style for night scenes under the current time, location, weather, and shooting scenario can be found in the following text, which will not be elaborated here.

[0098] The following combination Figure 2 The modules in the software structure of the electronic device are shown, and the image data processing method provided in the embodiments of this application is introduced.

[0099] The image data processing method provided in this application can be executed independently by an edge device or in a "edge-cloud combined" architecture. The edge-cloud combined architecture includes edge devices and cloud devices. The edge device can be understood as the electronic device mentioned above, and the cloud device can be understood as a server deployed in the cloud. This application does not limit the form, type, hardware components, or software structure of the cloud device.

[0100] Down Figure 4 A flowchart illustrating the image data processing method performed by the edge device is shown below; Figure 6 The diagram illustrates the process flow of an image data processing method executed by an "edge-cloud" architecture.

[0101] like Figure 4 As shown, the image data processing method provided in this application embodiment includes:

[0102] S401: The camera responds to user input, captures image data, and sends the image data to the camera application via the camera Hal.

[0103] The user launches the camera application by inputting a camera activation command. The electronic device responds to the user's command and launches the camera application. Once launched, the camera is controlled to capture image data, which is then provided to the camera application via the camera HA. ​​The camera application can then display the captured images on the screen, forming a camera preview interface.

[0104] In some embodiments, a user's actions to open the camera application may include: tapping the camera application icon on the display screen, pressing a specific physical button, inputting voice, and using air gestures.

[0105] The image data captured by the camera can be understood as: the image signal obtained by the photosensitive element after being processed by the ISP and DSP respectively.

[0106] S402, The camera application displays a camera preview interface based on image data.

[0107] The camera preview interface can display the image corresponding to the image data, i.e., the first image.

[0108] S403, The sensor sends ambient light intensity and location information to the camera application.

[0109] For example, the sensor may include an ambient light sensor, which operates to detect the current brightness of the environment and obtain the ambient light brightness. Optionally, the ambient light brightness detected by the ambient light sensor can be reported to multiple applications in the application layer through modules in the kernel layer and application framework layer, for example, it can be reported to a camera application.

[0110] As another example, the sensor may also include a GNSS chip, such as a Global Positioning System (GPS) or BeiDou, which can obtain the current location information of the electronic device. Optionally, the location information detected by the GNSS chip can also be reported to multiple applications in the application layer through modules in the kernel layer and application framework layer, for example, it can be reported to a camera application.

[0111] In some embodiments, the ambient light sensor can be configured to be normally on, meaning it starts running as soon as the electronic device is powered on, or it can be started running according to actual needs, such as when a user starts a camera application, the ambient light sensor starts running synchronously. This application does not limit the conditions under which hardware such as the ambient light sensor and GNSS chip starts running.

[0112] This application does not restrict the execution order of step S403. Figure 4 An exemplary execution sequence of step S403 is shown. Furthermore, the sensor can periodically report ambient light intensity and location information to the camera application; this periodicity can be continuous or intermittent, allowing the camera application to obtain the current ambient light intensity and location information of the electronic device in real time. This application does not limit the number of times or the period at which the sensor reports ambient light intensity and location information.

[0113] S404: The camera application obtains user data, weather information, time information, and city information.

[0114] For example, user data may include the user's current travel information, basic user data, and user preference information. The camera application can obtain user data through modules of the operating system; details regarding user data can be found above and will not be repeated here.

[0115] A user's current travel information can come from a ticketing application on an electronic device. The ticketing application can report the user's travel information to a module in the operating system, while the camera application can obtain the user's travel information from a module in the operating system.

[0116] Weather information can come from weather applications on electronic devices, or weather applications can report weather information to modules in the operating system. Camera applications can obtain weather information from modules in the operating system. Optionally, if the camera application obtains the user's travel information, it can obtain weather information for the origin, transit points, and destination based on the user's travel information, or it can obtain weather information corresponding to the location information, which refers to the current location of the electronic device. Alternatively, if the camera application does not obtain the user's travel information, i.e., the user's location is fixed, the camera application can obtain weather information corresponding to the location information.

[0117] Time information can come from calendar and clock apps on electronic devices. Calendar and clock apps can also report time information to modules in the operating system. Camera apps can obtain time information from modules in the operating system. Time information may include system clock and holiday information.

[0118] The camera app can obtain city information through modules of the operating system. A description of city information can be found above and will not be repeated here. The camera app can also obtain multiple user data, weather information, time information, and city information.

[0119] This application does not restrict the execution order of step S404. Figure 4 An example of one execution sequence for step S404 is shown.

[0120] It is understood that the camera application can obtain user data, weather information, time information, city information, ambient light intensity, and location information through steps S403 and S404. Among these, weather information, time information, city information, and location information can be collectively referred to as system information.

[0121] S405: The camera application sends ambient light intensity, user data, weather information, location information, time information, and city information to the camera Hal.

[0122] After obtaining user data, system information, and ambient light intensity, the camera application can send these to the camera Hal via the camera service, so that the camera Hal can perform the following steps S406 to S409.

[0123] S406 and camera Hal extract features from image data to construct the feature vectors required for scene recognition.

[0124] Image data captured by the camera can be reported to the camera hall in real time. For example, the camera hall can extract features from the image data of each frame to obtain a feature vector. Alternatively, the camera hall can extract features from the image data of a subset of frames to obtain a feature vector. A subset of frames can be understood as images extracted at intervals of several frames.

[0125] Feature vectors of image data can indicate aspects such as brightness, color distribution, and texture features. For example, a feature vector may include brightness features, color distribution, and texture features.

[0126] In some embodiments, the camera Hal can process image data using conventional image processing algorithms to obtain feature vectors. For example, conventional image processing algorithms may include algorithms for calculating the average brightness of the image, which can convert the image to a black and white image and calculate the average grayscale value of all pixels as a brightness feature; algorithms for calculating color histograms and color matrices, which can calculate the distribution of various color pixels in the image as color distribution; and algorithms for calculating gray-level co-occurrence matrix (GLCM) and local binary mode (LBP), which can calculate texture features.

[0127] In other embodiments, the camera Hal may employ a deep learning model module to process image data and obtain feature vectors. Exemplary deep learning models include convolutional neural networks (CNNs), autoencoders consisting of encoders and decoders, etc., as detailed in related technologies, and will not be elaborated upon here.

[0128] S407 and camera Hal obtain the scene category corresponding to the image based on the feature vector of the image data and the ambient light intensity.

[0129] Combination Figure 5 The camera Hal includes a perception algorithm, which can invoke the perception algorithm to obtain the scene category corresponding to the image based on the feature vector of the image data and the ambient light intensity. For example, the scene category may include moon, stage, fireworks, night scene, food, portrait, sports, night portrait, etc., and this application is not limited thereto.

[0130] In some embodiments, the perception algorithm may include a classification model. The camera Hal can invoke the classification model to classify the scene of the image based on the feature vector of the image data, thereby obtaining the scene category corresponding to the image. This scene category may be indicated by a scene category identifier (ID). In this way, the camera Hal can obtain the scene category corresponding to the image based on the feature vector of the image data.

[0131] In other embodiments, the perception algorithm may include a classification model and a post-processing algorithm. The camera Hal may invoke the classification model to classify the image based on the feature vectors of the image data, obtaining the scene category corresponding to the image, where the scene category may be indicated by a scene category identifier (ID). The camera Hal may invoke the post-processing algorithm to determine whether the image belongs to a night scene based on ambient light intensity. The camera Hal may comprehensively evaluate the scene category corresponding to the image based on the scene category obtained from the classification model and the night scene recognition result obtained from the post-processing algorithm, thereby enhancing the accuracy of image scene category recognition. For example, if the night scene recognition result is that the image belongs to a night scene, and the classification model determines that the scene category corresponding to the image is a portrait, then the evaluation result obtained by the camera Hal may be: the scene category corresponding to the image is a night scene portrait.

[0132] In other embodiments, the scene category corresponding to the image obtained by the camera Hal may also include an image label, which is used to indicate the image content, such as indicating the objects included in the image. For example, for an image, if the camera Hal identifies the night scene as a night scene, the camera Hal may also obtain an image label of "city".

[0133] Optionally, the classification model can be a traditional classifier, a neural network model, or the like, but this application does not limit the specific type of classification model.

[0134] Alternatively, to improve the comprehensiveness of the classification model in recognizing scene categories, the classification model can be a model capable of recognizing multiple scene categories. That is, a classification model can identify the scene category of images belonging to different scene categories and obtain the recognition result. To achieve this, the training phase of the classification model can use images of various scenes (at least including scenes such as the moon, stage, fireworks, night scene, food, portrait, sports, and night scene portrait) for training.

[0135] Alternatively, to facilitate edge deployment, the classification model can also be a lightweight model.

[0136] The S408 camera's Hal sensor uses user data, location information, weather information, city information, time information, and the scene category corresponding to the image to determine the optimal shooting mode and style for night scenes.

[0137] Among them, the optimal shooting mode for night scenes, also known as the night scene shooting mode, is a recommended shooting mode for this particular scene. It can be understood that images taken using this optimal shooting mode are more suitable for the current time and location, resulting in better image quality; hence, it is called the optimal shooting mode.

[0138] In some embodiments, the camera Hal determines the optimal shooting mode and style for night scene scenarios. Therefore, when the scene category corresponding to the image indicates that the image is a night scene, the camera Hal can perform step S408 and the following steps. For example, in combination with... Figure 5 Scenes such as the moon, stage, fireworks, night scene, and night portrait belong to the category of night scene scenes. Under these scene categories, the camera Hal can execute step S408 and the following steps.

[0139] In other embodiments, the camera Hal may perform step S408 and the following steps without limiting the scene category. Based on this, by performing step S408, the camera Hal can obtain the optimal shooting mode and shooting style for the scene indicated by the scene category corresponding to the image. This document focuses on night scene scenarios, but this does not constitute a limitation on the image data processing methods provided in the embodiments of this application.

[0140] For example, in combination Figure 5 The camera hal can include a night scene scene recommendation algorithm, which is used to obtain the optimal shooting mode for night scenes. If the scene category corresponding to the image belongs to a night scene, the camera hal calls the night scene scene recommendation algorithm based on user data, location information, weather information, city information, and time information to obtain the optimal shooting mode for the night scene. The optimal shooting mode for a night scene can be indicated by a night scene mode identifier (ID). For example, Figure 5 The night scene modes showcased include: Super Moon Mode, Stage Mode, Fireworks Mode, Night Scene Long Exposure Mode, Light Painting Mode, Night Scene Telephoto AI Ultra-HD Mode, City Distance Scene Mode, Street Scene Mode, and Blues Scene Mode.

[0141] Location information, weather information, city information, and time information are collectively referred to as system information in the following text.

[0142] In some embodiments, the night scene scene recommendation algorithm may include a classification model. The camera Hal can invoke the classification model to obtain the optimal shooting mode for the night scene based on user data, system information, and the scene category corresponding to the image.

[0143] Optionally, user data and system information can first undergo feature extraction to obtain feature vectors, and then feature vector fusion can be performed to obtain fused features. The camera hal can then call a classification model to determine the optimal shooting mode for night scene scenes based on the fused features and the scene category corresponding to the image. The camera hal can perform feature extraction and fusion operations. The night scene category corresponding to the image also includes image labels.

[0144] Feature extraction and fusion can be implemented in the following two ways:

[0145] In one implementation, the trip information, user preference information, and basic user information in the user data can be extracted as feature vectors separately to obtain feature vectors for each type of user data. These feature vectors can then be fused to obtain the fused feature of the user data. Similarly, the weather information, location information, city information, and time information in the system information can be extracted as feature vectors separately to obtain feature vectors for each type of system information. These feature vectors can then be fused to obtain the fused feature of the system information. Finally, the fused feature of the user data and the fused feature of the system information are fused to obtain the fused feature.

[0146] In another implementation, the trip information, user preference information, and user basic information in the user data can be extracted as feature vectors separately to obtain the feature vector of each type of user data; the weather information, location information, city information, and time information in the system information can be extracted as feature vectors separately to obtain the feature vector of each type of system information; then, the feature vectors of each type of user data and the feature vectors of each type of system information are fused to obtain the fused feature.

[0147] In some application scenarios, the optimal shooting mode for night scenes can include multiple night scene styles (i.e., shooting styles). For example, the city distant scene mode, the alleyway mode, and the blues scene mode can be configured with multiple night scene styles, combined with... Figure 5 The camera Hal can also include a night scene stylization recommendation algorithm. The camera Hal calls the night scene stylization recommendation algorithm based on user data, location information, weather information, city information and time information, as well as the night scene ID, to obtain the shooting style of the night scene.

[0148] The shooting style of night scenes can be indicated by a night scene style identifier (ID). For example, Figure 5 The nighttime scene styles showcased included: cyberpunk, black and gold, and soft-focus effects in urban landscapes; Chungking Express and CineStill 800T retro styles in alleyways; and blues styles in blues scenes.

[0149] In some embodiments, the night scene-based recommendation algorithm may include a classification model. The camera Hal can invoke the classification model to obtain the night scene style based on user data, system information, and the optimal shooting mode for the night scene.

[0150] Optionally, user data and system information can first undergo feature extraction to obtain feature vectors, and then feature vector fusion can be performed to obtain fused features. The camera Hal can then call a classification model based on the fused features and the optimal shooting mode for the night scene to obtain the night scene style. The camera Hal can perform feature extraction and fusion operations; the implementation methods for feature extraction and fusion can be found in the aforementioned content and will not be repeated here.

[0151] In one implementation, the camera Hal can use the optimal shooting mode for a night scene to determine whether the shooting mode includes a night scene style. If the optimal shooting mode for a night scene includes a night scene style, the camera Hal calls a classification model based on user data and system information to obtain the night scene style of the night scene. This night scene style conforms to the user's aesthetic and matches the current shooting environment, which means that it can ensure the quality of the image captured under the current shooting environment while also taking into account the user's preferences.

[0152] In one example, the scene category corresponding to the image is "moon." The camera Hal uses a night scene recommendation algorithm based on user data, location information, weather information, city information, time information, and the scene category "moon," to determine that the best shooting mode for night scenes is "Supermoon mode." However, the Supermoon mode does not differentiate between multiple shooting styles; therefore, the camera Hal does not obtain a specific night scene style.

[0153] In another example, the image corresponds to the scene category of night scene. The camera Hal calls a night scene-specific recommendation algorithm based on user data, location information, weather information, city information, time information, and the night scene category to determine the optimal shooting mode for the night scene as the urban distant scene mode. The urban distant scene mode can include various shooting styles. Therefore, the camera Hal calls a classification model based on user data and system information to determine the night scene style as black and gold within the urban distant scene. The night scene category can also include image tags.

[0154] In this embodiment, the fusion features are derived from user data and system information. The user data indicates the user's gender, age, current itinerary, and itinerary preferences. The system information indicates the weather, urban features, current time, and holiday information of the user's current location. Based on the fusion features and the scene category corresponding to the image, the camera Hal obtains the best shooting mode and shooting style for the night scene. It can provide different users with specific shooting modes and styles that are more suitable for the current time and current location, making the captured image more effective and achieving personalized night scene effects and stylized results for each user.

[0155] With user data further including user preference information when taking images, the camera Hal, based on fused features and the scene category corresponding to the image, obtains the best shooting mode and shooting style for night scenes. It can also ensure that the captured images further take into account the user's personal preferences and improve the user's shooting experience.

[0156] It should be noted that the user data on which the camera Hal obtains the best shooting mode and style for night scenes includes the user's current itinerary information, basic user data, and user preference information. The system information on which it is based includes location information, weather information, city information, and time information. It is the most comprehensive information of user data and system information, but this does not constitute a limitation on the content of the user data and system information on which the camera Hal obtains the best shooting mode and style for night scenes.

[0157] In some embodiments, user travel preference information in user data can indicate the user's travel purpose preferences, and in particular, can also indicate the user's preferences in terms of shooting scene, shooting time, and shooting style. Location information, weather information, and time information can indicate the geographical location and lighting conditions of the images taken by the user. Therefore, the camera Hal obtains the best shooting mode and shooting style for night scene based on the user's travel preference information, location information, weather information, and time information. This best shooting mode and shooting style can also ensure better image quality and achieve personalized night scene effects for each user.

[0158] In other embodiments, the user's image preference information in the user data can indicate the user's preference for the mode and style of the images taken. The camera Hal can obtain the best shooting mode and shooting style for night scene based on the user's travel preference information, location information, weather information, time information and the user's image preference information, and can also capture images that conform to the user's personal preferences.

[0159] In some embodiments, the classification model for obtaining the best shooting mode of the night scene (hereinafter referred to as the first classification model) and the classification model for obtaining the night scene style (hereinafter referred to as the second classification model) can be traditional classifiers, neural network models, etc., but this application does not limit the specific type of classification model.

[0160] Optionally, to facilitate edge deployment, the first classification model and / or the second classification model may also adopt a lightweight model.

[0161] To improve the comprehensiveness of the model, the training samples used in the training phase of the first classification model and the second classification model can be diversified.

[0162] For example, the training methods for the first classification model and the second classification model can be as follows:

[0163] Acquire tracking information (i.e., image metadata or tag data) of various types of user-captured images. The tracking information can indicate the time, location, shooting mode, scene classification, user preferences, and other information when the user captured the image. It can include basic information such as device shooting parameters, shooting time, and location, as well as user behavior tracking, i.e., user interaction data recorded through embedded points, such as shooting mode and scene classification. It can also include image content analysis tracking, such as image features.

[0164] Then, based on the image's dot information, we can analyze the distribution of historical shooting time periods, preferred shooting modes, and geographical location heatmaps to construct user profiles for the training samples of the first classification model. For example, a user profile could be: a young trend explorer aged 18 to 25 who likes to use automatic beauty mode to take photos in scenic spots. Alternatively, based on the image's dot information, we can analyze the distribution of historical shooting time periods, preferred filter types, and geographical location heatmaps to construct user profiles for the training samples of the second classification model. For example, a user profile could be: a young trend explorer aged 18 to 25 who is keen on quickly taking photos and selfies at popular photo spots using a high-saturation filter shooting style.

[0165] Among them, the geographic location heat map is used to indicate the distribution density and hotspot areas of the locations where the photos were taken.

[0166] Based on the user profiles of the training samples, train the initial model of the first classification model and the initial model of the second classification model until the training of the first and second classification models is completed.

[0167] For example, Table 1 shows the design of night scene recommendation, night scene gameplay recommendation, and night scene atmosphere effect recommendation schemes.

[0168] Table 1

[0169]

[0170] In Table 1, the GPS in the entry conditions of the urban distant scene mode (including its several stylizations) and the alleyway mode (including its several stylizations) can refer to the location information of the system information, the time can refer to the time information of the system information, the illuminance information can be determined by the weather information and the time information, and is used to indicate the luminous flux density of light shining on the surface of the object. The 2D image can include the scene category corresponding to the image, and optionally the scene category can also include the image label.

[0171] The sunset detection in the entry conditions of the Blues scene mode (including its various stylizations) can be determined by the location information of the system information and the scene category corresponding to the image. Optionally, the scene category may also include the image label.

[0172] In some embodiments, the shooting style in step S408 can also be an optional execution content, that is, step S408 can be replaced by: the camera Hal obtains the best shooting mode for the night scene based on user data, location information, weather information, city information and time information, as well as the scene category corresponding to the image. The specific implementation method can be found in the foregoing content, and will not be repeated here.

[0173] S409, Camera Hal sends the best shooting mode and shooting style for night scenes to the camera application.

[0174] In some embodiments, the camera Hal sends an identifier of the best shooting mode for night scenes, as well as an identifier of the shooting style for night scenes, to the camera application via the camera service.

[0175] The S410 camera application displays prompts and receives enable results on the camera preview screen.

[0176] In some embodiments, the prompt information can be used to advise the user that the current scene uses the optimal shooting mode and style to obtain the best image quality. Combined with... Figure 5 The camera Hal can perform UX recommendation interaction to prompt users to use the best shooting mode and shooting style for the current scene to obtain the best image effect. It can also receive the scene mode enable result returned by the user.

[0177] For example, a camera app can display a floating window on the camera preview screen, which may include a prompt message and two buttons. One button is for confirming, i.e., taking an image using the best shooting mode and style; the other button is for canceling, i.e., taking an image without using the best shooting mode and style. Users can interact with both buttons to confirm or cancel taking an image using the best shooting mode and style.

[0178] As another example, the camera application can display a floating window in the camera preview interface, which may include prompts. Users can confirm or cancel taking images using the optimal shooting mode and style by changing the orientation of the electronic device. Optionally, users can confirm or cancel taking images using the optimal shooting mode and style by flipping the electronic device. The orientation of the electronic device can be obtained from detection data from a gyroscope sensor.

[0179] As another example, the camera app can display a floating window in the camera preview interface, which may include prompts. Users can confirm or cancel taking an image using the best shooting mode and style by long-pressing, double-tapping, or single-clicking.

[0180] It is understood that the user's confirmation operation can be interpreted as an enable operation. Upon receiving the user's enable operation, the camera application can execute the following step S411. If the camera application does not receive the user's enable operation, it indicates that the user is not using the optimal shooting mode and style to capture the image, and the application can exit the process of this embodiment.

[0181] S411, The camera application sends an enable result to the camera Hal.

[0182] In some embodiments, the camera application sends an enable result to the camera Hal via the camera service.

[0183] The S412 camera's Hal process handles image data to produce images that meet the requirements of the optimal shooting mode and shooting style.

[0184] The camera Hal can be configured with various shooting modes, image processing algorithms, and shooting parameters for different shooting styles. When the camera Hal receives the enable result, the electronic device uses the shooting parameters corresponding to the optimal shooting mode and shooting style to acquire image data. This image data is then used as the image data to be processed. The camera Hal can use the corresponding algorithm to process the image data to obtain an image that meets the requirements of the optimal shooting mode and shooting style.

[0185] For example, in the Supermoon mode, the image processing algorithm optimizes the telephoto shooting algorithm to enhance the details of the moon; for the fireworks mode, the image processing algorithm uses a trajectory prediction algorithm to capture the dynamic effects of fireworks; and for the city night scene mode, the image processing algorithm uses a multi-frame synthesis algorithm to improve the image quality in low-light environments.

[0186] Combination Figure 5 The camera Hal calls the scene-based processing algorithm corresponding to the best shooting mode and the night scene stylization processing algorithm corresponding to the shooting style to process the image data to be processed, and obtain an image that meets the requirements of the best shooting mode and the shooting style.

[0187] In some embodiments, the camera Hal obtains the optimal shooting mode and shooting parameters for a night scene based on user data, location information, weather information, city information, time information, and the scene category corresponding to the image. The camera Hal receives the enable result, and the electronic device uses the shooting parameters corresponding to the optimal shooting mode to acquire image data. This image data is then used as the image data to be processed. The camera Hal can also use the algorithm corresponding to the optimal shooting mode to process the image data to be processed to obtain an image that meets the requirements of the optimal shooting mode.

[0188] S413, Camera Hal sends images to the camera application that meet the requirements of the optimal shooting mode and shooting style.

[0189] In some embodiments, the camera Hal sends image data of images requiring optimal shooting mode and shooting style to the camera application via a camera service.

[0190] S414 The camera application displays images that meet the requirements of the optimal shooting mode and shooting style in the camera preview interface.

[0191] The camera application receives an image that meets the requirements of the optimal shooting mode and style. It can then replace the image displayed in the camera preview with this second image, which meets the same requirements. The user can then input shooting commands in the camera preview to capture an image that meets the optimal shooting mode and style requirements.

[0192] In some embodiments, the camera application may also display a capsule in the camera preview interface. The capsule is a floating ellipse or rounded rectangle that can present various night scene scenes and night scene styles. Users can use the capsule to switch the image displayed in the camera preview interface to other night scene scenes or to images of other night scene styles, i.e., a third image.

[0193] Combination Figure 5 The camera preview interface displays images that meet the requirements of the optimal shooting mode and shooting style. It also displays capsules. The camera application adjusts the interaction through night scene décor and stylization processing to determine the night scene or night scene style that the user inputs through the capsule. The camera Hal can process the images that meet the requirements of the optimal shooting mode and shooting style into images that meet the night scene or night scene style that the user inputs through the capsule. The processed images can be returned to the camera application, which then displays the processed images, i.e., the night scene décor and stylized images, in the camera preview interface.

[0194] For example, the camera Hal processes images that meet the requirements of the optimal shooting mode and shooting style into images that meet the user's input via the capsule, in the following way: image data is acquired using shooting parameters corresponding to the switched night scene or night scene style, and the image data is processed using the algorithm corresponding to the switched night scene or night scene style to obtain an image that meets the switched night scene or night scene style.

[0195] Optionally, the electronic device can also update the night scene recommendation algorithm and the night scene style recommendation algorithm based on the operation information of the switched night scene scene or night scene style input by the user through the capsule. The operation information may include at least image data, the night scene scene before switching, the night scene style, and the night scene scene or night scene style after switching.

[0196] In this embodiment, the electronic device is equipped with multiple night scene scenes and styles, enriching the night scene options and enhancing the ability to distinguish complex night scene details. It can accurately differentiate between night scene types with significant differences, such as supermoons, fireworks displays, and urban light landscapes. It can also recognize different night scene types, including distant urban views, street scenes, and blues scenes. In particular, the multiple night scene styles provide users with diverse visual experiences. After the user launches the camera application, the electronic device enters the camera preview interface and, based on user data, system information, and image data, determines the optimal shooting mode and style for the night scene. This eliminates the need for complex manual operations, preserving the professional-grade depth of night scene rendering while freeing the user's hands.

[0197] More importantly, this solution achieves a leapfrog upgrade in night scene shooting from functional to intelligent by constructing a technical closed loop of scene perception, intelligent recommendation, and personalized rendering, bringing users a smarter and more efficient shooting experience.

[0198] like Figure 6 As shown in the embodiments of this application, another image data processing method includes:

[0199] S601: The camera responds to user operations, acquires image data, and sends the image data to the camera application via the camera Hal.

[0200] The implementation method of step S601 can be found in the content of step S401 above, and will not be repeated here.

[0201] S602, the camera application displays a camera preview interface based on image data.

[0202] The camera preview interface can display the image corresponding to the image data, i.e., the first image.

[0203] S603: The sensor sends ambient light intensity and location information to the camera application.

[0204] The implementation method of step S603 can be found in the content of step S403 above, and will not be repeated here.

[0205] S604, the camera application obtains user data, weather information, time information, and city information.

[0206] The implementation method of step S604 can be found in the content of step S404 above, and will not be repeated here.

[0207] S605: The camera application sends ambient light intensity, location information, user data, time information, city information, and weather information to the camera Hal.

[0208] The implementation method of step S605 can be found in the content of step S405 above, and will not be repeated here.

[0209] The S606 and camera Hal extract features from image data to construct the feature vectors required for scene recognition.

[0210] The implementation method of step S606 can be found in the content of step S406 above, and will not be repeated here.

[0211] S607 and camera Hal obtain the scene category corresponding to the image based on the feature vector of the image data and the ambient light intensity.

[0212] The implementation method of step S607 can be found in the content of step S407 above, and will not be repeated here.

[0213] The S608 camera's Hal sensor uses user data, location information, weather information, city information, time information, and the scene category corresponding to the image to determine the optimal shooting mode and style for night scenes.

[0214] The implementation method of step S608 can be found in the aforementioned step S408, and will not be repeated here.

[0215] The S609 camera Hal sends the best shooting mode and style for night scenes to the camera app.

[0216] The implementation method of step S609 can be found in the aforementioned step S409, and will not be repeated here.

[0217] The S610 camera application displays prompts and receives enable results on the camera preview screen.

[0218] The implementation method of step S610 can be found in the aforementioned step S410, and will not be repeated here.

[0219] S611, The camera application sends an enable result to the camera Hal.

[0220] The implementation method of step S611 can be found in the aforementioned step S411, and will not be repeated here.

[0221] The S612 camera Hal sends the optimal shooting mode identifier, shooting style identifier, and image data for night scenes to the cloud side via camera service and DMS.

[0222] The image data is obtained from the image captured by the camera's Hal after receiving an enable result. Based on this enable result, the camera determines the optimal shooting mode and style for capturing the image, and then acquires the image data currently being captured by the camera. Optionally, the camera can acquire image data using the shooting parameters corresponding to the optimal shooting mode and style.

[0223] In this embodiment, the edge device is used to obtain the optimal shooting mode and shooting style for the night scene, and the cloud-side device is used to process the image data to obtain an image that meets the requirements of the optimal shooting mode and shooting style. To this end, after receiving the enable result, the camera Hal can send the optimal shooting mode identifier, shooting style identifier, and image data for the night scene to the cloud side through the camera service and DMS.

[0224] S613 processes image data on the cloud side to obtain images that meet the requirements of the optimal shooting mode and shooting style.

[0225] Cloud-side devices can also be configured with various shooting modes and image processing algorithms for different shooting styles. The cloud-side devices can use the algorithms corresponding to the best shooting mode and shooting style to process image data and obtain images that meet the requirements of the best shooting mode and shooting style.

[0226] In this embodiment, since the image data processing is performed by the cloud-side device, which has stronger computing power and larger storage space, it can support more complex image processing algorithms and obtain images with better image quality.

[0227] S614, the cloud side sends images that meet the requirements of the best shooting mode and shooting style to the camera application through DMS and camera service.

[0228] The S615 camera application displays images that meet the requirements of the optimal shooting mode and shooting style in the camera preview interface.

[0229] The implementation method of step S615 can be found in the aforementioned step S414, and will not be repeated here. The image that meets the requirements of the optimal shooting mode and shooting style is the second image.

[0230] In this embodiment, the camera Hal sends the optimal shooting mode identifier, shooting style identifier, and image data for night scene to the cloud via camera service and DMS, which can ensure that user data does not leave the cloud and avoid data privacy issues caused by user data leakage.

[0231] In some embodiments, after the camera Hal receives the enable result, in addition to performing step S612, it can also process image data to obtain an image that meets the requirements of the optimal shooting mode and shooting style. Thus, the electronic device can obtain both the image obtained by the camera Hal that meets the requirements of the optimal shooting mode and shooting style, and the image from the cloud that meets the requirements of the optimal shooting mode and shooting style, resulting in two images that meet the requirements of the optimal shooting mode and shooting style. The electronic device can save both images and display the image of higher quality (usually the image from the cloud) in the camera application.

[0232] When a user inputs an image comparison operation, the camera application can display two images that meet the optimal shooting mode and shooting style requirements respectively in the camera preview interface. In one implementation, the camera preview interface displays one image that meets both requirements; the user inputs a switch image operation, and the camera application responds by displaying another image that meets both requirements in the camera preview interface. In another implementation, the camera application responds to the user's image comparison operation by displaying two images that meet both requirements in separate areas of the camera preview interface.

[0233] Optionally, when a user inputs an image capture operation, the electronic device can capture and save two images that meet the optimal shooting mode and shooting style requirements. When the user browses an image, they can input an image comparison operation to display two images that meet the optimal shooting mode and shooting style requirements on the image browsing interface. In one implementation, the image browsing interface displays one image that meets the optimal shooting mode and shooting style requirements. When the user inputs a switch image operation, the gallery application responds to the switch operation and displays the image browsing interface for another image that meets the optimal shooting mode and shooting style requirements. In another implementation, the camera application responds to the user's input image comparison operation and displays two images that meet the optimal shooting mode and shooting style requirements.

[0234] In other embodiments, to further enhance the diversity of image shooting modes and styles, the camera Hal obtains user data, system information, image data, and the scene category corresponding to the image. It can also send this data to the cloud side via camera service and DMS. The cloud-side device, based on user data, location information, weather information, city information, time information, and the scene category corresponding to the image, determines the optimal shooting mode and style for night scenes. Furthermore, the cloud-side device sends the optimal shooting mode and style identifier to the camera application via DMS and camera service. The camera application displays a prompt and receives an enable result on the camera preview interface. When the user confirms using the optimal shooting mode and style to capture an image, the camera application sends an enable result to the cloud side via camera service and DMS. The cloud side processes the image data to obtain an image that meets the requirements of the optimal shooting mode and style, and then sends the image to the camera application via DMS and camera service. The camera application can then display the image that meets the requirements of the optimal shooting mode and style on the camera preview interface.

[0235] Optionally, the scene category corresponding to the image can also be obtained by the cloud-side device. In this way, after the camera application or camera Hall obtains user data, system information, ambient light intensity, and image data, it can send these data to the cloud side via the camera service and DMS. The cloud-side device then extracts features from the image data to construct the feature vector required for scene recognition. Based on the feature vector and ambient light intensity, the scene category corresponding to the image is obtained. For detailed implementation methods, please refer to steps S406 and S407 above, which will not be repeated here.

[0236] In other embodiments, in the aforementioned scheme where the camera application displays a capsule on the camera preview interface, the user can use the capsule to switch the image displayed on the camera preview interface to other night scene scenes or images with other night scene styles. These other night scene scenes or images with different night scene styles can also be obtained from the cloud-side device. In one implementation, the user inputs a switching operation through the capsule. The camera application can send image data, along with the identifier of the night scene scene or night scene style of the image switched by the user, to the cloud-side device via the camera service and DMS. The cloud-side device processes the image data using the algorithm corresponding to the night scene scene or night scene style of the image switched by the user, obtains the image after the scene or style switch, and then sends it to the camera application via DMS and the camera service, where it is displayed on the camera preview interface.

[0237] In conjunction with the foregoing, the electronic device's gallery application provides an entry point for modifying image styles, allowing users to change the style of existing images. For example, these existing images can be images captured by the electronic device using the solution provided in this application's embodiments, or images captured by the electronic device using other methods; this application does not limit the scope of such limitations.

[0238] The scheme to modify the image style can be executed independently by the edge device, or it can be executed using an "edge-cloud combination" architecture, which includes edge devices and cloud devices. Figure 7 This diagram illustrates the process of modifying image style using an end-side device. Figure 8 The diagram illustrates the process of modifying the "edge-cloud integration" architecture using an image style.

[0239] like Figure 7 As shown, the image data processing method provided in this application embodiment includes:

[0240] S701, the gallery application responds to the first operation by displaying the image browsing interface.

[0241] For example, the first operation is an image browsing operation, which may be an operation of selecting an image in the interface of a gallery application, and the gallery application responds to the operation by displaying an image browsing interface for the selected image; the image browsing operation may also be an operation of clicking a button that links to the gallery application in the camera preview interface, and the gallery application responds to the operation by displaying an image browsing interface for the image previously captured by the electronic device.

[0242] Optionally, the image browsing interface displays the image selected by the user, as well as multiple buttons, one of which is an edit button for image editing, including but not limited to filter modification.

[0243] S702, the Gallery app responds to the second operation by displaying a list of filters.

[0244] The second operation is to confirm and modify the filter on the image. For example, the user can click the edit button, and the gallery application will display the image editing interface. This interface displays the image and also includes a filter modification button. The user clicks the filter modification button to complete the second operation. For example, in response to the second operation, the gallery application displays a filter list in the image editing interface. This filter list may include various filters, each corresponding to a specific shooting style. Optionally, the filter list may include various traditional filters, such as grayscale and black-and-white filters, as well as various night scene style filters, such as cyberpunk, black-and-gold, and soft-focus filters for urban landscapes, retro styles like Chungking Express and CineStill 800T for alleyways, and blues styles and sunrise pastel styles for blues scenes.

[0245] S703, the gallery application responds to the third operation by sending the filter identifier and image data selected in the third operation to the camera Hal.

[0246] The third operation is the selection of a filter. For example, the user clicks on a filter displayed in the filter list to select the filter. In response to the selection operation, the gallery application sends the filter identifier and image data selected in the third operation to the camera Hal.

[0247] In some embodiments, the night scene stylization processing algorithm of the camera Hal can be made available to the gallery application. The gallery application can call the night scene stylization processing algorithm of the camera Hal through the application framework layer to send the filter identifier and image data selected by the third operation to the camera Hal.

[0248] In other embodiments, the hardware abstraction layer may also configure a separate night scene stylization algorithm for the gallery application. When a user selects a filter, the gallery application, in response to this selection, can call the night scene stylization algorithm through the application framework layer to process the image data and obtain an image that meets the filter requirements. In this embodiment, the electronic device may not execute steps S704 and S705. The hardware abstraction layer configures a separate night scene stylization algorithm for the gallery application, obtains an image that meets the filter requirements, and sends the image that meets the filter requirements to the gallery application through the application framework layer.

[0249] The S704 camera Hal processes image data to obtain images that meet the filter requirements.

[0250] The camera Hal uses the corresponding filter identifier in the image stylization processing algorithm to process the image data and obtain an image that meets the filter requirements.

[0251] S705, Camera Hal sends images that meet the filter requirements to the gallery application.

[0252] The camera Hal sends images that meet the filter requirements to the gallery application through the application framework layer.

[0253] S706, the gallery application displays images that meet the filter requirements.

[0254] The gallery application displays images that meet the filter requirements in the image editing interface, that is, it replaces the original image with an image that meets the filter requirements.

[0255] In this embodiment, a variety of night scene styles are displayed to the user through a filter list, which can enrich the filter effects of night scene images. Users can switch the filters of night scene images without using complicated manual operations, thus improving the user experience.

[0256] like Figure 8 As shown in the embodiments of this application, another image data processing method includes:

[0257] S801, the gallery application responds to the first operation by displaying the image browsing interface.

[0258] The implementation method of step S801 can be found in the content of step S701 above, and will not be repeated here.

[0259] S802, the Gallery app responds to the second operation by displaying a list of filters.

[0260] The implementation method of step S802 can be found in the content of step S702 mentioned above, and will not be repeated here.

[0261] S803, the gallery application responds to the third operation by sending the filter identifier and image data selected in the third operation to the DMS.

[0262] For details on the third operation, please refer to the aforementioned step S703, which will not be repeated here.

[0263] In some embodiments, the gallery application sends the filter identifier and image data selected by the third operation to the DMS through a module in the application framework layer.

[0264] S804 and DMS send the filter identifier and image data selected in the third operation to the cloud side.

[0265] S805 processes image data on the cloud side to obtain images that meet the filter requirements.

[0266] The cloud-side device is equipped with a variety of processing algorithms. It can use the processing algorithm corresponding to the filter selected in the third operation to process the number of images and obtain images that meet the filter requirements.

[0267] S806: The cloud side sends images that meet the filter requirements to the gallery application via DMS.

[0268] In some embodiments, the cloud side sends images that meet the filter requirements to the gallery application via modules of DMS and the application framework layer.

[0269] S807, the gallery application displays images that meet the filter requirements.

[0270] The gallery application displays images that meet the filter requirements in the image editing interface, that is, it replaces the original image with an image that meets the filter requirements.

[0271] In this embodiment, a filter list displays various night scene styles to the user, enriching the filter effects for night scene images. Users can switch filters for night scene images without cumbersome manual operations, improving the user experience. Furthermore, cloud-based devices process image data to obtain images that meet filter requirements. Because cloud-based devices have stronger computing power and larger storage space, they can support more complex image processing algorithms, providing users with richer image filter effects, enhancing the emotion and atmosphere of the image, and improving its expressiveness.

[0272] In other embodiments, after receiving the third operation, in addition to executing step S803, the gallery application can also execute steps S703 to S705 with the camera Hal, or call the night scene stylization processing algorithm through the application framework layer to process the image data and obtain an image that meets the filter requirements. The night scene stylization processing algorithm is a hardware abstraction layer or can be configured separately for the gallery application. In this way, the gallery application can also obtain two images that meet the filter requirements. The gallery application can display the image of higher quality between the two images, which is usually the image from the cloud side.

[0273] Users can also control an electronic device to display two images by inputting comparison operations. For example, a gallery app displays an image that meets the filter requirements; the user inputs a switch image operation, and the camera app responds by displaying another image that meets the filter requirements. As another example, the gallery app displays an image that meets the filter requirements; the user inputs a switch image operation, and the camera app responds by displaying two images that meet the filter requirements, separated by region.

[0274] Another embodiment of this application provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps of any of the above methods.

[0275] Computer-readable storage media can be non-transitory computer-readable storage media, such as read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage devices.

[0276] Another embodiment of this application provides a computer program product containing instructions. When the computer program product is run on a computer or processor, it causes the computer or processor to perform one or more steps of any of the methods described above.

[0277] It should be understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0278] In summary, the above description is merely a preferred embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for processing image data, characterized in that, Applied to an electronic device, the electronic device including a camera, the method includes: In response to the camera application launch operation, the camera is activated to acquire image data and a camera preview interface is displayed; wherein, the camera preview interface displays a first image, which corresponds to the image data acquired by the camera; Based on the image data captured by the camera, the scene category corresponding to the image data is obtained; When the scene category is a night scene, the shooting mode and shooting style of the night scene are obtained based on user data and system information; the user data includes the user's travel preferences, and the system information includes the weather information, location information and current time of the user's current location; A second image is displayed on the camera preview interface, and the second image meets the requirements of the shooting mode and the shooting style.

2. The method according to claim 1, characterized in that, The process of obtaining the shooting mode and shooting style for the night scene based on user data and system information includes: Based on the user data and the system information, the shooting mode for the night scene is obtained; When the night scene shooting mode includes multiple shooting styles, the shooting style of the night scene is obtained based on the user data and the system information.

3. The method according to claim 1 or 2, characterized in that, Before displaying the second image on the camera preview interface, the method further includes: The image data captured by the camera is processed to obtain the image data of the second image; the camera captures the image data based on the shooting mode and shooting style of the night scene. And / or, acquire image data of the second image sent by the cloud-side device.

4. The method according to any one of claims 1 to 3, characterized in that, Before displaying the second image on the camera preview interface, the method further includes: A floating window is displayed on the camera preview interface to indicate the shooting mode and shooting style for the night scene; The step of displaying the second image on the camera preview interface includes: in response to a user's enable operation, displaying the second image on the camera preview interface, wherein the enable operation is used to confirm that the image is captured using the night scene shooting mode and shooting style.

5. The method according to claim 4, characterized in that, Also includes: The camera preview interface displays a capsule that includes various night scene scenes and / or various shooting styles for night scene scenes; In response to the user's selection of a night scene or shooting style, a second image is displayed on the camera preview interface, and the third image satisfies the requirements of the night scene or shooting style selected by the user.

6. The method according to claim 2, characterized in that, The process of obtaining the shooting mode for the night scene based on the user data and the system information includes: Obtain the fusion features of the user data and the system information; The first classification model is invoked based on the fusion features to obtain the shooting mode of the night scene; wherein, the first classification model is trained using the user profile of the training samples of the first classification model, and the user profile of the training samples of the first classification model is constructed based on the image's dot information, analysis of historical shooting time distribution, common shooting mode preferences and geographical location heat map.

7. The method according to claim 2 or 6, characterized in that, The process of obtaining the shooting style of the night scene based on the user data and the system information includes: Obtain the fusion features of the user data and the system information; The second classification model is invoked based on the fusion features to obtain the shooting style of the night scene; wherein, the second classification model is trained using the user profile of the training samples of the second classification model, and the user profile of the training samples of the second classification model is constructed based on the image dot information, the distribution of historical shooting time periods, the preference for commonly used filter types, and the geographical location heat map.

8. The method according to any one of claims 1 to 7, characterized in that, Also includes: In response to the image capture operation, the second image is saved; In response to an image browsing operation of the second image, an image browsing interface for the second image is displayed, the image browsing interface including an edit button; In response to the click operation of the edit button, the image editing interface is displayed. The image editing interface displays a filter list, which includes a variety of filters. Each filter corresponds to a shooting style, and the styles include a variety of shooting styles for night scenes. Based on the filter selected by the user and the image data of the second image, an image that meets the requirements of the filter selected by the user is displayed in the image editing interface.

9. The method according to claim 8, characterized in that, The method, based on the user-selected filter and the image data of the second image, before displaying an image that meets the requirements of the user-selected filter in the image editing interface, further includes: The image data of the second image is processed to obtain image data of an image that meets the filter requirements selected by the user; And / or, receive image data from a cloud-side device that meets the filter requirements selected by the user.

10. An electronic device, characterized in that, include: One or more processors, memory, cameras, and displays; The memory, the camera, and the display screen are coupled to one or more of the processors. The memory stores computer program code, which includes computer instructions. When the one or more processors execute the computer instructions, the electronic device performs the image data processing method as described in any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that, Used to store computer programs, which, when executed, are specifically used to implement the image data processing method as described in any one of claims 1 to 9.

12. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the image data processing method as described in any one of claims 1 to 9.

Images