Data processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a data processing method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the steps that first input of a user for a shooting control in a shooting preview interface is received; in response to the first input, storing the collected first image in a first storage space; generating a summary score according to the first score of the electronic equipment in the first time period and a second score of the electronic equipment in the second time period; wherein the first score is a score of an equipment state of the electronic equipment in a first time period, the second score is a score of an estimated equipment operation state of the electronic equipment executing an image processing task in a second time period, and the first time period is earlier than the second time period; and when the summary score is greater than or equal to a preset threshold value, reading the first image from the first storage space, and performing image processing on the first image to obtain a second image.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and specifically relates to a data processing method, device, electronic device and storage medium. Background Art

[0002] The camera system in electronic devices such as mobile phones and tablets not only needs to capture high-quality images, but also needs to perform complex image processing. Since resources such as processors and memory in electronic devices are limited, how to use these resources to optimize the performance of the camera system has become a focus of attention.

[0003] In the related art, the camera system tightly couples image data acquisition and image algorithm processing. This coupling method is prone to slow memory allocation due to resource competition when processing complex tasks, causing the image algorithm processing process to be stuck and the image processed by the image algorithm to be unable to be saved in time. As a result, the resource utilization rate in the electronic device is low, affecting the performance of the camera system. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a data processing method, device, electronic device and storage medium, which can improve resource utilization in electronic devices and optimize the performance of camera systems.

[0005] In a first aspect, an embodiment of the present application provides a data processing method, including:

[0006] Receive a first input from a user to a shooting control in a shooting preview interface;

[0007] In response to a first input, storing the acquired first image in a first storage space;

[0008] Generate a summary score according to a first score of the electronic device in a first time period and a second score in a second time period; wherein the first score is a score of a device state of the electronic device in the first time period, and the second score is a score of an estimated device operating state of the electronic device performing an image processing task in the second time period, and the first time period is earlier than the second time period;

[0009] When the summary score is greater than or equal to the preset threshold, the first image is read from the first storage space, and image processing is performed on the first image to obtain a second image.

[0010] In a second aspect, an embodiment of the present application provides a data processing system, including a user interaction module, a frame capture module and a control module, wherein a first storage space is set in the frame capture module; wherein,

[0011] A user interaction module, used to receive a first input from a user to a shooting control in a shooting preview interface;

[0012] The frame capture module is used to receive a first input from a user to a shooting control in a shooting preview interface; in response to the first input, capture a first image, and store the first image in a first storage space;

[0013] A control module, configured to generate a summary score based on a first input and according to a first score of the electronic device in a first time period and a second score in a second time period, wherein the first score is a score of a device state of the electronic device in the first time period, and the second score is a score of an estimated device operating state of the electronic device performing an image processing task in the second time period, and the first time period is earlier than the second time period; and when the summary score is greater than or equal to a preset threshold, send a first instruction to the frame capture module;

[0014] The frame capture module is used to read the first image from the first storage space based on the first instruction, perform image processing on the first image, and obtain the second image.

[0015] In a third aspect, an embodiment of the present application provides a data processing device, including:

[0016] A receiving module, used to receive a first input from a user to a shooting control in a shooting preview interface;

[0017] A storage module, configured to store the acquired first image into a first storage space in response to a first input;

[0018] A generating module, configured to generate a summary score according to a first score of the electronic device in a first time period and a second score in a second time period; wherein the first score is a score of a device state of the electronic device in the first time period, and the second score is a score of an estimated device operating state of the electronic device performing an image processing task in the second time period, and the first time period is earlier than the second time period;

[0019] The reading module is used to read the first image from the first storage space and perform image processing on the first image to obtain the second image when the summary score is greater than or equal to a preset threshold.

[0020] In a fourth aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and when the program or instruction is executed by the processor, the steps of the data processing method shown in the first aspect are implemented.

[0021] In a fifth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the data processing method shown in the first aspect are implemented.

[0022] In a sixth aspect, an embodiment of the present application provides a chip, the chip including a processor and a display interface, the display interface and the processor are coupled, and the processor is used to run programs or instructions to implement the steps of the data processing method shown in the first aspect.

[0023] In a seventh aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the steps of the data processing method shown in the first aspect.

[0024] In an embodiment of the present application, in response to a first input of a user to a shooting control in a shooting preview interface, a captured first image is stored in a first storage space; then, a summary score is generated based on a first score of the electronic device in a first time period and a second score of the electronic device in a second time period, wherein the first score is a score of a device state of the electronic device in the first time period, and the second score is a score of an estimated device operating state of the electronic device performing an image processing task in the second time period, and the first time period is earlier than the second time period; then, when the summary score is greater than or equal to a preset threshold, the first image is read from the first storage space, and image processing is performed on the first image to obtain a second image. In this way, the process of acquiring images and the process of executing image processing tasks can be separated. By summarizing the score of the device status of the electronic device in the first time period and the score of the estimated device operating status when executing the image processing task in the second time period, the resource situation in the electronic device for executing the image processing task is fully considered, and the first image will only be processed when the summarized score is greater than or equal to a preset threshold, which indicates that the electronic device has the amount of resources required to execute the image processing task. Compared with the method of performing image processing immediately after acquiring the first image, this method can effectively avoid congestion and jamming of the image processing process caused by concentrated resource demands, and the problem of being unable to save images processed by the image algorithm in time, and ensure the amount of resources required to execute the image processing task. While improving the resource utilization rate in the electronic device, it can effectively improve the performance of the camera system. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A structural diagram of a data processing system;

[0026] Figure 2 A schematic diagram of the structure of a data processing system provided for some embodiments of the present application;

[0027] Figure 3 A schematic diagram of the structure of a data processing system provided for some embodiments of the present application;

[0028] Figure 4 A flowchart of a data processing method provided for some embodiments of the present application;

[0029] Figure 5 A schematic diagram of the structure of a data processing device provided for some embodiments of the present application;

[0030] Figure 6 A schematic diagram of the structure of an electronic device provided for some embodiments of the present application;

[0031] Figure 7 A schematic diagram of the hardware structure of an electronic device provided for some embodiments of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0033] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0034] In related technologies, camera systems usually tightly couple the image data acquisition stage and the image algorithm processing stage, such as Figure 1 As shown, the data processing system includes a user interaction module 101, a frame capture module 102 and an algorithm processing module 103. Based on the data processing system, after the user interaction module 101 receives the input of the image taken by the user, the frame capture module 102 will collect the original image, that is, the RAW image, and call the image processing algorithm in the algorithm processing module 103 to perform image processing on the RAW image, that is, convert the RAW image into a YUV image, and then call the image processing algorithm in the algorithm processing module 103 to perform image processing on the YUV image. In this way, the YUV image can be converted into an image in JPEG (Joint Photographic Experts Group) format, and then the image in JPEG format is returned to the user interaction module 101 for the user to view.

[0035] From this we can see that Figure 1In the data processing system shown, after the frame capture module 102 captures the original image, it immediately follows the process of performing image processing on the original image. This coupling may lead to waste of resources or bottlenecks in the electronic device when processing complex tasks. It is easy to have a phenomenon in which memory allocation is slow due to resource competition, and the original image cannot be processed in time or the image processed by the image algorithm cannot be saved in time, resulting in a decrease in camera system performance. Moreover, as the complexity of the image processing algorithm increases and the computing power requirements increase, this phenomenon becomes more and more obvious. The image data acquisition stage and the image algorithm processing stage are often affected by both system load and real-time performance.

[0036] Therefore, in order to optimize the performance of the camera system, improve resource utilization in electronic devices, and ensure data processing efficiency, the present application is an embodiment that provides a system, method, device and electronic device that separates image data acquisition and image algorithm processing and can reasonably perform data processing tasks.

[0037] Based on this, the embodiments of the present application provide a data processing system, method, device and electronic device. Figures 2 to 4 , the data processing system and method provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0038] First, combine Figure 2 A data processing system provided in an embodiment of the present application is described in detail.

[0039] Figure 2 A schematic diagram of the structure of a data processing system provided for some embodiments of the present application.

[0040] like Figure 2 As shown, the data processing system 20 provided in the embodiment of the present application can be set in an electronic device.

[0041] Based on this, in some embodiments of the present application, the data processing system 20 includes a user interaction module 201, a frame capture module 202 and a control module 203, and a first storage space is set in the frame capture module 202, wherein the first storage space can be a universal flash storage (Universal Flash Storage, ufs), which is a storage system that encapsulates Nand Flash and Flash controller protocol interfaces together.

[0042] Based on this, the user interaction module 201 is used to receive a first input from a user to a shooting control in a shooting preview interface; the frame capture module 202 is used to receive a first input from a user to a shooting control in a shooting preview interface; in response to the first input, a first image is captured and the first image is stored in a first storage space; the control module is used to generate a summary score based on the first input, according to a first score of the electronic device in a first time period and a second score in a second time period, wherein the first score is a score of the device status of the electronic device in the first time period, and the second score is a score of the estimated device operating status of the electronic device performing an image processing task in the second time period, and the first time period is earlier than the second time period; when the summary score is greater than or equal to a preset threshold, a first instruction is sent to the frame capture module; the frame capture module 202 is used to read the first image from the first storage space based on the first instruction, perform image processing on the first image, and obtain the second image.

[0043] Therefore, the process of acquiring images and the process of executing image processing tasks can be separated by adding the control module 203 and the first storage space. Specifically, by storing the acquired first image in the first storage space, the process of image processing immediately after the first image is acquired can be effectively avoided. In addition, the control module 203 summarizes the score of the device status of the electronic device in the first time period and the score of the estimated device operating status of the image processing task in the second time period, so as to fully consider the resource situation of the electronic device for executing the image processing task, and the first image will be processed only when the summarized score is greater than or equal to the preset threshold, which indicates that the electronic device has the amount of resources required to execute the image processing task. Compared with the method of performing image processing immediately after the first image is acquired, the image processing process congestion and jamming caused by concentrated resource demand and the problem of being unable to save the image processed by the image algorithm in time can be effectively avoided, the amount of resources required to execute the image processing task is guaranteed, the flexible scheduling of image processing tasks is improved, and the camera system performance can be effectively improved while improving the resource utilization rate in the electronic device.

[0044] Combine the following Figure 2 , the data processing system 20 in the embodiment of the present application is described in detail.

[0045] The user interaction module 201 may include functions of a camera application and functions of an album application.

[0046] Among them, the functions of the camera application may include displaying an interface for interacting with the user, such as a shooting preview interface, an image display interface, etc., and receiving interactive operations of the user with the interface.

[0047] Alternatively, if Figure 3As shown, the user interaction module 201 can also be divided into a user interaction module for the camera application and a user interaction module for the album application. Among them, the user interaction module for the camera application can be used for the camera preview and shooting operation. The user interaction module for the album application can be used for the image viewed by the user in the album application, which needs urgent priority processing, so that the final effect image after algorithm optimization can be quickly presented to the user. In this way, the priority of the image processing task of the image viewed by the user can be improved through interaction with the user.

[0048] The frame grabbing module 202 can be used for image data capture and caching to ensure the integrity and real-time performance of the image data. Specifically, according to the camera usage scenario requirements, such as High Dynamic Range Imaging (HDRI or HDR) scenes, night scenes, portrait scenes, sports scenes, etc., a sufficient number of RAW images that meet the requirements can be obtained from the camera sensor. The frame grabbing module 202 can obtain these RAW images through the camera standard interface, and collect and organize the reference frame selection and image data and related parameters in a targeted manner, and save the collected RAW images to the first storage space for static storage for subsequent algorithm processing. This process does not perform overload algorithm processing, thereby ensuring the real-time smoothness of the camera preview.

[0049] The control module 203 can be used to summarize the score of the device status of the electronic device in the first time period, and the score of the estimated device operating status of the electronic device in the second time period to perform the image processing task, to obtain the summary score of the electronic device to perform the image processing task in the next time, and then, based on the summary score, to determine whether the electronic device can perform the image processing task in the next time. In this way, by comprehensively evaluating whether the electronic device can perform the image processing task through data of multiple categories and multiple time periods, the accuracy of determining the timing of triggering the execution of the image processing task can be improved, and the image processing process congestion caused by concentrated resource demand can be effectively avoided. The problem of being unable to save the image processed by the image algorithm in time, the amount of resources required to perform the image processing task is guaranteed, and the resource utilization rate in the electronic device can be improved while effectively improving the performance of the camera system.

[0050] In addition, in some embodiments of the present application, the data processing system 20 may further include a second storage space 204. The second storage space 204 is used to store the second image that has been processed for the user to view.

[0051] In addition, in some embodiments of the present application, the data processing system 20 may further include an algorithm processing module 205. Specifically, the algorithm processing module 205 may be used to provide the frame capture module 202 with an image processing algorithm for performing image processing on the first image, and the image processing algorithm may include any image processing algorithm required by any frame capture module 202, such as an image basic image quality processing algorithm, an effect algorithm, and an encoding processing algorithm for images in JPG format.

[0052] It should be noted that the algorithm processing module 205 can also asynchronously perform image processing on the acquired first image through the image processing algorithm after the frame capture module 202, so as to avoid interference with the real-time capture process.

[0053] The following is a detailed description of the module functions of the modules in the data processing system 20 when they cooperate with each other to execute the data processing process, as shown below.

[0054] In some embodiments of the present application, the control module is specifically used to obtain device status data of the electronic device within a first time period, wherein the device status data includes at least one of the following: device power, device load, device memory, device temperature, device charging status parameters, and device screen-off status parameters; and generate a first score based on the device status data and the importance value corresponding to the device status data.

[0055] Exemplarily, a weighted sum may be performed on the at least two device status data and the importance value corresponding to each of the at least two device status data, and the value obtained by the weighted sum may be determined as the first score.

[0056] And, obtaining first device operation data of the electronic device within a third time period, the first device operation data including at least one of the following: computing power for performing an image processing task, user behavior data related to the image processing task, and device energy consumption for performing the image processing task; based on the first device operation data, determining second device operation data within a second time period, the second device operation data including at least one of the following: estimated power for performing the image processing task, estimated user behavior data related to the image processing task, and estimated device energy consumption for performing the image processing task; generating a second score according to the second device operation data and the importance value corresponding to the second device operation data.

[0057] In this way, the control module can determine whether to trigger the frame capture module to perform the image processing task based on the summary score of the first score and the second score, fully considering the resource situation in the electronic device for performing the image processing task. This can effectively avoid congestion and jamming of the image processing process caused by concentrated resource demands, and the inability to save images processed by the image algorithm in time, and ensure the amount of resources required to perform the image processing task. While improving the resource utilization in the electronic device, it can effectively improve the performance of the camera system.

[0058] In some embodiments of the present application, the control module is further configured to, when the summary score is less than a preset threshold, send a second instruction to the frame capture module;

[0059] The frame capture module is used to perform at least one of the following operations based on the second instruction: stop the operation of reading the first image from the first storage space, and suspend the operation of image processing on the first image.

[0060] Here, since the summary score is less than the preset threshold, it means that the electronic device is not suitable for performing the image processing task in the next time. At this time, the operations of stopping reading the first image and pausing the image processing of the first image can be performed until the summary score is greater than or equal to the preset threshold, and then the image processing task can be performed.

[0061] In some embodiments of the present application, the data processing system 20 further includes a second storage space, which is a resident memory in the electronic device;

[0062] The frame capture module is also used to send the second image to the second storage space;

[0063] A second storage space is used to store a second image and send the second image to the user interaction module;

[0064] The user interaction module is used to switch the display from the shooting preview interface to the image display interface, and the image display interface is used to display the second image. It should be noted that the image display interface in the embodiment of the present application can be understood as the interface of the album application, that is, it is used to display the image that has been stored in the album application, and the image is an image that has been processed.

[0065] In this way, when it is determined that the electronic device can currently perform the image processing task, after the user clicks on the input for taking the image, the user does not need to click on the input for viewing the taken image, and the operation of capturing the first image and continuously displaying the second image after the first image is processed can be achieved. This effectively avoids the congestion and jamming of the image processing process caused by concentrated resource demands, and the problem of being unable to save the image processed by the image algorithm in time, ensures the amount of resources required to perform the image processing task, and reduces the user's operation of viewing images while improving resource utilization in the electronic device.

[0066] In some embodiments of the present application, the embodiments of the present application can also provide users with a function of previewing RAW images, and the user can select which RAW images need urgent priority processing. In this way, while ensuring the amount of resources required to perform image processing tasks and improving resource utilization in electronic devices, the user can be quickly presented with the effect image required by the user after image processing. Here, it can be achieved by interacting with the user to increase the priority of the image processing task of the image seen by the user. Based on this, the user interaction module is also used to receive the user's second input for viewing the image control in the shooting preview interface when the first image includes at least two first images.

[0067] The frame capture module is also used to, in response to a second input, read at least two first images from the first storage space; and send at least two first images to the user interaction module.

[0068] The user interaction module is also used to switch the display from the shooting preview interface to the image preview interface, where the image preview interface is used to display at least two first images; receive a third input from the user to select a target image from the at least two first images; and send the target image to the frame capture module in response to the third input. It should be noted that the image preview interface in the embodiment of the present application can be used to display images that are not stored in the album application, and is still waiting for the user to select whether to perform image processing or whether to store it in the album application. The frame capture module is also used to perform image processing on the target image to obtain a second image.

[0069] It should be noted that, in this embodiment, after the user selects the target image, the user interaction module may also send the target image to the control module 203. The control module 203 generates a summary score of the electronic device executing the processing of the target image according to the first score of the electronic device executing the image processing task in the first time period and the second score of the image processing task expected to be executed in the second time period for the target image. When the summary score is greater than or equal to the preset threshold, the control module 203 sends a first instruction to the frame capture module to trigger the frame capture module to perform image processing on the target image to obtain a second image, and then displays the second image required by the user through the user interaction module 201.

[0070] In this way, the amount of resources required to execute image processing tasks can be guaranteed, and while the resource utilization in electronic devices is improved, the effect images required by users and after image processing can be quickly presented to users, thereby improving the efficiency of users taking images using electronic devices.

[0071] Based on the above-mentioned data processing system, the following Figure 4 A data processing method provided in an embodiment of the present application is described in detail.

[0072] Figure 4 A flowchart of a data processing method provided for some embodiments of the present application.

[0073] like Figure 4 As shown, the data processing method provided in the embodiment of the present application can be applied to Figure 2 or Figure 3 The data processing system shown, based on this, the data processing method may include steps 410 to 440, as specifically shown below.

[0074] Step 410, receiving a first input from a user to a shooting control in a shooting preview interface;

[0075] Step 420, in response to the first input, storing the acquired first image in a first storage space;

[0076] Step 4300, generating a summary score according to a first score of the electronic device in a first time period and a second score in a second time period, wherein the first score is a score of a device state of the electronic device in the first time period, and the second score is a score of an estimated device operating state of the electronic device performing an image processing task in the second time period, and the first time period is earlier than the second time period;

[0077] Step 440 , when the summary score is greater than or equal to the preset threshold, read the first image from the first storage space, and perform image processing on the first image to obtain a second image.

[0078] In this way, the process of acquiring images and the process of executing image processing tasks can be separated. By summarizing the score of the device status of the electronic device in the first time period and the score of the estimated device operating status when executing the image processing task in the second time period, the resource situation in the electronic device for executing the image processing task is fully considered, and the first image will only be processed when the summarized score is greater than or equal to a preset threshold, which indicates that the electronic device has the amount of resources required to execute the image processing task. Compared with the method of performing image processing immediately after acquiring the first image, this method can effectively avoid congestion and jamming of the image processing process caused by concentrated resource demands, and the problem of being unable to save images processed by the image algorithm in time, and ensure the amount of resources required to execute the image processing task. While improving the resource utilization in the electronic device, the performance of the camera system can be effectively improved.

[0079] The above steps are described in detail below, as shown below.

[0080] First, step 410 is involved. In some embodiments of the present application, the shooting preview interface in the embodiments of the present application may be an interface in a camera application, or an interface in an application that provides a shooting function in an electronic device. In this way, the user can click on the shooting control in the shooting preview interface to achieve image shooting.

[0081] Receiving involves step 420, and the captured first image can be stored in the first storage space, thereby effectively avoiding the process of performing image processing immediately after the first image is captured. Compared with the method of performing image processing immediately after the first image is captured, the congestion and jamming of the image processing process caused by concentrated resource demands and the problem of being unable to save the image processed by the image algorithm in time can be effectively avoided, thereby improving the flexible scheduling of image processing tasks.

[0082] In addition, in some embodiments of the present application, the first image includes at least two first images. Based on this, after step 420, the data processing method may further include steps 4601 to 4606, as specifically shown below.

[0083] Step 4601: receiving a second input from the user to a control for viewing an image in the shooting preview interface.

[0084] Step 4602: In response to a second input, read at least two first images from a first storage space.

[0085] Step 4603: Switch the display from the shooting preview interface to the image preview interface, where the image preview interface is used to display at least two first images.

[0086] Step 4604: Receive a third input from a user selecting a target image from at least two first images.

[0087] Step 4605, in response to the third input, performing image processing on the target image to obtain a second image;

[0088] Step 4606, display the second image on the image preview interface.

[0089] In this way, users can be provided with the function of previewing RAW images and select which RAW images need urgent priority processing. In this way, the amount of resources required to execute image processing tasks can be guaranteed, while the resource utilization rate in electronic devices is improved, and the user can be quickly presented with the desired effect image after image processing.

[0090] Furthermore, with respect to step 430, in some embodiments of the present application, step 430 may specifically include step 4301 and step 4302 to determine the first score, as shown below.

[0091] Step 4301, obtaining device status data of the electronic device in a first time period, wherein the device status data includes at least one of the following: device power, device load, device memory, device temperature, device charging status parameters, and device screen-off status parameters.

[0092] Exemplarily, the device power is recorded as A, the device load is recorded as B, the device memory is recorded as C, the device temperature is recorded as D, the device charging status parameter is recorded as E, and the device screen-off status parameter is recorded as F.

[0093] Step 4302: Generate a first score based on the device status data and the importance value corresponding to the device status data.

[0094] Exemplarily, the importance value corresponding to the device power is recorded as WA, the importance value corresponding to the device load is recorded as WB, the importance value corresponding to the device memory is recorded as WC, the importance value corresponding to the device temperature is recorded as WD, the importance value corresponding to the device charging state parameter is recorded as WE, and the importance value corresponding to the device screen off state parameter is recorded as WF. Based on this, the first score Score1 can be calculated by the following formula (1):

[0095] Score1=A*WA+B*WB+C*WC+D*WD+E*WD+F*WF (1)

[0096] It should be noted that the above formula (1) can adjust the number of objects participating in the weighted summation according to different device status data.

[0097] In some embodiments of the present application, step 430 may specifically include steps 4303 to 4305 to determine the second score, as shown below.

[0098] Step 4303, obtaining first device operation data of the electronic device within a third time period, where the first device operation data includes at least one of the following: computing power for performing image processing tasks, user behavior data related to the image processing tasks, and device energy consumption for performing image processing tasks.

[0099] The value range of the third time period may be the value range of the first time period. In this way, the first device operation data may be obtained while the device status data is obtained, which shortens the time for calculating the summary score and improves the efficiency of image processing.

[0100] Step 4304: Determine the second device operation data in the second time period based on the first device operation data, wherein the second device operation data includes at least one of the following: estimated power to perform the image processing task, estimated user behavior data related to the image processing task, and estimated device energy consumption to perform the image processing task.

[0101] Exemplarily, the estimated power G for performing an image processing task, the estimated user behavior data H related to the image processing task, and the estimated device energy consumption I for performing the image processing task.

[0102] Step 4305: Generate a second score based on the second device operation data and the importance value corresponding to the second device operation data.

[0103] For example, based on the estimated effort WG for performing the image processing task, the estimated user behavior data WH related to the image processing task, and the estimated device energy consumption WI for performing the image processing task, the second score Score2 can be calculated by the following formula (2):

[0104] Score2=G*WG+H*WH+I*WI (2)

[0105] It should be noted that the above formula (2) can adjust the number of objects participating in the weighted summation according to different equipment operation data.

[0106] Based on this, in the embodiment of the present application, the sum of the first score and the second score can be used as the summary score.

[0107] For example, still taking the above-mentioned examples from step 4301 to step 4305 as an example for explanation, the summary score in the embodiment of the present application can be calculated by the following formula (3):

[0108] Total score = Score1 + Score2 (3)

[0109] In this way, in step 440, when the summary score is greater than or equal to the preset threshold, the first image is read from the first storage space, and image processing is performed on the first image to obtain the second image.

[0110] Exemplarily, if the summary score is 20, which is less than the preset threshold of 80, it means that the electronic device is not suitable for performing image processing tasks in the next time, and at least one of the following operations can be performed: stop reading the first image from the first storage space, and suspend the image processing operation of the first image.

[0111] If the summary score is 90, which is greater than the preset threshold of 80, it means that the electronic device is suitable for performing image processing tasks in the next time and there will be no device lag. At this time, the first image is read from the first storage space, and the first image is processed to obtain the second image.

[0112] In addition, in some embodiments of the present application, the data processing method may further include step 450, and when the summary score is less than a preset threshold, perform at least one of the following operations: stop reading the first image from the first storage space, and suspend image processing of the first image.

[0113] Here, since the summary score is less than the preset threshold, it means that the electronic device is not suitable for performing the image processing task in the next time. At this time, the operations of stopping reading the first image and pausing the image processing of the first image can be performed until the summary score is greater than or equal to the preset threshold, and then the image processing task can be performed.

[0114] In addition, in some embodiments of the present application, after step 440, the data processing method may further include steps 4601 and 4602.

[0115] Step 4601, storing the second image in a second storage space, where the second storage space is a resident memory in the electronic device.

[0116] Step 4602: Switch the display from the shooting preview interface to the image display interface, where the image display interface is used to display the second image read from the second storage space.

[0117] In this way, the process of acquiring images and the process of executing image processing tasks can be separated. By summarizing the scores of electronic devices executing image processing tasks, the resource conditions in the electronic device for executing image processing tasks can be fully considered. Compared with the method of performing image processing immediately after acquiring the first image, the congestion and jamming of the image processing process caused by concentrated resource demands and the inability to save images processed by image algorithms in time can be effectively avoided. The amount of resources required to execute image processing tasks is guaranteed, and the camera system performance can be effectively improved while improving the resource utilization rate in electronic devices.

[0118] It should be noted that the data processing method provided in the embodiments of the present application can be executed by electronic devices such as mobile phones, tablet computers, laptop computers, PDAs, wearable devices, etc. In some embodiments of the present application, the data processing method provided in the embodiments of the present application is described by taking the electronic device as the execution subject to execute the data processing method as an example.

[0119] The data processing method provided in the embodiment of the present application can be executed by a data processing device. In the embodiment of the present application, a data processing device executing the data processing method is taken as an example to illustrate the device of the data processing method provided in the embodiment of the present application.

[0120] The present application also provides a data processing device. Figure 5 Provide detailed explanation.

[0121] Figure 5 A schematic diagram of the structure of a data processing device provided for some embodiments of the present application.

[0122] like Figure 5 As shown, the data processing device 50 can be applied to an electronic device, and the data processing device 50 can specifically include:

[0123] The receiving module 501 is used to receive a first input from a user to a shooting control in a shooting preview interface;

[0124] The storage module 502 is used to store the collected first image in a first storage space in response to the first input;

[0125] A generating module 503 is used to generate a summary score according to a first score of the electronic device in the first time period and a second score in the second time period; wherein the first score is a score of the device state of the electronic device in the first time period, and the second score is a score of the estimated device operation state of the electronic device performing the image processing task in the second time period, and the first time period is earlier than the second time period;

[0126] The reading module 504 is configured to read the first image from the first storage space and perform image processing on the first image to obtain a second image when the summary score is greater than or equal to a preset threshold.

[0127] The data processing device 50 in the embodiment of the present application is described in detail below, as shown below.

[0128] In some embodiments of the present application, the data processing device 50 may further include an acquisition module, which is used to acquire device status data of the electronic device in the first time period, wherein the device status data includes at least one of the following: device power, device load, device memory, device temperature, device charging status parameter, and device screen off status parameter;

[0129] The generating module 503 is used to generate a first score according to the device status data and the importance value corresponding to the device status data.

[0130] In some embodiments of the present application, the data processing device 50 may further include an acquisition module, which is used to acquire first device operation data of the electronic device in a third time period, where the first device operation data includes at least one of the following: computing power for performing an image processing task, user behavior data related to the image processing task, and energy consumption of a device for performing an image processing task;

[0131] The data processing device 50 may further include a determination module, configured to determine second device operation data within a second time period based on the first device operation data, the second device operation data including at least one of the following: an estimated effort to perform the image processing task, estimated user behavior data related to the image processing task, and an estimated device energy consumption to perform the image processing task;

[0132] The generating module 503 may also be configured to generate a second score according to the second device operating data and the importance value corresponding to the second device operating data.

[0133] In some embodiments of the present application, the data processing device 50 may further include an execution module for performing at least one of the following operations when the summary score is less than a preset threshold: stopping the operation of reading the first image from the first storage space, and suspending the operation of image processing on the first image.

[0134] In some embodiments of the present application, the data processing device 50 may further include a display module; wherein,

[0135] The storage module 502 is further used to store the second image in a second storage space, where the second storage space is a resident memory in the electronic device;

[0136] The display module is used to switch the display from the shooting preview interface to the image display interface, and the image display interface is used to display the second image read from the second storage space.

[0137] In some embodiments of the present application, the receiving module 501 may also be used to receive a second input of a user to a view image control in the shooting preview interface when the first image includes at least two first images;

[0138] The reading module 504 may also be configured to, in response to a second input, read at least two first images from the first storage space;

[0139] The data processing device 50 may further include a display module, which is used to switch the display from the shooting preview interface to the image preview interface, and the image preview interface is used to display at least two first images;

[0140] The receiving module 501 may also be used to receive a third input from a user selecting a target image from at least two first images;

[0141] The data processing device 50 may further include a processing module, configured to perform image processing on the target image in response to the third input to obtain a second image;

[0142] The display module may also be used to display the second image on the image preview interface.

[0143] The data processing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0144] The data processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0145] The data processing device provided in the embodiment of the present application can achieve the following Figure 2 The various processes implemented in the data processing method embodiment shown achieve the same technical effect, and will not be described again here to avoid repetition.

[0146] Based on this, the data processing device provided in the embodiment of the present application stores the captured first image in the first storage space in response to the user's first input to the shooting control in the shooting preview interface; then, based on the first score of the electronic device in the first time period and the second score in the second time period, generates a summary score, wherein the first score is the score of the device status of the electronic device in the first time period, and the second score is the score of the estimated device operating status of the electronic device performing the image processing task in the second time period, and the first time period is earlier than the second time period; then, when the summary score is greater than or equal to the preset threshold, the first image is read from the first storage space, and image processing is performed on the first image to obtain the second image. In this way, the process of acquiring images and the process of executing image processing tasks can be separated. By summarizing the score of the device status of the electronic device in the first time period and the score of the estimated device operating status when executing the image processing task in the second time period, the resource situation in the electronic device for executing the image processing task is fully considered, and the first image will only be processed when the summarized score is greater than or equal to a preset threshold, which indicates that the electronic device has the amount of resources required to execute the image processing task. Compared with the method of performing image processing immediately after acquiring the first image, this method can effectively avoid congestion and jamming of the image processing process caused by concentrated resource demands, and the problem of being unable to save images processed by the image algorithm in time, and ensure the amount of resources required to execute the image processing task. While improving the resource utilization rate in the electronic device, it can effectively improve the performance of the camera system.

[0147] Optional, such as Figure 6 As shown, an embodiment of the present application also provides an electronic device 60, including a processor 601 and a memory 602, wherein the memory 602 stores programs or instructions that can be executed on the processor 601, and when the program or instructions are executed by the processor 601, the various steps of the above-mentioned data processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0148] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0149] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0150] The electronic device 700 includes but is not limited to: a radio frequency unit 701, a network module 702, an audio output unit 703, an input unit 704, a sensor 705, a display unit 706, a user input unit 707, an interface unit 708, a memory 709, a processor 710 and other components.

[0151] Those skilled in the art will appreciate that the electronic device 700 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 710 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system.

[0152] Figure 7 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0153] Among them, in an embodiment of the present application, the processor 710 is used to receive a first input from a user to a shooting control in a shooting preview interface. The processor 710 can also be used to store the captured first image in a first storage space in response to the first input. The processor 710 can also be used to generate a summary score based on a first score of the electronic device in a first time period and a second score in a second time period, wherein the first score is a score of the device state of the electronic device in the first time period, and the second score is a score of the estimated device operating state of the electronic device performing an image processing task in the second time period, and the first time period is earlier than the second time period. The processor 710 can also be used to read the first image from the first storage space and perform image processing on the first image to obtain a second image when the summary score is greater than or equal to a preset threshold.

[0154] The electronic device 700 is described in detail below, as shown below.

[0155] In some embodiments of the present application, the processor 710 is configured to obtain device status data of the electronic device in a first time period, wherein the device status data includes at least one of the following: device power, device load, device memory, device temperature, device charging status parameter, and device screen-off status parameter;

[0156] A first score is generated based on the device status data and the importance value corresponding to the device status data.

[0157] In some embodiments of the present application, the processor 710 is configured to obtain first device operation data of the electronic device in a third time period, where the first device operation data includes at least one of the following: computing power for performing an image processing task, user behavior data related to the image processing task, and energy consumption of a device for performing the image processing task;

[0158] Determining, based on the first device operation data, second device operation data within a second time period, the second device operation data comprising at least one of the following: an estimated effort to perform the image processing task, an estimated user behavior data related to the image processing task, and an estimated device energy consumption to perform the image processing task;

[0159] A second score is generated according to the second device operation data and the importance value corresponding to the second device operation data.

[0160] In some embodiments of the present application, the processor 710 is used to perform at least one of the following operations when the summary score is less than a preset threshold: stop reading the first image from the first storage space, and suspend image processing of the first image.

[0161] In some embodiments of the present application, the processor 710 is configured to store the second image in a second storage space, where the second storage space is a resident memory in the electronic device;

[0162] The display unit 706 is used to switch the display from the shooting preview interface to the image display interface, and the image display interface is used to display the second image read from the second storage space.

[0163] In some embodiments of the present application, the user input unit 707 is used to receive a second input of a user to a view image control in the shooting preview interface when the first image includes at least two first images;

[0164] The processor 710 is configured to read at least two first images from the first storage space in response to a second input;

[0165] A display unit 706, used to switch the display from the shooting preview interface to the image preview interface, where the image preview interface is used to display at least two first images;

[0166] The user input unit 707 may also be used to receive a third input from a user selecting a target image from at least two first images;

[0167] The processor 710 may also be configured to, in response to a third input, perform image processing on the target image to obtain a second image;

[0168] The display unit 706 may also be configured to display the second image on the image preview interface.

[0169] It should be understood that the input unit 704 may include a graphics processing unit (GPU) 7041 and a microphone 7042, and the graphics processor 7041 processes the image data of a static image or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 706 may include a display panel, and the display panel may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 707 includes a touch panel 7071 and at least one of other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 may include two parts: a touch detection device and a touch display. Other input devices 7072 may include, but are not limited to, a physical keyboard, function keys (such as a volume display button, a switch button, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0170] The memory 709 can be used to store software programs and various data. The memory 709 can mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area can store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 709 can include a volatile memory or a non-volatile memory, or the memory 709 can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 709 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0171] The processor 710 may include one or more processing units; optionally, the processor 710 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless display signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 710.

[0172] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned data processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0173] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0174] In addition, an embodiment of the present application further provides a chip, which includes a processor and a display interface, the display interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned data processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0175] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0176] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned data processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0177] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0178] In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.

[0179] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0180] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A data processing method, applied to an electronic device, characterized in that: include: Receive a first input from a user to a shooting control in a shooting preview interface; In response to the first input, storing the acquired first image in a first storage space; Generate a summary score according to a first score of the electronic device in a first time period and a second score in a second time period; wherein the first score is a score of a device state of the electronic device in the first time period, and the second score is a score of an estimated device operating state of the electronic device performing an image processing task in the second time period, and the first time period is earlier than the second time period; When the summary score is greater than or equal to a preset threshold, the first image is read from the first storage space, and image processing is performed on the first image to obtain a second image.

2. The method according to claim 1, characterized in that The method further comprises: Acquire device status data of the electronic device in the first time period, wherein the device status data includes at least one of the following: device power, device load, device memory, device temperature, device charging status parameter, and device screen-off status parameter; The first score is generated according to the device status data and the importance value corresponding to the device status data.

3. The method according to claim 1, characterized in that The method further comprises: Acquire first device operation data of the electronic device in a third time period, where the first device operation data includes at least one of the following: computing power for performing an image processing task, user behavior data related to the image processing task, and energy consumption of a device for performing the image processing task; Determining, based on the first device operation data, second device operation data within the second time period, the second device operation data comprising at least one of the following: an estimated effort to perform an image processing task, estimated user behavior data related to the image processing task, and an estimated device energy consumption to perform the image processing task; The second score is generated according to the second device operation data and the importance value corresponding to the second device operation data.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: When the summary score is less than the preset threshold, at least one of the following operations is performed: stopping the operation of reading the first image from the first storage space, and pausing the operation of image processing on the first image.

5. The method according to claim 1, characterized in that The method further comprises: storing the second image in a second storage space, where the second storage space is a resident memory in the electronic device; The display is switched from the shooting preview interface to an image display interface, where the image display interface is used to display the second image read from the second storage space.

6. The method according to claim 1, characterized in that The first images include at least two first images; After storing the acquired first image in the first storage space in response to the first input, the method further includes: Receiving a second input from a user on a control for viewing an image in the shooting preview interface; In response to the second input, reading the at least two first images from the first storage space; Switching the display from the shooting preview interface to an image preview interface, where the image preview interface is used to display the at least two first images; receiving a third input of a user selecting a target image from the at least two first images; In response to the third input, performing image processing on the target image to obtain the second image; The second image is displayed on the image preview interface.

7. A data processing system, characterized in that: It includes a user interaction module, a frame capture module and a control module, wherein a first storage space is set in the frame capture module; wherein, The user interaction module is used to receive a first input from a user to a shooting control in the shooting preview interface; The frame capture module is used to capture a first image in response to the first input, and store the first image in the first storage space; The control module is configured to generate a summary score based on the first input and according to a first score of the electronic device in a first time period and a second score in a second time period, wherein the first score is a score of a device state of the electronic device in the first time period, and the second score is a score of an estimated device operating state of the electronic device performing an image processing task in the second time period, and the first time period is earlier than the second time period; and send a first instruction to the frame capture module when the summary score is greater than or equal to a preset threshold; The frame capture module is used to read the first image from the first storage space based on the first instruction, perform image processing on the first image, and obtain a second image.

8. The system according to claim 7, characterized in that The control module is specifically configured to obtain device status data of the electronic device within the first time period, wherein the device status data includes at least one of the following: device power, device load, device memory, device temperature, device charging status parameter, and device screen-off status parameter; and generate the first score according to the device status data and the importance value corresponding to the device status data; And, obtaining first device operation data of the electronic device within a third time period, the first device operation data including at least one of the following: computing power for performing an image processing task, user behavior data related to the image processing task, and device energy consumption for performing the image processing task; based on the first device operation data, determining second device operation data within the second time period, the second device operation data including at least one of the following: estimated power for performing an image processing task, estimated user behavior data related to the image processing task, and estimated device energy consumption for performing the image processing task; generating the second score according to the second device operation data and the importance value corresponding to the second device operation data.

9. The system according to claim 7, characterized in that The control module is further configured to send a second instruction to the frame capture module when the summary score is less than the preset threshold; The frame capture module is used to perform at least one of the following operations based on the second instruction: stop the operation of reading the first image from the first storage space, and suspend the operation of image processing on the first image.

10. The system according to claim 7, characterized in that The data processing system further includes a second storage space, wherein the second storage space is a resident memory in the electronic device; The frame capture module is further used to send the second image to the second storage space; The second storage space is used to store the second image and send the second image to the user interaction module; The user interaction module is used to switch the display from the shooting preview interface to the image display interface, and the image display interface is used to display the second image.

11. The system according to claim 10, characterized in that The user interaction module is further configured to, when the first image includes at least two first images, receive a second input from a user on a control for viewing an image in the shooting preview interface; The frame capture module is further used for, in response to the second input, reading the at least two first images from the first storage space; and sending the at least two first images to the user interaction module; The user interaction module is further configured to switch the display from the shooting preview interface to an image preview interface, wherein the image preview interface is configured to display the at least two first images; receive a third input from a user selecting a target image from the at least two first images; and send the target image to the frame capture module in response to the third input; The frame capture module is also used to perform image processing on the target image to obtain the second image.

12. A data processing device, applied to an electronic device, characterized in that: include: A receiving module, used to receive a first input from a user to a shooting control in a shooting preview interface; A storage module, configured to store the acquired first image into a first storage space in response to the first input; A generating module, configured to generate a summary score according to a first score of the electronic device in a first time period and a second score in a second time period; wherein the first score is a score of a device state of the electronic device in the first time period, and the second score is a score of an estimated device operating state of the electronic device performing an image processing task in the second time period, and the first time period is earlier than the second time period; A reading module is used to read the first image from the first storage space and perform image processing on the first image to obtain a second image when the summary score is greater than or equal to a preset threshold.

13. An electronic device, characterized in that: include: A processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the data processing method according to any one of claims 1 to 6.

Images