Data processing method and device, electronic equipment and storage medium

Abstract

The application discloses a data processing method and device, electronic equipment and storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: receiving a first input of a user on a shooting control in a shooting preview interface; storing a collected first image into a first storage space in response to the first input; generating a summary score according to a first score of the electronic equipment within a first time period and a second score of the electronic equipment within a second time period; wherein the first score is a score of the equipment state of the electronic equipment within the first time period, the second score is a score of the estimated equipment running state of the electronic equipment in executing an image processing task within the second time period, and the first time period is earlier than the second time period; and in the case that the summary score is greater than or equal to a preset threshold, reading the first image from the first storage space and performing image processing on the first image to obtain a second image.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and particularly relates to a data processing method and device, electronic equipment and a storage medium. BACKGROUND

[0002] The camera system in electronic equipment such as mobile phones and tablet computers not only needs to capture high-quality images, but also needs to perform complex image processing processes. Since the processor, memory and other resources in the electronic equipment are limited, how to utilize these resources to optimize the performance of the camera system has become the focus.

[0003] In the related art, the camera system tightly couples image data acquisition and image algorithm processing together. This coupling mode is prone to slow memory allocation due to resource competition when processing complex tasks, causing the image algorithm processing process to be stuck, and the image processed by the image algorithm cannot be saved in time. As a result, the resource utilization rate in the electronic equipment is low, and the performance of the camera system is affected. SUMMARY

[0004] The embodiments of the application provide a data processing method and device, electronic equipment and a storage medium, which can improve the resource utilization rate in the electronic equipment and optimize the performance of the camera system.

[0005] In a first aspect, the embodiments of the application provide a data processing method, comprising:

[0006] receiving a first input of a user on a shooting control in a shooting preview interface;

[0007] in response to the first input, storing a first image collected to a first storage space;

[0008] generating a summary score according to a first score of the electronic equipment in a first time period and a second score of the electronic equipment in a second time period; the first score is a score of the device state of the electronic equipment in the first time period, the second score is a score of the estimated device running state of the electronic equipment in the second time period, and the first time period is earlier than the second time period;

[0009] in a case where the summary score is greater than or equal to a preset threshold, reading the first image from the first storage space and performing image processing on the first image to obtain a second image.

[0010] In a second aspect, the embodiments of the application provide a data processing system, comprising a user interaction module, a frame grabbing module and a control module, and the frame grabbing module is provided with a first storage space; wherein,

[0011] the user interaction module is configured to receive a first input of a user on a shooting control in a shooting preview interface;

[0012] The frame grabbing module is configured to receive a first input of a user on a shooting control in a shooting preview interface; in response to the first input, collect a first image and store the first image in a first storage space;

[0013] The control module is configured to generate a summary score based on the first input, according to a first score of the electronic device in a first time period and a second score of the electronic device in a second time period, the first score being a score of a device state of the electronic device in the first time period, the second score being a score of an estimated device running state of the electronic device in the second time period for performing an image processing task, the first time period being earlier than the second time period; and send a first instruction to the frame grabbing module in a case where the summary score is greater than or equal to a preset threshold.

[0014] The frame grabbing module is configured to read the first image from the first storage space based on the first instruction, perform image processing on the first image, and obtain a second image.

[0015] In a third aspect, an embodiment of the present application provides a data processing apparatus, comprising:

[0016] The receiving module is configured to receive a first input of a user on a shooting control in a shooting preview interface;

[0017] The storage module is configured to store the collected first image in a first storage space in response to the first input;

[0018] The generation module is configured to generate a summary score according to a first score of the electronic device in a first time period and a second score of the electronic device in a second time period; wherein the first score is a score of a device state of the electronic device in the first time period, the second score is a score of an estimated device running state of the electronic device in the second time period for performing an image processing task, and the first time period is earlier than the second time period.

[0019] The reading module is configured to read the first image from the first storage space and perform image processing on the first image to obtain a second image in a case where the summary score is greater than or equal to a preset threshold.

[0020] In a fourth aspect, an embodiment of the present application provides an electronic device, which comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and the program or instruction is executed by the processor to implement the steps of the data processing method shown in the first aspect.

[0021] In a fifth aspect, an embodiment of the present application provides a readable storage medium, and the readable storage medium stores a program or instruction, and the program or instruction is executed by a processor to implement the steps of the data processing method shown in the first aspect.

[0022] In a sixth aspect, an embodiment of the present application provides a chip, which comprises a processor and a display interface, the display interface being coupled with the processor, and the processor being configured to run programs or instructions to implement the steps of the data processing method according to the first aspect.

[0023] In a seventh aspect, an embodiment of the present application provides a computer program product stored in a storage medium, which is executed by at least one processor to implement the steps of the data processing method according to the first aspect.

[0024] In the embodiment of the present application, the first image collected is stored in the first storage space in response to the first input of the user to the shooting control in the shooting preview interface; then, the summary score is generated according to the first score of the electronic device in the first time period and the second score of the electronic device in the second time period, wherein the first score is the score of the device state of the electronic device in the first time period, the second score is the score of the estimated device running state of the electronic device in the second time period for executing the image processing task, and the first time period is earlier than the second time period; and then, the first image is read from the first storage space and the image processing is performed on the first image to obtain the second image in the case where the summary score is greater than or equal to the preset threshold. In this way, the process of collecting the image and the process of executing the image processing task can be separated, the score of the device state of the electronic device in the first time period and the score of the estimated device running state of the electronic device in the second time period for executing the image processing task are summarized, the resource situation in the electronic device for executing the image processing task is fully considered, and the first image is processed only when the summary score is greater than or equal to the preset threshold, that is, when the electronic device has the required resource amount for executing the image processing task. Compared with the way of performing the image processing immediately after the first image is collected, the image processing process congestion caused by the concentration of resource requirements can be effectively avoided, the problem that the image processed by the image algorithm cannot be saved in time cannot be solved, the required resource amount for executing the image processing task can be ensured, the resource utilization rate in the electronic device can be improved, and the camera system performance can be effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 FIG. 1 is a structural schematic diagram of a data processing system according to some embodiments of the present application;

[0026] Figure 2 FIG. 1 is a structural schematic diagram of a data processing system according to some embodiments of the present application;

[0027] Figure 3 FIG. 1 is a structural schematic diagram of a data processing system according to some embodiments of the present application;

[0028] Figure 4 FIG. 1 is a structural schematic diagram of a data processing system according to some embodiments of the present application;

[0029] Figure 5 A structural schematic diagram of a data processing apparatus provided for some embodiments of the present application is shown in FIG. 1.

[0030] Figure 6 A structural schematic diagram of an electronic device provided for some embodiments of the present application is shown in FIG. 2.

[0031] Figure 7 A hardware structural schematic diagram of an electronic device provided for some embodiments of the present application is shown in FIG. 3. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0033] The terms "first", "second", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by "first", "second", and the like are generally of a kind, and are not limited to the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / ", generally represents a "or" relationship between the front and rear associated objects.

[0034] In the related art, a camera system usually tightly couples an image data acquisition stage and an image algorithm processing stage together, as shown in FIG. 1. Figure 1 As shown in FIG. 1, a data processing system includes a user interaction module 101, a frame grabbing module 102, and an algorithm processing module 103. Based on the data processing system, after the user interaction module 101 receives the input of the user shooting image, the frame grabbing module 102 will collect the original image, i.e., the RAW image, and call the image processing algorithm in the algorithm processing module 103 to perform image processing on the RAW image, i.e., convert the RAW image into a YUV image, and then call the image processing algorithm in the algorithm processing module 103 to perform image processing on the YUV image. In this way, the YUV image can be converted into a JPEG (Joint Photographic Experts Group) format image, and then the JPEG format image is returned to the user interaction module 101 for the user to view.

[0035] Therefore, as shown in FIG. 2, the data processing system provided by the present application includes a user interaction module 201, a frame grabbing module 202, and an algorithm processing module 203. Figure 1In the data processing system shown, the process of performing image processing on the original image immediately follows the acquisition of the original image by the frame capture module 102. This coupling may lead to the waste of resources or bottlenecks in electronic devices when processing complex tasks. It is easy to cause slow memory allocation due to resource contention, and the inability to process the original image or save the image after image algorithm processing in a timely manner, resulting in a decline in camera system performance. Moreover, as the complexity of image processing algorithms increases and the computing power requirements increase, this phenomenon becomes more and more obvious. The image data acquisition stage and the image algorithm processing stage are often affected by both system load and real-time performance.

[0036] Therefore, in order to optimize camera system performance, improve resource utilization in electronic devices, and ensure data processing efficiency, this application provides a system, method, apparatus, and electronic device that separates image data acquisition and image algorithm processing and can reasonably perform data processing tasks.

[0037] Based on this, embodiments of this application provide a data processing system, method, apparatus, and electronic device. The following description, in conjunction with the appendix... Figures 2 to 4 The data processing system and method provided in this application will be described in detail through specific embodiments and application scenarios.

[0038] First, combined Figure 2 A data processing system provided in the embodiments of this application will be described in detail.

[0039] Figure 2 This is a schematic diagram of the structure of a data processing system provided for some embodiments of this application.

[0040] like Figure 2 As shown, the data processing system 20 provided in this application embodiment can be installed in an electronic device.

[0041] Based on this, in some embodiments of this application, the data processing system 20 includes a user interaction module 201, a frame capture module 202, and a control module 203. The frame capture module 202 is provided with a first storage space, which can be Universal Flash Storage (ufs), a storage system that encapsulates Nand Flash and Flash controller protocol interfaces together.

[0042] Based on this, the user interaction module 201 is used to receive the user's first input to the shooting control in the shooting preview interface; the frame capture module 202 is used to receive the user's first input to the shooting control in the shooting preview interface; in response to the first input, it captures a first image and stores the first image in a first storage space; the control module is used to generate a summary score based on the first input, according to the first score of the electronic device in a first time period and the second score in a second time period, wherein the first score is the score of the electronic device status in the first time period, and the second score is the score of the estimated device operating status of the electronic device performing image processing tasks in the second time period, and the first time period is earlier than the second time period; if the summary score is greater than or equal to a preset threshold, it sends a first instruction to the frame capture module; the frame capture module 202 is used to read the first image from the first storage space based on the first instruction, perform image processing on the first image, and obtain a second image.

[0043] Therefore, by adding a control module 203 and a first storage space, the process of acquiring images and the process of executing image processing tasks can be separated. Specifically, by storing the acquired first image in the first storage space, the image processing process is effectively avoided immediately after acquiring the first image. Furthermore, the control module 203 summarizes the device status score of the electronic device in the first time period and the estimated device operating status score of executing the image processing task in the second time period. This fully considers the resource situation in the electronic device for executing the image processing task. The first image will only be processed when the summarized score is greater than or equal to a preset threshold, indicating that the electronic device has the resources required to execute the image processing task. Compared with the method of performing image processing immediately after acquiring the first image, this method can effectively avoid the problem of image processing congestion and lag caused by concentrated resource demand, and the inability to save the image processed by the image algorithm in a timely manner. It ensures the amount of resources required to execute the image processing task, improves the flexible scheduling of image processing tasks, and effectively improves the performance of the camera system while improving the resource utilization rate in the electronic device.

[0044] The following is combined Figure 2 The data processing system 20 in the embodiments of this application will be described in detail.

[0045] The user interaction module 201 may include the functions of the camera application and the functions of the photo album application.

[0046] The camera application's functions may include displaying an interface for user interaction, such as a shooting preview interface or an image display interface, as well as receiving user interaction operations with that interface.

[0047] Optionally, such as Figure 3As shown, the user interaction module 201 can also be divided into a user interaction module of a camera application and a user interaction module of a photo album application. The user interaction module of the camera application can be used for preview and shooting operations of the camera. The user interaction module of the photo album application can be used for the image that the user views in the photo album application to be urgently prioritized for processing, so that the user can quickly be presented with the final effect image after algorithm optimization. In this way, the priority of the image processing task of the image seen by the user can be improved through interaction with the user.

[0048] The frame grabbing module 202 can be used for image data capture and caching to ensure the integrity and real-time performance of the image data. Specifically, RAW images of sufficient number of frames that meet the requirements of the camera usage scenarios, such as High Dynamic Range Imaging (HDRI or HDR) scenarios, night scene scenarios, portrait scenarios, motion scenarios, etc., can be obtained from the camera sensor. The frame grabbing module 202 can obtain these RAW images through a standard camera interface, and save the collected RAW images to the first storage space for static storage for subsequent algorithm processing, which does not do the reloaded algorithm processing, and ensures the real-time performance of the camera preview.

[0049] The control module 203 can be used to aggregate the score of the device state of the electronic device in the first time period, and the score of the estimated device running state of the electronic device in the second time period for performing the image processing task, to obtain the aggregated score of the electronic device for performing the image processing task in the next time, and then determine whether the electronic device can perform the image processing task in the next time based on the aggregated score. In this way, the accuracy of determining the timing of triggering the image processing task can be improved by comprehensively evaluating whether the electronic device can perform the image processing task based on multiple types and multiple time periods of data, which can effectively avoid the problem of image processing process congestion caused by concentrated resource demand, and cannot save the image processed by the image algorithm in time, and ensure the amount of resources required for performing the image processing task, which can improve the resource utilization rate of the electronic device while effectively improving the performance of the camera system.

[0050] In addition, in some embodiments of the present application, the data processing system 20 can further include a second storage space 204. The second storage space 204 is used to store the second image that has been processed by the image processing.

[0051] In addition, in some embodiments of the present application, the data processing system 20 can further include an algorithm processing module 205. Specifically, the algorithm processing module 205 can be configured to provide the grabbing module 202 with an image processing algorithm for image processing of the first image, which can include any image processing algorithm required by the grabbing module 202, such as an image basic quality processing algorithm, an effect algorithm, an encoding processing algorithm for a JPG format image, and the like.

[0052] It should be noted that the algorithm processing module 205 can also asynchronously process the captured first image through an image processing algorithm after the grabbing module 202, thereby avoiding interference with the real-time capture process.

[0053] The functions of each module in the data processing system 20 when cooperating to perform the data processing process will be described in detail below, as follows.

[0054] In some embodiments of the present application, the control module is specifically configured to obtain device state data of the electronic device within a first time period, wherein the device state data includes at least one of the following: device power, device load, device memory, device temperature, device charging state parameter, and device screen-off state parameter; and generate a first score according to the device state data and an importance value corresponding to the device state data.

[0055] For example, at least two device state data and an importance value corresponding to each of the at least two device state data can be weighted and summed, and the weighted and summed value is determined as the first score.

[0056] In addition, the first device running data of the electronic device within a third time period is obtained, and the first device running data includes at least one of the following: computing power for executing an image processing task, user behavior data related to the image processing task, and device energy consumption for executing the image processing task; based on the first device running data, second device running data within the second time period is determined, and the second device running data includes at least one of the following: estimated computing power for executing the image processing task, estimated user behavior data related to the image processing task, and estimated device energy consumption for executing the image processing task; and a second score is generated according to the second device running data and an importance value corresponding to the second device running data.

[0057] In this way, the control module can determine whether to trigger the frame grabbing module to perform the image processing task based on the aggregated score of the first score and the second score, fully considering the resource condition of the electronic device for performing the image processing task, which can effectively avoid the problem of image processing process congestion caused by concentrated resource demand, the inability to timely save the image processed by the image algorithm, and the like, and ensure the resource amount required for performing the image processing task, thereby improving the resource utilization of the electronic device and effectively improving the camera system performance.

[0058] In some embodiments of the present application, the control module is further configured to, in a case where the aggregated score is less than a preset threshold, send a second instruction to the frame grabbing module;

[0059] The frame grabbing module is configured to, based on the second instruction, perform at least one of the following operations: stopping reading the first image from the first storage space, and pausing image processing of the first image.

[0060] Here, since the aggregated score is less than the preset threshold, it indicates that the electronic device is not suitable for performing the image processing task in the next time, at this time, the operations of stopping reading the first image and pausing the image processing of the first image can be performed until the aggregated score is greater than or equal to the preset threshold, and then the image processing task is performed.

[0061] In some embodiments of the present application, the data processing system 20 further includes a second storage space, and the second storage space is a resident memory in the electronic device;

[0062] The frame grabbing module is further configured to send the second image to the second storage space.

[0063] The second storage space is configured to store the second image and send the second image to the user interaction module.

[0064] The user interaction module is configured to switch from displaying the shooting preview interface to displaying an image display interface, and the image display interface is configured to display the second image. It should be noted that the image display interface in the embodiments of the present application can be understood as the interface of the album application, that is, the interface for displaying the image that has been stored in the album application, and the image has been processed by the image processing.

[0065] In this way, when it is determined that the electronic device can currently perform the image processing task, after the user clicks the input of shooting the image, the operation of collecting the first image and continuously displaying the second image processed by the image processing of the first image can be implemented without the user clicking the input of viewing the shot image, which can effectively avoid the problem of image processing process congestion caused by concentrated resource demand, the inability to timely save the image processed by the image algorithm, and the like, and ensure the resource amount required for performing the image processing task, thereby improving the resource utilization of the electronic device and reducing the operation of the user viewing the image.

[0066] In some embodiments of the present application, the embodiments of the present application can also provide a user with a preview of a RAW image and a function of selecting which RAW pictures need to be processed in emergency priority, so that the user can be quickly presented with the desired effect picture after image processing while ensuring the amount of resources required to perform image processing tasks and improving resource utilization in the electronic device. Here, the priority of the image processing task of the picture seen by the user can be improved through interaction with the user. Based on this, the user interaction module is also used to receive a second input of the user on the view image control in the shooting preview interface when the first image includes at least two first images.

[0067] The frame grabbing module is also used to read the at least two first images from the first storage space in response to the second input; and send the at least two first images to the user interaction module.

[0068] The user interaction module is also used to switch the display from the shooting preview interface to an image preview interface, and the image preview interface is used to display the at least two first images; receive a third input of the user selecting a target image from the at least two first images; and send the target image to the frame grabbing module in response to the third input. It should be noted that the image preview interface in the embodiments of the present application can be used to display images that are not stored in the album application and are still waiting for the user to select whether to perform image processing or whether to store in the album application. The frame grabbing module is also used to perform image processing on the target image to obtain a second image.

[0069] It should be noted that in this embodiment, after the user selects the target image, the user interaction module can also send the target image to the control module 203. The control module 203 generates a summary score of the electronic device for processing the target image according to the first score of the electronic device performing the image processing task in the first time period and the second score of the electronic device performing the image processing task in the second time period, and in the case that the summary score is greater than or equal to a preset threshold, the control module 203 sends a first instruction to the frame grabbing module to trigger the frame grabbing module to perform image processing on the target image to obtain a second image, and then display the second image required by the user through the user interaction module 201.

[0070] Therefore, the user can be quickly presented with the desired effect picture after image processing while ensuring the amount of resources required to perform image processing tasks and improving resource utilization in the electronic device, thereby improving the efficiency of the user using the electronic device to take pictures.

[0071] Based on the above-mentioned data processing system, the following will be combined Figure 4 A data processing method provided by the embodiments of the present application will be described in detail.

[0072] Figure 4 A flowchart of a data processing method provided for some embodiments of the present application.

[0073] As shown in Figure 4 , the data processing method provided by the embodiments of the present application can be applied to a data processing system as shown in Figure 2 or Figure 3 , based on which the data processing method can include steps 410 to 440, as shown below.

[0074] Step 410, receiving a first input of a user on a shooting control in a shooting preview interface;

[0075] Step 420, in response to the first input, storing the collected first image to a first storage space;

[0076] Step 4300, generating a summary score according to a first score of the electronic device in a first time period and a second score of the electronic device in a second time period, wherein the first score is a score of the device state of the electronic device in the first time period, the second score is a score of the estimated device running state of the electronic device in the second time period for performing an image processing task, and the first time period is earlier than the second time period;

[0077] Step 440, in the case where the summary score is greater than or equal to a preset threshold, reading the first image from the first storage space and performing image processing on the first image to obtain a second image.

[0078] In this way, the process of collecting images and the process of performing image processing tasks can be separated, and by summarizing the score of the device state of the electronic device in the first time period and the estimated device running state of the electronic device in the second time period for performing an image processing task, the resource situation in the electronic device for performing the image processing task is fully considered, and the first image is processed only when the summary score is greater than or equal to the preset threshold, which means that the electronic device has the required amount of resources for performing the image processing task. Compared with the way of collecting the first image and then immediately performing image processing, the image processing process can be effectively avoided from being congested and jammed due to the concentration of resource demand, and the problem of being unable to save the image processed by the image algorithm in time can be solved, ensuring the required amount of resources for performing the image processing task, improving the resource utilization rate in the electronic device, and effectively improving the camera system performance.

[0079] The above steps are described in detail as follows.

[0080] Firstly, in relation to step 410, in some embodiments of the present application, the shooting preview interface in the embodiments of the present application can be an interface in a camera application or an interface in an application in the electronic device that provides a shooting function. In this way, the user can click the shooting control in the shooting preview interface to implement shooting of the image.

[0081] Receiving, in relation to step 420, can effectively avoid the process of image processing immediately after the first image is captured by storing the captured first image in the first storage space. Compared with the way of image processing immediately after the first image is captured, the image processing process congestion caused by the concentration of resource demand can be effectively avoided, and the problem that the image processed by the image algorithm cannot be saved in time can be solved, and the flexibility of image processing task scheduling is improved.

[0082] In addition, in some embodiments of the present application, the first image includes at least two first images, based on which, after step 420, the data processing method can further include steps 4601 to 4606, as shown below.

[0083] Step 4601, receiving a second input of the user to the viewing image control in the shooting preview interface.

[0084] Step 4602, in response to the second input, reading the at least two first images from the first storage space.

[0085] Step 4603, switching the display from the shooting preview interface to the image preview interface, and the image preview interface is used to display the at least two first images.

[0086] Step 4604, receiving a third input of the user selecting a target image from the at least two first images.

[0087] Step 4605, in response to the third input, performing image processing on the target image to obtain a second image;

[0088] Step 4606, displaying the second image in the image preview interface.

[0089] Thus, the user can be provided with a preview of the RAW image, and the function of selecting which RAW image needs to be processed urgently by the user, so that the required resource amount for executing the image processing task can be ensured, the resource utilization rate of the electronic device can be improved, and the user can quickly be presented with the effect picture required by the user after image processing.

[0090] Further, in relation to step 430, in some embodiments of the present application, step 430 can specifically include step 4301 and step 4302, and the first score is determined, as shown below.

[0091] Step 4301: Obtain device status data of the electronic device during the first time period, wherein the device status data includes at least one of the following: device power, device load, device memory, device temperature, device charging status parameters, and device screen off status parameters.

[0092] For example, the device power is denoted as A, the device load as B, the device memory as C, the device temperature as D, the device charging status parameter as E, and the device screen off status parameter as F.

[0093] Step 4302: Generate a first score based on the device status data and the importance value corresponding to the device status data.

[0094] For example, the importance value corresponding to the device's battery level is denoted as WA, the importance value corresponding to the device's load is denoted as WB, the importance value corresponding to the device's memory is denoted as WC, the importance value corresponding to the device's temperature is denoted as WD, the importance value corresponding to the device's charging status parameters is denoted as WE, and the importance value corresponding to the device's screen-off status parameters is denoted as WF. Based on this, the first score Score1 can be calculated using the following formula (1):

[0095] Score1=A*WA+B*WB+C*WC+D*WD+E*WD+F*WF (1)

[0096] It should be noted that the number of objects participating in the weighted summation can be adjusted according to the different equipment status data in the above formula (1).

[0097] In some embodiments of this application, step 430 may specifically include steps 4303 to 4305, determining a second score, as shown below.

[0098] Step 4303: Obtain the first device operation data of the electronic device during the third time period. The first device operation data includes at least one of the following: computing power for performing image processing tasks, user behavior data related to image processing tasks, and device energy consumption for performing image processing tasks.

[0099] The value range of the third time period can be the same as that of the first time period. This allows for the acquisition of both device status data and initial device operation data, shortening the time required to calculate and summarize the score and improving image processing efficiency.

[0100] Step 4304: Based on the first device operation data, determine the second device operation data for the second time period. The second device operation data includes at least one of the following: estimated power for performing the image processing task, estimated user behavior data related to the image processing task, and estimated device energy consumption for performing the image processing task.

[0101] Exemplarily, the estimated computing power G for performing the image processing task, the estimated user behavior data H related to the image processing task, and the estimated device energy consumption I for performing the image processing task.

[0102] At step 4305, a second score is generated according to the second device running data and the importance value corresponding to the second device running data.

[0103] Exemplarily, the estimated computing power WG for performing the image processing task, the estimated user behavior data WH related to the image processing task, and the estimated device energy consumption WI for performing the image processing task, based on which, the second score Score2 can be calculated by formula (2) as follows:

[0104] Score2 = G * WG + H * WH + I * WI (2)

[0105] It should be noted that the above formula (2) can adjust the number of objects participating in weighted summation according to different device running data.

[0106] Based on this, the sum of the first score and the second score can be used as the summary score in the embodiments of the present application.

[0107] Exemplarily, still taking the example in the above steps 4301 to 4305, the summary score in the embodiments of the present application can be calculated by formula (3) as follows:

[0108] Summary score = Score1 + Score2 (3)

[0109] Thus, at step 440, in the case that the summary score is greater than or equal to the preset threshold, the first image is read from the first storage space, and the image processing is performed on the first image to obtain the second image.

[0110] Exemplarily, if the summary score is 20, i.e., less than the preset threshold 80, it indicates that the electronic device is not suitable for performing the image processing task in the next time, and at least one of the following operations can be performed: stopping the operation of reading the first image from the first storage space, and pausing the operation of image processing on the first image.

[0111] If the summary score is 90, i.e., greater than the preset threshold 80, it indicates that the electronic device is suitable for performing the image processing task in the next time, and there is no device lag, at this time, the first image is read from the first storage space, and the image processing is performed on the first image to obtain the second image.

[0112] In addition, in some embodiments of the present application, the data processing method can further include step 450: in the case that the summary score is less than the preset threshold, performing at least one of the following operations: stopping reading the first image from the first storage space, and pausing image processing of the first image.

[0113] Here, since the summary score is less than the preset threshold, it indicates that the electronic device is not suitable for performing the image processing task in the next time, at this time, the operations of stopping reading the first image and pausing image processing of the first image can be performed until the summary score is greater than or equal to the preset threshold, and then the image processing task is performed.

[0114] In addition, in some embodiments of the present application, after step 440, the data processing method can further include step 4601 and step 4602.

[0115] Step 4601: storing the second image into a second storage space, the second storage space being a resident memory in the electronic device.

[0116] Step 4602: switching the display from the shooting preview interface to an image display interface, the image display interface being used to display the second image read from the second storage space.

[0117] Therefore, the process of collecting images and the process of performing image processing tasks can be separated, and the summary score of the electronic device performing the image processing task fully considers the resource condition of the electronic device for performing the image processing task, compared with the mode of performing image processing immediately after collecting the first image, the image processing process congestion caused by the concentration of resource demand can be effectively avoided, the problem that the image processed by the image algorithm cannot be saved in time can be solved, the resource amount required for performing the image processing task is ensured, the resource utilization rate of the electronic device is improved, and the camera system performance can be effectively improved.

[0118] It should be noted that the data processing method provided by the embodiments of the present application can be executed by a mobile phone, a tablet computer, a notebook computer, a palm computer, a wearable device, or the like. In some embodiments of the present application, the data processing method is executed by an electronic device as an execution subject, and the data processing method provided by the embodiments of the present application is described.

[0119] The data processing method provided by the embodiments of the present application can be executed by a data processing device. In the embodiments of the present application, the data processing method is executed by a data processing device, and the data processing method provided by the embodiments of the present application is described.

[0120] The present application further provides a data processing device. The data processing device will be described in detail in combination with Figure 5

[0121] ​Figure 5 A structural schematic diagram of a data processing apparatus provided for some embodiments of the present application.

[0122] As shown in Figure 5 The data processing apparatus 50 can be applied to an electronic device, and specifically can include:

[0123] The receiving module 501 is configured to receive a first input of a user on a shooting control in a shooting preview interface.

[0124] The storage module 502 is configured to, in response to the first input, store a first image collected to a first storage space.

[0125] The generation module 503 is configured to generate a summary score according to a first score of the electronic device in a first time period and a second score in a second time period, wherein the first score is a score of a device state of the electronic device in the first time period, the second score is a score of an estimated device running state of the electronic device in the second time period performing an image processing task, and the first time period is earlier than the second time period.

[0126] The reading module 504 is configured to, in a case where the summary score is greater than or equal to a preset threshold, read the first image from the first storage space and perform image processing on the first image to obtain a second image.

[0127] The data processing apparatus 50 in the embodiments of the present application will be described in detail as follows.

[0128] In some embodiments of the present application, the data processing apparatus 50 can further include an acquisition module configured to acquire device state data of the electronic device in the first time period, wherein the device state data includes at least one of the following: device power, device load, device memory, device temperature, device charging state parameter, and device screen-off state parameter.

[0129] The generation module 503 is configured to generate the first score according to the device state data and an importance value corresponding to the device state data.

[0130] In some embodiments of the present application, the data processing apparatus 50 can further include an acquisition module configured to acquire first device running data of the electronic device in a third time period, wherein the first device running data includes at least one of the following: computing power for performing the image processing task, user behavior data related to the image processing task, and device energy consumption for performing the image processing task.

[0131] The data processing apparatus 50 can further include a determination module configured to determine, based on the first device running data, second device running data in a second time period, the second device running data comprising at least one of: an estimated computing power for performing the image processing task, estimated user behavior data related to the image processing task, and an estimated device energy consumption for performing the image processing task.

[0132] The generation module 503 can be further configured to generate a second score according to the second device running data and an importance value corresponding to the second device running data.

[0133] In some embodiments of the present application, the data processing apparatus 50 can further include an execution module configured to, in a case where the aggregated score is less than a preset threshold, perform at least one of: stopping reading the first image from the first storage space, and pausing image processing of the first image.

[0134] In some embodiments of the present application, the data processing apparatus 50 can further include a display module, and the display module is configured to:

[0135] The storage module 502 is further configured to store the second image into a second storage space, the second storage space being a resident memory in the electronic device.

[0136] The display module is configured to switch from displaying the shooting preview interface to displaying an image display interface, the image display interface being configured to display the second image read from the second storage space.

[0137] In some embodiments of the present application, the receiving module 501 can be further configured to, in a case where the first image comprises at least two first images, receive a second input of the user on a view image control in the shooting preview interface.

[0138] The reading module 504 can be further configured to, in response to the second input, read the at least two first images from the first storage space.

[0139] The data processing apparatus 50 can further include a display module configured to switch from displaying the shooting preview interface to displaying an image preview interface, the image preview interface being configured to display the at least two first images.

[0140] The receiving module 501 can be further configured to receive a third input of the user selecting a target image from the at least two first images.

[0141] The data processing apparatus 50 can further include a processing module configured to, in response to the third input, perform image processing on the target image to obtain a second image.

[0142] The display module can be further configured to display the second image in the image preview interface.

[0143] The data processing apparatus in the embodiments of the present application can be an electronic device, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and can also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a cashier machine, or a self-service machine, etc., and the embodiments of the present application are not limited in this regard.

[0144] The data processing apparatus in the embodiments of the present application can be a device with an operating system. The operating system can be an Android operating system, an IOS operating system, or other possible operating systems, and the embodiments of the present application are not limited in this regard.

[0145] The data processing apparatus provided in the embodiments of the present application can implement each process of the data processing method embodiment shown in Figure 2 and achieve the same technical effects. To avoid repetition, details are not described herein.

[0146] Based on this, the data processing apparatus provided by the embodiments of the present application stores the collected first image into the first storage space in response to the first input of the user to the shooting control in the shooting preview interface; then, generates a summary score according to a first score of the electronic device in a first time period and a second score of the electronic device in a second time period, wherein the first score is the score of the device state of the electronic device in the first time period, the second score is the score of the estimated device running state of the electronic device in the second time period, and the first time period is earlier than the second time period; and then, in the case where the summary score is greater than or equal to a preset threshold, reads the first image from the first storage space and performs image processing on the first image to obtain a second image. In this way, the process of collecting images and the process of executing image processing tasks can be separated, the score of the device state of the electronic device in the first time period and the score of the estimated device running state of the electronic device in the second time period are summarized, the resource situation in the electronic device for executing the image processing task is fully considered, and the first image is processed only when the summary score is greater than or equal to the preset threshold, that is, when the electronic device has the required resource amount for executing the image processing task. Compared with the way of collecting the first image and then immediately performing image processing, the image processing process congestion caused by the concentration of resource requirements can be effectively avoided, the problem that the image processed by the image algorithm cannot be saved in time can be solved, the required resource amount for executing the image processing task is ensured, the resource utilization rate in the electronic device is improved, and the camera system performance can be effectively improved.

[0147] Optionally, as shown in Figure 6 The embodiments of the present application also provide an electronic device 60, which includes a processor 601 and a memory 602, the memory 602 stores programs or instructions executable on the processor 601, the programs or instructions are executed by the processor 601 to implement the steps of the above-mentioned data processing method embodiments, and the same technical effects can be achieved. To avoid repetition, it will not be described here.

[0148] It should be noted that the electronic device in the embodiments of the present application includes the above-mentioned mobile electronic device and non-mobile electronic device.

[0149] Figure 7 A hardware structure schematic diagram of an electronic device is provided for the embodiments of the present application.

[0150] The electronic device 700 includes but is not limited to: a radio frequency unit 701, a network module 702, an audio output unit 703, an input unit 704, a sensor 705, a display unit 706, a user input unit 707, an interface unit 708, a memory 709, a processor 710, etc.

[0151] Those skilled in the art can understand that the electronic device 700 can also include a power supply (such as a battery) for supplying power to each component, and the power supply can be logically connected to the processor 710 through a power management system, so that the power management system can realize the functions of managing charging, discharging, and power consumption management.

[0152] Figure 7 The electronic device structure shown in the figure is not a limitation on the electronic device, and the electronic device can include more or fewer components than the figure, or combine certain components, or different component arrangements, which are not described here.

[0153] In the embodiments of the present application, the processor 710 is configured to receive a first input of a user on a shooting control in a shooting preview interface. The processor 710 is further configured to store a first image collected in response to the first input to a first storage space. The processor 710 is further configured to generate a summary score according to a first score of the electronic device in a first time period and a second score of the electronic device in a second time period, wherein the first score is a score of the device state of the electronic device in the first time period, the second score is a score of the estimated device running state of the electronic device in the second time period performing image processing tasks, and the first time period is earlier than the second time period. The processor 710 is further configured to read the first image from the first storage space and perform image processing on the first image to obtain a second image in the case that the summary score is greater than or equal to a preset threshold.

[0154] The electronic device 700 is described in detail as follows.

[0155] In some embodiments of the present application, the processor 710 is configured to obtain device state data of the electronic device in a first time period, wherein the device state data includes at least one of the following: device power, device load, device memory, device temperature, device charging state parameter, device screen-off state parameter.

[0156] According to the device state data and the importance value corresponding to the device state data, a first score is generated.

[0157] In some embodiments of the present application, the processor 710 is configured to obtain first device running data of the electronic device in a third time period, and the first device running data includes at least one of the following: computing power for executing image processing tasks, user behavior data related to image processing tasks, and device energy consumption for executing image processing tasks.

[0158] Based on the first device running data, second device running data in the second time period is determined, and the second device running data includes at least one of the following: estimated computing power for executing image processing tasks, estimated user behavior data related to image processing tasks, and estimated device energy consumption for executing image processing tasks.

[0159] generate a second score according to the second device running data and the importance degree value corresponding to the second device running data.

[0160] In some embodiments of the present application, the processor 710 is configured to, in a case where the aggregated score is less than a preset threshold, perform at least one of the following operations: stopping reading the first image from the first storage space, and pausing image processing of the first image.

[0161] In some embodiments of the present application, the processor 710 is configured to store the second image to a second storage space, the second storage space being a resident memory in the electronic device.

[0162] The display unit 706 is configured to switch from displaying the shooting preview interface to displaying an image display interface, the image display interface being configured to display the second image read from the second storage space.

[0163] In some embodiments of the present application, the user input unit 707 is configured to, in a case where the first image includes at least two first images, receive a second input of a user on a view image control in the shooting preview interface.

[0164] The processor 710 is configured to, in response to the second input, read the at least two first images from the first storage space.

[0165] The display unit 706 is configured to switch from displaying the shooting preview interface to displaying an image preview interface, the image preview interface being configured to display the at least two first images.

[0166] The user input unit 707 is further configured to receive a third input of a user selecting a target image from the at least two first images.

[0167] The processor 710 is further configured to, in response to the third input, perform image processing on the target image to obtain the second image.

[0168] The display unit 706 is further configured to display the second image in the image preview interface.

[0169] It should be understood that the input unit 704 can include a graphics processor (GPU) 7041 and a microphone 7042, the graphics processor 7041 processes image data of a still image or a video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 706 can include a display panel, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 707 includes at least one of a touch panel 7071 and other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 can include two parts of a touch detection device and a touch display. The other input devices 7072 can include, but are not limited to, a physical keyboard, function keys (such as volume display keys, on-off keys, etc.), trackballs, mice, joysticks, and the like, which will not be described here.

[0170] The memory 709 can be used to store software programs and various data, and can mainly include a first storage area storing programs or instructions and a second storage area storing data. The first storage area can store an operating system, at least one application program or instruction required by a function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory 709 can include a volatile memory or a non-volatile memory, or the memory 709 can include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synch link dynamic random access memory (SLDRAM), and a direct memory bus random access memory (DRRAM). The memory 709 in the embodiments of the present application includes but is not limited to these and any other suitable types of memory.

[0171] The processor 710 can include one or more processing units; optionally, the processor 710 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to operating systems, user interfaces, and application programs, and the modem processor mainly processes wireless display signals, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 710.

[0172] The embodiment of the present application further provides a readable storage medium, and the readable storage medium stores programs or instructions, the programs or instructions are executed by a processor to realize various processes of the above-mentioned data processing method embodiment, and the same technical effects can be achieved, and details are not repeated here to avoid repetition.

[0173] The processor is a processor in the electronic device in the above-mentioned embodiment. The readable storage medium includes a computer readable storage medium, such as a computer readable memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0174] In addition, the embodiment of the present application further provides a chip, and the chip includes a processor and a display interface, the display interface is coupled with the processor, and the processor is used to run programs or instructions to realize various processes of the above-mentioned data processing method embodiment, and the same technical effects can be achieved, and details are not repeated here to avoid repetition.

[0175] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0176] The embodiment of the present application provides a computer program product, the program product is stored in a storage medium, the program product is executed by at least one processor to realize various processes of the above-mentioned data processing method embodiment, and the same technical effects can be achieved, and details are not repeated here to avoid repetition.

[0177] It should be noted that in this paper, the term "include", "contain" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0178] Moreover, it is noted that the scope of the methods and apparatus of the present embodiments are not limited by the order of execution or sequences, as some steps can, in some embodiments, occur in different orders, concurrently, or with prior steps partial or complete ones at the same time. Furthermore, reference to a certain feature is not meant to exclude all other features, thus the description is not exhaustive.

[0179] Those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a plurality of instructions for making a terminal (which can be a mobile phone, computer, server, or network equipment, etc.) execute the method of each embodiment of the present application.

[0180] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-mentioned specific embodiments, the above-mentioned specific embodiments are only illustrative, not restrictive, and those skilled in the art can make many forms without departing from the scope of the present application and the protection scope of the claims under the inspiration of the present application, which all belong to the protection scope of the present application.

Claims

1. A data processing method applied to an electronic device, characterized in that, The method comprises: receiving a first input of a user on a shooting control in a shooting preview interface; in response to the first input, storing a first image collected to a first storage space; generating an aggregated score according to a first score of the electronic device in a first time period and a second score of the electronic device in a second time period; the first score is a score of the device state of the electronic device in the first time period, and the second score is a score of the estimated device running state of the electronic device in the second time period for performing image processing tasks; the first time period is earlier than the second time period; in the case where the aggregated score is greater than or equal to a preset threshold, reading the first image from the first storage space and performing image processing on the first image to obtain a second image.

2. The method of claim 1, wherein, The method further comprises: obtaining device state data of the electronic device in the first time period, wherein the device state data comprises at least one of the following: device power, device load, device memory, device temperature, device charging state parameter, device screen-off state parameter; generating the first score according to the device state data and an importance value corresponding to the device state data.

3. The method of claim 1, wherein, The method further comprises: obtaining first device running data of the electronic device in a third time period, the first device running data comprising at least one of the following: computing power for performing image processing tasks, user behavior data related to the image processing tasks, and device energy consumption for performing the image processing tasks; determining second device running data in the second time period based on the first device running data, the second device running data comprising at least one of the following: estimated computing power for performing image processing tasks, estimated user behavior data related to the image processing tasks, and estimated device energy consumption for performing the image processing tasks; generating the second score according to the second device running data and an importance value corresponding to the second device running data.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: in the case where the aggregated score is less than the preset threshold, performing at least one of the following operations: stopping the operation of reading the first image from the first storage space, and pausing the operation of image processing on the first image.

5. The method of claim 1, wherein, The method further comprises: storing the second image to a second storage space, the second storage space being a resident memory in the electronic device; switching from the shooting preview interface to display an image display interface, the image display interface being used to display the second image read from the second storage space.

6. The method of claim 1, wherein, The first image comprises at least two first images; After the response to the first input, storing the first image collected to the first storage space, the method further comprises: receiving a second input of a user on a view image control in the shooting preview interface; in response to the second input, reading the at least two first images from the first storage space; switching from the shooting preview interface to display an image preview interface, the image preview interface being used to display the at least two first images; receiving a third input of a user selecting a target image from the at least two first images; In response to the third input, the target image is image-processed to obtain the second image; The second image is displayed on the image preview interface.

7. A data processing system, characterized by The data processing system comprises a user interaction module, a frame grabbing module and a control module, and the frame grabbing module is provided with a first storage space; wherein, The user interaction module is configured to receive a first input of a user on a shooting control in a shooting preview interface; The frame grabbing module is configured to collect a first image in response to the first input and store the first image in the first storage space; The control module is configured to generate a summary score based on the first input, according to a first score of the electronic device in a first time period and a second score of the electronic device in a second time period, the first score being a score of the device state of the electronic device in the first time period, the second score being a score of the estimated device running state of the electronic device in the second time period for performing image processing tasks, the first time period being earlier than the second time period; and send a first instruction to the frame grabbing module in the case that the summary score is greater than or equal to a preset threshold; The frame grabbing module is configured to read the first image from the first storage space based on the first instruction, and image-process the first image to obtain a second image.

8. The system of claim 7, wherein, The control module is specifically configured to obtain device state data of the electronic device in the first time period, wherein the device state data comprises at least one of the following: device power, device load, device memory, device temperature, device charging state parameter, device screen-off state parameter; and generate the first score according to the device state data and an importance value corresponding to the device state data; In addition, the control module is configured to obtain first device running data of the electronic device in a third time period, the first device running data comprising at least one of the following: computing power for performing image processing tasks, user behavior data related to the image processing tasks, and device energy consumption for performing the image processing tasks; determine second device running data in the second time period based on the first device running data, the second device running data comprising at least one of the following: estimated computing power for performing image processing tasks, estimated user behavior data related to the image processing tasks, and estimated device energy consumption for performing the image processing tasks; and generate the second score according to the second device running data and an importance value corresponding to the second device running data.

9. The system of claim 7, wherein, The control module is further configured to send a second instruction to the frame grabbing module in the case that the summary score is less than the preset threshold; The frame grabbing module is configured to perform at least one of the following operations based on the second instruction: stop reading the first image from the first storage space, and pause image processing of the first image.

10. The system of claim 7, wherein, The data processing system further comprises a second storage space, which is a resident memory in the electronic device; The frame grabbing module is further configured to send the second image to the second storage space; The data processing system further comprises a second storage space, which is a resident memory in the electronic device; The second storage space is configured to store the second image and send the second image to the user interaction module. The user interaction module is configured to switch the display from the shooting preview interface to an image display interface, and the image display interface is configured to display the second image.

11. The system of claim 10, wherein, The user interaction module is further configured to, in a case where the first image includes at least two first images, receive a second input of a user on a view image control in the shooting preview interface. The frame grabbing module is further configured to, in response to the second input, read the at least two first images from the first storage space, and send the at least two first images to the user interaction module. The user interaction module is further configured to switch the display from the shooting preview interface to an image preview interface, and the image preview interface is configured to display the at least two first images, receive a third input of a user on selection of a target image from the at least two first images, and send the target image to the frame grabbing module in response to the third input. The frame grabbing module is further configured to perform image processing on the target image to obtain the second image.

12. A data processing apparatus applied to an electronic device, characterized in that, The method comprises the following steps: The receiving module is configured to receive a first input of a user on a shooting control in a shooting preview interface. The storage module is configured to, in response to the first input, store a first image collected to a first storage space. The generating module is configured to generate a summary score according to a first score of the electronic device in a first time period and a second score of the electronic device in a second time period, wherein the first score is a score of a device state of the electronic device in the first time period, the second score is a score of an estimated device running state of the electronic device in the second time period for performing an image processing task, and the first time period is earlier than the second time period. The reading module is configured to, in a case where the summary score is greater than or equal to a preset threshold, read the first image from the first storage space, and perform image processing on the first image to obtain a second image.

13. An electronic device, comprising: The processor, the memory and the program or instructions stored in the memory and executable on the processor, when executed by the processor, implement the steps of the data processing method of any one of claims 1-6. The processor, the memory and the program or instructions stored in the memory and executable on the processor, when executed by the processor, implement the steps of the data processing method of any one of claims 1-6.

Images