Image processing control method and device and electronic equipment

Abstract

The invention discloses an image processing control method and device and electronic equipment. The method comprises the steps that the working mode of the electronic equipment is detected, the working mode comprises a photographing mode and a non-photographing mode, and the photographing mode is the mode between the moment when the electronic equipment receives a photographing instruction and the moment when an image is generated; under the condition that the electronic equipment is in a photographing mode, first control information or second control information is sent to an image processing chip, the first control information is used for indicating the image processing chip to process a photographed image and a preview image according to a first rule, and the second control information is used for indicating the image processing chip to process the preview image according to a second rule; the first rule is a processing rule for performing time-phased processing on the photographed image and the preview image; the second control information is used for indicating the image processing chip to process the photographed image and stop processing the preview image.

Description

Technical Field

[0001] The present application belongs to the field of image processing, and specifically relates to an image processing control method, device and electronic equipment. Background Art

[0002] Mobile phone imaging has been one of the important research directions for major manufacturers in recent years as a means of technological innovation and improving user experience. As the current terminal functions and features are concentrated in the same application processor (AP), the AP needs to process both the photographed image and the previewed image. When the photographed image and the previewed image exist at the same time, the photographed image needs to be processed first, and then the previewed image. This results in multiple frames of images in preview mode being unable to be processed, affecting the image processing efficiency. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide an image processing control method, device and electronic device to solve the problem of low processing efficiency when a photographed image and a preview image exist at the same time.

[0004] In order to solve the above technical problems, this application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides an image processing control method, which is applied to an application processor AP, including:

[0006] Detecting an operating mode of the electronic device, the operating mode including a photographing mode and a non-photographing mode, the photographing mode being a mode between a time when the electronic device receives a photographing instruction and a time when an image is generated;

[0007] When the electronic device is in a photographing mode, first control information or second control information is sent to the image processing chip, wherein the first control information is used to instruct the image processing chip to process the photographed image and the preview image according to a first rule, and the first rule is a processing rule for processing the photographed image and the preview image in different time periods; the second control information is used to instruct the image processing chip to process the photographed image and stop processing the preview image.

[0008] In a second aspect, an embodiment of the present application provides an image processing control method, which is applied to an image processing chip, comprising:

[0009] receiving first control information or second control information sent by the AP, wherein the first control information is used to instruct the image processing chip to process the photographed image and the preview image according to a first rule, wherein the first rule is a processing rule for processing the photographed image and the preview image in different time periods; and the second control information is used to instruct the image processing chip to process the photographed image and stop processing the preview image;

[0010] The image is processed according to the first control information or the second control information.

[0011] In a third aspect, an embodiment of the present application provides an image processing control device, which is applied to an AP, including:

[0012] A detection module is used to detect an operating mode of the electronic device, wherein the operating mode includes a photographing mode and a non-photographing mode, wherein the photographing mode is a mode between the time when the electronic device receives a photographing instruction and the time when an image is generated;

[0013] The first sending module is used to send first control information or second control information to the image processing chip when the electronic device is in a camera mode, the first control information is used to instruct the image processing chip to process the camera image and the preview image according to a first rule, and the first rule is a processing rule for processing the camera image and the preview image in different time periods; the second control information is used to instruct the image processing chip to process the camera image and stop processing the preview image.

[0014] In a fourth aspect, an embodiment of the present application provides an image processing control device, which is applied to an image processing chip, including:

[0015] a first receiving module, configured to receive first control information or second control information sent by an AP, wherein the first control information is used to instruct the image processing chip to process the photographed image and the preview image according to a first rule, wherein the first rule is a processing rule for processing the photographed image and the preview image in different time periods; and the second control information is used to instruct the image processing chip to process the photographed image and stop processing the preview image;

[0016] The second processing module is used to process the image according to the first control information or the second control information.

[0017] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented, or the steps of the method described in the second aspect are implemented.

[0018] In a sixth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented, or the steps of the method described in the second aspect are implemented.

[0019] In the seventh aspect, an embodiment of the present application further provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the method described in the first aspect, or to implement the method described in the second aspect.

[0020] In an eighth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method as described in the first aspect or the method as described in the second aspect.

[0021] In an embodiment of the present application, the AP detects the working mode of the electronic device, and when the electronic device is in the photographing mode, sends control information to the image processing chip, instructing the image processing chip to process the photographed image and the preview image according to the first rule, or instructing the image processing chip to process the photographed image and stop processing the preview image, thereby reasonably utilizing the resources of the image processing chip, improving the image processing efficiency, reducing the image processing cost, and providing users with a better experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is one of the flowcharts of the image processing control method of the embodiment of the present application;

[0023] Figure 2 This is the second flow chart of the image processing control method according to the embodiment of the present application;

[0024] Figure 3 It is one of the structural schematic diagrams of the image processing control device in the embodiment of the present application;

[0025] Figure 4 This is the second structural schematic diagram of the image processing control device according to the embodiment of the present application;

[0026] Figure 5 It is one of the structural schematic diagrams of the electronic device according to the embodiment of the present application;

[0027] Figure 6 This is the second structural schematic diagram of the electronic device according to the embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0029] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0030] The image processing control method, device and electronic device provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0031] like Figure 1 As shown, an embodiment of the present application provides an image processing control method, which is applied to an AP, and the method includes:

[0032] Step 101: Detecting an operating mode of an electronic device, wherein the operating mode includes a photographing mode and a non-photographing mode, wherein the photographing mode is a mode between the time when the electronic device receives a photographing instruction and the time when an image is generated;

[0033] Step 102: When the electronic device is in a photographing mode, send first control information or second control information to the image processing chip, wherein the first control information is used to instruct the image processing chip to process the photographed image and the preview image according to a first rule, and the first rule is a processing rule for processing the photographed image and the preview image in different time periods; the second control information is used to instruct the image processing chip to process the photographed image and stop processing the preview image.

[0034] In this embodiment, the electronic device may include an AP and an image processing chip. The AP is the main chip of the electronic device, which can process data tasks of the device, control the user interface, process graphics and audio, and manage network communications. The image processing chip can be an external chip, and it can also be understood that the image processing chip is a newly added chip on the electronic device for image processing. The AP and the image processing chip are connected to perform data transmission.

[0035] Optionally, the image processing chip may be an external chip equipped with a neural network processor (Neural-network Processing Unit, NPU). For example, if the NPU is set in the image processing chip, the NPU may perform image processing.

[0036] When a user turns on the camera of an electronic device and enters the preview mode, the preview algorithm of the NPU in the image processing chip operates normally and processes each frame of the preview image once, for example, processing the preview image according to the current parameters of the camera.

[0037] The AP determines whether the electronic device is currently in a photo mode, where the photo mode refers to the intermediate state from when the user clicks the photo button (i.e., the electronic device receives a photo command) to when the photo JPEG image is generated. Since NPU resources are used when taking photos, there will be a preemption and sharing of NPU resources. The non-photo mode refers to the mode before the user turns on the camera and does not click the photo button, or the mode after the user clicks the photo button and generates a photo JPEG image. It should be noted that the preview image is generated in real time, and a preview image will be generated even in the photo mode. The generated preview image is cached after processing, and the photo image is a further processing of the preview image cached at the current moment when the photo command is received. Since the photo image is output to the device, when processing the photo image, it is necessary not only to process the screen parameters such as the filter set by the user, but also to process the format, size, resolution and other parameters of the image output, so the processing time (or resources) of the preview image is usually less than the processing time (or resources) of the photo image.

[0038] When the AP detects that the electronic device is in the camera mode, it can send the first control information to the image processing chip to instruct the image processing chip to process the camera image and the preview image according to the first rule. The first rule is a processing rule for processing the camera image and the preview image in different time periods. The purpose of processing according to the first rule is to avoid the image processing chip from processing one image all the time, causing the other image to wait too long, resulting in low image processing efficiency. Optionally, the first rule can be a time-divided processing rule, that is, the AP instructs the image processing chip to process the camera image and the preview image in different time periods through the first control information.

[0039] Alternatively, when the AP detects that the electronic device is in a photo-taking mode, it may send a second control message to the image processing chip, instructing the image processing chip to process the photo image and suspend processing of the preview image. At this time, the preview image is processed by the AP. Since the electronic device is in a photo-taking mode at this time, the processing complexity of the photo image is relatively high. In order to avoid the user having to wait for a long time for the image and affecting the user experience, the image processing chip gives priority to processing the photo image.

[0040] It should be noted that the processing of the photographed image and the processing of the preview image described in the embodiments of the present application may refer to adjusting the image parameters (such as size, color, resolution, size of objects in the image, etc.) according to parameters set by the user, the camera's own parameters, etc.

[0041] In an embodiment of the present application, the AP detects the working mode of the electronic device, and when the electronic device is in the photographing mode, sends control information to the image processing chip, instructing the image processing chip to process the photographed image and the preview image according to the first rule, or instructing the image processing chip to process the photographed image and stop processing the preview image, thereby reasonably utilizing the resources of the image processing chip, improving the image processing efficiency, reducing the image processing cost, and providing users with a better experience.

[0042] As an optional embodiment, the first rule includes:

[0043] Divide the photographed image into N regions, where N is greater than or equal to 2;

[0044] After the first area is processed and before the second area starts to be processed, a preview image is processed;

[0045] The first area and / or the second area includes: an image area among the N areas.

[0046] In this embodiment, the AP instructs the image processing chip to process the photographed image and the preview image according to the first rule. The image processing chip can divide the photographed image into multiple regions, such as dividing a large image into multiple slices, and each slice is processed in turn. After a single slice is processed, the image processing chip processes a frame of the preview image, and after the preview image is processed, the next slice is processed until all slices are processed.

[0047] It should be noted that the NPU for processing the photo image and the NPU for processing the preview image can be the same or different.

[0048] As an optional embodiment, the method further includes:

[0049] Determining processing resources required for the photographed image and processing resources required for the preview image;

[0050] The sending the first control information or the second control information to the image processing chip includes:

[0051] If the processing resources required for the preview image or the processing resources required for the photographed image meet the first condition, sending second control information to the image processing chip;

[0052] The first condition includes one of the following:

[0053] The processing resources required for the preview image are greater than or equal to the processing resources required for the photographed image;

[0054] The processing resources required for the preview image are greater than or equal to a first preset value; the first preset value may be preconfigured or predefined, the first preset value may be a fixed value, or may be set according to the processing resources required for the photographed image or the capabilities of the image processing chip, such as being set to half of the processing resources required for the photographed image, or being set to two-thirds of the resources corresponding to the capabilities of the image processing chip, and so on.

[0055] The processing resources required for the photographed image are greater than or equal to a second preset value, which may be preconfigured or predefined, and may be a fixed value, or may be set according to the capabilities of the image processing chip, such as being set to half of the processing resources required for the photographed image. The processing resources required for the photographed image are greater than or equal to the second preset value, which may indicate that the image processing chip has no capability or is insufficient to support processing of the preview image. In order to avoid too low processing efficiency of the preview image, the image processing chip may only process the photographed image, and the AP may process the preview image.

[0056] In this embodiment, when the AP determines that the electronic device is in the photographing mode, it can determine the time (or resources) required to process the photographed image and the time (or resources) required to process the preview image. If the time (or resources) required to process the preview image is long, for example, the time (or resources) required to process the preview image is greater than or equal to the time (or resources) required to process the photographed image, or the time (or resources) required to process the preview image is greater than or equal to a preset value, or the time (or resources) required to process the photographed image is greater than or equal to a preset value. In order to avoid affecting the processing performance or efficiency of the photographed image or the preview image, the second control information is sent to the image processing chip to instruct the image processing chip to only process the photographed image and turn off the processing function of the preview image. At this time, the preview image is processed by the AP.

[0057] Optionally, the method further includes: in a case of sending second control information to the image processing chip, the AP processes the preview image.

[0058] In this embodiment, when the AP instructs the image processing chip to process only the photographed image, the AP processes the preview image, which can ensure that both the photographed image and the preview image can be processed in time, thereby improving the image processing efficiency.

[0059] Optionally, the method further includes: when the electronic device is in a non-photographing mode, sending third control information to the image processing chip, wherein the third control information is used to instruct the image processing chip to process the preview image.

[0060] In this embodiment, if the AP detects that the electronic device is in a non-photographing mode and there is no need to process the photographed image, the AP sends a third control information to the image processing chip to instruct the image processing chip to process the preview image. If the AP is processing the preview image at this time, the AP stops processing the preview image.

[0061] The following example illustrates the specific implementation process of image processing by the AP and the image processing chip.

[0062] Example 1: The AP sends first control information to the image processing chip, and the image processing chip performs image processing, including:

[0063] Step 11: The user opens the camera and enters the preview.

[0064] After the user enters the preview interface, the NPU preview algorithm processing of the image processing chip starts to run normally, and each frame of the preview image is processed once.

[0065] Step 12: The AP determines whether it is currently in a photo taking state (or photo taking mode).

[0066] If you are currently in the photo-taking state (the intermediate state from when the user clicks to take a photo to when the JPEG of the photo is generated is called the photo-taking state), since NPU resources are also used when taking photos, you will face the preemption and sharing of NPU resources.

[0067] Step 13: If the electronic device is in a non-photographing state (or non-photographing mode), the NPU of the image processing chip processes each frame of the preview image normally, such as adjusting the image parameters (such as size, color, resolution, size of objects in the image, etc.) according to the parameters set by the user, camera parameters, the currently used photography mode, etc.

[0068] When not taking photos, there is no preemption of NPU resources, and the NPU is only used to accelerate the processing of each frame of the preview algorithm.

[0069] Step 14: If the electronic device is in the photographing state (or photographing mode), it is determined whether the current slice has been processed.

[0070] The electronic device is currently in the photo mode, and the NPU needs to reuse resources. If the processing strategy is not used, there will be problems such as preview freezes and slow photo taking. This step divides the photo processing process into multiple slices. For example, a 4000*3000 image is divided into 10 slices, each of which is a 4000*300 image. When the NPU of the image processing chip processes the photo image, each slice is processed in turn.

[0071] Step 15: Wait for the current slice to complete processing.

[0072] The accelerated processing of each frame of preview image requires waiting for the processing of a single slice of the camera image to be completed and the next slice to be processed. If the current slice is not processed, the processing of the preview image continues to wait.

[0073] Step 16: Wake up the preview image processing.

[0074] After the processing of the currently processed camera image slice is completed, if there is a processing task of the preview image waiting, the processing of the next slice will not be started, so that it is in a waiting state, and the algorithm of the preview image will be awakened first to process the preview image.

[0075] Step 17: Continue to trigger the next slice processing.

[0076] After processing a frame of preview image, continue to trigger the next slice processing, and cycle in sequence.

[0077] Step 18: Photo shooting is completed.

[0078] After the photo is taken, the system returns to the normal preview state, and the NPU preview algorithm of the image processing chip starts to operate normally. Each frame of the preview image is processed once, and the system returns to step 11.

[0079] In this example, the external image processing chip based on camera acceleration performs preview effect acceleration calculation during preview and generates additional metadata (meta). The camera image is divided into multiple slices for processing. After each slice is processed, the preview image is accelerated. After the preview image is accelerated, the next slice of the camera image is processed again until all slices are processed. This implementation method can time-share multiplexing to accelerate the preview NPU algorithm and the camera NPU algorithm, reasonably utilize the NPU resources of the external chip, reduce costs, and enable users to have a better experience.

[0080] Example 2: The AP sends the second control information to the image processing chip, and the image processing chip performs image processing, including:

[0081] Step 21: The user opens the camera and enters the preview.

[0082] After the user enters the preview interface, the NPU preview algorithm processing of the image processing chip starts to run normally, and each frame of the preview image is processed once.

[0083] Step 22: The AP determines whether it is currently in a photographing state (or photographing mode).

[0084] If you are currently in the photo-taking state (the intermediate state from when the user clicks to take a photo to when the JPEG of the photo is generated is called the photo-taking state), since NPU resources are also used when taking photos, you will face the preemption and sharing of NPU resources.

[0085] In this step, if the algorithm processing of the preview image occupies more NPU resources, the AP is switched to process the preview image when processing the photographed image.

[0086] Step 23: If the electronic device is in a non-photographing state (or non-photographing mode), the NPU of the image processing chip performs normal processing acceleration on each frame of the preview image.

[0087] When not taking photos, there is no preemption of NPU resources, and the NPU is only used to accelerate the processing of each frame of the preview algorithm.

[0088] Step 24: If the electronic device is in a photographing state (or photographing mode), the AP sends a second control message to the image processing chip, instructing the image processing chip to turn off the NPU acceleration processing of the preview image and only process the photographed image.

[0089] When the camera is currently in photo mode, the image processing chip turns off the preview image processing function of the NPU to avoid a significant impact on the photo performance.

[0090] Step 25: Switch to AP to process the preview image.

[0091] In the photo mode, the processing of the preview image is switched to the AP, that is, the AP performs algorithm acceleration of the preview image, and the image processing chip performs algorithm acceleration processing of the photo image.

[0092] Step 26: Photo shooting is completed.

[0093] After the photo is taken, the normal preview state is returned, and the NPU preview algorithm processing of the image processing chip starts to operate normally, and each frame of the preview image is processed once, that is, returning to step 21.

[0094] In this example, when taking a photo, if processing the preview image occupies more NPU resources and has a greater impact on the photo performance, the AP is switched to process the preview image when taking a photo, and the external chip only processes the photo image. When processing the preview image occupies more NPU resources or the load is very high, the photo performance is optimized.

[0095] It should be noted that, in the embodiments of the present application, the processing of photographed images and preview images may refer to: processing the images according to the parameters set by the user in the camera or pre-set parameters (such as size, filter, resolution, beauty effect, etc.), which is not specifically limited here.

[0096] In an embodiment of the present application, the AP detects the working mode of the electronic device, and when the electronic device is in the photographing mode, sends control information to the image processing chip, instructing the image processing chip to process the photographed image and the preview image according to the first rule, or instructing the image processing chip to process the photographed image and stop processing the preview image, thereby reasonably utilizing the resources of the image processing chip, improving the image processing efficiency, reducing the image processing cost, and providing users with a better experience.

[0097] like Figure 2 As shown, the embodiment of the present application also provides an image processing control method, which is applied to an image processing chip, and the method includes:

[0098] Step 201: receiving first control information or second control information sent by an AP, wherein the first control information is used to instruct an image processing chip to process a photographed image and a preview image according to a first rule, wherein the first rule is a processing rule for processing the photographed image and the preview image in different time periods; and the second control information is used to instruct the image processing chip to process the photographed image and stop processing the preview image;

[0099] Step 202: Process an image according to the first control information or the second control information.

[0100] In this embodiment, the electronic device may include an AP and an image processing chip. The AP is the main chip of the electronic device, which can process data tasks of the device, control the user interface, process graphics and audio, and manage network communications. The image processing chip can be an external chip, and it can also be understood that the image processing chip is a newly added chip on the electronic device for image processing. The AP and the image processing chip are connected to perform data transmission.

[0101] When a user turns on the camera of an electronic device and enters the preview mode, the preview algorithm of the NPU in the image processing chip operates normally and processes each frame of the preview image once, for example, processing the preview image according to the current parameters of the camera.

[0102] The AP determines whether the electronic device is currently in a photo mode, where the photo mode refers to the intermediate state between when the user clicks the photo button (i.e., the electronic device receives a photo command) and when the photo JPEG image is generated. The non-photo mode refers to the mode before the user turns on the camera and does not click the photo button, or the mode after the user clicks the photo button and the photo JPEG image is generated.

[0103] When the AP detects that the electronic device is in the camera mode, it can send a first control message to the image processing chip to instruct the image processing chip to process the camera image and the preview image according to a first rule. The first rule is a processing rule for processing the camera image and the preview image in different time periods. The purpose of following the first rule is to avoid the image processing chip from processing one image all the time, causing the other image to wait too long, resulting in low image processing efficiency. Optionally, the first rule can be a time-divided processing rule, that is, the AP instructs the image processing chip to process the camera image and the preview image in different time periods through the first control message. After receiving the first control message, the image processing chip processes the camera image and the preview image according to the first control message.

[0104] Alternatively, the AP detects that the electronic device is in the photo mode and can send a second control message to the image processing chip to instruct the image processing chip to process the photo image and suspend the processing of the preview image. Since the electronic device is in the photo mode at this time, the processing complexity of the photo image is relatively high. In order to avoid the user waiting for a long time for the image and affecting the user experience, the image processing chip prioritizes the photo image. After receiving the second control message, the image processing chip processes the photo image according to the second control message. In this case, the AP processes the preview image.

[0105] In an embodiment of the present application, the image processing chip receives control information sent by the AP, and processes the photographed image and the preview image according to the first rule based on the control information, or the image processing chip processes the photographed image and stops processing the preview image, thereby rationally utilizing the resources of the image processing chip, improving image processing efficiency, reducing image processing costs, and enabling users to obtain a better experience.

[0106] Optionally, the first rule includes:

[0107] Divide the photographed image into N regions, where N is greater than or equal to 2;

[0108] After the first area is processed and before the second area starts to be processed, a preview image is processed;

[0109] The first area and / or the second area includes: an image area among the N areas.

[0110] In this embodiment, the image processing chip receives the first control information and can divide the photographed image into multiple regions, such as dividing a large image into multiple slices, and processing each slice in turn. After a single slice is processed, the image processing chip processes a preview image, and after the preview image is processed, the next slice is processed until all slices are processed.

[0111] Optionally, if the image processing chip receives the second control information, the preview image processing function is turned off and only the photographed image is processed, and the preview image is processed by the AP at this time, so that both the photographed image and the preview image can be processed in time, thereby improving the image processing efficiency.

[0112] Optionally, the method further includes:

[0113] receiving third control information sent by the AP, where the third control information is used to instruct the image processing chip to process the preview image;

[0114] The preview image is processed according to the third control information.

[0115] In this embodiment, if the AP detects that the electronic device is in a non-photographing mode and does not need to process the photographed image, the AP sends third control information to the image processing chip to instruct the image processing chip to process the preview image.

[0116] In an embodiment of the present application, the image processing chip receives control information sent by the AP, and processes the photographed image and the preview image according to the first rule based on the control information, or the image processing chip processes the photographed image and stops processing the preview image, thereby rationally utilizing the resources of the image processing chip, improving image processing efficiency, reducing image processing costs, and enabling users to obtain a better experience.

[0117] like Figure 3 As shown, the embodiment of the present application further provides an image processing control device 300, which is applied to an AP, including:

[0118] The detection module 310 is used to detect the working mode of the electronic device, where the working mode includes a photographing mode and a non-photographing mode, where the photographing mode is a mode between the time when the electronic device receives a photographing instruction and the time when an image is generated;

[0119] The first sending module 320 is used to send first control information or second control information to the image processing chip when the electronic device is in the camera mode. The first control information is used to instruct the image processing chip to process the camera image and the preview image according to a first rule, and the first rule is a processing rule for processing the camera image and the preview image in different time periods; the second control information is used to instruct the image processing chip to process the camera image and stop processing the preview image.

[0120] Optionally, the first rule includes:

[0121] Divide the photographed image into N regions, where N is greater than or equal to 2;

[0122] After the first area is processed and before the second area starts to be processed, a preview image is processed;

[0123] The first area and / or the second area includes: an image area among the N areas.

[0124] Optionally, the device further comprises:

[0125] A determination module, used to determine the processing resources required for the photographed image and the processing resources required for the preview image;

[0126] The first sending module is specifically used for:

[0127] If the processing resources required for the preview image or the processing resources required for the photographed image meet the first condition, sending second control information to the image processing chip;

[0128] The first condition includes one of the following:

[0129] The processing resources required for the preview image are greater than or equal to the processing resources required for the photographed image;

[0130] The processing resources required for the preview image are greater than or equal to a first preset value;

[0131] The processing resources required for the photographed image are greater than or equal to a second preset value.

[0132] Optionally, the device further comprises:

[0133] The first processing module is configured to enable the AP to process the preview image when sending second control information to the image processing chip.

[0134] Optionally, the device further comprises:

[0135] The third sending module is used to send third control information to the image processing chip when the electronic device is in a non-photographing mode, wherein the third control information is used to instruct the image processing chip to process the preview image.

[0136] In an embodiment of the present application, the AP detects the working mode of the electronic device, and when the electronic device is in the photographing mode, sends control information to the image processing chip, instructing the image processing chip to process the photographed image and the preview image according to the first rule, or instructing the image processing chip to process the photographed image and stop processing the preview image, thereby reasonably utilizing the resources of the image processing chip, improving the image processing efficiency, reducing the image processing cost, and providing users with a better experience.

[0137] like Figure 4 As shown, the embodiment of the present application further provides an image processing control device 400, which is applied to an image processing chip, including:

[0138] The first receiving module 410 is used to receive the first control information or the second control information sent by the AP, wherein the first control information is used to instruct the image processing chip to process the photographed image and the preview image according to the first rule, and the first rule is a processing rule for processing the photographed image and the preview image in different time periods; the second control information is used to instruct the image processing chip to process the photographed image and stop processing the preview image;

[0139] The second processing module 420 is configured to process an image according to the first control information or the second control information.

[0140] Optionally, the first rule includes:

[0141] Divide the photographed image into N regions, where N is greater than or equal to 2;

[0142] After the first area is processed and before the second area starts to be processed, a preview image is processed;

[0143] The first area and / or the second area includes: an image area among the N areas.

[0144] Optionally, the device further comprises:

[0145] A second receiving module, used for receiving third control information sent by the AP, wherein the third control information is used for instructing the image processing chip to process the preview image;

[0146] The third processing module is used to process the preview image according to the third control information.

[0147] In an embodiment of the present application, the image processing chip receives control information sent by the AP, and processes the photographed image and the preview image according to the first rule based on the control information, or the image processing chip processes the photographed image and stops processing the preview image, thereby rationally utilizing the resources of the image processing chip, improving image processing efficiency, reducing image processing costs, and enabling users to obtain a better experience.

[0148] The image processing control device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0149] The image processing control device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0150] The image processing control device provided in the embodiment of the present application can achieve Figure 1 to Figure 2 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0151] Optional, such as Figure 5 As shown, an embodiment of the present application further provides an electronic device 500, which may include an AP and / or an image processing chip, and includes a processor 501, a memory 502, and a program or instruction stored in the memory 502 and executable on the processor 501. When the program or instruction is executed by the processor 501, each process of the above-mentioned image processing control method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0152] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0153] Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0154] The electronic device 600 includes but is not limited to: a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609, and a processor 610.

[0155] Those skilled in the art will appreciate that the electronic device 600 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 610 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 6 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0156] Wherein, in the case where the electronic device is an AP:

[0157] The processor 610 is used to detect the working mode of the electronic device, where the working mode includes a photographing mode and a non-photographing mode, and the photographing mode is a mode between the time when the electronic device receives a photographing instruction and the time when an image is generated;

[0158] The interface unit 608 is used to: when the electronic device is in a photographing mode, send first control information or second control information to the image processing chip, the first control information is used to instruct the image processing chip to process the photographed image and the preview image according to a first rule, and the first rule is a processing rule for processing the photographed image and the preview image in different time periods; the second control information is used to instruct the image processing chip to process the photographed image and stop processing the preview image.

[0159] Optionally, the first rule includes:

[0160] Divide the photographed image into N regions, where N is greater than or equal to 2;

[0161] After the first area is processed and before the second area starts to be processed, a preview image is processed;

[0162] The first area and / or the second area includes: an image area among the N areas.

[0163] Optionally, the processor 610 is further configured to determine processing resources required for the photographed image and processing resources required for the preview image;

[0164] The interface unit 608 is specifically used for:

[0165] If the processing resources required for the preview image or the processing resources required for the photographed image meet the first condition, sending second control information to the image processing chip;

[0166] The first condition includes one of the following:

[0167] The processing resources required for the preview image are greater than or equal to the processing resources required for the photographed image;

[0168] The processing resources required for the preview image are greater than or equal to a first preset value;

[0169] The processing resources required for the photographed image are greater than or equal to a second preset value.

[0170] Optionally, the processor 610 is further configured to:

[0171] In case of sending the second control information to the image processing chip, the AP processes the preview image.

[0172] Optionally, the interface unit 608 is further configured to:

[0173] When the electronic device is in a non-photographing mode, third control information is sent to the image processing chip, where the third control information is used to instruct the image processing chip to process a preview image.

[0174] Wherein, in the case where the electronic device is an image processing chip:

[0175] The interface unit 608 is used to: receive the first control information or the second control information sent by the AP, the first control information is used to instruct the image processing chip to process the photographed image and the preview image according to the first rule, and the first rule is a processing rule for processing the photographed image and the preview image in different time periods; the second control information is used to instruct the image processing chip to process the photographed image and stop processing the preview image;

[0176] The processor 610 is configured to process an image according to the first control information or the second control information.

[0177] Optionally, the first rule includes:

[0178] Divide the photographed image into N regions, where N is greater than or equal to 2;

[0179] After the first area is processed and before the second area starts to be processed, a preview image is processed;

[0180] The first area and / or the second area includes: an image area among the N areas.

[0181] Optionally, the interface unit 608 is further used to: receive third control information sent by the AP, where the third control information is used to instruct the image processing chip to process the preview image;

[0182] The processor 610 is further configured to: process the preview image according to the third control information.

[0183] It should be understood that in the embodiment of the present application, the input unit 604 may include a graphics processor (GPU) 6041 and a microphone 6042, and the graphics processor 6041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 606 may include a display panel 6061, and the display panel 6061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 607 includes a touch panel 6071 and at least one of other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 may include two parts: a touch detection device and a touch controller. Other input devices 6072 may include, but are not limited to, a physical keyboard, function keys (such as a volume control button, a switch button, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0184] The memory 609 can be used to store software programs and various data. The memory 609 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 609 may include a volatile memory or a non-volatile memory, or the memory 609 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 609 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0185] The processor 610 may include one or more processing units; optionally, the processor 610 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 610.

[0186] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned image processing control method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0187] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0188] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing control method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0189] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0190] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned image processing control method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0191] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0192] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

Claims

1. An image processing control method, characterized in that: Applied to application processor AP, including: Detecting an operating mode of the electronic device, the operating mode including a photographing mode and a non-photographing mode, the photographing mode being a mode between a time when the electronic device receives a photographing instruction and a time when an image is generated; When the electronic device is in a photographing mode, first control information or second control information is sent to the image processing chip, wherein the first control information is used to instruct the image processing chip to process the photographed image and the preview image according to a first rule, and the first rule is a processing rule for processing the photographed image and the preview image in different time periods; the second control information is used to instruct the image processing chip to process the photographed image and stop processing the preview image.

2. The method according to claim 1, characterized in that The first rule includes: Divide the photographed image into N regions, where N is greater than or equal to 2; After the first area is processed and before the second area starts to be processed, a preview image is processed; The first area and / or the second area includes: an image area among the N areas.

3. The method according to claim 1, characterized in that The method further comprises: Determining processing resources required for the photographed image and processing resources required for the preview image; The sending the first control information or the second control information to the image processing chip includes: If the processing resources required for the preview image or the processing resources required for the photographed image meet the first condition, sending second control information to the image processing chip; The first condition includes one of the following: The processing resources required for the preview image are greater than or equal to the processing resources required for the photographed image; The processing resources required for the preview image are greater than or equal to a first preset value; The processing resources required for the photographed image are greater than or equal to a second preset value.

4. The method according to claim 1 or 3, characterized in that: The method further comprises: In case of sending the second control information to the image processing chip, the AP processes the preview image.

5. The method according to claim 1, characterized in that The method further comprises: When the electronic device is in a non-photographing mode, third control information is sent to the image processing chip, where the third control information is used to instruct the image processing chip to process a preview image.

6. An image processing control method, characterized in that: Applied to image processing chips, including: receiving first control information or second control information sent by the AP, wherein the first control information is used to instruct the image processing chip to process the photographed image and the preview image according to a first rule, wherein the first rule is a processing rule for processing the photographed image and the preview image in different time periods; and the second control information is used to instruct the image processing chip to process the photographed image and stop processing the preview image; The image is processed according to the first control information or the second control information.

7. The method according to claim 6, characterized in that The first rule includes: Divide the photographed image into N regions, where N is greater than or equal to 2; After the first area is processed and before the second area starts to be processed, a preview image is processed; The first area and / or the second area includes: an image area among the N areas.

8. The method according to claim 6, characterized in that The method further comprises: receiving third control information sent by the AP, where the third control information is used to instruct the image processing chip to process the preview image; The preview image is processed according to the third control information.

9. An image processing control device, characterized in that: include: A detection module is used to detect an operating mode of the electronic device, wherein the operating mode includes a photographing mode and a non-photographing mode, wherein the photographing mode is a mode between the time when the electronic device receives a photographing instruction and the time when an image is generated; The first sending module is used to send first control information or second control information to the image processing chip when the electronic device is in a camera mode, the first control information is used to instruct the image processing chip to process the camera image and the preview image according to a first rule, and the first rule is a processing rule for processing the camera image and the preview image in different time periods; the second control information is used to instruct the image processing chip to process the camera image and stop processing the preview image.

10. The device according to claim 9, characterized in that The first rule includes: Divide the photographed image into N regions, where N is greater than or equal to 2; After the first area is processed and before the second area starts to be processed, a preview image is processed; The first area and / or the second area includes: an image area among the N areas.

11. The device according to claim 9, characterized in that The device also includes: A determination module, used to determine the processing resources required for the photographed image and the processing resources required for the preview image; The first sending module is specifically used for: If the processing resources required for the preview image or the processing resources required for the photographed image meet the first condition, sending second control information to the image processing chip; The first condition includes one of the following: The processing resources required for the preview image are greater than or equal to the processing resources required for the photographed image; The processing resources required for the preview image are greater than or equal to a first preset value; The processing resources required for the photographed image are greater than or equal to a second preset value.

12. The device according to claim 9 or 11, characterized in that The device also includes: The first processing module is used for the AP to process the preview image when sending the second control information to the image processing chip.

13. The device according to claim 9, characterized in that The device also includes: The third sending module is used to send third control information to the image processing chip when the electronic device is in a non-photographing mode, wherein the third control information is used to instruct the image processing chip to process the preview image.

14. An image processing control device, characterized in that: include: a first receiving module, configured to receive first control information or second control information sent by an AP, wherein the first control information is used to instruct the image processing chip to process the photographed image and the preview image according to a first rule, wherein the first rule is a processing rule for processing the photographed image and the preview image in different time periods; and the second control information is used to instruct the image processing chip to process the photographed image and stop processing the preview image; The second processing module is used to process the image according to the first control information or the second control information.

15. The device according to claim 14, characterized in that The first rule includes: Divide the photographed image into N regions, where N is greater than or equal to 2; After the first area is processed and before the second area starts to be processed, a preview image is processed; The first area and / or the second area includes: an image area among the N areas.

16. The device according to claim 14, characterized in that The device also includes: A second receiving module, used for receiving third control information sent by the AP, wherein the third control information is used for instructing the image processing chip to process the preview image; The third processing module is used to process the preview image according to the third control information.

17. An electronic device, characterized in that: It includes a processor and a memory, the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, it implements the steps of the image processing control method as described in any one of claims 1 to 5, or implements the steps of the image processing control method as described in any one of claims 6 to 8.

18. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, it implements the steps of the image processing control method as described in any one of claims 1 to 5, or implements the steps of the image processing control method as described in any one of claims 6 to 8.

Images