Image processing method and electronic equipment

Abstract

The embodiment of the invention provides an image processing method and electronic equipment. In the method, the electronic equipment can determine that the dump condition is met under the condition that the performance is insufficient or a multi-frame fusion technology needs to be frequently used. Under the condition that the dump condition is met, the electronic equipment can execute dump operation on the collected multiple frames of original images, and the multiple frames of original images are stored in the ROM from the RAM. And the electronic equipment also stores the metadata respectively corresponding to the multiple frames of original images into the ROM. Subsequently, when the electronic equipment meets the synthesis condition, the multiple frames of original images are loaded into the RAM from the ROM to synthesize the images. By implementing the technical scheme provided by the invention, performance reduction of the electronic equipment can be avoided when a multi-frame fusion technology is used.

Description

Technical Field

[0001] The present application relates to the field of terminal and image processing technology, and in particular to an image processing method and an electronic device. Background Art

[0002] Multi-frame fusion technology allows electronic devices to use multiple frames of original images taken continuously to synthesize an image (the photo that is finally displayed to the user). When using multi-frame fusion technology, electronic devices can fuse the advantages of multiple frames of original images. For example, in low-light conditions, electronic devices can use long-exposure original images and short-exposure images to synthesize an image through multi-frame fusion technology. Long-exposure original images can capture more light through long exposure, and short-exposure images can avoid image blur. Then, multi-frame averaging noise reduction can effectively reduce noise. The image obtained through multi-frame fusion technology can significantly improve the quality of the film. Overcome common limitations and problems in single-frame shooting, such as low light conditions or excessive noise.

[0003] How to reasonably use multi-frame fusion technology to enable electronic devices to obtain high-quality images is worth discussing. Summary of the invention

[0004] The embodiments of the present application provide an image processing method and an electronic device for optimizing the process of synthesizing multiple frames of original images into one image.

[0005] In a first aspect, the present application provides an image processing method, which is applied to an electronic device, wherein the electronic device includes a memory and a first memory other than the memory, and the method includes: starting a camera, capturing an original image through the camera of the electronic device, and recording the original image into the memory; receiving multiple instructions for shooting images, the multiple instructions including a first instruction for shooting a first image received at a first time, and an instruction for shooting a second image received at a second time; the second time is earlier or later than the first time; performing a dump operation when a dump condition is met; the dump operation includes: recording the identifiers of M frames of original images matching the first time and the identifiers of metadata corresponding to the M frames of original images respectively, into first synthetic information; dumping the first original image in the M frames of original images from the memory to the first memory; dumping the original images in the M frames of original images except the first original image in The original images of the M frames have been dumped into the first memory before the dump operation is performed; whether each original image in the M frames has been dumped into the first memory before the dump operation is performed is determined by query data; the query data includes an identifier of the original image stored in the first memory; when the dump operation is performed, the metadata corresponding to the M frames of original images are stored in the first memory; the metadata corresponding to one frame of original image is used to indicate the acquisition information of the one frame of original image and the shooting information corresponding to the first time; the original images in the M frames of original image except the first original image are also used to synthesize the second image, and M is an integer greater than 1; after the dump operation is performed, when the synthesis condition is met, the M frames of original image and the metadata corresponding to the M frames of original image are obtained from the first memory through the first synthesis information, and loaded into the memory to synthesize the first image.

[0006] In the above embodiment, satisfying the dump condition means that the current performance of the electronic device is insufficient. At this time, using the multi-frame fusion technology will cause problems such as excessive memory usage and excessive processor load. This will cause the performance of the electronic device to decline to the point of being stuck. By performing a dump operation, the original image and metadata that were originally required to be recorded in the memory can be stored in a memory (first memory) outside the memory, releasing the storage pressure of the memory, suspending the multi-frame fusion technology, and avoiding increasing the load on the processor. When the synthesis condition is met later (indicating that the electronic device has good performance and supports multi-frame fusion technology), the original image and metadata are loaded into the memory to synthesize the image. In this way, the performance of the electronic device can be avoided from being degraded when using the multi-frame fusion technology.

[0007] In combination with the first aspect, in some embodiments, the method also includes: when the dump condition is not met, synthesizing the first image through the M frames of original images in the memory and the metadata corresponding to the M frames of original images respectively; when the dump operation is not performed, the metadata corresponding to the M frames of original images respectively are stored in the memory.

[0008] In the above embodiment, failure to meet the dump condition means that the electronic device currently has good performance, and the use of multi-frame fusion technology will not cause the electronic device to freeze. Therefore, there is no need to perform a dump operation, and the electronic device can use the processor to directly synthesize the image through the original image and metadata in the memory. In the process of synthesizing the image, the memory plays a role of temporary storage, providing the electronic device with the data and instructions required for synthesizing the image.

[0009] In combination with the first aspect, in some embodiments, satisfying the dump condition includes: the electronic device is in a continuous shooting mode; wherein the continuous shooting mode is used for the electronic device to continuously shoot at least two images when a shooting control is long pressed.

[0010] In the above embodiment, the continuous shooting mode means that the electronic device needs to continuously shoot multiple images in a short time and needs to frequently use the multi-frame fusion technology. Entering the continuous shooting mode can be regarded as a dump condition.

[0011] In combination with the first aspect, in some embodiments, before obtaining the M frame original image and the metadata corresponding to the M frame original image respectively from the first memory through the first synthesis information, the method also includes: obtaining the first synthesis information from the synthesis information queue; the method also includes: obtaining the second synthesis information from the synthesis information queue; the second synthesis information includes the identifier of the E frame original image matching the second time, and the identifier of the metadata corresponding to the E frame original image respectively; the second time is the time when the second instruction is received, and E is equal to M; through the second synthesis information, the E frame original image and the metadata corresponding to the E frame original image respectively are obtained from the first memory, and loaded into the memory to synthesize the second image.

[0012] In the above embodiment, the existence of the synthesis information queue enables the electronic device to store multiple synthesis information. When the synthesis condition is met, the synthesis of the image is realized by the synthesis information in the synthesis information queue, and no image loss will be caused.

[0013] In combination with the first aspect, in some embodiments, satisfying the synthesis condition includes at least one of the following: the electronic device turns off the camera, the camera runs in the background, and the performance of the electronic device satisfies a preset condition.

[0014] In the above embodiment, the performance of the electronic device meets the preset conditions, which means that the performance of the electronic device is good, and even if the multi-frame fusion technology is used, the performance will not be reduced to a freezing state. The good performance of the electronic device includes but is not limited to at least one of the following: small memory usage (for example, the occupancy rate is less than 50% of the total memory capacity), small processor load (the number of tasks to be processed is less than a preset number, for example, 10).

[0015] In combination with the first aspect, in some embodiments, the query data is constructed through a dictionary tree; the dictionary tree includes a root node and X leaf nodes, and the leaf node is a node in the dictionary tree that does not have a child node; the X indicates the number of original images dumped into the first memory; a path between the root node and a leaf node is used to determine the identification of a frame of the original image, and the identification of the original image determined by different paths is different.

[0016] In the above embodiment, the path from the root node to the leaf node in the dictionary tree represents the identifier of the original image, which means that the height of the dictionary tree is determined by the length of the identifier of the original image. Generally speaking, the identifiers of the original images are of equal length, so the heights of the paths in the dictionary tree are consistent, and each time the identifier of the original image is searched, the search efficiency is relatively uniform.

[0017] In combination with the first aspect, in some embodiments, when the number of cameras that capture the original image is equal to 1, the identification of the frame of original image is represented by the acquisition time of the frame of original image; when the number of cameras that capture the original image is greater than 1, the identification of the frame of original image includes the acquisition time of the frame of original image plus the serial number of the camera that captures the frame of original image; the nodes other than the root node on the path are all used to record the characters in the identification of the frame of original image, and the path includes a first node and a second node, the first node is used to record the first character in the identification of the frame of original image, and the second node is used to record the second character in the identification of the frame of original image; on the path, the first node is closer to the root node than the second node, then compared with the second character, the first character in the identification of the frame of original image is in a higher position; the X leaf nodes in the dictionary tree are also used to construct a serialized linked list, which includes linked list nodes of X frames of original images; when the X leaf nodes and the X linked list nodes are sorted from front to back, the i-th linked list node is obtained through the i-th leaf node, and the i-th linked list node is used to determine the information of the i-th original image, the linked list node of the i-1-th original image, and the linked list node of the i+1-th original image; the first acquisition time of the i-1-th original image is before the second acquisition time of the i-th original image, or the first acquisition time is the same as the second acquisition time and the first serial number is less than the second serial number; the first serial number is the serial number of the camera that acquires the i-1-th original image; the second serial number is the serial number of the camera that acquires the i-th original image, and i is an integer greater than 1 and less than X.

[0018] In the above embodiment, the construction of the serialized linked list is conducive to more intuitively displaying the identifiers of each original image recorded in the dictionary tree.

[0019] In combination with the first aspect, in some embodiments, the method further includes: before dumping the first original image in the M frames of original images from the memory to the first memory, determining that the identifier of the first original image is not recorded in the dictionary tree; wherein, determining that the identifier of the first original image is not recorded in the dictionary tree specifically includes: determining the maximum identifier from the identifiers of the M frames of original images, searching the maximum identifier through the dictionary tree and determining the target identifier, and when the maximum identifier is recorded in the dictionary tree, the target identifier is the maximum identifier; when the maximum identifier is not recorded in the dictionary tree, the target identifier is the minimum identifier among the brother identifiers recorded in the dictionary tree, and the brother identifier is greater than the maximum identifier; in the serialized linked list, starting from the linked list node where the target identifier is located, traversing the linked list nodes in the serialized linked list forward, and when the second linked list node is not determined before the first linked list node is determined, determining that the identifier of the first original image is not recorded in the dictionary tree; the identifier of the original image recorded by the first linked list node is less than the identifier of the first original image; the identifier of the original image recorded by the second linked list node is equal to the identifier of the first original image.

[0020] In the above embodiment, the dictionary tree is used to locate a target identifier, and then the query for the identifier of the first image is queried from the target identifier forward through the serialized linked list, and the query for the dictionary tree is converted into a query for the serialized linked list. Since the structure of the serialized linked list is simpler than the dictionary tree, the query will be faster. This is conducive to realizing a quick query for the identifier of the first image.

[0021] In combination with the first aspect, in some embodiments, after dumping the first original image in the M frames of original images from the memory to the first storage, the method also includes: updating the dictionary tree and the serialized linked list, the update including: adding a new node in the dictionary tree to obtain a first path, the first path is used to determine the identification of the first original image, adding the information of the first new node to the child node distribution information of the parent node corresponding to the first new node, and adding the linked list node of the first original image to the serialized linked list; when the number of new nodes is 1, the new node is the first new node, and when the number of new nodes is greater than 1, the first new node is the new node closest to the root node among the new nodes.

[0022] In the above embodiment, after dumping the original image into the first memory, the dictionary tree and the serialization linked list need to be updated. This facilitates the subsequent dumping of the same original image, which can be stored in the first memory, and the electronic device does not need to dump the same original image again, and can directly reuse it.

[0023] In combination with the first aspect, in some embodiments, the method also includes: after synthesizing the first image, subtracting 1 from the usage count; the usage count is used to indicate the number of times the first original image is used to synthesize an image; when the usage count is 0, deleting the new node in the updated dictionary tree, deleting the information of the new node from the child node distribution information of the parent node, and deleting the linked list node of the first original image from the updated serialized linked list.

[0024] In the above embodiment, the usage count of the original image can conveniently record the reuse of the original image. A usage count of 0 indicates that the original image has completed image synthesis and no more images need to be synthesized using the original image. The identifier of the original image can be deleted from the dictionary tree to simplify the dictionary tree.

[0025] In a second aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method implemented in the first aspect.

[0026] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on an electronic device, enable the electronic device to execute the method implemented in the first aspect.

[0027] In a fourth aspect, an embodiment of the present application provides a chip system, which is applied to an electronic device. The chip system includes one or more processors, and the processor is used to call computer instructions so that the electronic device executes the method implemented in the first aspect.

[0028] In a fifth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the method implemented in the first aspect.

[0029] It is understandable that the electronic device provided in the second aspect, the computer storage medium provided in the third aspect, the chip system provided in the fourth aspect, and the computer program product provided in the fifth aspect are all used to execute the method provided in the embodiment of the present application. Therefore, other beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of a composite image when no dump is performed is shown; Figure 2 A set of exemplary user interfaces for burst mode is shown; Figure 3A schematic diagram showing repeated dumping due to reuse of the original image when shooting with a single camera; Figure 4 A schematic diagram showing the non-dumping due to reuse of the original image when shooting with a single camera; Figure 5 A schematic diagram showing the situation in which the original image is lost without repeated dumping when shooting with multiple cameras; Figure 6 A schematic diagram showing that when multiple cameras are shooting, the original image is not dumped repeatedly and is not lost; Figure 7 An exemplary red-black tree when shooting with a single camera is shown; Figure 8 An exemplary red-black tree for multi-camera shooting is shown; Fig. 9 An exemplary red-black tree for multi-camera shooting is shown; Fig.10 An exemplary dictionary tree involved in single-camera shooting is shown; Fig.11 An exemplary dictionary tree involved in multi-camera shooting is shown; Fig.12 shows a schematic diagram involved in traversing a dictionary tree; Fig.13 An exemplary flow chart of implementing an image processing method for capturing images when multiple cameras are used for capturing images is shown; Fig.14 A schematic diagram is shown in which a target identification is determined; Fig.15 A schematic diagram of determining a brother node is shown; Fig.16 A schematic diagram is shown when determining a target identifier based on a brother node; Fig.17 A schematic diagram is shown when adding a new node to the dictionary tree; Fig.18 shows a schematic diagram involved in deleting a node in a dictionary tree; Fig.19 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In some embodiments, the electronic device loads multiple frames of original images captured by the camera into a random access memory (RAM). When an instruction to capture an image is detected, the central processing unit (CPU) of the electronic device can quickly read the multiple frames of original images from the RAM to synthesize an image. The central processing unit can be referred to as a processor.

[0032] Here, RAM is the memory of the electronic device, and the data and instructions stored therein can be directly read by the processor of the electronic device to quickly process the data and instructions. Therefore, loading the original image into the memory can quickly use the original image to synthesize the image provided to the user. The original image refers to the initial image data generated by the camera sensor after capturing light. The original image has not undergone any advanced processing or optimization and is not directly presented to the user.

[0033] like Figure 1 As shown, after starting the camera, the electronic device continuously captures the original image through the camera and obtains the capture information of each frame of the original image. The electronic device loads the original image captured by the camera into the RAM. Specifically, it can be recorded in the buffer area 1 of the RAM, and the buffer area 1 supports the storage of multiple frames of original images. The electronic device obtains the original image frame by frame from the buffer area 1 of the RAM, and after simple processing of the original image, a preview image can be displayed on the screen. Here, simple processing does not include multi-frame fusion processing, which is an image processing technology based on processing a single frame of the original image to obtain a preview image.

[0034] At time 1, an instruction to shoot image 1 is received. In response to the instruction, the electronic device synthesizes image 1 by using M frames of original images matching time 1 in the RAM and metadata corresponding to the M frames of original images in the RAM.

[0035] It should be noted here that the metadata corresponding to a frame of original image can be used to indicate the acquisition information of the frame of original image and the shooting information at the shooting time. Taking the shooting time as time 1 for explanation, the metadata corresponding to the frame of original image includes the acquisition information corresponding to the frame of original image and the shooting information at time 1. Among them, the acquisition information includes the information when acquiring a frame of original image, including the parameters of the camera that acquires the frame of original image, the shooting environment parameters and other parameters. Among them, the parameters of the camera may include the focus distance, exposure time and other parameters when acquiring the frame of original image. Environmental parameters include the ambient light intensity when acquiring the frame of original image, etc. The shooting information at time 1 includes but is not limited to one or more of the following: ambient light intensity at time 1, white balance parameters at time 1, etc.

[0036] For continuous shooting scenarios such as high-speed continuous shooting that require taking a large number of images in a short period of time, the electronic device can also create a synthesis information queue in the memory to record the synthesis information of the image to be synthesized. In this way, it can support the electronic device to record the synthesis information of the image into the synthesis information queue when there is no time to synthesize the image, and then obtain the original image and metadata required to synthesize the image based on the synthesis information of the image to synthesize the image. In this way, in high-speed continuous shooting scenarios where multiple images need to be synthesized continuously, there will be no image loss due to untimely processing.

[0037] In a continuous shooting scenario, in response to an instruction to shoot an image, the electronic device determines the address information of the M frames of original images for synthesizing the image, and records it in the synthesis information of the image. In addition, the electronic device generates metadata corresponding to the M frames of original images, and then records the address information of the metadata in the synthesis information of the image. Then, the synthesis information is recorded in the synthesis information queue. Subsequently, the electronic device can obtain the synthesis information from the synthesis information queue in the synthesis order, and obtain the original image and metadata required for the synthesis image through the synthesis information to synthesize the image.

[0038] refer to Figure 1 Middle (1) - Figure 1 In the continuous shooting scene shown at (4), the black arrow at (1) indicates that at time 11, an instruction to shoot image 11 is received, the black arrow at (2) indicates that at time 12, an instruction to shoot image 12 is received, the black arrow at (3) indicates that at time 13, an instruction to shoot image 13 is received, and the black arrow at (4) indicates that at time 14, an instruction to shoot image 14 is received.

[0039] The electronic device may record synthesis information a1 of image 11, synthesis information a2 of image 12, synthesis information a3 of image 13, and synthesis information a4 of image 14 into synthesis information queue a. Subsequently, the electronic device obtains synthesis information from synthesis queue a to synthesize images according to the synthesis order.

[0040] The synthesis order includes but is not limited to at least one of the following: images with earlier shooting time are synthesized first, and images with later shooting time are synthesized first. The shooting time of the image is the time when the instruction to shoot the image is received. For example, the images with later shooting time are synthesized first in consideration of the following usage scenario: after taking multiple photos, the user will first click on the photo in the lower left corner, and then continue to slide to view the previously taken photos.

[0041] For example, if the synthesis information queue a is a first-in, last-out queue, images shot later are synthesized first. Then, synthesis information a4 is first obtained from the synthesis information queue a, and image 14 is synthesized using synthesis information a4. Then, synthesis information a3 is obtained to synthesize image 13. Then, synthesis information a2 is obtained to synthesize image 12. Then, synthesis information a1 is obtained to synthesize image 11.

[0042] The synthesis information for synthesizing an image includes the address information of the M frames of original images and the metadata corresponding to each frame of the original image required for synthesizing the image. The address information is used to determine the storage address of the original image and metadata required for synthesizing the image in the memory. Through the address information, the M frames of original images and the metadata corresponding to each frame of the original image in the M frames of original images can be obtained to synthesize the image.

[0043] It should be noted here that the aforementioned Figure 1 In the example, M is 4. In actual situations, the number of frames of the original image used to synthesize an image can be other values, such as 10 frames, 5 frames, etc., which is not limited in the embodiment of the present application.

[0044] The multi-frame fusion technology needs to use multiple frames of original images to synthesize an image at a time. The data volume of one frame of the original image (for example, 15MB) is large, which requires more memory space. In addition, a large number of image processing algorithms are involved in the synthesis process, which will increase the processor load. When the performance of an electronic device is insufficient, if the multi-frame fusion technology is still used, the performance of the electronic device may further decline. Or, when the multi-frame fusion technology needs to be used frequently, the performance of the electronic device may decline.

[0045] In order to reasonably use the multi-frame fusion technology and avoid the performance degradation of the electronic device due to the multi-frame fusion technology, causing freezes and affecting the normal operation of the electronic device. In the case where the performance of the electronic device is insufficient or the multi-frame fusion technology needs to be used frequently, the electronic device determines that the dump condition is met.

[0046] When it is determined that the dumping condition is met, the electronic device performs a dumping operation to dump the data (including at least the original image and metadata) originally recorded or required to be recorded in the RAM and used for synthesizing the image to another memory (referred to as memory 1) other than the RAM. The dumped data is deleted in the RAM to release memory space. When the synthesis condition is met, the data required for synthesizing the image is loaded from memory 1 to the RAM to synthesize the image.

[0047] It should be noted that the memory 1 may be a read-only memory (ROM), a flash memory, etc. The following description will be made by taking the memory 1 as a ROM as an example.

[0048] The performance of an electronic device can be characterized by at least one of the following indicators, but not limited to: the temperature of the electronic device, the memory usage, and the processor load. The higher the temperature of the electronic device, the worse the performance. The temperature of the electronic device can be represented by the temperature of the processor or the temperature of the back shell of the electronic device. The larger the memory usage, the smaller the available memory space, and the worse the performance of the electronic device. The greater the processor load, the worse the performance of the electronic device.

[0049] Here, the frequently used multi-frame fusion technology mentioned above includes: the electronic device is in a continuous shooting mode. The continuous shooting mode is used for the electronic device to continuously shoot at least two images. For the relevant content related to the continuous shooting mode, please refer to the following Figure 2 Description.

[0050] It should be noted here that it is not limited to continuous shooting mode, nor is it limited to scenes where multi-frame fusion technology is frequently used. Other image processing tasks that consume a lot of power and consume a lot of time (such as AI portrait processing) can also meet the dumping conditions.

[0051] like Figure 2 As shown in (1), in the continuous shooting mode, when the user long presses the shooting control 201, the electronic device can receive multiple image shooting instructions successively, and one instruction is used to shoot one image. At this time, in response to the operation of the shooting control 201, the electronic device can use the multi-frame fusion technology multiple times to obtain multiple images. And the images shot in the continuous shooting mode are recorded in the album. An example of the album can be Figure 2 In response to an operation (such as a click operation) on the continuous shooting album 202, the electronic device can display the images taken in the continuous shooting mode. An exemplary user interface involved in the display can refer to the following Figure 2 As shown in (3).

[0052] In some possible implementations, when the electronic device performs a dump operation, the electronic device may generate an image file in the ROM for each image to be synthesized, where the image file is used to record the original image and metadata required to synthesize an image.

[0053] When performing the dump operation, for the continuous shooting scene, the electronic device can still create a synthesis information queue a and record the synthesis information queue a into the ROM to record the synthesis information a of the image. The synthesis information a of an image includes the address of the image file synthesized with the image. Through the address of the image file, the original image and metadata required for synthesizing the image can be obtained to synthesize the image.

[0054] like Figure 3 As shown, for Figure 1 Middle (1) - Figure 1In the continuous shooting scene shown in (4), if the electronic device cannot synthesize the images in time, the original images (M frames) and metadata (one frame of original data corresponds to one metadata) required for synthesizing images 11-14 can be dumped into ROM in the order of shooting time. Then they are recorded in image files 1-image files 4 respectively. For images with earlier shooting time, the address of the image file can be first recorded in the synthesis information queue a. After the dumping conditions are met, the original image and metadata required for synthesizing the image are obtained through the synthesis information queue a and loaded into RAM to synthesize the image.

[0055] It should be noted that in a continuous shooting scenario, part of the original image used when shooting two adjacent images can be the same, which means that part of the original image can be reused to synthesize different images. Figure 3 As shown, in the above implementation, for the case where the original image that can be reused is repeatedly dumped, the number of times an original image is reused is the number of times it is repeatedly dumped. Figure 3 As shown, the original images used in the composite image 11 include original images N-original images N+3, and the original images used in the composite image 12 include original images N+1-original images N+4. Among them, the original images that are repeatedly dumped include original images N+1-original images N+3. The original images of the composite image 13 also include original images N+2-original images N+4 that are repeatedly dumped, and the original image of the composite image 14 also includes original image N+5 that is repeatedly dumped. The original images N+2 and N+3 are also repeatedly dumped twice.

[0056] Repeated dumping not only wastes ROM storage space, but also wastes computing resources. In addition, the original image has a large amount of data, which takes a long time to dump, and repeated dumping will also increase the dumping time.

[0057] In order to avoid repeated dumping when the original image is reused, an image processing method is proposed. In this method, when the electronic device dumps the original image from RAM to ROM, it first determines whether the original image has been recorded in the ROM. If it has been recorded in the ROM, it means that the original image is a reused original image and no longer needs to be dumped. If it has not been recorded in the ROM, the electronic device records the original image in the ROM.

[0058] In the implementation of the image processing method involved in the embodiment of the present application, after starting the camera, the electronic device collects the original image through the camera and records the original image into the RAM. At time 1, an instruction to capture image 1 (recorded as instruction 1) is received, and when the dump condition is met, the electronic device performs the dump operation.

[0059] The dump operation (dump operation 1) is performed, including: determining the identifier of the M frames of original images matching time 1 in the memory, and recording them in the synthesis information 1. Dumping the original image 1 in the M frames of original images from the RAM to the ROM, and deleting the original image 1 from the memory after the dump. The other original images in the M frames of original images except the original image 1 are called original images 2. The original image 2 is the image in the M frames of images that has been dumped into the ROM before the dump operation 1 is performed. Whether each original image in the M frames of original images has been dumped into the ROM before the dump operation 1 is performed is determined by querying data. The query data includes the identifier of the original image recorded in the ROM. The synthesis information 1 also includes the identifier of the metadata corresponding to each original image in the M frames of original images. When the dump operation 1 is performed, the metadata corresponding to each original image in the M frames of original images is stored in the ROM. The original image 2 is also used to synthesize the image 2, and the instruction for shooting the image 2 (recorded as instruction 2) is earlier or later than the instruction 1. M is an integer greater than 1.

[0060] It should be noted here that the M frames of original images matching time 1 include but are not limited to at least one of the following: the M frames of images in the memory whose acquisition time is closest to time 1, and the M frames of images whose acquisition time is closest to time 1 among the original images whose acquisition time is less than or equal to time 1.

[0061] After the dump operation is performed, when the synthesis conditions are met, the M frames of original images and metadata corresponding to each frame of the M frames of original images are obtained from the ROM through the synthesis information 1, and loaded into the ROM to synthesize the image 1.

[0062] An example of image 1 and image 2 can be Figure 4 At least two images among images 11 to 14. Figure 4 As shown, for Figure 1 Middle (1) - Figure 1 In the continuous shooting scene shown in (4), if the electronic device cannot synthesize the images in time, the original images (M frames) and metadata (one frame of original data corresponds to one metadata) required for synthesizing images 11-14 can be dumped into ROM in the order of shooting time. When dumping the original images from RAM to ROM, the original images already in ROM are no longer dumped repeatedly to avoid wasting ROM space. For example, the original images required for synthesizing image 12 include original images N+1-original images N+4. When dumping the original images required for synthesizing image 12 into ROM, original images N+1-original images N+3 have been recorded in ROM. Original images N+1-original images N+3 can be reused and no dumping is performed. Only original image N+4 needs to be dumped into ROM. The reasons why original images N+1-original images N+3 have been recorded in ROM include: the electronic device has executed the operation of dumping the original images required for synthesizing image 11 (including original images N-original images N+3) into ROM.

[0063] When dumping, the electronic device also generates synthesis information of the image and records it in the ROM. When there are many images to be synthesized, the synthesis information of each image can also be recorded in a synthesis information queue (such as synthesis information queue 1). For example, Figure 4 In the example, the synthesis information of images 11 to 14 can be recorded in the synthesis information queue 1. When the synthesis condition is met, the synthesis information is obtained from the synthesis information queue according to the synthesis order. Then, the original image and metadata required for the synthesis image are obtained through the synthesis information to synthesize the image.

[0064] In the image processing method, the synthesis information of the image includes the identifier of the original image synthesized with the image and the identifier of the metadata. The identifier of the original image is used to indicate the original image, and the storage address of the original image in the ROM can be determined by the identifier of the original image. The identifier of the metadata is used to determine the storage address of the metadata in the ROM. In some possible cases, the name of the synthesis information can carry the name of the image to be synthesized, so as to construct a corresponding relationship between the image to be synthesized and the synthesis information.

[0065] The electronic device may record the synthesis information of image 1 (recorded as synthesis information 1) and the synthesis information of image 2 (recorded as synthesis information 2) into synthesis information queue 1. When the synthesis condition is met, the electronic device obtains synthesis information 1 from synthesis information queue 1 to synthesize image 1.

[0066] The electronic device may also obtain synthesis information 2 from the synthesis information queue. Synthesis information 2 includes an identifier of the original image of frame E matching time 2, and an identifier of metadata corresponding to each original image in the original image of frame E. Time 2 is the time when instruction 2 (an instruction for synthesizing image 2) is received, and E is equal to M. Through synthesis information 2, the original image of frame E and metadata corresponding to each original image in the original image of frame E are obtained from ROM and loaded into RAM to synthesize image 2.

[0067] The synthesis information queue may be a first-in-first-out queue or a first-in-last-out queue. For the synthesis information 1 and synthesis information 2 mentioned above, if the synthesis information queue is a first-in-last-out queue, and the synthesis information 1 enters the synthesis information queue before the synthesis information 2, the electronic device synthesizes the image 2 based on the synthesis information 2 first, and then synthesizes the image 1 based on the synthesis information 1.

[0068] In some possible implementations, the acquisition timestamp of the original image (referred to as acquisition time) is used as the identifier of the original image. However, in a scenario where multiple cameras are used to shoot, if the acquisition time is used as the identifier of the original image, the dump will fail. Because when the M frames of original images that synthesize an image come from multiple cameras, different original images acquired by different cameras can have the same acquisition time, which means that different original images can have the same identifier. When a frame of the original image (recorded as original image a) is recorded in the ROM, different original images with the same timestamp as the original image a will be mistakenly regarded as the original image a by the electronic device and cannot be recorded in the ROM. This will cause the original image of the synthesized image to be lost, further leading to synthesis failure. Synthesis failure includes: the synthesized image quality is poor or the image is not synthesized. Synthesis failure is presented to the user side, which means shooting failure, affecting the user experience.

[0069] like Figure 5 As shown in FIG, it is a continuous shooting scene captured by multiple cameras (including camera 1 and camera 2). Figure 5 At (1), in response to an instruction to capture image 11 at time 11, the electronic device determines the identifiers of the M frames of original images that match time 11 (here, the acquisition time). These include original image N-original image N+3 and original image K. The acquisition time of original image N and original image K is the same. The electronic device first dumps original image N-original image N+3 into ROM. Subsequently, before dumping original image K, the electronic device determines that the identifier of original image N already exists in ROM and is the same as the identifier of original image K. It mistakenly determines that original image K has been dumped into ROM, and no longer performs the dump operation on original image K, resulting in the loss of original image K. Subsequently, the electronic device no longer synthesizes image 11, or the synthesized image 11 is of poor quality.

[0070] It should be noted here that Figure 4 and Figure 5 All are continuous shooting scenes. Figure 4 It is a continuous shooting scene shot by a single camera. Figure 5 It is a continuous shooting scene under multi-camera shooting. Figure 5 For the description of other contents in the Figure 4 , I will not go into details here.

[0071] In some other possible implementations, in order to improve the probability of successful dumping, and thus improve the probability of successful synthesis of images. At the same time, the simplicity of the identification of the original image is maintained. When the number of cameras that collect the original image is equal to 1, the identification of a frame of the original image is represented by the acquisition time of the frame of the original image. When the number of cameras that collect the original image is greater than 1, the identification of a frame of the original image includes: the acquisition time of the frame of the original image plus the serial number of the camera that collects the frame of the original image. For example, the acquisition time of a frame of the original image is "123", and the serial number of the camera that collects the frame of the original image is 2, then the identification of the frame of the original image is "1232". Not limited to this, when the number of cameras that collect the original image is greater than 1, the identification of a frame of the original image may also include: the serial number of the camera that collects the frame of the original image plus the acquisition time of the frame of the original image. For example, the acquisition time of a frame of the original image is "123", and the serial number of the camera that collects the frame of the original image is "2", then the identification of the frame of the original image is 2123.

[0072] Figure 6 As shown in FIG. 1 , another continuous shooting scene captured by multiple cameras (including camera 1 and camera 2) is shown. Figure 5 , although the acquisition time of the original image N and the original image K is the same. However, the serial number of camera 2 is introduced into the identification of the original image K, and the serial number of camera 1 is introduced into the identification of the original image N. Then the identifications of the original image N and the original image K are different. The electronic device can dump both the original image N and the original image K into the ROM and successfully synthesize the image 1.

[0073] It should be noted that the electronic device uses numbers between "0-9" to represent the identifier of the original image. Compared with using other letters to represent the identifier of the original image, the process of constructing the query data can be simplified. It can also simplify the process of searching for the identifier of the original image in the query data.

[0074] In some possible implementations, a red-black tree can be used to construct query data. All nodes except leaf nodes in the red-black tree can indicate the identity of a frame of the original image. The red-black tree is a balanced binary search tree that can still guarantee high query efficiency in the worst case. There are certain rule restrictions when constructing the red-black tree. These rule restrictions can make the height of a red-black tree containing P nodes at most 2×log(P+1), and the worst query time is O(logP). The use of a red-black tree can quickly complete the query of the identity of the original image based on the query data. Determine whether the identity of a frame of the original image is recorded in the red-black tree, and then determine whether the original image of the frame has been dumped into the ROM.

[0075] The rules and restrictions when constructing a red-black tree include but are not limited to: the root node is a black node, the child nodes of any red node must be black, and the number of black nodes contained in all paths from any node to each of its leaf nodes is the same.

[0076] like Figure 7 An exemplary red-black tree for single camera shooting is shown. All nodes except leaf nodes in the red-black tree can indicate the identification of a frame of original image, including: All nodes except leaf nodes in the red-black tree can be used to record the identification of the original image. One node records the identification of one original image.

[0077] The rule restrictions of the red-black tree make the red-black tree have a faster query efficiency, but it also brings defects to the red-black tree, including but not limited to the following defects.

[0078] Defect 1: After the red-black tree is constructed, the process of deleting and updating nodes is cumbersome, which is not conducive to the update of the red-black tree. For example, when a frame of the original image is no longer used for synthesizing an image, the node that records the corresponding identifier of the frame of the original image needs to be deleted from the red-black tree. However, during the deletion process, operations such as balancing rotation and recoloring are required to make the red-black tree after deleting the node re-adapt to the aforementioned rules and restrictions.

[0079] Defect 2: The rule restrictions of the red-black tree will make the red-black tree taller as the number of nodes increases, and the query efficiency will also decrease.

[0080] Defect 3: The identifiers recorded in each node in the red-black tree are not serialized. When querying, it is not possible to quickly query the identifiers of other nodes by relying on the identifier recorded in one node.

[0081] Defect 4: When the data structure that needs to be recorded in the nodes of the red-black tree changes, all nodes that record data in the entire red-black tree need to be modified. Figure 7 In the red-black tree shown in FIG. 1 , if one more dimension of data needs to be added to the red-black tree, for example, the serial number of the camera that collected the original image, all nodes that have been recorded and identified need to be modified. Figure 8 As shown, the electronic device can modify the identifier recorded by each node from the acquisition time to the acquisition time plus the serial number of the camera. Fig. 9 As shown, a pointer can be added in each node to point to a linked list recording the serial number of the camera.

[0082] Considering that the identifiers of the original images are actually ordered when the images are taken continuously, and in order to overcome the defects of the red-black tree mentioned above, in some other possible implementations, a new container is proposed for constructing query data, which is a dictionary tree. By utilizing the orderliness between the identifiers of multiple frames of original images, a dictionary tree that is conducive to query can be constructed.

[0083] The dictionary tree includes a root node and X leaf nodes, and the leaf node is a node in the dictionary tree that does not have child nodes. X indicates the number of original images dumped into the ROM. A path from the root node to the leaf node is used to determine the identification of a frame of original image, and the identification of the original image determined by different paths is different. All nodes except the root node on a path are used to record the characters in the identification of a frame of original image. A node on a path is used to record a character in the identification of a frame of original image. A path includes node 1 and node 2. Among them, node 1 is used to record character 1 in the identification of a frame of original image, and node 2 is used to record character 2 in the identification of the frame of original image. On this path, node 1 is closer to the root node than node 2, so compared with character 2, character 1 is in a higher position in the identification of the frame of original image. An example of a dictionary tree can be referred to. Fig.10 The content shown in . Fig.10 In the dictionary tree shown in (1), the dictionary tree is constructed by taking the identification of a frame of original image as 4 characters as an example. Including the root node, the height of the dictionary tree is 5. The nodes between the root node and the leaf nodes of the dictionary tree can be called internal nodes.

[0084] In some possible implementations, the child nodes of any dictionary tree node (referred to as node) in the dictionary tree gradually increase in size when they are sorted from front to back. This allows the X leaf nodes in the dictionary tree to be used to construct a serialized linked list, which includes a linked list node of X frames of original images, which indicates the result of the identification of the X frames of original images being arranged from front to back, and the identification of the X frames of original images gradually increases in size from front to back. Fig.10 As shown in (1), the serialized linked list constructed by the X leaf nodes in the dictionary tree indicates the identifiers 4694, 5431, 5472, 5477, 5479, 5483, 5633 and 5671 from the front to the back.

[0085] like Fig.10 As shown in (2), a non-root node in the dictionary tree (dictionary tree node, referred to as node) can be used to record the value of the node, the distribution information of the node's child nodes, the leaf node mark, the pointer to the child node, and the pointer to the linked list node.

[0086] The value of a node can be represented by a character from "0-9", and its character type is char.

[0087] The distribution information of child nodes can be represented by a 10-bit binary number, whose character type is bit. If the f-th bit in the 10-bit binary number is 1, it means that there is a child node with value f among the child nodes of the node. If the f-th bit is 0, it means that there is no child node with value f among the child nodes of the node. f is a digit between 0 and 9.

[0088] The leaf node flag (isLeaf) is 0 or 1, which indicates whether the node is a leaf node. 0 means that the node is not a leaf node, and 1 means that the node is a leaf node.

[0089] The pointer to the child node is represented by an array. The array has 10 elements, denoted as children

[10] . The f-th element in the array records the pointer f, which points to the child node of the node with the value f.

[0090] The pointer pointing to the linked list node is used to point to the linked list node of the leaf node (which is also the linked list node of the original image indicated by the leaf node). The linked list node of a leaf node includes the value of the linked list node, and the value of the linked list node is the identifier of the image indicated by the linked list node. The linked list node of a leaf node may also include a pointer pointing to the original image indicated by the linked list node (for example, original image a), and the pointer may be represented as an image address. The linked list node of a leaf node may also include a pointer pointing to the previous linked list node and a pointer pointing to the next linked list node. The pointer of the previous linked list node can be used to determine the identifier and address of the previous frame of the original image of the original image a. The pointer of the next linked list node can be used to determine the identifier and address of the next frame of the original image of the original image a.

[0091] A node can only have one of the two pointers. The two pointers are: a pointer to a child node and a pointer to a linked list node. Whether a node is a leaf node determines what kind of pointer the node has, or one of the two pointers a node has is null. If a node is a leaf node, it has no child nodes but has a linked list node, the pointer to the child node is null, and the pointer to the linked list node is not null. If a node is not a leaf node, it has child nodes but no linked list node, the pointer to the child node is not null, and the pointer to the linked list node is null.

[0092] It should be noted here that the linked list nodes are also used to construct the aforementioned serialized linked list. When the X leaf nodes in the dictionary tree and the linked list nodes of the X original images in the serialized linked list are arranged from front to back, the i-th leaf node is used to record the pointer to the i-th linked list node, and the i-th linked list node is used to record the information of the i-th original image, the linked list node of the i-1th original image, and the linked list node of the i+1th original image. The acquisition time of the i-1th original image (recorded as acquisition time 1) is before the acquisition time of the i-th original image (recorded as acquisition time 2), or, the acquisition time 1 is the same as the acquisition time 2 and the sequence number 1 is less than the sequence number 2. Sequence number 1 is the sequence number of the camera that acquires the i-1th original image. Sequence number 2 is the sequence number of the camera that acquires the i-th original image, and i is an integer greater than 1 and less than X. The i-1th original image can also be called the previous original image of the i-th original image. The i+1th original image can also be called the next original image of the i-th original image.

[0093] Based on the above content, we can know that compared with the red-black tree, the dictionary tree will be shorter. The height of the dictionary tree depends on the length of the original image logo. If the logo includes U characters, the height of the dictionary tree is U+1. The query efficiency of the dictionary tree is high, usually O(U). This overcomes the defect 2 of the red-black tree.

[0094] The dictionary tree arranges each character in the identifier of a frame node into different nodes, which is conducive to the expansion of the dictionary tree. When the data structure that needs to be recorded by the node in the dictionary tree changes, it can be changed by adding or reducing the nodes on the path. For example, if you need to add one-dimensional data (such as the serial number of the camera that captured the original image), you only need to add the serial number to the leaf node, and the serialization list also needs to be adaptively modified. This overcomes the defect of the red-black tree 4.

[0095] like Fig.11 As shown in FIG. 1 , it is an exemplary dictionary tree involved in shooting with multiple cameras (camera 1 and camera 2). Fig.10 The dictionary tree shown is modified, and the modification method includes: adding a child node after each leaf node of the original dictionary tree, and the newly added child node is used as a new leaf node. The value of the new leaf node can be used to record the serial number of the camera carried in the identifier of the original image. The original leaf node in the original dictionary tree is modified to an internal node, and its pointer to the child node and the pointer to the linked list node are modified at the same time. Each new leaf node in the new dictionary tree can construct a new serialized linked list. For example, the new serialized linked list indicates the identifiers 46941, 54312, 54721, 54722, 54772, 54792, 54832, 56332 and 56712 from front to back.

[0096] It should be noted here that Fig.10The dictionary tree shown in can also be an example of a dictionary tree built by an electronic device in a single camera shooting scenario. Fig.11 The dictionary tree shown in can also be an example of a dictionary tree built by an electronic device in a multi-camera shooting scenario. Fig.11 In the dictionary, the value of the leaf node in the tree can be used to record the serial number of the camera in the identifier corresponding to a frame of the original image.

[0097] By traversing the nodes in the dictionary tree, the electronic device can determine whether the identifier of the original image to be dumped into the ROM (identifier to be checked) is in the dictionary tree. The process of determining the identifier to be checked includes: the pointer to the child node recorded in the node enables the electronic device to complete the query quickly. The query process includes: after the electronic device determines in the dictionary tree the node to which the highest-order character of the identifier to be checked belongs. Through the pointer to the child node recorded in the node to which the highest-order character belongs (children1

[10] ), it can be determined whether the node to which the second-highest-order character in the identifier to be checked belongs exists. The determination process includes: looping through the pointers in children1

[10] until it is determined that children[char3] exists and children[char3] is not empty, then it is determined that the node to which the second-highest-order character in the identifier to be checked belongs exists. char3 is the value of the second-highest-order character. If the node to which the second-highest-order character belongs exists in the child node of the node to which the highest-order character belongs, the node to which the second-highest-order character belongs is located through the pointer to the child node. Then, through the pointer pointing to the child node recorded in the node to which the second highest character belongs, the node to which the next character belongs, and so on, it is determined whether the identifier to be checked is recorded in the dictionary tree.

[0098] refer to Fig.12 , take the identifier to be checked as "54772" as an example. The electronic device first determines the node 5 that is closest to the root node. The value of node 5 is the highest bit in "54772". Then, through the pointer pointing to the child node recorded in node 5, it is determined that the pointer children[4] exists and is not empty. The child node 4 of node 5 (referred to as node 4) is located through children[4]. Here, the value of node 4 is the second highest bit 4 in "54772". Subsequently, the electronic device determines that the pointer children[7] exists and is not empty through the pointer pointing to the child node recorded in node 4. The child node 7 of node 4 (referred to as node 7) is located through children[7]. Here, the value of node 7 is the third highest bit 7 in "54772". By analogy, the electronic device can determine the path where the identifier to be checked "54772" is located. And through the pointer pointing to the original image indicated by 54772 recorded in the linked list node (denoted as linked list node a) of the leaf node on the path, the original image indicated by the identifier "54772" to be checked is located.

[0099] Not limited to the above method, the process of determining the identifier to be checked may also include: the distribution information of the sub-nodes recorded in the node and the pointer pointing to the sub-nodes enable the electronic device to complete the query faster. The query process includes: after the electronic device determines in the dictionary tree the node to which the highest-order character of the identifier to be checked belongs. Through the distribution of the sub-nodes of the node to which the highest-order character belongs, the node to which the second-highest-order character in the identifier to be checked belongs can be determined. If there is a node to which the second-highest-order character belongs in the sub-node of the node to which the highest-order character belongs, the node to which the second-highest-order character belongs is located through the pointer pointing to the sub-node. Then, the node to which the next-highest-order character belongs is found through the sub-node distribution information of the node to which the second-highest-order character belongs, and so on, to determine whether the identifier to be checked is recorded in the dictionary tree.

[0100] like Fig.12 As shown, the identification to be checked is "54772" as an example. The electronic device first determines the node 5 closest to the root node. The value of node 5 is the highest bit in "54772". Then, through the child node distribution information of node 5 (0001010000B, from the highest bit to the lowest bit, it is the 9th, 8th, 7th, 6th, 5th, 4th, 3rd, 2nd, 1st bit), it is determined that the 4th bit is 1, then node 5 has a child node 4. Then, through the pointer children[4] pointing to child node 4 in node 5, the child node 4 of node 5 (referred to as node 4) is located. Here, the value of node 4 is the second highest bit 4 in "54772". Subsequently, the electronic device determines that the 7th bit is 1 through the child node distribution information of node 4 (0110001000B, from the highest bit to the lowest bit, it is the 9th, 8th, 7th, 6th, 5th, 4th, 3rd, 2nd, 1st bit), then node 4 has a child node 7. Then, through the pointer children[7] pointing to child node 7 in node 4, the child node 7 of node 4 (referred to as node 7) is located. Here, the value of node 7 is the third highest bit 7 in "54772". By analogy, the electronic device can determine the path where the identifier "54772" to be checked is located. And through the pointer pointing to the original image indicated by 54772 recorded in the linked list node (recorded as linked list node a) of the leaf node on the path, the original image indicated by the identifier "54772" to be checked is located.

[0101] It should be noted here that, continue to refer to Figure 10-12 As shown in the figure, the structure of the dictionary tree does not need to be modified when deleting and updating nodes. This overcomes the defect 1 of the red-black tree.

[0102] It should also be noted that the identifiers recorded in the dictionary tree can also be reflected in the serialized linked list. The serialized linked list is serialized, and when querying, you can rely on the identifier recorded by a node to quickly query the identifiers of other nodes through the serialized linked list. This overcomes the defect of the red-black tree 3. For example, refer to Fig.12As shown, the electronic device can locate the identifier "54722" and address of the previous frame of the original image indicated by the identifier "54772" to be checked through the pointer pointing to the previous linked list node recorded in the linked list node a (the pointer pointing to 54722).

[0103] If the identifier to be searched is not in the dictionary tree, the electronic device can also determine the target brother node by querying the dictionary tree. The target brother node is the smallest brother node of the highest-order unrecorded character in the identifier to be searched, or, when the highest-order unrecorded character does not have a smallest brother node, the target brother node is the smallest brother node of the lowest-order recorded character. The smallest brother node of a character has the same parent node as the node to which the character belongs, and is the smallest node among the child nodes of the parent node that is larger than the node to which the character belongs. Continue to refer to Fig.12 As shown, taking the identifier to be checked "55772" as an example, the highest unrecorded character in the identifier to be checked "55772" is the second highest bit 5, and its smallest brother node is node 6.

[0104] Subsequently, the electronic device can determine a target identifier through the target brother node, and the target identifier is the smallest identifier among the brother identifiers of the identifier to be checked. The brother identifier of the identifier to be checked is an identifier in the serialized linked list that is larger than the identifier to be checked. For example, through the distribution information of the child nodes of node 6 (0010001000B, from the highest bit to the lowest bit, respectively, the 9th, 8th, 7th, 6th, 5th, 4th, 3rd, 2nd, 1st bit), it is determined that the smallest child node of node 6 is 3, and then "563*" can be determined. Subsequently, the electronic device determines that the value of the smallest child node of node 3 is 3 through the dictionary tree, and then "5633*" can be determined. And so on, the identifier "56332" is determined as the target identifier.

[0105] Subsequently, through the linked list node where the target identifier is located, it is possible to query in the serialized linked list whether other identifiers to be checked are recorded in the dictionary tree. Among them, the original images indicated by other identifiers to be checked can be used to synthesize the same image with the original image indicated by the identifier to be checked.

[0106] In this way, compared to the method of obtaining other identifiers to be checked and re-querying in the dictionary tree when the identifier to be checked is not in the dictionary tree, some computing power can be saved. Because the identifiers of the original images with continuous acquisition time will be close in position in the dictionary tree, and the identifiers of the original images of each frame used to synthesize an image are usually continuous. It can be seen that the target identifier and the identifier to be checked usually have the same prefix. If the identifier to be checked is not in the dictionary tree, the method of returning the target identifier can save the time of searching the dictionary tree for the same prefix.

[0107] When shooting with a single camera, the M frames of images required to synthesize an image come from the single camera. When shooting with multiple cameras, the M frames of images required to synthesize an image come from multiple cameras. In the following text, two cameras (camera 1 and camera 2) are used as an example to illustrate that among the M frames of images, M1 frames of images come from camera 1 and M2 frames of images come from camera 2. Both M1 and M2 are numbers greater than or equal to 1. Fig.13 The exemplary flow chart of the image processing method for capturing images when multiple cameras are used is shown. The description of the process can refer to the following steps S101, S102, S103a-S105a, and S103b-S105b.

[0108] S101. Start the camera, the electronic device obtains original image data 1 through camera 1 in the camera and records it in RAM, and obtains original image data 2 through camera 2 in the camera and records it in RAM.

[0109] The original image data 1 includes the original image captured by the camera 1 and the capture information corresponding to the original image.

[0110] The original image data 2 includes the original image captured by the camera 2 and the capture information corresponding to the original image.

[0111] S102 . Receive multiple instructions for capturing images, including an instruction for capturing image 1 received at time 1 .

[0112] In some possible cases, the instruction is generated by a camera application and sent to a processor of the electronic device. Receiving multiple instructions for capturing an image includes the processor receiving an instruction for capturing an image sent by the camera application. One instruction is used to generate one image.

[0113] The multiple instructions include the aforementioned instruction for capturing image 1 received at time 1. In addition to instruction 1, the multiple instructions may also include an instruction for capturing image 2 received at time 2. Time 2 is before or after time 1.

[0114] After receiving an instruction 1 for capturing an image 1, the electronic device may perform an operation for synthesizing the image 1 based on the instruction 1 (see step S103b-step S105b). Alternatively, the electronic device may perform a dump operation based on the instruction 2 to store the original image and metadata required for synthesizing the image 1 into the ROM, and then synthesize the image 1.

[0115] It should be noted here that the dump operation takes a certain amount of time. When the electronic device performs the dump operation, if a new instruction to take an image is received, the electronic device can first record the new instruction in the instruction queue. After the dump operation is completed, the electronic device obtains the new instruction from the instruction queue. Then perform the dump operation on the new instruction. The instruction queue can be a first-in-first-out queue, or a first-in-last-out queue. When the instruction queue is a first-in-first-out queue, it means that the instruction received first in the instruction queue can be responded to first. When the instruction queue is a first-in-last-out queue, it means that the instruction received later in the instruction queue can be responded to first. Among them, the response instruction refers to performing a dump operation on the instruction.

[0116] When the dump condition is met, the following steps S103a to S105a are executed: After the dump operation is executed, the original image and metadata required for synthesizing the image are stored in the ROM, and then loaded into the RAM for image synthesis when the condition is met.

[0117] S103a. Determine the identifiers of the M1 frame original image and the M2 frame original image that match time 1, and record them in the synthesis information 1. The synthesis information 1 also includes the identifiers of the metadata corresponding to the M1 frame original image and the M2 frame original image. The synthesis information 1 is also used to indicate that the M1 frame original image and the M2 frame original image are used for synthesizing image 1.

[0118] When shooting with multiple cameras, the identification of the original image is represented by the acquisition time of the original image plus the serial number of the camera that acquired the original image.

[0119] The metadata corresponding to the original image does not involve multiplexing. In some possible cases, the metadata identifier may be the address of the metadata, which is used to find the metadata from the ROM.

[0120] The metadata corresponding to a frame of original image of the synthesized image 1 may be generated based on the acquisition information corresponding to the frame of original image and the shooting information at time 1 when the electronic device determines to use the frame of original image to synthesize the image 1.

[0121] S104a. The original images of the M1 frame and the M2 frame that are not recorded in the ROM are dumped from the RAM to the ROM, and the metadata corresponding to each frame of the M1 frame and the M2 frame are recorded in the ROM. Whether an original image frame is recorded in the ROM is queried through the dictionary tree.

[0122] The original image of the M1 frame original image and the M2 frame original image that is not recorded in the ROM is recorded as original image 1. The original images other than original image 1 in the M1 frame original image and the M2 frame original image are called original image 2. Here, original image 2 has been dumped into the ROM before executing dump operation 1. This means that original image 2 is reused, and in addition to being used to synthesize image 2, it can also be used to synthesize other images (such as image 2). Among them, dump operation 1 is a dump operation executed in response to instruction 1.

[0123] For each original image in the M1 frame original image and the M2 frame original image, the electronic device needs to determine whether the identifier of each original image frame has been recorded in the dictionary tree, so as to determine the original image 1 and the original image 2 therein.

[0124] Determining whether the identifier of each frame of the original image has been recorded in the dictionary tree, so as to determine the original image 1 and the original image 2, includes: the electronic device first determines the maximum identifier (a to-be-checked identifier) ​​in the M frames of the original image (M1 frame original image and M2 frame original image), searches the maximum identifier through the dictionary tree and determines the target identifier. In the case where the maximum identifier is recorded in the dictionary tree, the target identifier is the maximum identifier. In the case where the maximum identifier is not recorded in the dictionary tree, the target identifier is the minimum identifier among the brother identifiers recorded in the dictionary tree. Among them, the brother identifier is all identifiers recorded in the dictionary tree that are greater than the maximum identifier. After determining the target identifier, in the serialized linked list, starting from the linked list node where the target identifier is located, the linked list nodes in the serialized linked list are traversed forward. During the traversal, if the linked list node 2 is not determined before the linked list node 1 is determined, it is determined that the identifier of the original image 1 is not recorded in the dictionary tree. The identifier of the original image recorded in the linked list node 1 is less than the identifier of the original image 1, and the identifier of the original image recorded in the linked list node 2 is equal to the identifier of the original image 1. When the linked list node 3 is determined, it is determined that the identifier of the original image 1 has been recorded in the dictionary tree. The identifier of the original image recorded in the linked list node 3 is equal to the identifier of the original image 3.

[0125] Among them, searching the maximum identifier and determining the target identifier through the dictionary tree includes: when searching the characters in the maximum identifier in the dictionary tree, start searching from the highest character to the lowest character in the maximum identifier. Each time a character (the c-th character) is searched, not only the node to which the character belongs must be determined, but also the smallest brother node of the character must be determined as the target brother node. Let c=c+1, continue to search for the c-th character, not only the node to which the c-th character belongs must be determined. It is also necessary to determine the smallest brother node of the c-th character, and use the smallest brother node of the c-th character as the target brother node to update the target brother node. If the c-th character does not have a brother node, the target brother node is not updated with the smallest brother node of the c-th character, and the current target brother node remains unchanged. Let c=c+1 again, continue to search for the c-th character, until it is determined that the c-th character is not in the dictionary tree, or, it is determined that all characters in the maximum identifier are in the dictionary tree, and the maximum identifier is used as the target identifier. If the cth character is not in the dictionary tree, but the cth character has a smallest brother node, it is still necessary to use the smallest brother node of the cth character to update the target brother node and return the target brother node. Subsequently, the electronic device can determine the target identifier through the target brother node.

[0126] In the above content, determining the node to which the c-th character belongs, and then determining the minimum brother node of the c-th character, includes: using the formula lowbit(bit0&~(1<<(char0+1)-1) ) to determine the minimum brother node of the c-th character. The char0 in the formula represents the value of the c-th character, which is also the value of the node to which the c-th character belongs. Bit0 represents the child node information of the parent node of the node to which the c-th character belongs. 1<<(char0+1)-1) is used to generate a binary number, in which the lowest char0+1 bits in the binary number are all 0, and bit0 is a 10-digit number. In the formula, 1<<(char+1)-1) also means setting the lowest char0+1 0s in 0000000000B to 1. For example, if char0 is equal to 7, then 1<<(char+1)-1) is expressed as 0011111111B. "~" represents the negation operation, and "&" represents the AND operation. "bit0&~(1<<(char0+1)-1)" can be used to represent all child nodes greater than char0 in the child nodes of the node to which the c-th character belongs. lowbit(x)=x&(-x)=2 r , used to determine that the lowest bit 1 in x is at the rth position. Here, lowbit(bit0&~(1<<(char0+1)-1) ) is used to determine the smallest child node r whose value is greater than char0 among the child nodes indicated by bit0, and r is the smallest sibling node of the cth character.

[0127] Here, lowbit (bit0&~(1<<(char0+1)-1) ) can be realized by bit operation. Electronic devices can perform bit operation processing quickly. Usually, it only takes one clock cycle to complete.

[0128] It should be noted here that if the c-th character does not belong to a node, bit0 represents the child node information of the node to which the previous character of the c-th character belongs.

[0129] The electronic device can determine the target identifier through the target brother node, including: setting the target brother node as the node to be processed, obtaining the child node information of the node to be processed, and recording it as bit1. Then executing lowbit (bit1) to determine the smallest child node of the node to be processed. If the smallest child node is not a leaf node, perform operation 1, and operation 1 includes: updating the smallest child node to the node to be processed, obtaining the child node information of the node to be processed, and recording it as bit1, and then executing lowbit (bit1) to determine the smallest child node of the node to be processed. If the smallest child node is not a leaf node, perform operation 1 again until the leaf node is determined. The identifier recorded in the linked list node of the leaf node is used as the target identifier.

[0130] It should be noted here that if the result of lowbit(bit&~(1<<(char+1)-1) ) is 0, it means that the c-th character has no brother node.

[0131] refer to Fig.14 In the example in FIG. 1 , the identifiers of the M frames of the original image of image 1 are respectively "54312, 54721, 54722, 54752". The maximum identifier is: "54752". The electronic device searches the dictionary tree for the maximum identifier "54752" and determines the target identifier "54772". The determination process can refer to the following Fig.15 as well as Fig.16 Description.

[0132] like Fig.15 As shown in (1), the query starts from the highest bit 5 in "54752". When the node to which the second highest bit 4 belongs (node ​​4) is queried, the value of node 4 and the child node information of node 4 are used based on the above formula to determine that the smallest brother node of node 4 is node 6. The determination process includes: executing lowbit(0001010000B&~(1<<(4+1)-1) ) equals 2 6 Then the elder brother node of node 4 is node 6. Let node 6 be the target elder brother node. Among them, 0001010000B is the child node information of the parent node of node 4.

[0133] refer to Fig.15As shown in (2), for the third highest bit 7, when the node to which the third highest bit 7 belongs is found (node ​​7), the value of node 7 and the child node information of node 7 are used based on the above formula to determine that the smallest brother node of node 7 is node 8. The determination process includes: executing lowbit(0110001000B&~(1<<(7+1)-1) ) equals 2 8 Then the smallest brother node of node 7 is node 8. Then the smallest brother node 8 is used to update the target brother node 6 as the new target brother node. Among them, 0110001000B is the child node information of the parent node of node 7.

[0134] refer to Fig.16 As shown in (1), when the fourth high bit 5 is found, it is determined that the dictionary tree does not include the node to which the character 5 belongs, and the smallest brother node of the character 5 is determined to be node 7. The determination process includes: executing lowbit(1010000100B&~(1<<(5+1)-1) ) equals 2 7 . Then the smallest brother node of character 5 is node 7. Then use Fig.16 The brother node 7 shown in (1) updates the target brother node 7 as the new target brother node. Among them, 1010000100B is the child node information of the node to which the third highest 7 belongs.

[0135] Follow-up, confirm Fig.16 The child node information of the target brother node shown in (2) is 0000000100B. Then determine that lowbit (0000000100B) = 2 2 The smallest child node determined by the target brother node is node 2, and node 2 is a leaf node. Subsequently, the identifier "54772" recorded in the linked list node of node 2 is used as the target identifier.

[0136] refer to Fig.17 As shown in (1), in the serialized linked list, starting from the linked list node where "54772" is located, traversing the linked list nodes in the serialized linked list forward, it can be determined that the identifiers "54312, 54721, 54722" have all been recorded in the dictionary tree.

[0137] It can be seen that the original image indicated by the identifiers "54312, 54721, 54722" is original image 2. The original image indicated by the identifier "54752" is original image 1.

[0138] After step S104a, the electronic device also needs to add the identifier of the original image 1 to the dictionary tree to update the dictionary tree and the serialization linked list.

[0139] Among them, the update includes: adding a new node in the trie to obtain path 1, where path 1 is used to determine the identifier of the original image 1, adding the information of the new node 1 to the child node distribution information of the parent node corresponding to the new node 1, updating the pointer children[char2] in the pointer of the parent node corresponding to the new node 1 that points to the child node to point to the new node 1, and adding the linked list node of the original image 1 to the serialized linked list. When the number of new nodes is 1, the new node is the new node 1. When the number of new nodes is greater than 1, the new node 1 is the new node closest to the root node among the new nodes. Among them, char2 represents the value of the new node 1.

[0140] Among them, adding the information of the new node 1 to the child node distribution information of the parent node corresponding to the new node 1 includes: executing bit2 = bit2|(1<<char2). "=" represents assignment or update. The bit2 before update is the child node distribution information of the parent node corresponding to the new node 1. bit2|(1<<char2) means setting the char2-th bit in the bit2 before update from 0 to 1.

[0141] Reference Fig.17 As shown in (2) of the reference, taking the addition of the identifier "54752" to the trie as an example for illustration. The nodes to be added to the trie here are node 5 and node 2, which are used to represent the fourth character 5 and the fifth character 2 in the identifier "54752" respectively. Among the added new nodes, the new node closest to the root node is node 5. Add the information of node 5 to the child node information of the parent node (node 7) corresponding to node 5. The addition process includes: updating the child node information of node 7 (denoted as bit3), executing bit3 = 1010000100B|(1<<5), and updating the child node information of node 7 from 1010000100B to 1010100100B. Setting the 5th bit from 0 to 1 indicates that there is node 5 among the child nodes included in node 7. Point the children[5] in the pointer of node 7 that points to the child node to the child node 5 of node 7.

[0142] S105a. After determining that the synthesis condition is satisfied, obtain the original image of M1 frame and the original image of M2 frame from the ROM through the synthesis information 1, and the metadata corresponding to each frame of the original image, and load them into the RAM to synthesize the image 1.

[0143] Among them, the satisfaction of the synthesis condition includes but is not limited to at least one of the following: exiting the continuous shooting mode, the electronic device exiting the camera, switching the camera to run in the background, detecting an operation of the user viewing the image, and the good performance of the electronic device, that is, even using the multi-frame fusion technology will not cause the performance to drop to freezing. For the description of the synthesis information queue 1, reference can be made to the foregoing Figure 3-Figure 5The good performance of the electronic device includes but is not limited to at least one of the following: small memory usage, small processor load, etc.

[0144] For example, the synthesis condition may be satisfied when: the electronic device has good performance, and the electronic device exits the camera or switches the camera to the background. After the electronic device exits the camera or switches the camera to the background, if the electronic device is still performing a high-load task (such as playing a game), the image can be synthesized when the high-load task is completed and the electronic device has good performance.

[0145] The camera involved in the embodiments of the present application can also be understood as a camera application.

[0146] The synthesis information 1 records the identifiers of the M frames of original images (M1 frame original image and M2 frame original image), which can be used to determine the address of the original image, thereby obtaining the original image. Determining the address of the original image includes: determining the leaf node of the original image in the dictionary tree based on the identifier of the original image, and finding the original image in the ROM through the pointer pointing to the original image recorded in the linked list node of the leaf node. The pointer pointing to the original image can determine the address of the original image.

[0147] The synthesis information 1 records the metadata identifiers corresponding to the M frame original images (M1 frame original image and M2 frame original image), which can be used to determine the address of the metadata, thereby obtaining the metadata. The metadata identifier can be used to determine the address of the metadata, including: the metadata identifier is the address of the metadata.

[0148] After acquiring M frames and metadata corresponding to the original images of each frame from the ROM, they are loaded into the RAM to synthesize the image 1.

[0149] If the dump condition is not met, the dump is not performed. The following steps SS103b to S105b in the figure are performed. The image is synthesized by the original image in the memory and the metadata.

[0150] S103b. Determine synthesis information a, which includes the address information of the M1 frame original image and its metadata and the M2 frame original image and its original data matching time 1 in the RAM. Synthesis information a is also used to indicate that the M1 frame original image and the M2 frame original image are used to synthesize image 1.

[0151] S104b. Based on the address information in the synthesis information a, the original image of frame M1 and metadata corresponding to each original image are obtained from the RAM, and the original image of frame M2 and metadata corresponding to each original image are obtained from the RAM.

[0152] S105b. Synthesize image 1 based on the M1 frame original image and metadata corresponding to each original image and the M2 frame original image and metadata corresponding to each original image.

[0153] The contents involved in step S103b-step S105b can refer to the above Figure 1 The description of the related contents will not be repeated here.

[0154] It should be noted here that, since the original image may be reused, the electronic device may set a usage count (initialized to 0) for each frame of the original image. The usage count of an original image frame is used to indicate the number of times the original image frame is used to synthesize an image. Each time the electronic device queries the frame of image, the usage count of the original image frame can be increased by 1. After the electronic device uses the original image to synthesize an image once, the usage count of the original image frame is reduced by 1. When the usage count of an original image frame is 0, the identifier of the original image frame is deleted in the dictionary tree.

[0155] Deleting the identifier of the original image of the frame in the dictionary tree includes: determining all nodes (nodes to be deleted) that have no child nodes on the path corresponding to the identifier indicating the original image of the frame in the dictionary tree. And determining the highest-order node to be deleted and leaf nodes (leaf nodes to be deleted) among the nodes to be deleted. Iteratively deleting the nodes to be deleted, starting from the leaf nodes to be deleted to the highest-order node to be deleted. Delete the bit information of the highest-order node to be deleted from the child node distribution information of the parent node of the highest-order node to be deleted, set the pointer pointing to the highest-order node to be deleted in the parent node of the highest-order node to be deleted to null, and delete the linked list node of the leaf node to be deleted from the serialized linked list. Deleting the linked list node of the leaf node to be deleted includes: updating the linked list node of the previous leaf node of the leaf node to be deleted, and the linked list node of the next leaf node of the leaf node to be deleted. Make the pointer pointing to the next linked list node in the linked list node of the previous leaf node point to the linked list node of the next leaf node, and make the pointer pointing to the previous linked list node in the linked list node of the next leaf node point to the linked list node of the previous leaf node.

[0156] like Fig.18 As shown in the figure, it is a schematic diagram of deleting the identifier "54312" in the dictionary tree. The nodes on the path indicating the identifier "54312" that have no child nodes include node 2, node 1, and node 3. Among them, the highest-order node to be deleted is node 3, and the child node information of the parent node (node ​​4) corresponding to node 3 needs to be updated (from 0110001000B to 0110000000B). Delete the linked list node of the leaf node to be deleted (node ​​2) from the serialized linked list.

[0157] It should be noted here that, referring to the above Figure 10-12 , Figure 14-18The dictionary tree shown. The electronic device can also add -INF (infinitesimal) linked list nodes and INF (infinity) linked list nodes at the beginning and end of the serialized linked list. This allows the electronic device to always return a linked list node when searching in the serialized linked list. For example, if the identifier to be queried is greater than the maximum identifier of the original image recorded in the dictionary tree, the INF linked list node is returned.

[0158] It should also be noted that the storage time of the original image and metadata recorded in the ROM is time-limited. If the storage time is greater than the preset time (for example, 1 hour), the electronic device has not used the original image and metadata in the ROM for image synthesis. The electronic device can delete the original image and metadata whose storage time is greater than the preset time, and update the dictionary tree and serialization linked list. The calculation time of the preset time may include: starting from the time the original image and metadata are recorded in the ROM, or starting the timing after the electronic device is turned off. If the shutdown time reaches the preset time, the original image and metadata in the ROM are deleted after the power is turned on, and the dictionary tree and serialization linked list are updated.

[0159] It should also be noted that the establishment time of the dictionary tree includes: when it is determined that the dump condition is met, the dictionary tree can be created. Initially, there is only a root node in the dictionary tree. After the electronic device dumps the original image into the ROM, it can be recorded in the dictionary tree based on the identifier of the dumped original image. Dump one frame and record one frame to update the dictionary tree.

[0160] The following is an introduction to an exemplary electronic device provided in an embodiment of the present application.

[0161] Fig.19 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0162] The following is a detailed description of the embodiment using an electronic device as an example. It should be understood that the electronic device may have more than Fig.19 More or fewer components may be shown, two or more components may be combined, or there may be a different configuration of components. Fig.19 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0163] The electronic device may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194 and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0164] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0165] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.

[0166] The non-volatile memory may include a read-only memory (ROM) and a flash memory.

[0167] The random access memory is a volatile memory used to temporarily store data and instructions being processed. As a memory, the random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0168] The non-volatile memory may also store executable programs and data of users and applications, etc. The executable programs and data in the non-volatile memory may be loaded into the random access memory for direct reading and writing by the processor 110 .

[0169] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0170] In the embodiment of the present application, the processor 110 can call the computer instructions stored in the internal memory 121 to enable the electronic device to execute the method in the embodiment of the present application.

[0171] The present application also provides a chip system, which includes at least one processor for implementing the functions involved in the method executed by the electronic device in any of the above embodiments.

[0172] In one possible design, the chip system also includes a memory, which is used to store program instructions and data, and the memory is located inside or outside the processor.

[0173] The chip system may be composed of the chip, or may include the chip and other discrete devices.

[0174] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0175] Optionally, the memory in the chip system may be one or more. The memory may be integrated with the processor or may be separated from the processor, which is not limited in the embodiment of the present application.

[0176] Exemplarily, the memory can be a non-transitory processor, such as a read-only memory ROM, which can be integrated with the processor on the same chip or can be set separately on different chips. The embodiments of the present application do not specifically limit the type of memory and the setting method of the memory and the processor.

[0177] The present application also provides a computer program product, which includes: a computer program (also referred to as code, or instruction), which enables a computer to execute the method executed by the electronic device in any of the above embodiments when the computer program is executed.

[0178] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program (also referred to as code or instruction). When the computer program is executed, the computer executes the method executed by the electronic device in any of the above embodiments.

[0179] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0180] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0181] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items.

[0182] The terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of technical features indicated. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0183] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0184] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. An image processing method, characterized in that: Applied to an electronic device, the electronic device includes a memory and a first memory other than the memory, the method includes: Starting a camera, capturing an original image through the camera of the electronic device, and recording the original image into the memory; receiving a plurality of instructions for capturing images, the plurality of instructions comprising a first instruction for capturing a first image received at a first time and an instruction for capturing a second image received at a second time; the second time being earlier or later than the first time; In the case where the dump condition is met, a dump operation is performed; the dump operation includes: recording the identifiers of the M frames of original images matching the first time and the identifiers of the metadata respectively corresponding to the M frames of original images into the first synthesis information; dumping the first original image in the M frames of original images from the memory to the first memory; the original images other than the first original image in the M frames of original images have been dumped to the first memory before the dump operation is performed; whether each frame of the original image in the M frames of original images has been dumped to the first memory before the dump operation is performed is determined by query data; the query data includes the identifiers of the original images stored in the first memory; when the dump operation is performed, the metadata respectively corresponding to the M frames of original images are stored in the first memory; the metadata corresponding to one frame of original image is used to indicate the acquisition information of the one frame of original image and the shooting information corresponding to the first time; the original images other than the first original image in the M frames of original images are also used to synthesize the second image, and M is an integer greater than 1; After executing the dump operation, when the synthesis condition is met, the M frames of original images and metadata corresponding to the M frames of original images are obtained from the first memory through the first synthesis information, and loaded into the memory to synthesize the first image.

2. The method according to claim 1, characterized in that The method further comprises: When the dump condition is not met, the first image is synthesized by using the M frames of original images in the memory and the metadata corresponding to the M frames of original images respectively; when the dump operation is not performed, the metadata corresponding to the M frames of original images respectively are stored in the memory.

3. The method according to claim 1, characterized in that Meeting the dump condition includes: the electronic device is in a continuous shooting mode; wherein the continuous shooting mode is used for the electronic device to continuously shoot at least two images when a shooting control is long pressed.

4. The method according to claim 3, characterized in that Before acquiring the M frames of original images and metadata respectively corresponding to the M frames of original images from the first memory using the first synthesis information, the method further includes: Obtaining the first synthetic information from a synthetic information queue; The method further comprises: The second synthesis information is obtained from the synthesis information queue; the second synthesis information includes the identifier of the E frame original image matching the second time, and the identifier of the metadata corresponding to the E frame original image; the second time is the time when the second instruction is received, and the E is equal to the M; The E frame original image and metadata respectively corresponding to the E frame original image are acquired from the first memory through the second synthesis information, and loaded into the memory to synthesize the second image.

5. The method according to claim 4, characterized in that Meeting the synthesis condition includes at least one of the following: turning off the camera, running the camera in the background, and the performance of the electronic device meeting a preset condition.

6. The method according to any one of claims 1 to 5, characterized in that The query data is constructed through a dictionary tree; the dictionary tree includes a root node and X leaf nodes, and the leaf node is a node in the dictionary tree that does not have a child node; the X indicates the number of original images dumped into the first memory; a path between the root node and a leaf node is used to determine the identification of a frame of the original image, and the identification of the original image determined by different paths is different.

7. The method according to claim 6, characterized in that When the number of cameras that capture the original image is equal to 1, the identification of the original image frame is represented by the acquisition time of the original image frame; when the number of cameras that capture the original image is greater than 1, the identification of the original image frame includes the acquisition time of the original image frame plus the serial number of the camera that captures the original image frame; All nodes except the root node on the path are used to record characters in the identifier of the frame of original image, and the path includes a first node and a second node, the first node is used to record a first character in the identifier of the frame of original image, and the second node is used to record a second character in the identifier of the frame of original image; On the one path, the first node is closer to the root node than the second node, and compared with the second character, the first character is at a higher position in the identifier of the one frame of the original image; The X leaf nodes in the dictionary tree are also used to construct a serialized linked list, which includes linked list nodes of X frames of original images; when the X leaf nodes and the X linked list nodes are sorted from front to back, the i-th linked list node is obtained through the i-th leaf node, and the i-th linked list node is used to determine the information of the i-th frame original image, the linked list node of the i-1-th frame original image, and the linked list node of the i+1-th frame original image; the first acquisition time of the i-1-th frame original image is before the second acquisition time of the i-th frame original image, or the first acquisition time is the same as the second acquisition time and the first serial number is less than the second serial number; the first serial number is the serial number of the camera that acquires the i-1-th frame original image; The second serial number is the serial number of the camera that captures the i-th frame of the original image, and i is an integer greater than 1 and less than X.

8. The method according to claim 7, characterized in that The method further comprises: Before dumping a first original image in the M frames of original images from the memory to the first memory, determining that an identifier of the first original image is not recorded in the dictionary tree; The step of determining that the identifier of the first original image is not recorded in the dictionary tree specifically includes: Determine the maximum identifier from the identifiers of the M frames of original images, search the maximum identifier through the dictionary tree and determine the target identifier, if the maximum identifier is recorded in the dictionary tree, the target identifier is the maximum identifier; if the maximum identifier is not recorded in the dictionary tree, the target identifier is the minimum identifier among the brother identifiers recorded in the dictionary tree, and the brother identifier is greater than the maximum identifier; In a serialized linked list, starting from the linked list node where the target identifier is located, traverse the linked list nodes in the serialized linked list forward, and when the second linked list node is not determined before the first linked list node is determined, determine that the identifier of the first original image is not recorded in the dictionary tree; the identifier of the original image recorded by the first linked list node is less than the identifier of the first original image; and the identifier of the original image recorded by the second linked list node is equal to the identifier of the first original image.

9. The method according to claim 8, characterized in that After dumping the first original image in the M frames of original images from the memory to the first memory, the method further includes: Update the dictionary tree and the serialized linked list, the update comprising: adding a new node in the dictionary tree to obtain a first path, the first path is used to determine the identification of the first original image, adding the information of the first new node to the child node distribution information of the parent node corresponding to the first new node, and adding the linked list node of the first original image to the serialized linked list; when the number of new nodes is 1, the new node is the first new node, and when the number of new nodes is greater than 1, the first new node is the new node closest to the root node among the new nodes.

10. The method according to claim 9, characterized in that The method further comprises: After synthesizing the first image, the usage count is reduced by 1; the usage count is used to indicate the number of times the first original image is used to synthesize an image; When the usage count is 0, the new node is deleted from the updated dictionary tree, the information of the new node is deleted from the child node distribution information of the parent node, and the linked list node of the first original image is deleted from the updated serialized linked list.

11. An electronic device, characterized in that: include: One or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method as described in any one of claims 1-10.

12. A computer-readable storage medium comprising computer instructions, characterized in that: When the computer instructions are executed on an electronic device, the electronic device is caused to execute the method as claimed in any one of claims 1 to 10.

13. A chip system, which is applied to electronic equipment, characterized in that: The chip system includes one or more processors, and the processor is used to call computer instructions so that the electronic device executes the method as described in any one of claims 1-10.

14. A computer program product comprising instructions, characterized in that When the computer program product is executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 10.

Images