Image Processing Method and Electronic Device

Abstract

Embodiments of this application provide an image processing method and an electronic device. In this method, the electronic device can determine that the dump condition is met when the performance is insufficient or when the multi-frame fusion technology needs to be frequently used. When the dump condition is met, the electronic device can perform a dump operation on the collected multiple original images, and store the multiple original images from the RAM to the ROM. The electronic device also stores the metadata corresponding to the multiple original images to the ROM. Subsequently, when the electronic device meets the synthesis condition, the multiple original images are loaded from the ROM to the RAM to synthesize an image. Implementing the technical solution provided by this application can avoid the performance degradation of the electronic device when using the multi-frame fusion technology.

Description

Technical Field

[0001] This application relates to the technical fields of terminals and image processing, and particularly to an image processing method and an electronic device. Background Art

[0002] The multi-frame fusion technology enables an electronic device to synthesize multiple original images captured continuously into one image (the photo finally shown to the user). When using the multi-frame fusion technology, the electronic device can fuse the advantageous parts of multiple original images. For example, under low light conditions, the electronic device can use a long-exposure original image and a short-exposure image to synthesize one image through the multi-frame fusion technology. The long-exposure original image can capture more light through long exposure, and the short-exposure image can avoid image blurring. Then, through multi-frame average noise reduction, noise can be effectively reduced. The image obtained through the multi-frame fusion technology can significantly improve the quality of the final image. It overcomes common limitations and problems in single-frame shooting, such as low light conditions or excessive noise.

[0003] How to reasonably use the multi-frame fusion technology to enable the electronic device to obtain high-quality images is worthy of discussion. Summary of the Invention

[0004] Embodiments of this application provide an image processing method and an electronic device for optimizing the process of synthesizing multiple original images into one image.

[0005] In a first aspect, the present application provides an image processing method, which is applied to an electronic device. The electronic device includes a memory and a first memory other than the memory. The method includes: starting a camera, collecting an original image through a camera of the electronic device, and recording the original image in the memory; receiving a plurality of instructions for taking images, the plurality of instructions including a first instruction for taking a first image received at a first time and an instruction for taking a second image received at a second time; the second time being earlier than or later than the first time; performing a dump operation when a dump condition is satisfied; the dump operation including: recording identifiers of M frames of original images matching the first time and identifiers of metadata respectively corresponding to the M frames of original images in first composite information; dumping a first original image among the M frames of original images from the memory to the first memory; the original images other than the first original image among the M frames of original images have been dumped to the first memory before the dump operation is performed; whether each frame of the M frames of original images has been dumped to the first memory before the dump operation is performed is determined by querying data; the query data includes identifiers of original images already stored in the first memory; when the dump operation is performed, the metadata respectively corresponding to the M frames of original images are stored in the first memory; the metadata corresponding to one frame of original image is used to indicate the acquisition information of the one frame of original image and the shooting information corresponding to the first time; the original images other than the first original image among the M frames of original images are also used to synthesize the second image; M is an integer greater than 1; after the dump operation is performed, when a synthesis condition is satisfied, the M frames of original images and the metadata respectively corresponding to the M frames of original images are obtained from the first memory through the first composite information and loaded into the memory to synthesize the first image.

[0006] In the above embodiment, satisfying the dump condition means that the current performance of the electronic device is insufficient. At this time, using the multi-frame fusion technology will cause problems such as excessive memory occupancy and excessive processor load. As a result, the performance of the electronic device drops and there is lag. Performing the dump operation can store the original images and metadata that originally needed to be recorded in the memory into a memory other than the memory (the first memory), release the storage pressure of the memory, pause the multi-frame fusion technology, and avoid increasing the processor load. When the synthesis condition is satisfied later (indicating that the performance of the electronic device is good and supports the multi-frame fusion technology), the original images and metadata are loaded into the memory to synthesize the image. In this way, the performance degradation of the electronic device can be avoided when using the multi-frame fusion technology.

[0007] In combination with the first aspect, in some embodiments, the method further includes: when the dump condition is not satisfied, synthesizing the first image through the M frames of original images in the memory and the metadata respectively corresponding to the M frames of original images; when the dump operation is not performed, the metadata respectively corresponding to the M frames of original images are stored in the memory.

[0008] In the above embodiments, not meeting the dump condition means that the electronic device has good current performance, and using the multi-frame fusion technology will not cause the electronic device to degrade to a stuck state. Therefore, there is no need to perform a dump operation, and the electronic device can directly synthesize an image through the original image and metadata in the memory using the processor. During the process of synthesizing the image, the memory plays a role of temporary storage, providing the data and instructions required for the electronic device to synthesize the image.

[0009] In combination with the first aspect, in some embodiments, meeting the dump condition includes: the electronic device is in a continuous shooting mode; wherein, the continuous shooting mode is used for the electronic device to continuously shoot at least two images when the shooting control is long-pressed.

[0010] In the above embodiments, the continuous shooting mode means that the electronic device needs to continuously shoot multiple images in a short period of time and needs to frequently use the multi-frame fusion technology. Entering the continuous shooting mode can be used as a dump condition.

[0011] In combination with the first aspect, in some embodiments, before obtaining the M original images and the metadata respectively corresponding to the M original images from the first memory through the first synthesis information, the method further includes: obtaining the first synthesis information from a synthesis information queue; the method further includes: obtaining the second synthesis information from the synthesis information queue; the second synthesis information includes the identifiers of the E original images matching the second time, and the identifiers of the metadata respectively corresponding to the E original images; the second time is the time when the second instruction is received, and E is equal to M; through the second synthesis information, obtaining the E original images and the metadata respectively corresponding to the E original images from the first memory and loading them into the memory to synthesize the second image.

[0012] In the above embodiments, the existence of the synthesis information queue enables the electronic device to store multiple synthesis information. When the synthesis condition is met, the synthesis of the image is realized through the synthesis information in the synthesis information queue, which will not cause image loss.

[0013] In combination with the first aspect, in some embodiments, meeting the synthesis condition includes at least one of the following: the electronic device turns off the camera, runs the camera in the background, and the performance of the electronic device meets a preset condition.

[0014] In the above embodiments, the performance of the electronic device meeting the preset condition means that the performance of the electronic device is good, and even using the multi-frame fusion technology will not cause the performance to degrade to a stuck state. The good performance of the electronic device includes but is not limited to at least one of the following: small memory occupancy (for example, the occupancy rate is less than 50% of the overall memory capacity), and small processor load (the number of tasks to be processed is less than a preset number, for example, 10 items).

[0015] In combination with the first aspect, in some embodiments, the query data is constructed by a trie tree; the trie tree includes a root node and X leaf nodes, where a leaf node is a node in the trie tree that does not have child nodes; the X indicates the number of original images dumped into the first memory; a path between the root node and a leaf node is used to determine the identifier of a frame of the original image, and the identifiers of the original images determined by different paths are different.

[0016] In the above embodiments, the path between the root node and the leaf node in the trie tree represents the identifier of the original image, which means that the height of the trie tree is determined by the length of the identifier of the original image. Generally speaking, the identifiers of the original images are of the same length, so the heights of the paths in the trie tree are the same, and the search efficiency is relatively unified each time the identifier of the original image is searched.

[0017] In combination with the first aspect, in some embodiments, when the number of cameras for collecting the original images is equal to 1, the identifier of a frame of the original image is represented by the acquisition time of the frame of the original image; when the number of cameras for collecting the original images is greater than 1, the identifier of a frame of the original image includes the acquisition time of the frame of the original image plus the serial number of the camera that collected the frame of the original image; the nodes on the path except the root node are all used to record the characters in the identifier of the frame of the original image. The path includes a first node and a second node. The first node is used to record the first character in the identifier of the frame of the original image, and the second node is used to record the second character in the identifier of the frame of the original image; on the path, the first node is closer to the root node than the second node, so compared with the second character, the first character is at a higher position in the identifier of the frame of the original image; the X leaf nodes in the trie tree are also used to construct a serialization linked list, and the serialization linked list includes linked list nodes of X frames of the original image; when the X leaf nodes and the X linked list nodes are sorted from front to back, the i-th linked list node is obtained through the i-th leaf node, and the i-th linked list node is used to determine the information of the i-th frame of the original image, the linked list node of the (i - 1)-th frame of the original image, and the linked list node of the (i + 1)-th frame of the original image; the first acquisition time of the (i - 1)-th frame of the original image is before the second acquisition time of the i-th frame of the original image, or, the first acquisition time is the same as the second acquisition time and the first serial number is less than the second serial number; the first serial number is the serial number of the camera that collected the (i - 1)-th frame of the original image; the second serial number is the serial number of the camera that collected the i-th frame of the original image, and i is an integer greater than 1 and less than X.

[0018] In the above embodiments, the construction of the serialization linked list is beneficial to more intuitively display the identifiers of the original images recorded in the trie tree.

[0019] In combination with the first aspect, in some embodiments, the method further includes: before dumping the first original image in the M-frame original images from the memory to the first memory, determining that the identification of the first original image is not recorded in the trie tree; wherein, determining that the identification of the first original image is not recorded in the trie tree specifically includes: determining the maximum identification from the identifications of the M-frame original images, searching for the maximum identification through the trie tree and determining the target identification, when the maximum identification is recorded in the trie tree, the target identification is the maximum identification; when the maximum identification is not recorded in the trie tree, the target identification is the minimum identification among the sibling identifications recorded in the trie tree, and the sibling identification is greater than the maximum identification; in the serialization linked list, starting from the linked list node where the target identification is located, traversing the linked list nodes in the serialization linked list forward, when it is determined that no second linked list node is determined before the first linked list node, determining that the identification of the first original image is not recorded in the trie tree; the identification of the original image recorded by the first linked list node is less than the identification of the first original image; the identification of the original image recorded by the second linked list node is equal to the identification of the first original image.

[0020] In the above embodiments, the trie tree is used to locate a target identification, and then whether the identification of the first image exists is queried forward from the target identification through the serialization linked list, converting the query for the trie tree into a query for the serialization linked list. Since the structure of the serialization linked list is simpler than that of the trie tree, the query will be faster. It is beneficial to achieve a fast query for the identification of the first image.

[0021] In combination with the first aspect, in some embodiments, after dumping the first original image in the M-frame original images from the memory to the first memory, the method further includes: updating the trie tree and the serialization linked list, and the update includes: adding a new node in the trie tree to obtain a first path, the first path being used to determine the identification of the first original image, adding the information of the first new node to the child node distribution information of the parent node corresponding to the first new node, and adding the linked list node of the first original image to the serialization linked list; when the number of new nodes is 1, the new node is the first new node, and when the number of new nodes is greater than 1, the first new node is the new node closest to the root node.

[0022] In the above embodiments, after dumping the original image to the first memory, it is necessary to update the trie tree and the serialization linked list. It is convenient for subsequent dumping of the same original image so that the same original image can exist in the first memory, and the electronic device does not need to dump the same original image again and can directly reuse it.

[0023] In combination with the first aspect, in some embodiments, the method further includes: after synthesizing the first image, decrementing the usage count; the usage count is used to represent the number of times the first original image is used for synthesizing an image; when the usage count is 0, deleting the new node from the updated trie, deleting the information of the new node from the child node distribution information of the parent node, and deleting the linked list node of the first original image from the updated serialized linked list.

[0024] In the above embodiments, the usage count of the original image can conveniently record the reuse situation of the original image. A usage count of 0 indicates that the original image has completed the synthesis of the image, and there is no longer an image that needs to use the original image for synthesis. The identifier of the original image can be deleted from the trie to simplify the trie.

[0025] In a second aspect, an embodiment of the present application provides an electronic device, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to cause the electronic device to execute the method implemented in the first aspect.

[0026] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, including instructions, when the instructions run on an electronic device, causing the electronic device to execute the method implemented in the first aspect.

[0027] In a fourth aspect, an embodiment of the present application provides a chip system, which is applied to an electronic device, and the chip system includes one or more processors, and the processors are used to call computer instructions to cause the electronic device to execute the method implemented in the first aspect.

[0028] In a fifth aspect, an embodiment of the present application provides a computer program product containing instructions, when the computer program product runs on an electronic device, causing the electronic device to execute the method implemented in the first aspect.

[0029] It can be understood that the electronic device provided in the second aspect, the computer storage medium provided in the third aspect, the chip system provided in the fourth aspect, and the computer program product provided in the fifth aspect are all used to execute the method provided in the embodiments of the present application. Therefore, the other beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, and will not be elaborated here. Description of the Drawings

[0030] Figure 1 Shows a schematic diagram of a synthesized image when no dumping is performed;

[0031] Figure 2 Shows a set of exemplary user interfaces in continuous shooting mode;

[0032] Figure 3 Shows a schematic diagram of repeated dumping due to the reuse of the original image during single-camera shooting;

[0033] Figure 4 Shows a schematic diagram of non-repeated dumping due to the reuse of the original image during single-camera shooting;

[0034] Figure 5 Shows a schematic diagram of non-repeated dumping but with loss of the original image during multi-camera shooting;

[0035] Figure 6 Shows a schematic diagram of non-repeated dumping and no loss of the original image during multi-camera shooting;

[0036] Figure 7 Shows an exemplary red-black tree involved in single-camera shooting;

[0037] Figure 8 Shows an exemplary red-black tree involved in multi-camera shooting;

[0038] Figure 9 Shows an exemplary red-black tree involved in multi-camera shooting;

[0039] Figure 10 Shows an exemplary trie tree involved in single-camera shooting;

[0040] Figure 11 Shows an exemplary trie tree involved in multi-camera shooting;

[0041] Figure 12 Shows a schematic diagram involved in traversing the trie tree;

[0042] Figure 13 Shows an exemplary flowchart involved in image shooting when implementing an image processing method during multi-camera shooting;

[0043] Figure 14 Shows a schematic diagram involved in determining the target identifier;

[0044] Figure 15 Shows a schematic diagram of determining the elder brother node;

[0045] Figure 16 Shows a schematic diagram involved in determining the target identifier based on the elder brother node;

[0046] Figure 17 Shows a schematic diagram involved in adding a node to the trie tree;

[0047] Figure 18 Shows a schematic diagram involved in deleting a node from the trie tree;

[0048] Figure 19 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0049] In some embodiments, the electronic device loads multiple frames of original images collected by the camera into a random access memory (RAM). When detecting an instruction to capture an image, the central processing unit (CPU) of the electronic device can quickly read the multiple frames of original images from the RAM to synthesize an image. The central processing unit can be abbreviated as the processor.

[0050] Here, the RAM serves as the memory of the electronic device, and the data and instructions stored therein can be directly read by the processor of the electronic device to quickly process the data and instructions. Therefore, loading the original images into the memory can quickly synthesize the images provided to the user using the original images. The original image refers to the initial image data generated by the camera sensor after capturing light, and the original image has not undergone any advanced processing or optimization and is not directly presented to the user.

[0051] As Figure 1 shown, after starting the camera, the electronic device continuously collects original images through the camera and obtains the acquisition information of each frame of the original images. The electronic device loads the original images collected by the camera into the RAM. Specifically, it can be recorded in buffer area 1 of the RAM, and this buffer area 1 supports storing multiple frames of original images. The electronic device sequentially obtains the original images from buffer area 1 of the RAM. After performing simple processing on the original images, a preview image can be displayed on the screen. Here, the simple processing does not include multi-frame fusion processing and is an image processing technology for obtaining a preview image after processing based on a single frame of the original image.

[0052] At time 1, an instruction to capture image 1 is received. In response to this instruction, the electronic device synthesizes image 1 through M frames of original images in the RAM that match time 1, and the metadata respectively corresponding to the M frames of original images in the RAM.

[0053] It should be noted here that the metadata corresponding to a frame of original image can be used to indicate the acquisition information of the frame of original image and the shooting information at the shooting time. Taking the shooting time as time 1 for illustration, the metadata corresponding to the frame of original image includes the acquisition information corresponding to the frame of original image and the shooting information at time 1. Among them, the acquisition information includes the information when acquiring a frame of original image, including parameters such as the parameters of the camera for acquiring the frame of original image and the shooting environment parameters. Among them, the parameters of the camera can include parameters such as the focus distance and exposure time when acquiring the frame of original image. The environment parameters include the ambient light intensity when acquiring the frame of original image. The shooting information at time 1 includes, but is not limited to, one or more of the following: the ambient light intensity at time 1, the white balance parameters at time 1, etc.

[0054] In continuous shooting scenarios such as high-speed continuous shooting where a large number of images need to be taken in a short time, the electronic device can also create a composite information queue in the memory to record the composite information of the images to be composed. In this way, it can support the electronic device to record the composite information of the image into the composite information queue first when it is too late to compose the image, and then obtain the original image and metadata required for composing the image based on the composite information of the image to compose the image. In this way, in the high-speed continuous shooting scenario where multiple images need to be continuously composed, images will not be missing due to untimely processing.

[0055] In a continuous shooting scenario, in response to an instruction to shoot an image, the electronic device determines the address information of M frames of original images for composing the image and records it in the composite information of the image. Moreover, the electronic device generates the metadata corresponding to each of the M frames of original images, and then records the address information of the metadata into the composite information of the image. Then, the composite information is recorded into the composite information queue. Subsequently, the electronic device can obtain the composite information from the composite information queue in the composite order, and obtain the original image and metadata required for composing the image through the composite information to compose the image.

[0056] Reference Figure 1 In (1)- Figure 1 Regarding the continuous shooting scenario shown in (4) in (1), the black arrow in (1) indicates that at time 11, an instruction to shoot image 11 is received, the black arrow in (2) indicates that at time 12, an instruction to shoot image 12 is received, the black arrow in (3) indicates that at time 13, an instruction to shoot image 13 is received, and the black arrow in (4) indicates that at time 14, an instruction to shoot image 14 is received.

[0057] The electronic device can record the composite information a1 of image 11, the composite information a2 of image 12, the composite information a3 of image 13, and the composite information a4 of image 14 into the composite information queue a. Subsequently, in the composite order, the electronic device obtains the composite information from the composite queue a to compose the image.

[0058] The synthesis order includes but is not limited to at least one of the following: synthesizing the images with earlier shooting times first, or synthesizing the images with later shooting times first. Here, the shooting time of an image is the time when the instruction to shoot the image is received. For example, synthesizing the images with later shooting times first is considered for the following usage scenario: after the user has taken multiple photos, they will first click on the photo in the lower left corner and then continue to slide to view the previously taken photos.

[0059] For example, if the synthesis information queue a is a last-in-first-out queue and the images with later shooting times are synthesized first, then the synthesis information a4 is first obtained from the synthesis information queue a, and image 14 is synthesized through the synthesis information a4. Then, the synthesis information a3 is obtained to synthesize image 13. Next, the synthesis information a2 is obtained to synthesize image 12. And then, the synthesis information a1 is obtained to synthesize image 11.

[0060] The synthesis information for synthesizing an image includes the M original frames required for synthesizing the image and the address information of the metadata corresponding to each original frame. This address information is used to determine the storage addresses in the memory of the original images and metadata required for synthesizing the image. Through this address information, the M original frames and the metadata corresponding to each of the M original frames can be obtained to synthesize the image.

[0061] It should be noted here that Figure 1 the above example uses M as 4. In actual situations, the number of original frames used to synthesize an image can be other values. For example, 10 frames, 5 frames, etc., and the embodiments of the present application do not limit this.

[0062] The multi-frame fusion technology needs to use multiple original frames to synthesize an image at one time. The data volume of one original frame (for example, 15MB) is relatively large, which requires more memory space. Moreover, a large number of image processing algorithms are involved in the synthesis process, which will increase the processor load. When the performance of the electronic device is insufficient, if the multi-frame fusion technology is still used, it may cause the performance of the electronic device to further decline. Or, when the multi-frame fusion technology needs to be used frequently, it may cause the performance of the electronic device to decline.

[0063] In order to use the multi-frame fusion technology reasonably, avoid the performance of the electronic device from declining due to the multi-frame fusion technology, causing lags and affecting the normal operation of the electronic device. When the performance of the electronic device is insufficient or the multi-frame fusion technology needs to be used frequently, the electronic device determines that the dump condition is met.

[0064] When it is determined that the dump condition is met, the electronic device performs a dump operation, and dumps the data (including at least the original image and metadata) originally recorded or to be recorded in the RAM for synthesizing an image to another memory (denoted as Memory 1) other than the RAM. And the dumped data is deleted in the RAM to free up memory space. When the synthesis condition is met, the data required for synthesizing the image is loaded from Memory 1 to the RAM to synthesize the image.

[0065] Here, it should be noted that Memory 1 can be a read-only memory (ROM), a flash memory, etc. In the following text, Memory 1 is taken as an example of ROM for illustration.

[0066] The performance of the electronic device can be characterized by at least one of the following indicators: the temperature of the electronic device, the memory occupancy, and the processor load. The higher the temperature of the electronic device, the worse the performance. Among them, the temperature of the electronic device can be represented by the temperature of the processor or the temperature of the back shell of the electronic device. The larger the memory occupancy, the smaller the available memory space, and the worse the performance of the electronic device. The more the processor load, the worse the performance of the electronic device.

[0067] Here, the frequently used multi-frame fusion technology mentioned above includes: the electronic device is in the continuous shooting mode. Among them, the continuous shooting mode is used for the electronic device to continuously shoot at least two images. For the relevant content involved in the continuous shooting mode, reference can be made to the following description of Figure 2 of.

[0068] Here, it should be noted that it is not limited to the continuous shooting mode, nor to the scenario of frequently using the multi-frame fusion technology. Other image processing tasks with high power consumption and high time consumption (such as AI portrait processing) can also meet the dump condition.

[0069] As Figure 2 shown in (1) below, in the continuous shooting mode, when the user long-presses the shooting control 201, the electronic device can successively receive multiple instructions for shooting images, and one instruction is used to shoot one image. At this time, in response to the operation on the shooting control 201, the electronic device can use the multi-frame fusion technology multiple times to obtain multiple images. And the images taken in the continuous shooting mode are recorded in the album. An example of this album can be Figure 2 the continuous shooting album 202 shown in (2) below. In response to the operation (such as a click operation) on the continuous shooting album 202, the electronic device can display the images taken in the continuous shooting mode. An exemplary user interface involved in the display can be referred to Figure 2 shown in (3) below.

[0070] In some possible implementation manners, when the electronic device performs a dump operation, for each image to be synthesized, the electronic device may generate an image file in the ROM, and an image file is used to record the original images and metadata required for synthesizing one image.

[0071] When performing the dump operation, for a continuous shooting scenario, the electronic device can still create a synthesis information queue a and record the synthesis information queue a in the ROM for recording the synthesis information a of the image. The synthesis information a of one image includes the address of the image file for synthesizing the image. Through the address of the image file, the original images and metadata required for synthesizing the image can be obtained to synthesize the image.

[0072] As Figure 3 shown, for Figure 1 the continuous shooting scenario shown in (1)- Figure 1 in (4) above, if the electronic device cannot synthesize the image in time, it can, in the order of shooting time, respectively dump the original images (M frames) and metadata (one piece of original data corresponds to one piece of metadata) required for synthesizing images 11 - 14 to the ROM, and then record them to image files 1 - 4 respectively. For the image with an earlier shooting time, the address of the image file can be first recorded in the synthesis information queue a. After the dump condition is met, the original images and metadata required for synthesizing the image are obtained through the synthesis information queue a and loaded into the RAM to synthesize the image.

[0073] It should be noted here that in the continuous shooting scenario, some of the original images used when shooting two adjacent images can be the same, which means that some of the original images can be reused for synthesizing different images. As Figure 3 shown, in the above implementation manner, for the original images that can be reused, there is a situation of repeated dumping. The number of times an original image is reused is the number of times it is repeatedly dumped. As Figure 3 shown, the original images used for synthesizing image 11 include original images N - N + 3, and the original images used for synthesizing image 12 include original images N + 1 - N + 4. Among them, the repeatedly dumped original images include original images N + 1 - N + 3. There are also repeatedly dumped original images N + 2 - N + 4 in the original images of image 13, and there are also repeatedly dumped original images N + 5 in the original images of image 14. Original image N + 2 and original image N + 3 are also repeatedly dumped 2 times.

[0074] Repeated dumping not only causes waste of the storage space of the ROM, but also causes waste of computing resources. Moreover, the data volume of the original images is large, and it takes a long time to dump them. Repeated dumping will also increase the dumping time.

[0075] To avoid repeated dumping when the original image is reused, an image processing method is proposed. In this method, when the electronic device dumps the original image from the RAM to the ROM, it first determines whether the original image has been recorded in the ROM. If it has been recorded in the ROM, it means that the original image is an original image for which reuse exists, and no further dumping is performed. If it has not been recorded in the ROM, the electronic device records the original image in the ROM.

[0076] When implementing the image processing method involved in the embodiments of the present application, after starting the camera, the electronic device acquires the original image through the camera and records the original image in the RAM. At time 1, upon receiving the instruction to capture Image 1 (denoted as Instruction 1), under the condition of meeting the dumping condition, the electronic device performs the dumping operation.

[0077] Among them, performing the dumping operation (Dumping Operation 1) includes: determining the identifiers of the M frames of original images in the memory that match time 1 and recording them in Synthesis Information 1. Dumping the original image 1 in the M frames of original images from the RAM to the ROM and deleting the original image 1 from the memory after dumping. The other original images in the M frames of original images except the original image 1 are called original image 2. The original image 2 is the image that has been dumped to the ROM before performing Dumping Operation 1 in the M-frame images. Whether each frame of the original images in the M frames of original images has been dumped to the ROM before time 1 is determined by querying the data. The query data includes the identifiers of the original images already recorded in the ROM. The synthesis information 1 also includes the identifiers of the metadata corresponding to each frame of the original images in the M frames of original images. When performing Dumping Operation 1, the metadata corresponding to each frame of the original images in the M frames of original images is stored in the ROM. The original image 2 is also used to synthesize Image 2, and the instruction to capture Image 2 (denoted as Instruction 2) is earlier or later than Instruction 1. M is an integer greater than 1.

[0078] It should be noted here that the M frames of original images that match time 1 include, but are not limited to, at least one of the following: the M frames of images in the memory with the acquisition time closest to time 1, and among the original images with the acquisition time less than or equal to time 1, the M frames of images with the acquisition time closest to time 1.

[0079] After performing the dumping operation, under the condition of meeting the synthesis condition, through the synthesis information 1, the M frames of original images and the metadata corresponding to each frame of the original images in the M frames of original images are obtained from the ROM and loaded into the ROM to synthesize Image 1.

[0080] An example of Image 1 and Image 2 can be Figure 4 at least two of Images 11 - 14 in Figure 4 As shown in Figure 1 in Figure 1In the continuous shooting scenario shown at (4), if the electronic device cannot synthesize images in time, the original images (M frames) and metadata (one piece of original data corresponds to one piece of metadata) required for synthesizing images 11 - 14 can be dumped into the ROM in the order of shooting time. When dumping the original images from the RAM to the ROM, the original images that already exist in the ROM are not dumped repeatedly to avoid wasting the space of the ROM. For example, the original images required for synthesizing image 12 include original images N + 1 - N + 4. When dumping the original images required for synthesizing image 12 to the ROM, original images N + 1 - N + 3 have been recorded in the ROM, and original images N + 1 - N + 3 can be reused without being dumped again, and only original image N + 4 needs to be dumped to the ROM. The reason why original images N + 1 - N + 3 have been recorded in the ROM is that the electronic device has performed an operation of dumping the original images (including original images N - N + 3) required for synthesizing image 11 to the ROM.

[0081] When performing the dump, the electronic device also generates the synthesis information of the images and records it in the ROM. When there are many images to be synthesized, the synthesis information of each image can also be recorded in a synthesis information queue (such as synthesis information queue 1). For example, Figure 4 in it, the synthesis information of images 11 - 14 can be recorded in synthesis information queue 1. When the synthesis conditions are met, the synthesis information is obtained from the synthesis information queue in the synthesis order. Then, the original images and metadata required for synthesizing the image are obtained through the synthesis information to synthesize the image.

[0082] In the image processing method, the synthesis information of the image includes the identifier of the original image for synthesizing the image and the identifier of the metadata. Among them, the identifier of the original image is used to indicate the original image, and the storage address of the original image in the ROM can be determined through the identifier of the original image. The identifier of the metadata is used to determine the storage address of the metadata in the ROM. In some possible cases, the name of the synthesis information can carry the name of the image to be used to establish the correspondence between the image to be synthesized and the synthesis information.

[0083] The electronic device can record the synthesis information of image 1 (denoted as synthesis information 1) and the synthesis information of image 2 (denoted as synthesis information 2) in synthesis information queue 1. When the synthesis conditions are met, the electronic device obtains synthesis information 1 from synthesis information queue 1 to synthesize image 1.

[0084] The electronic device can also obtain the synthetic information 2 from the synthetic information queue. The synthetic information 2 includes the identifier of the E-frame original image that matches time 2, and the identifiers of the metadata corresponding to each frame of the original image in the E-frame original image. Time 2 is the time when instruction 2 (the instruction for synthetic image 2) is received, and E is equal to M. Through the synthetic information 2, the E-frame original image and the metadata corresponding to each frame of the original image in the E-frame original image are obtained from the ROM and loaded into the RAM to synthesize the synthetic image 2.

[0085] The synthetic information queue can be a first-in-first-out queue, or a first-in-last-out queue, etc. For the aforementioned synthetic information 1 and synthetic information 2, if the synthetic information queue is a first-in-last-out queue, and the synthetic information 1 enters the synthetic information queue earlier than the synthetic information 2, the electronic device first synthesizes the synthetic image 2 based on the synthetic information 2, and then synthesizes the synthetic image 1 based on the synthetic information 1.

[0086] In some possible implementation manners, the acquisition timestamp of the original image (abbreviated as the acquisition time) is used as the identifier of the original image. However, in the scenario of multi-camera shooting, if the acquisition time is used as the identifier of the original image, it will cause the dump to fail. Because when the M-frame original images for synthesizing an image come from multiple cameras, different original images captured by different cameras can have the same acquisition time, which means different original images can have the same identifier. When a frame of original image (denoted as original image a) is recorded into the ROM, different original images with the same timestamp as original image a will be misrecognized by the electronic device as original image a and cannot be recorded into the ROM. This will cause the original images for synthesizing the image to be lost, further resulting in the synthesis failure. The synthesis failure includes: the synthesized image has poor quality or no image is synthesized. When the synthesis failure is presented to the user side, it is the shooting failure, which affects the user experience.

[0087] As Figure 5 shown, it is a continuous shooting scenario under multi-camera (including camera 1 and camera 2) acquisition. See Figure 5 In (1) therein, in response to the instruction to shoot the image 11 at time 11, the electronic device determines the identifiers of the M-frame original images that match time 11 (here it is the acquisition time). It includes the original image N - the original image N + 3 and the original image K. Among them, the acquisition times of the original image N and the original image K are the same. The electronic device first dumps the original image N - the original image N + 3 into the ROM. Subsequently, before dumping the original image K, the electronic device determines that the identifier of the original image N already exists in the ROM and is the same as the identifier of the original image K, misjudges that the original image K has been dumped into the ROM, and no longer performs the dump operation for the original image K, resulting in the loss of the original image K. Subsequently, the electronic device no longer synthesizes the image 11, or the synthesized image 11 has poor quality.

[0088] Here it should be noted that Figure 4 and Figure 5All are continuous shooting scenarios. Figure 4 It is a continuous shooting scenario under single-camera shooting. Figure 5 It is a continuous shooting scenario under multi-camera shooting. For Figure 5 The relevant descriptions of other content in Figure 4 can refer to the foregoing

[0089] In some other possible implementation manners, in order to increase the probability of successful dumping, and thus increase the probability of successfully synthesizing an image. At the same time, maintain the simplicity of the identifier of the original image. When the number of cameras for collecting the original image is equal to 1, the identifier of a frame of the original image is represented by the acquisition time of this frame of the original image. When the number of cameras for collecting the original image is greater than 1, the identifier of a frame of the original image includes: the acquisition time of this frame of the original image plus the serial number of the camera that collected this frame of the original image. For example, if the acquisition time of a frame of the original image is "123" and the serial number of the camera that collected this frame of the original image is 2, then the identifier of this frame of the original image is "1232". Without limitation, when the number of cameras for collecting the original image is greater than 1, the identifier of a frame of the original image can also include: the serial number of the camera that collected this frame of the original image plus the acquisition time of this frame of the original image. For example, if the acquisition time of a frame of the original image is "123" and the serial number of the camera that collected this frame of the original image is "2", then the identifier of this frame of the original image is 2123.

[0090] Figure 6 As shown, it is a continuous shooting scenario under the collection of another multi-camera (including Camera 1 and Camera 2). Compared with Figure 5 , although the acquisition times of the original image N and the original image K are the same. However, the serial number of Camera 2 is introduced into the identifier of the original image K, and the serial number of Camera 1 is introduced into the identifier of the original image N. Then the identifiers of the original image N and the original image K are different. The electronic device can dump both the original image N and the original image K into the ROM, and successfully synthesize Image 1.

[0091] It should be noted here that the electronic device uses numbers between "0-9" to represent the identifiers of the original images. Compared with using other letters to represent the identifiers of the original images, it can simplify the process of constructing query data. It can also simplify the process of querying the identifiers of the original images in the query data.

[0092] In some possible implementations, a red-black tree can be used to construct query data. Among the nodes of the red-black tree except the leaf nodes, each node can indicate the identifier of a frame of original image. A red-black tree is a balanced binary search tree, which can still ensure a relatively high query efficiency even in the worst case. There are certain rule restrictions when constructing a red-black tree. These rule restrictions can make the height of a red-black tree with P nodes at most 2×log(P + 1), and the worst-case query time is O(logP). Using a red-black tree can quickly complete the query of the identifier of the original image based on the query data, determine whether the identifier of a frame of original image is recorded in the red-black tree, and further determine whether the original image of this frame has been dumped into the ROM.

[0093] The rule restrictions when constructing a red-black tree include but are not limited to: the root node is a black node, the child node of any red node must be black, and the number of black nodes included in all paths from any node to each of its leaf nodes is the same.

[0094] Such as Figure 7 shows an exemplary red-black tree when a single camera is shooting. Among the nodes of the red-black tree except the leaf nodes, each node can indicate the identifier of a frame of original image, including: among the nodes of the red-black tree except the leaf nodes, each node can be used to record the identifier of the original image. One node records the identifier of one original image.

[0095] The rule restrictions of the red-black tree contribute to its relatively high query efficiency, but also bring defects to the red-black tree, including but not limited to the following defects.

[0096] Defect 1: After the red-black tree is constructed, the processes of deleting nodes and updating nodes are relatively cumbersome, which is not conducive to the update of the red-black tree. For example, when a frame of original image is no longer used for synthesizing an image, the node recording the identifier corresponding to this frame of original image needs to be deleted from the red-black tree. However, during the deletion process, operations such as balancing rotation and recoloring are required to make the red-black tree after deleting the node re-adapt to the aforementioned rule restrictions.

[0097] Defect 2: The rule restrictions of the red-black tree will cause the more nodes in the red-black tree, the higher the red-black tree, and the query efficiency will also decrease.

[0098] Defect 3: The identifiers recorded by each node in the red-black tree are not serialized, and when querying, it is impossible to quickly query the identifiers of other nodes relying on the identifier recorded by one node.

[0099] Defect 4: When the data structure that the nodes in the red-black tree need to record changes, all the nodes in the entire red-black tree that record data need to be modified. For example, in Figure 7In the red-black tree shown, if one more dimension of data needs to be added to the red-black tree, for example, the serial number of the camera that captured the original image, then all the nodes with recorded identifiers need to be modified. As Figure 8 shown, the electronic device can change the identifier recorded in each node from the capture time to the capture time plus the serial number of the camera. Or, as Figure 9 shown, pointers can be added to each node to point to a linked list that records the serial number of the camera.

[0100] Considering that when continuously capturing images, the identifiers of the original images are actually in order, and at the same time to overcome the defects involved in the aforementioned red-black tree. In some other possible implementation manners, a new container is proposed for constructing query data, and this new container is a trie tree. By using the orderliness between the identifiers of multiple frames of original images, a trie tree that is conducive to querying can be constructed.

[0101] The trie tree includes a root node and X leaf nodes. A leaf node is a node in the trie tree that does not have child nodes. X indicates the number of original images dumped into the ROM. A path between the root node and a leaf node is used to determine the identifier of a frame of original image, and the identifiers of different frames of original images determined by different paths are different. The nodes other than the root node on a path are all used to record the characters in the identifier of a frame of original image. A node on a path is used to record one character in the identifier of a frame of original image. A path includes node 1 and node 2. Among them, node 1 is used to record character 1 in the identifier of a frame of original image, and node 2 is used to record character 2 in the identifier of this frame of original image. On this path, if node 1 is closer to the root node than node 2, then compared with character 2, character 1 is in a higher position in the identifier of this frame of original image. An example of the trie tree can be referred to Figure 10 the content shown in. Figure 10 In the trie tree shown in (1) in, taking the identifier of a frame of original image as 4 characters as an example for construction. Counting the root node, the height of this trie tree is 5. The nodes between the root node and the leaf nodes of the trie tree can be called internal nodes.

[0102] In some possible implementation manners, when the child nodes of any trie tree node (simply referred to as a node) in the trie tree are sorted from front to back, they gradually increase. This enables the X leaf nodes in the trie tree to be used to construct a serialized linked list, and this serialized linked list includes the linked list nodes of X frames of original images, which indicates the result of the identifiers of X frames of original images arranged from front to back, and the identifiers of X frames of original images gradually increase from front to back. As Figure 10 shown in (1) in, the serialized linked list constructed by the X leaf nodes in the trie tree indicates the identifiers 4694, 5431, 5472, 5477, 5479, 5483, 5633, and 5671 from front to back in sequence.

[0103] AsFigure 10 As shown in (2) in the middle, a non-root node in the trie (trie node, simply referred to as a node) can be used to record the value of the node, the distribution information of the node's child nodes, the leaf node flag, the pointer to the child node, and the pointer to the linked list node.

[0104] Among them, the value of the node can be represented by a single character in "0-9", and its character type is char.

[0105] The distribution information of the child nodes can be represented by a 10-bit binary number, and its character type is bit. If the f-th bit in the 10-bit binary number is 1, it means that there is a child node with the value f among the child nodes of the node. If the f-th bit is 0, it means that there is no child node with the value f among the child nodes of the node. f is a single digit in "0-9".

[0106] The leaf node flag (isLeaf) is 0 or 1, and is used to indicate whether the node is a leaf node. 0 means that the node is not a leaf node, and 1 means that the node is a leaf node.

[0107] The pointer to the child node is represented by an array. This array has 10 elements, denoted as children

[10] . The f-th element in the array records the pointer f, and this pointer f points to the child node with the value f among the child nodes of the node.

[0108] The pointer to the linked list node is used to point to the linked list node of the leaf node (which is also the linked list node of the original image indicated by the leaf node). The linked list node of a leaf node includes the value of the linked list node, and the value of this linked list node is the identifier of the image indicated by the linked list node. The linked list node of a leaf node can also include a pointer to the original image indicated by the linked list node (such as the original image a), and this pointer can be represented as the image address. The linked list node of a leaf node can also include a pointer to the previous linked list node and a pointer to the next linked list node. The pointer to the previous linked list node can be used to determine the identifier and address of the previous frame of the original image a. The pointer to the next linked list node can be used to determine the identifier and address of the next frame of the original image a.

[0109] A node can only have one of the two pointers. Among them, the two pointers are: the pointer to the child node, the pointer to the linked list node. Whether a node is a leaf node determines what kind of pointer the node has, or, one of the two pointers of a node is null. If a node is a leaf node, then the node has no child nodes but has a linked list node, the pointer to the child node is null, and the pointer to the linked list node is non-null. If a node is not a leaf node, then the node has child nodes but no linked list node, the pointer to the child node is non-null, and the pointer to the linked list node is null.

[0110] It should be noted here that the linked list nodes are also used to construct the serialization linked list mentioned above. When arranging the X leaf nodes in the trie and the X linked list nodes of the original images in the serialization linked list from front to back, the i-th leaf node is used to record the pointer pointing to the i-th linked list node, and the i-th linked list node is used to record the information of the i-th original image, the linked list node of the (i - 1)-th original image, and the linked list node of the (i + 1)-th original image. The acquisition time of the (i - 1)-th original image (denoted as acquisition time 1) is before the acquisition time of the i-th original image (denoted as acquisition time 2), or acquisition time 1 is the same as acquisition time 2 and the serial number 1 is less than the serial number 2. The serial number 1 is the serial number of the camera that acquires the (i - 1)-th original image. The serial number 2 is the serial number of the camera that acquires the i-th original image, and i is an integer greater than 1 and less than X. The (i - 1)-th original image can also be referred to as the previous original image of the i-th original image. The (i + 1)-th original image can also be referred to as the next original image of the i-th original image.

[0111] Based on the foregoing, it can be seen that compared with the red-black tree, the trie will be shorter. The height of the trie depends on the length of the identifier of the original image. If the identifier includes U characters, the height of the trie is U + 1. The query efficiency of the trie is relatively high, usually O(U). This overcomes the defect 2 of the red-black tree.

[0112] The trie arranges each character in the identifier of a frame node into different nodes respectively, which is beneficial to the expansion of the trie. When the data structure that the nodes in the trie need to record changes, it can be changed by adding or reducing the nodes on the path. For example, if one more dimension of data (such as the serial number of the camera that acquires the original image) needs to be added, only the serial number needs to be added after the leaf node, and at the same time, the serialization linked list also needs to be adaptively modified. This overcomes the defect 4 of the red-black tree.

[0113] As Figure 11 shown, it is an exemplary trie involved in shooting by multiple cameras (camera 1 and camera 2). It can be obtained by modifying the trie Figure 10 shown above. The modification methods include: adding child nodes after each leaf node in the original trie, and the added child nodes serve as new leaf nodes. The value of the new leaf node can be used to record the serial number of the camera carried in the identifier of the original image. Modify the original leaf nodes in the original trie to internal nodes, and at the same time modify the pointers pointing to the child nodes and the pointers pointing to the linked list nodes. Each new leaf node in the new trie can construct a new serialization linked list. For example, the new serialization linked list sequentially indicates the identifiers 46941, 54312, 54721, 54722, 54772, 54792, 54832, 56332, and 56712 from front to back.

[0114] It should be noted here that Figure 10The trie tree shown can also be an example of the trie tree built by the electronic device in a single-camera shooting scenario. Figure 11 The trie tree shown can also be an example of the trie tree built by the electronic device in a multi-camera shooting scenario. Figure 11 In this case, the value of the leaf node in the trie tree can be used to record the serial number of the camera in the identifier corresponding to a frame of the original image.

[0115] By traversing the nodes in the trie tree, the electronic device can determine whether the identifier of the original image to be dumped into the ROM (the identifier to be queried) is in the trie tree. The process of determining the identifier to be queried includes: the pointers recorded in the node pointing to the child nodes enable the electronic device to complete the query relatively quickly. The query process includes: when the electronic device determines in the trie tree the node to which the highest-order character of the identifier to be queried belongs. Through the pointer (children1

[10] ) pointing to the child nodes recorded in the node to which the highest-order character belongs, it can be determined whether the node to which the second-highest-order character in the identifier to be queried belongs exists. The determination process includes: looping through the pointers in children1

[10] until it is determined that there is children[char3] and children[char3] is not null, then it is determined that the node to which the second-highest-order character in the identifier to be queried belongs exists. char3 is the value of the second-highest-order character. If there is a node to which the second-highest-order character belongs among the child nodes of the node to which the highest-order character belongs, then the node to which the second-order character belongs is located through the pointer pointing to the child node. Then, through the pointer pointing to the child node recorded in the node to which the second-highest-order character belongs, the node to which the next character belongs is determined, and so on, to determine whether the identifier to be queried is recorded in the trie tree.

[0116] Reference Figure 12 Taking the identifier to be queried as "54772" as an example for illustration. The electronic device first determines the node 5 closest to the root node. The value of node 5 is the highest order in "54772". Then, it is determined through the pointers pointing to the child nodes recorded in node 5 that the pointer children[4] exists and is not null. Through children[4], the child node 4 of node 5 (abbreviated as node 4) is located. Here, the value of node 4 is the second-highest order 4 in "54772". Subsequently, the electronic device determines through the pointers pointing to the child nodes recorded in node 4 that the pointer children[7] exists and is not null. Through children[7], the child node 7 of node 4 (abbreviated as node 7) is located. Here, the value of node 7 is the third-highest order 7 in "54772". And so on, the electronic device can determine the path where the identifier to be queried "54772" is located. And through the pointer pointing to the original image indicated by 54772 recorded in the linked list node (denoted as linked list node a) of the leaf node on this path, the original image indicated by the identifier to be queried "54772" is located.

[0117] Not limited to the above manner, the process of determining the identifier to be queried may further include: the distribution information of the child nodes recorded in the node and the pointers pointing to the child nodes enable the electronic device to complete the query more quickly. The query process includes: when the electronic device determines in the trie the node to which the highest-bit character of the identifier to be queried belongs. Based on the distribution of the child nodes of the node to which the highest-bit character belongs, the node to which the second-highest-bit character of the identifier to be queried belongs can be determined. If there is a node to which the second-highest-bit character belongs among the child nodes of the node to which the highest-bit character belongs, the node to which the second-bit character belongs is located through the pointer pointing to the child node. Then, based on the distribution information of the child nodes of the node to which the second-highest-bit character belongs, the node to which the next character belongs is found, and so on, to determine whether the identifier to be queried is recorded in the trie.

[0118] As Figure 12 shown, taking the identifier to be queried as "54772" as an example for illustration. The electronic device first determines the node 5 closest to the root node. The value of node 5 is the highest bit in "54772". Then, based on the distribution information of the child nodes of node 5 (0001010000B, from the highest bit to the lowest bit, the 9th, 8th, 7th, 6th, 5th, 4th, 3rd, 2nd, and 1st bits in sequence), it is determined that the 4th bit is 1, so node 5 has a child node 4. Then, through the pointer children[4] in node 5 pointing to the child node 4, the child node 4 of node 5 (abbreviated as node 4) is located. Here, the value of node 4 is the second-highest bit 4 in "54772". Subsequently, the electronic device determines that the 7th bit is 1 based on the distribution information of the child nodes of node 4 (0110001000B, from the highest bit to the lowest bit, the 9th, 8th, 7th, 6th, 5th, 4th, 3rd, 2nd, and 1st bits in sequence), so node 4 has a child node 7. Then, through the pointer children[7] in node 4 pointing to the child node 7, the child node 7 of node 4 (abbreviated as node 7) is located. Here, the value of node 7 is the third-highest bit 7 in "54772". And so on, the electronic device can determine the path where the identifier to be queried "54772" is located. And through the pointer pointing to the original image indicated by 54772 recorded in the linked list node (denoted as linked list node a) of the leaf node on this path, the original image indicated by the identifier to be queried "54772" is located.

[0119] It should be noted here that, continuing to refer to Figures 10 - 12 the content shown, when deleting and updating nodes, there is no need to modify the structure of the trie. This overcomes defect 1 of the red-black tree.

[0120] It should also be noted that the identifiers recorded in the trie can also be reflected in the serialized linked list. The serialized linked list is serialized, and when querying, relying on the identifier recorded in one node, the identifiers of other nodes can be quickly queried through the serialized linked list. This overcomes defect 3 of the red-black tree. For example, referring to Figure 12As shown in the figure, the electronic device can locate the identifier "54722" and the address of the previous frame of the original image indicated by the identifier to be queried "54772" through the pointer (the pointer pointing to 54722) recorded in the linked list node a that points to the previous linked list node.

[0121] If the identifier to be queried is not in the trie, by querying the trie, the electronic device can also determine the target elder brother node. The target elder brother node is the smallest elder brother node of the character that is not recorded in the highest bit of the identifier to be queried. Or, in the case where there is no smallest elder brother node for the character that is not recorded in the highest bit, the target elder brother node is the smallest elder brother node of the character that is recorded in the lowest bit. The smallest elder brother node of a character has the same parent node as the node to which the character belongs, and is the smallest node among the child nodes of the parent node that is larger than the node to which the character belongs. Continue to refer to Figure 12 As shown in the figure, taking the identifier to be queried as "55772" as an example for illustration, the character that is not recorded in the highest bit of the identifier to be queried "55772" is the second highest bit 5, and its smallest elder brother node is node 6.

[0122] Subsequently, the electronic device can determine a target identifier through the target elder brother node. The target identifier is the smallest identifier among the elder brother identifiers of the identifier to be queried. The elder brother identifiers of the identifier to be queried are the identifiers larger than the identifier to be queried in the serialized linked list. For example, by determining the distribution information (0010001000B, from the highest bit to the lowest bit, which are the 9th, 8th, 7th, 6th, 5th, 4th, 3rd, 2nd, and 1st bits in sequence) of the child nodes of node 6, it is determined that the smallest child node of node 6 is 3, then "563*" can be determined. Subsequently, the electronic device then determines the value of the smallest child node of node 3 through the trie, then "5633*" can be determined. By analogy, the identifier "56332" is determined as the target identifier.

[0123] Subsequently, through the linked list node where the target identifier is located, it can be queried in the serialized linked list whether other identifiers to be queried are recorded in the trie. Among them, the original images indicated by other identifiers to be queried can be used to synthesize the same image as the original image indicated by the identifier to be queried.

[0124] In this way, compared with the method of obtaining other identifiers to be queried and querying them again in the trie when the identifier to be queried is not in the trie, some computing power can be saved. Because the positions of the identifiers of the original images with continuous acquisition times in the trie are close, and the identifiers of the respective frames of the original images used to synthesize one image are usually continuous. It can be seen that the target identifier and the identifier to be queried usually have the same prefix. If the identifier to be queried is not in the trie, the method of returning the target identifier can save the time for querying the same prefix in the trie.

[0125] When shooting with a single camera, the M frames of images required to synthesize one image are from this single camera. When shooting with multiple cameras, the M frames of images required to synthesize one image are from multiple cameras. In the following text, two cameras (camera 1 and camera 2) are taken as an example for illustration. It is noted that among the M frames of images, M1 frames of images are from camera 1 and M2 frames of images are from camera 2. Both M1 and M2 are numbers greater than or equal to 1. Figure 13 shows an exemplary flowchart involved in shooting an image when implementing an image processing method during multi-camera shooting. The description of this process can refer to the following steps S101, step S102, step S103a - step S105a, and step S103b - step S105b.

[0126] S101. Start the camera. The electronic device obtains the original image data 1 through camera 1 in the camera and records it in the RAM, and obtains the original image data 2 through camera 2 in the camera and records it in the RAM.

[0127] The original image data 1 includes the original image collected by camera 1 and the acquisition information corresponding to the original image.

[0128] The original image data 2 includes the original image collected by camera 2 and the acquisition information corresponding to the original image.

[0129] S102. Receive multiple instructions for shooting an image, including the instruction for shooting image 1 received at time 1.

[0130] In some possible cases, the instructions are generated by the camera application and sent to the processor of the electronic device. Receiving multiple instructions for shooting an image includes the processor receiving the instruction for shooting an image sent by the camera application. One instruction is used to generate one image.

[0131] The multiple instructions include the instruction for shooting image 1 received at the aforementioned time 1. In addition to instruction 1, it may also include the instruction for shooting image 2 received at time 2. Time 2 is before or after time 1.

[0132] After receiving an instruction 1 for shooting image 1, the electronic device can perform the operation of synthesizing image 1 based on this instruction 1 (see steps S103b - step S105b). Or, the electronic device can perform a dump operation based on this instruction 2, store the original image and metadata required for synthesizing image 1 in the ROM, and then synthesize image 1 later.

[0133] It should be noted here that the dump operation takes a certain amount of time. When the electronic device performs the dump operation and receives a new instruction to capture an image, the electronic device can first record the new instruction in the instruction queue. After the dump operation ends, the electronic device then obtains the new instruction from the instruction queue and performs the dump operation for the new instruction. The instruction queue can be a first-in, first-out queue or a first-in, last-out queue. In the case of a first-in, first-out queue, it means that the instruction received first in the instruction queue can be responded to first. In the case of a first-in, last-out queue, it means that the instruction received last in the instruction queue can be responded to first. Here, responding to an instruction means performing the dump operation for that instruction.

[0134] When the dump condition is met, the following steps S103a to S105a in the figure are executed. After performing the dump operation and storing the original images and metadata required for image synthesis in the ROM, when the conditions are met, they are then loaded into the RAM for image synthesis.

[0135] S103a. Determine the identifiers of the M1-frame original image and the M2-frame original image that match time 1 and record them in synthesis information 1. The synthesis information 1 also includes the identifiers of the metadata corresponding to the M1-frame original image and the M2-frame original image. The synthesis information 1 is also used to indicate that the M1-frame original image and the M2-frame original image are used for synthesizing image 1.

[0136] When shooting with multiple cameras, the identifier of the original image is represented by the acquisition time of the original image plus the serial number of the camera that acquired the original image.

[0137] The metadata corresponding to the original image is not involved in multiplexing. In some possible cases, the identifier of the metadata can be the address of the metadata, which is used to find the metadata in the ROM.

[0138] The metadata corresponding to one frame of the original image for synthesizing image 1 can be generated based on the acquisition information corresponding to the frame of the original image and the shooting information at time 1 when the electronic device determines to use the frame of the original image for synthesizing image 1.

[0139] S104a. Transfer the original images that have not been recorded in the ROM among the M1-frame original image and the M2-frame original image from the RAM to the ROM, and record the metadata corresponding to each frame of the original image among the M1-frame original image and the M2-frame original image in the ROM. Whether a frame of the original image is recorded in the ROM is queried through a trie tree.

[0140] The original images in the M1-frame original image and the M2-frame original image that are not recorded in the ROM are denoted as the original image 1. The other original images in the M1-frame original image and the M2-frame original image except the original image 1 are called the original image 2. Here, the original image 2 has been dumped into the ROM before the dump operation 1 is executed. This means that the original image 2 is reused and can be used for synthesizing other images (such as image 2) in addition to being used for synthesizing the image 2. Among them, the dump operation 1 is a dump operation executed in response to the instruction 1.

[0141] For each frame of the original images in the M1-frame original image and the M2-frame original image, the electronic device needs to determine whether the identifier of each frame of the original image has been recorded in the trie, so as to determine the original image 1 and the original image 2 among them.

[0142] Determining whether the identifier of each frame of the original image has been recorded in the trie, so as to determine the original image 1 and the original image 2 among them, includes: The electronic device first determines the maximum identifier (a to-be-searched identifier) in the M-frame original images (the M1-frame original image and the M2-frame original image), searches for the maximum identifier through the trie and determines the target identifier. When the maximum identifier is recorded in the trie, the target identifier is the maximum identifier. When the maximum identifier is not recorded in the trie, the target identifier is the minimum identifier among the sibling identifiers recorded in the trie. Among them, the sibling identifier is all the identifiers in the trie that are greater than the maximum identifier. After determining the target identifier, in the serialized linked list, starting from the linked list node where the target identifier is located, traverse the linked list nodes in the serialized linked list forward. During the traversal, when it is determined that the linked list node 2 has not been determined before the linked list node 1, it is determined that the identifier of the original image 1 is not recorded in the trie. The identifier of the original image recorded by the linked list node 1 is less than the identifier of the original image 1, and the identifier of the original image recorded by the linked list node 2 is equal to the identifier of the original image 1. When the linked list node 3 is determined, it is determined that the identifier of the original image 1 is recorded in the trie. The identifier of the original image recorded by the linked list node 3 is equal to the identifier of the original image 3.

[0143] Among them, finding the maximum identifier and determining the target identifier through the trie tree includes: when querying the characters in the maximum identifier in the trie tree, start searching from the highest-order character to the lowest-order character in the maximum identifier. For each character searched (the c-th character), not only the node to which the character belongs needs to be determined, but also the smallest older brother node of the character needs to be determined as the target older brother node. Let c = c + 1, and continue to search for the c-th character. Not only the node to which the c-th character belongs needs to be determined, but also the smallest older brother node of the c-th character needs to be determined, and the smallest older brother node of the c-th character is used as the target older brother node to update the target older brother node. If the c-th character does not have an older brother node, the smallest older brother node of the c-th character is not used to update the target older brother node, and the current target older brother node remains unchanged. Then let c = c + 1, and continue to search for the c-th character until it is determined that the c-th character is not in the trie tree, or it is determined that all characters in the maximum identifier are in the trie tree, and the maximum identifier is used as the target identifier. If the c-th character is not in the trie tree, but the c-th character has a smallest older brother node, the smallest older brother node of the c-th character still needs to be used to update the target older brother node, and the target older brother node is returned. Subsequently, the electronic device can determine the target identifier through the target older brother node.

[0144] In the foregoing content, determining the node to which the c-th character belongs and then determining the smallest older brother node of the c-th character includes: using the formula lowbit(bit0&~(1<<(char0+1)-1) ) to determine the smallest older brother node of the c-th character. char0 in the formula represents the value of the c-th character, which is also the value of the node to which the c-th character belongs. bit0 represents the child node information of the parent node of the node to which the c-th character belongs. 1<<(char0+1)-1) is used to generate a binary number, and the lowest char0 + 1 bits in the binary number are all 0. bit0 is a 10-bit number. In the formula, 1<<(char+1)-1) also means setting the lowest char0 + 1 zeros in 0000000000B to 1. For example, if char0 is equal to 7, then 1<<(char+1)-1) is represented as 0011111111B. "~" represents the bitwise negation operation, and "&" represents the bitwise AND operation. "bit0&~(1<<(char0+1)-1)" can be used to represent all child nodes greater than char0 in the child nodes of the node to which the c-th character belongs. lowbit(x) = x&(-x) = 2 r , which is used to determine that the lowest 1 in x is in the r-th bit. Here, lowbit(bit0&~(1<<(char0+1)-1) ) is used to determine the smallest child node r with a value greater than char0 in the child nodes indicated by bit0, and r is the smallest older brother node of the c-th character.

[0145] Here, lowbit(bit0&~(1<<(char0+1)-1) ) can be implemented through bitwise operations, and the electronic device can execute bitwise operations quickly. Usually, it only takes one clock cycle to complete.

[0146] It should be noted here that if the c-th character does not have a corresponding node, bit0 represents the child node information of the node to which the character before the c-th character belongs.

[0147] The electronic device can determine the target identifier through the target elder brother node, including: setting the target elder brother node as the node to be processed, obtaining the child node information of the node to be processed, denoted as bit1. Then execute lowbit(bit1) to determine the smallest child node of the node to be processed. If the smallest child node is not a leaf node, execute Operation 1. Operation 1 includes: updating the smallest child node as the node to be processed, obtaining the child node information of the node to be processed, denoted as bit1, and then executing lowbit(bit1) to determine the smallest child node of the node to be processed. If the smallest child node is not a leaf node, execute Operation 1 again until a leaf node is determined. Use the identifier recorded in the linked list node of the leaf node as the target identifier.

[0148] It should be noted here that if the result of lowbit(bit&~(1<<(char+1)-1) ) is 0, it means that the c-th character does not have an elder brother node.

[0149] Reference Figure 14 In the example in, taking the identifiers of the M-frame original images of Image 1 as "54312, 54721, 54722, 54752" as an example for illustration. The largest identifier is: "54752". The electronic device searches for the largest identifier "54752" through the trie tree and determines the target identifier "54772". The determination process can refer to the following description of Figure 15 and Figure 16 .

[0150] As Figure 15 shown in (1) in, start querying from the highest bit 5 in "54752". When querying the node (node 4) to which the second highest bit 4 belongs, based on the foregoing formula through the value of node 4 and the child node information of node 4, determine that the smallest elder brother node of node 4 is node 6. The determination process includes: executing lowbit(0001010000B&~(1<<(4+1)-1) ) equals 2 6 . Then the elder brother node of node 4 is node 6. Set node 6 as the target elder brother node. Among them, 0001010000B is the child node information of the parent node of node 4.

[0151] Reference Figure 15As shown in (2) below, when querying the third highest bit 7 and finding the node (node 7) to which the third highest bit 7 belongs, based on the value of node 7 and the child node information of node 7 using the aforementioned formula, it is determined that the smallest older brother node of node 7 is node 8. The determination process includes: executing lowbit(0110001000B & ~(1 << (7 + 1) - 1)) equals 2 8 . Then the smallest older brother node of node 7 is node 8. Then use the smallest older brother node 8 to update the target older brother node 6 as the new target older brother node. Among them, 0110001000B is the child node information of the parent node of node 7.

[0152] Reference Figure 16 As shown in (1) below, when querying the fourth highest bit 5, it is determined that the trie does not include the node to which the character 5 belongs, and the smallest older brother node of the character 5 is determined to be node 7. The determination process includes: executing lowbit(1010000100B & ~(1 << (5 + 1) - 1)) equals 2 7 . Then the smallest older brother node of the character 5 is node 7. Then use Figure 16 the older brother node 7 shown in (1) below to update the target older brother node 7 as the new target older brother node. Among them, 1010000100B is the child node information of the node to which the third highest bit 7 belongs.

[0153] Subsequently, determine Figure 16 the child node information of the target older brother node shown in (2) below is 0000000100B. Then determine lowbit(0000000100B) = 2 2 . Then the smallest child node determined by the target older brother node is node 2, and this node 2 is a leaf node. Subsequently, use the identifier "54772" recorded in the linked list node of node 2 as the target identifier.

[0154] Reference Figure 17 As shown in (1) below, in the serialized linked list, starting from the linked list node where "54772" is located, traverse the linked list nodes in the serialized linked list forward. It can be determined that the identifiers "54312, 54721, 54722" have all been recorded in the trie.

[0155] Thus, it can be seen that the original images indicated by the identifiers "54312, 54721, 54722" are the original image 2. The original image indicated by the identifier "54752" is the original image 1.

[0156] After step S104a, the electronic device also needs to add the identifier of the original image 1 to the trie to update the trie and the serialized linked list.

[0157] Among them, the update includes: adding a new node in the trie to obtain path 1, where path 1 is used to determine the identifier of the original image 1, adding the information of new node 1 to the child node distribution information of the parent node corresponding to new node 1, updating the pointer children[char2] in the pointer of the parent node corresponding to new node 1 that points to the child node to point to new node 1, and adding the linked list node of the original image 1 to the serialized linked list. When the number of new nodes is 1, the new node is new node 1; when the number of new nodes is greater than 1, new node 1 is the new node closest to the root node among the new nodes. Here, char2 represents the value of new node 1.

[0158] Among them, adding the information of new node 1 to the child node distribution information of the parent node corresponding to new node 1 includes: executing bit2 = bit2|(1<<char2). "=" represents assignment or update. The bit2 before update is the child node distribution information of the parent node corresponding to new node 1. bit2|(1<<char2) means setting the char2-th bit in the bit2 before update from 0 to 1.

[0159] Reference Figure 17 As shown in (2) of the reference [], taking adding the identifier "54752" to the trie as an example for illustration. Here, the nodes to be added to the trie are node 5 and node 2, which are used to represent the fourth character 5 and the fifth character 2 in the identifier "54752" respectively. Among the added new nodes, the new node closest to the root node is node 5. Add the information of node 5 to the child node information of the parent node (node 7) corresponding to node 5. The addition process includes: updating the child node information of node 7 (denoted as bit3), executing bit3 = 1010000100B|(1<<5), and updating the child node information of node 7 from 1010000100B to 1010100100B. Setting the 5th bit from 0 to 1 indicates that there is node 5 among the child nodes included in node 7. Point the pointer children[5] in node 7 that points to the child node to the child node 5 of node 7.

[0160] S105a. After determining that the synthesis condition is met, obtain the original image of M1 frame and the original image of M2 frame from the ROM through the synthesis information 1, and the metadata corresponding to each frame of the original image, and load them into the RAM to synthesize image 1.

[0161] Among them, meeting the synthesis condition includes but is not limited to at least one of the following: exiting the continuous shooting mode, the electronic device exiting the camera, switching the camera to run in the background, detecting an operation of the user viewing an image, and the good performance of the electronic device, that is, even using the multi-frame fusion technology will not cause the performance to drop to jitter. For the description of the synthesis information queue 1, reference can be made to the foregoing Figures 3 - 5Description of it and related content. Among them, good performance of the electronic device includes, but is not limited to, at least one of the following: small memory occupancy, small processor load, etc.

[0162] For example, the conditions for synthesis can be: good performance of the electronic device, and the electronic device exits the camera or switches the camera to the background. After the electronic device exits the camera or switches the camera to the background, if the electronic device is still performing a high-load task (such as playing a game), it can wait until the high-load task ends and the electronic device has good performance before synthesizing the image.

[0163] The camera involved in the embodiments of the present application can also be understood as a camera application.

[0164] The synthesis information 1 records the identifiers of M frames of original images (M1 frame of original image and M2 frame of original image), which can be used to determine the address of the original image, so as to obtain the original image. Determining the address of the original image includes: determining the leaf node of the original image in the trie based on the identifier of the original image, and finding the original image in the ROM through the pointer to the original image recorded in the linked list node of the leaf node. The pointer to the original image can determine the address of the original image.

[0165] The synthesis information 1 records the identifiers of the metadata corresponding to M frames of original images (M1 frame of original image and M2 frame of original image) respectively, which can be used to determine the address of the metadata, so as to obtain the metadata. The identifier of the metadata can be used to determine the address of the metadata, including: the identifier of the metadata is the address of the metadata.

[0166] After obtaining M frames from the ROM and the metadata corresponding to each frame of the original image, load them into the RAM to synthesize Image 1.

[0167] In the case where the dump condition is not met, no dump is performed. Execute the following steps SS103b - S105b in the figure. Synthesize the image through the original image and metadata in the memory.

[0168] S103b. Determine the synthesis information a. The synthesis information a includes the address information of the M1 frame of the original image and its metadata and the M2 frame of the original image and its original data that match the time 1 in the RAM. The synthesis information a is also used to indicate that the M1 frame of the original image and the M2 frame of the original image are used to synthesize Image 1.

[0169] S104b. Based on the address information in the synthesis information a, obtain the M1 frame of the original image and the metadata corresponding to each original image from the RAM, and obtain the M2 frame of the original image and the metadata corresponding to each original image from the RAM.

[0170] S105b. Synthesize Image 1 based on the M1 frame of the original image and the metadata corresponding to each original image and the M2 frame of the original image and the metadata corresponding to each original image.

[0171] The content involved in steps S103b - S105b can be referred to the description of the foregoing Figure 1 and its related content, which will not be elaborated here.

[0172] Here, it should be noted that since the original images are reused, the electronic device can set a usage count (initially 0) for each frame of the original image. The usage count of a frame of the original image is used to represent the number of times the frame of the original image is used for synthesizing an image. Each time the electronic device queries the frame of the image, the usage count of the frame of the original image can be incremented by 1. After the electronic device uses the original image to synthesize an image once, the usage count of the frame of the original image is decremented by 1. When the usage count of a frame of the original image is 0, the identifier of the frame of the original image is deleted in the trie tree.

[0173] Deleting the identifier of the frame of the original image in the trie tree includes: determining all nodes (nodes to be deleted) on the path indicating the identifier corresponding to the frame of the original image in the trie tree that have no child nodes. And determining the highest - order node to be deleted and the leaf node (leaf node to be deleted) among the nodes to be deleted. Iteratively delete the nodes to be deleted, starting from the leaf node to be deleted and deleting up to the highest - order node to be deleted. Delete the bit information of the highest - order node to be deleted in the child - node distribution information of the parent node of the highest - order node to be deleted, set the pointer in the parent node of the highest - order node to the highest - order node to be deleted to be null, and, delete the linked - list node of the leaf node to be deleted from the serialized linked list. Deleting the linked - list node of the leaf node to be deleted includes: updating the linked - list node of the leaf node before the leaf node to be deleted and the linked - list node of the leaf node after the leaf node to be deleted. So that the pointer in the linked - list node of the previous leaf node to the next linked - list node points to the linked - list node of the next leaf node, and, so that the pointer in the linked - list node of the next leaf node to the previous linked - list node points to the linked - list node of the previous leaf node.

[0174] As Figure 18 shown, it is a schematic diagram of deleting the identifier "54312" in the trie tree. The nodes on the path indicating the identifier "54312" that have no child nodes include node 2, node 1, and node 3. Among them, the highest - order node to be deleted is node 3, and it is also necessary to update the child - node information of the corresponding parent node (node 4) of node 3 (from 0110001000B to 0110000000B). Delete the linked - list node of the leaf node to be deleted (node 2) from the serialized linked list.

[0175] Here, it should be noted that referring to the foregoing Figures 10 - 12 、 Figures 14 - 18The trie tree shown. The electronic device can also add - INF (infinitesimal) linked list nodes and INF (infinity) linked list nodes at the beginning and end of the serialized linked list respectively. This enables the electronic device to always return a linked list node when querying in the serialized linked list. For example, if the identifier to be queried is greater than the maximum identifier representing the original image recorded in the trie tree, the INF linked list node is returned.

[0176] It should also be noted that the storage time of the original image and metadata recorded in the ROM is time - limited. If after the storage time exceeds the preset time (for example, 1 hour), the electronic device has not used the original image and metadata in the ROM for image synthesis. Then the electronic device can delete the original image and metadata whose storage time exceeds the preset time, and update the trie tree and the serialized linked list. The calculation time of the preset time can include: starting from when the original image and metadata are recorded in the ROM, or starting to count time after the electronic device shuts down. If the shutdown time reaches the preset time, then after power - on, the original image and metadata in the ROM are deleted, and the trie tree and the serialized linked list are updated.

[0177] It should also be noted that the timing for building the trie tree includes: when it is determined that the dump condition is met, the trie tree can be created. Initially, there is only a root node in the trie tree. After the electronic device dumps the original image to the ROM, the identifier of the dumped original image can be recorded in the trie tree. One frame is recorded and updated in the trie tree for each dumped frame.

[0178] The following introduces the exemplary electronic device provided in the embodiments of the present application.

[0179] Figure 19 is a schematic structural diagram of the electronic device provided in the embodiments of the present application.

[0180] The following takes the electronic device as an example to specifically illustrate the embodiments. It should be understood that the electronic device can have more or fewer components than those Figure 19 shown, can combine two or more components, or can have different component configurations. Figure 19 The various components shown in

[0181] The electronic device may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0182] The internal memory 121 may include one or more random access memories (RAMs) and one or more non-volatile memories (NVMs).

[0183] The random access memory may include a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation of DDR SDRAM is generally called DDR5 SDRAM), etc.

[0184] The non-volatile memory may include a read-only memory (ROM) and a flash memory.

[0185] Random access memory is a volatile memory used for temporarily storing data and instructions being processed. As the internal memory, random access memory can be directly read from and written to by the processor 110, and can be used to store the operating system or executable programs (such as machine instructions) of other running programs, and can also be used to store data of users and application programs, etc.

[0186] Non-volatile memory can also store executable programs and store data of users and application programs, etc. The executable programs and data in non-volatile memory can be loaded into random access memory for direct reading from and writing to by the processor 110.

[0187] It can be understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than those illustrated, or combine certain components, or split certain components, or have different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0188] In the embodiments of this application, the processor 110 can call computer instructions stored in the internal memory 121 to cause the electronic device to execute the methods in the embodiments of this application.

[0189] This application also provides a chip system, which includes at least one processor for implementing the functions involved in the methods executed by the electronic device in any one of the above embodiments.

[0190] In a possible design, the chip system further includes a memory for storing program instructions and data, and the memory is located inside or outside the processor.

[0191] The chip system can be composed of chips or can include chips and other discrete devices.

[0192] Optionally, the processor in the chip system can be one or more. The processor can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor that realizes its functions by reading software code stored in the memory.

[0193] Optionally, the memory in the chip system can also be one or more. The memory can be integrated with the processor or can be separately arranged from the processor, which is not limited in the embodiments of this application.

[0194] Exemplarily, the memory can be a non-transitory processor, such as a read-only memory (ROM), which can be integrated with the processor on the same chip or can be separately provided on different chips. The embodiments of the present application do not make specific limitations on the type of the memory and the setting manner of the memory and the processor.

[0195] The present application also provides a computer program product, which includes a computer program (which can also be referred to as code or instruction). When the computer program runs, it causes a computer to execute the method executed by the electronic device in any of the above embodiments.

[0196] The present application also provides a computer-readable storage medium, which stores a computer program (which can also be referred to as code or instruction). When the computer program runs, it causes a computer to execute the method executed by the electronic device in any of the above embodiments.

[0197] As described above, the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. However, such modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0198] As used in the above embodiments, depending on the context, the term "when..." can be interpreted to mean "if...", or "after...", or "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if detecting (the stated condition or event)" can be interpreted to mean "if determining...", or "in response to determining...", or "when detecting (the stated condition or event)", or "in response to detecting (the stated condition or event)".

[0199] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and the appended claims of the present application, the singular forms "a", "an", "the", "above", "said", "this" are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more of the listed items.

[0200] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0201] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state drive), etc.

[0202] Those of ordinary skill in the art can understand all or part of the processes in the methods of the above embodiments. The processes can be completed by relevant hardware instructed by a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The aforementioned storage medium includes various media that can store program codes, such as ROM or random access memory RAM, magnetic disks, or optical discs.

Claims

1. An image processing method, characterized in that: Applied to an electronic device, the electronic device includes a memory and a first memory other than the memory, the method includes: Starting a camera, capturing an original image through the camera of the electronic device, and recording the original image into the memory; receiving a plurality of instructions for capturing images, the plurality of instructions comprising a first instruction for capturing a first image received at a first time and an instruction for capturing a second image received at a second time; the second time being earlier or later than the first time; In the case where the dump condition is met, a dump operation is performed; the dump operation includes: recording the identifiers of the M frames of original images matching the first time and the identifiers of the metadata respectively corresponding to the M frames of original images into the first synthesis information; dumping the first original image in the M frames of original images from the memory to the first memory; the original images other than the first original image in the M frames of original images have been dumped to the first memory before the dump operation is performed; whether each frame of the original image in the M frames of original images has been dumped to the first memory before the dump operation is performed is determined by query data; the query data includes the identifiers of the original images stored in the first memory; when the dump operation is performed, the metadata respectively corresponding to the M frames of original images are stored in the first memory; the metadata corresponding to one frame of original image is used to indicate the acquisition information of the one frame of original image and the shooting information corresponding to the first time; the original images other than the first original image in the M frames of original images are also used to synthesize the second image, and M is an integer greater than 1; After executing the dump operation, when the synthesis condition is met, the M frames of original images and metadata corresponding to the M frames of original images are obtained from the first memory through the first synthesis information, and loaded into the memory to synthesize the first image.

2. The method according to claim 1, characterized in that The method further comprises: When the dump condition is not met, the first image is synthesized by using the M frames of original images in the memory and the metadata corresponding to the M frames of original images respectively; when the dump operation is not performed, the metadata corresponding to the M frames of original images respectively are stored in the memory.

3. The method according to claim 1, characterized in that Meeting the dump condition includes: the electronic device is in a continuous shooting mode; wherein the continuous shooting mode is used for the electronic device to continuously shoot at least two images when a shooting control is long pressed.

4. The method according to claim 3, characterized in that Before acquiring the M frames of original images and metadata respectively corresponding to the M frames of original images from the first memory using the first synthesis information, the method further includes: Obtaining the first synthetic information from a synthetic information queue; The method further comprises: Acquire second synthesis information from the synthesis information queue; the second synthesis information includes an identifier of the E frame original image matching the second time, and identifiers of metadata corresponding to the E frame original images; the second time is the time when the second instruction is received, and the E is equal to the M; The E frame original image and metadata respectively corresponding to the E frame original image are acquired from the first memory through the second synthesis information, and loaded into the memory to synthesize the second image.

5. The method according to claim 4, characterized in that Meeting the synthesis condition includes at least one of the following: turning off the camera, running the camera in the background, and the performance of the electronic device meeting a preset condition.

6. The method according to any one of claims 1 to 5, characterized in that The query data is constructed through a dictionary tree; the dictionary tree includes a root node and X leaf nodes, and the leaf node is a node in the dictionary tree that does not have a child node; the X indicates the number of original images dumped into the first memory; a path between the root node and a leaf node is used to determine the identification of a frame of the original image, and the identification of the original image determined by different paths is different.

7. The method according to claim 6, characterized in that When the number of cameras that capture the original image is equal to 1, the identification of the original image frame is represented by the acquisition time of the original image frame; when the number of cameras that capture the original image is greater than 1, the identification of the original image frame includes the acquisition time of the original image frame plus the serial number of the camera that captures the original image frame; All nodes except the root node on the path are used to record characters in the identifier of the frame of original image, and the path includes a first node and a second node, the first node is used to record a first character in the identifier of the frame of original image, and the second node is used to record a second character in the identifier of the frame of original image; On the one path, the first node is closer to the root node than the second node, and compared with the second character, the first character is at a higher position in the identifier of the one frame of the original image; The X leaf nodes in the dictionary tree are also used to construct a serialized linked list, which includes linked list nodes of X frames of original images; when the X leaf nodes and the X linked list nodes are sorted from front to back, the i-th linked list node is obtained through the i-th leaf node, and the i-th linked list node is used to determine the information of the i-th frame original image, the linked list node of the i-1-th frame original image, and the linked list node of the i+1-th frame original image; the first acquisition time of the i-1-th frame original image is before the second acquisition time of the i-th frame original image, or the first acquisition time is the same as the second acquisition time and the first serial number is less than the second serial number; the first serial number is the serial number of the camera that acquires the i-1-th frame original image; The second serial number is the serial number of the camera that captures the i-th frame of the original image, and i is an integer greater than 1 and less than X.

8. The method according to claim 7, characterized in that The method further comprises: Before dumping a first original image in the M frames of original images from the memory to the first memory, determining that an identifier of the first original image is not recorded in the dictionary tree; The step of determining that the identifier of the first original image is not recorded in the dictionary tree specifically includes: Determine the maximum identifier from the identifiers of the M frames of original images, search the maximum identifier through the dictionary tree and determine the target identifier, if the maximum identifier is recorded in the dictionary tree, the target identifier is the maximum identifier; if the maximum identifier is not recorded in the dictionary tree, the target identifier is the minimum identifier among the brother identifiers recorded in the dictionary tree, and the brother identifier is greater than the maximum identifier; In a serialized linked list, starting from the linked list node where the target identifier is located, traverse the linked list nodes in the serialized linked list forward, and when the second linked list node is not determined before the first linked list node is determined, determine that the identifier of the first original image is not recorded in the dictionary tree; the identifier of the original image recorded by the first linked list node is less than the identifier of the first original image; and the identifier of the original image recorded by the second linked list node is equal to the identifier of the first original image.

9. The method according to claim 8, characterized in that After dumping the first original image in the M frames of original images from the memory to the first memory, the method further includes: Update the dictionary tree and the serialized linked list, the update comprising: adding a new node in the dictionary tree to obtain a first path, the first path is used to determine the identification of the first original image, adding the information of the first new node to the child node distribution information of the parent node corresponding to the first new node, and adding the linked list node of the first original image to the serialized linked list; when the number of new nodes is 1, the new node is the first new node, and when the number of new nodes is greater than 1, the first new node is the new node closest to the root node among the new nodes.

10. The method according to claim 9, characterized in that The method further comprises: After synthesizing the first image, the usage count is reduced by 1; the usage count is used to indicate the number of times the first original image is used to synthesize an image; When the usage count is 0, the new node is deleted from the updated dictionary tree, the information of the new node is deleted from the child node distribution information of the parent node, and the linked list node of the first original image is deleted from the updated serialized linked list.

11. An electronic device, characterized in that: include: One or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method as described in any one of claims 1-10.

12. A computer-readable storage medium comprising computer instructions, characterized in that: When the computer instructions are executed on an electronic device, the electronic device is caused to execute the method as claimed in any one of claims 1 to 10.

13. A chip system, which is applied to electronic equipment, characterized in that: The chip system includes one or more processors, and the processor is used to call computer instructions so that the electronic device executes the method as described in any one of claims 1-10.

14. A computer program product comprising instructions, characterized in that When the computer program product is executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 10.

Images