Image processing method and related device based on end-cloud cooperation

Abstract

The application provides an image processing method and related device based on end-cloud cooperation. In the method, an electronic device acquires a first image to be processed. The electronic device sends a first region image of the first image to a server through a first request message. The electronic device acquires a second image to be processed. The electronic device sends a first difference image to the server through a second request message. The first difference image is a difference image of a second region image and the first region image. The server restores the second region image according to the first difference image and the first region image. The server performs image processing on the second region image to acquire a refined image of the second region image. The server sends a second response message to the electronic device. The second response message is used to indicate the refined image of the second region image. The electronic device determines a refined image of the second image based on the refined image of the second region image. In this way, the data flow of image transmission and the processing delay can be effectively reduced, and efficient image processing can be realized through cooperation with the cloud.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronics, and in particular to an image processing method based on end-cloud cooperation and related devices. BACKGROUND

[0002] With the increasing demand of users for photographing, the requirement for the computing power of the central processing unit (CPU) of a terminal device such as a mobile phone in image processing is higher and higher. Due to hardware limitations, the application performance of the CPU in the terminal device such as the mobile phone is limited in improvement.

[0003] With the development of cloud computing technology, transferring the computing power of the end side to the cloud has gradually become a direction of technical evolution. At present, for the scene of refining multiple images, how to realize efficient image processing in cooperation with the cloud while ensuring low consumption of data traffic and low processing time delay remains to be studied. SUMMARY

[0004] The embodiments of the present application provide an image processing method based on end-cloud cooperation, which can effectively reduce the data traffic of image transmission and reduce the processing time delay, realize efficient image processing in cooperation with the cloud, so that the user can view the image refined by the cloud in time on the end side, and effectively improve the user experience.

[0005] In a first aspect, the embodiments of the present application provide a cloud camera processing method, the system comprising an electronic device and a server, the method comprising: the electronic device obtaining a first image to be processed; the electronic device sending a first region image of the first image to the server through a first request message; the first region image comprising an image in part or all regions of the first image; the electronic device obtaining a second image to be processed, the second region image comprising an image in part or all regions of the second image; the electronic device sending a first difference image to the server through a second request message, the first difference image being a difference image of the second region image of the second image and the first region image of the first image; the server restoring the second region image according to the first difference image and the first region image; the server performing image processing on the second region image to obtain a refined image of the second region image; the server sending a second response message to the electronic device, the second response message being used to indicate the refined image of the second region image; and the electronic device determining a refined image of the second image based on the refined image of the second region image.

[0006] The embodiment of the present application is implemented, for the second image to be processed, the terminal side can upload the image and the first image difference image, the first image has been uploaded to the cloud or uploaded to the cloud at the same time as the second image before the second image; the second image is restored by the cloud (i.e. server) according to the difference image, and the second image is refined. By cooperating with the cloud to refine the second image, the requirement and load of the algorithm of the terminal side can be reduced; compared with uploading the second image, uploading the difference image can reduce the data amount of image transmission between the terminal side and the cloud, thereby effectively reducing the data flow of image transmission and reducing the processing delay, cooperating with the cloud to realize efficient image processing, so that the user can view the image refined by the cloud in time on the terminal side, and effectively improve the user experience.

[0007] In an implementation manner, the second response message includes a second difference image, and the second difference image is a difference image between the refined image of the second region map and the second region map; the method further includes: restoring, by the electronic device, the refined image of the second region map according to the second difference image and the second region map. The embodiment of the present application is implemented, the cloud side issues the difference image before and after the refinement of the second region map to the terminal side, and the terminal side can restore the refined image of the second region map based on the difference image. Compared with directly issuing the refined image of the second region map, issuing the difference image can reduce the data amount of image transmission between the terminal side and the cloud, thereby effectively reducing the data flow of image transmission and reducing the processing delay, so that the user can view the image refined by the cloud in time on the terminal side.

[0008] In an implementation manner, the method further includes: performing, by the server, image processing on the first region map to obtain a refined image of the first region map; sending, by the server, a first response message to the electronic device, the first response message being used to indicate the refined image of the first region map; and determining, by the electronic device, a refined image of the first image based on the refined image of the first region map. The embodiment of the present application is implemented, the cloud side issues the difference image before and after the refinement of the first region map to the terminal side, and the terminal side can restore the refined image of the first region map based on the difference image. Compared with directly issuing the refined image of the first region map, issuing the difference image can reduce the data amount of image transmission between the terminal side and the cloud, thereby effectively reducing the data flow of image transmission and reducing the processing delay, so that the user can view the image refined by the cloud in time on the terminal side.

[0009] In an implementation, the electronic device is provided with a continuous shooting function, and the first image and the second image are two images captured by the continuous shooting function; the method further includes: the electronic device detecting a first input operation for starting the continuous shooting function; the electronic device obtaining the first image to be processed, and the electronic device obtaining the second image to be processed, including: in response to the first input operation, the electronic device obtaining the first image and the second image by the continuous shooting function; the first image is a first image by the continuous shooting function, and the second image is a non-first image by the continuous shooting function. By implementing the embodiments of the present application, the similarity of the images captured by the continuous shooting function is generally high; in the scenario of capturing images by the continuous shooting function, the terminal side can upload the first image by the continuous shooting function to the cloud side; for the non-first image by the continuous shooting function, for example, the second image, the terminal side can upload the difference image between the image and the first image to the cloud side. Compared with directly uploading the second image, uploading the difference image can effectively reduce the data traffic of image transmission and reduce the processing delay.

[0010] In an implementation, the electronic device obtaining the first image to be processed includes: in response to a detected first shooting instruction, the camera of the electronic device capturing the first image; the electronic device obtaining the second image to be processed includes: in response to a detected second shooting instruction, the camera of the electronic device capturing the second image; the shooting time interval between the second image and the first image is less than a time threshold, and / or, the similarity between the second image and the first image is greater than a similarity threshold. By implementing the embodiments of the present application, in the scenario of continuously capturing a single image multiple times, when the shooting time interval between the second image and the first image is less than the time threshold, and / or, the similarity between the second image and the first image is greater than the similarity threshold, the terminal side can upload the difference image between the image and the first image to the cloud side. Compared with directly uploading the second image, uploading the difference image can effectively reduce the data traffic of image transmission and reduce the processing delay.

[0011] In an implementation, the method further includes: the electronic device obtaining a third image to be processed; the electronic device sending, to the server, a third difference image through a third request message, the third difference image being a difference image of a third region image of the third image and the first region image; the third region image including images in part or all regions of the third image; the server restoring the third region image according to the third difference image and the first region image; the server performing image processing on the third region image to obtain a retouched image of the third region image; the server sending, to the electronic device, a third response message, the third response message being used to indicate the retouched image of the third region image; and the electronic device determining a retouched image of the third image based on the retouched image of the third region image. In the embodiment, the reference image (i.e., the first image) used for difference of the third image is an image of which a non-difference image has been uploaded to the cloud. In the embodiment, after the first image is uploaded to the cloud, for other images to be processed, the terminal side uploads, to the cloud, a difference image of the image to be processed and the first image. For example, in a scenario of using a continuous shooting function, for a non-first image of the continuous shooting, the terminal side uploads, to the cloud, a difference image of the image and a first image of the continuous shooting.

[0012] In an implementation, the method further includes: the electronic device obtaining a third image to be processed; the electronic device sending, to the server, a third difference image through a third request message, the third difference image being a difference image of a third region image of the third image and the second region image; the third region image including images in part or all regions of the third image; the server restoring the third region image according to the third difference image and the second region image; the server performing image processing on the third region image to obtain a retouched image of the third region image; the server sending, to the electronic device, a third response message, the third response message being used to indicate the retouched image of the third region image; and the electronic device determining a retouched image of the third image based on the retouched image of the third region image. In the embodiment, the reference image (i.e., the second image) used for difference of the third image is an image of which a corresponding difference image has been uploaded to the cloud. In the embodiment, for the third image to be processed, the terminal side uploads, to the cloud, a difference image of the third image and a last image to be processed of the third image. For example, in a scenario of using a continuous shooting function, for a non-first image of the continuous shooting, the terminal side uploads, to the cloud, a difference image of the image and a last image of the continuous shooting.

[0013] In an implementation, for a third image to be processed, the terminal side can upload, to the cloud, a difference image of the third image and any image that has been uploaded to the cloud.

[0014] In an implementation manner, the electronic device sends the first difference image to the server through the second request message, and the method comprises: in a case that a data amount of the first difference image is less than a data amount of the second region map, the electronic device sends the first difference image and an image identifier of the first region map to the server through the first request message; in a case that a data amount of the second difference image is less than a data amount of the refined map of the second region map, the second response message comprises the second difference image and the image identifier of the second region map, and the second difference image is a difference image of the refined map of the second region map and the second region map; in a case that the data amount of the second difference image is less than the data amount of the refined map of the second region map, the second response message comprises the refined map of the second region map. By implementing the embodiment of the application, after the terminal side and the cloud side perform difference on a specific image, if a data amount of a difference image corresponding to the specific image is smaller than that of the specific image, the difference image is sent; otherwise, the specific image is directly sent. In this way, the data amount of the image transmitted between the terminal side and the cloud side can be guaranteed to be as small as possible, and thus the data flow of image transmission and the processing delay can be effectively reduced.

[0015] In an implementation manner, when the refinement mode of the image is the full-image refinement mode, the region map of the image comprises image content in all regions of the image; when the refinement mode of the second image is the full-image refinement mode, the second image and the second region map are the same, and the refined map of the second region map is the refined map of the second image. By implementing the embodiment of the application, the refinement mode of the to-be-processed image is identified, the target subject image (i.e., the region map) is cut out from the to-be-processed image for the to-be-processed image in the subject refinement mode, and only the target subject image / the difference image corresponding to the target subject image is sent to the cloud. In this way, for the subject refinement mode, the data amount of the uploaded image and the returned image between the terminal and the cloud is further reduced, and thus the network flow of the transmitted image and the processing delay are reduced.

[0016] In an implementation manner, when the refinement mode of the image is the subject refinement mode, the region map of the image comprises a target subject in the image, and the size of the region map of the image is smaller than the size of the image; when the refinement mode of the second image is the subject refinement mode, before the electronic device sends the first difference image to the server through the second request message, the method further comprises: the electronic device acquires the position of the second region map in the second image, and cuts out the second region map from the second image; and the electronic device determines the refined map of the second image based on the refined map of the second region map, comprising: based on the position of the second region map in the second image, the electronic device replaces the second region map in the second image with the refined map of the second region map to acquire the refined map of the second image. By implementing the embodiment of the application, when the to-be-processed image adopts the subject refinement mode, the refined map of the target subject image (i.e., the region map) is used to replace the target subject image in the to-be-processed image, and thus the refined map of the to-be-processed image can be acquired.

[0017] In an implementation manner, the first request message comprises first retouching parameters, the first retouching parameters are used to indicate at least one image processing manner adopted by the second image; the server performs image processing on the second region map of the second image to obtain a retouched map of the second region map, comprising: the server performs image processing on the second image according to the at least one image processing manner indicated by the first retouching parameters to obtain the retouched map of the second region map.

[0018] In an implementation manner, the electronic device pre-stores a correspondence relationship between a plurality of image processing manners and retouching modes, the retouching modes comprise a full map retouching mode and a main body retouching mode; the plurality of image processing manners comprise a plurality of shooting modes, the plurality of shooting modes comprise a portrait mode, and the retouching mode corresponding to the portrait mode is the main body retouching mode; the image processing manner adopted by the image is used to determine the retouching mode of the image. According to the image processing manner adopted by the image to be processed, the retouching mode of the image can be determined by implementing the embodiment of the application.

[0019] In an implementation manner, the first request message further comprises an image identifier of the first region map; the second request message further comprises a reference image identifier and a source image identifier of the first difference image, the reference image of the first difference image is the first region map, and the source image of the first difference image is the second region map; the server restores the second region map according to the first difference image and the first region map, comprising: the server restores the second region map according to the first difference image and the first region map indicated by the reference image identifier, and determines that the image identifier of the restored second region map is the source image identifier; the second response message comprises the image identifier of the second region map. According to the embodiment of the application, when the difference image of the second region map is uploaded, the reference image identifier of the difference image is uploaded, so that the cloud determines the reference image used to restore the second region map; and the source image (namely, the second region map) identifier of the difference image is uploaded, so that the cloud determines the image identifier of the restored second region map.

[0020] In an implementation manner, the second request message further comprises an algorithm identifier, the algorithm identifier indicates a first restoration algorithm; the server restores the second region map according to the first difference image and the first region map, comprising: the server restores the second region map according to the first difference image, the first region map and the first restoration algorithm.

[0021] In an implementation manner, the image types include a differential image and a non-differential image; the first request message further includes a first image type, and the first image type is used to indicate that the first region image is a non-differential image; the second request message further includes a second image type, and the second image type is used to indicate that the first differential image is a differential image; and the server restores the second region image according to the first differential image and the first region image, including: when the second image type is determined as a differential image, the server restores the second region image according to the first differential image and the first region image. By implementing the embodiment of the application, when an image is uploaded, an image type is also uploaded, so that the cloud end determines whether the image uploaded by the terminal side needs to be restored differentially.

[0022] In an implementation manner, the electronic device sends a first region image of a first image to a server through a first request message, including: when the electronic device determines that the first image is not a non-first image in a continuous shooting scene, the electronic device sends a first region image in a retouched region of the first image to the server through the first request message; and the electronic device sends a first differential image to the server through a second request message, including: when the electronic device determines that a second image is a non-first image in a continuous shooting scene, the electronic device sends the first differential image to the server through the second request message.

[0023] In an implementation manner, when the second image meets any one of three conditions, the second image is a non-first image in a continuous shooting scene, and the three conditions include: the second image is a non-first photo in a continuous shooting function; a time interval between the second image and the first image is less than a time threshold; and a similarity between the second image and the first image is greater than a similarity threshold.

[0024] In an implementation manner, when any one of at least one image processing mode adopted by the second image corresponds to a subject retouch mode, the retouch mode of the second image is the subject retouch mode.

[0025] In an implementation manner, the electronic device pre-stores a correspondence relationship between a plurality of subject types and a plurality of retouch modes, and a first subject type in the plurality of subject types corresponds to a subject retouch mode; and the determination of the retouch mode of the second image as the subject retouch mode includes: identifying a target subject of the second image as the first subject type, and determining the retouch mode of the second image as a subject retouch mode corresponding to the first subject type.

[0026] In an implementation manner, the plurality of image processing modes include part or all of the following: a plurality of shooting modes, a plurality of filters, a plurality of character special effects, a plurality of beauty options, high-definition repair, and high-definition portrait.

[0027] In an implementation manner, the first image and the second image are preview images captured by a camera of the electronic device; the method further comprises: the electronic device displays the refined image of the first image in a preview frame; and the electronic device displays the refined image of the second image in the preview frame.

[0028] In an implementation manner, the method further comprises: in a case where multiple images are selected, detecting a second input operation, the second input operation being used to trigger image processing on the multiple images; the multiple images include the first image and the second image; and the electronic device obtaining the first image to be processed and the electronic device obtaining the second image to be processed comprises: in response to the second input operation, the electronic device obtains the first image to be processed and the second image to be processed.

[0029] In a second aspect, the embodiments of the present application provide an end-cloud collaborative image processing method, applied to an electronic device, the method comprising: the electronic device obtaining a first image to be processed; the electronic device sending a first region image of the first image to a server through a first request message; the first region image comprising an image in part or all regions of the first image; the electronic device obtaining a second image to be processed, a second region image comprising an image in part or all regions of the second image; the electronic device sending a first difference image to the server through a second request message, the first difference image being a difference image between the second region image of the second image and the first region image of the first image; the second request message being used to instruct to restore the second region image according to the first difference image and obtain a refined image of the second region image; the electronic device receiving a second response message sent by the server, the second response message being used to instruct the refined image of the second region image; and the electronic device determining a refined image of the second image based on the refined image of the second region image.

[0030] By implementing the embodiments of the present application, for the second image to be processed, the end side can upload the image and a difference image of the first image, the first image being uploaded to the cloud side before the second image or being uploaded to the cloud side at the same time as the second image; the second image is restored by the cloud side (i.e. the server) according to the difference image, and the second image is refined. By cooperating with the cloud side to refine the second image, the requirement for the computing power and the load of the end side can be reduced; compared with uploading the second image, uploading the difference image can reduce the data volume of the image transmission between the end side and the cloud side, thereby effectively reducing the data flow of the image transmission and the processing delay, and realizing efficient image processing by cooperating with the cloud side, so that the user can view the image refined by the cloud side in time on the end side, and the user experience is effectively improved.

[0031] In an implementation manner, the second response message comprises a second difference image, the second difference image being a difference image between the refined image of the second region image and the second region image; and the method further comprises: the electronic device restoring the refined image of the second region image according to the second difference image and the second region image.

[0032] In an implementation manner, the first request message is used to instruct the server to obtain a refined map of the first region map; the method further comprises: the electronic device receives a first response message sent by the server, and the first response message is used to instruct the refined map of the first region map; and the electronic device determines the refined map of the first image based on the refined map of the first region map.

[0033] In an implementation manner, the electronic device is provided with a continuous shooting function, and the first image and the second image are two images shot in one continuous shooting function; the method further comprises: the electronic device detects a first input operation for starting the continuous shooting function; and the electronic device obtains the first image to be processed and the second image to be processed, comprising: in response to the first input operation, the electronic device obtains the first image and the second image in the continuous shooting; the first image is the first image in the continuous shooting, and the second image is a non-first image in the continuous shooting.

[0034] In an implementation manner, the electronic device obtains the first image to be processed, comprising: in response to a detected first shooting instruction, the camera of the electronic device shoots the first image; and the electronic device obtains the second image to be processed, comprising: in response to a detected second shooting instruction, the camera of the electronic device shoots the second image; the shooting time interval between the second image and the first image is less than a time threshold, and / or the similarity between the second image and the first image is greater than a similarity threshold.

[0035] In an implementation manner, the method further comprises: the electronic device obtains a third image to be processed; the electronic device sends a third difference image to the server through a third request message, the third difference image being a difference image of a third region map of the third image and a first region map of the first image; the third region map comprises images in part or all regions of the third image; the third request message is used to instruct to restore the third region map according to the third difference image and obtain a refined map of the third region map; the electronic device receives a third response message sent by the server, and the third response message is used to instruct the refined map of the third region map; and the electronic device determines the refined map of the third image based on the refined map of the third region map.

[0036] In an implementation manner, the method further comprises: the electronic device obtains a third image to be processed; the electronic device sends a third difference image to the server through a third request message, the third difference image being a difference image of a third region map of the third image and a second region map of the second image; the third region map comprises images in part or all regions of the third image; the third request message is used to instruct to restore the third region map according to the third difference image and obtain a refined map of the third region map; the electronic device receives a third response message sent by the server, and the third response message is used to instruct the refined map of the third region map; and the electronic device determines the refined map of the third image based on the refined map of the third region map.

[0037] In an implementation manner, the electronic device sends the first difference image to the server through the second request message, including: in a case that a data amount of the first difference image is less than a data amount of the second region map, the electronic device sends the first difference image and an image identifier of the first region map to the server through the first request message; in a case that a data amount of the second difference image is less than a data amount of the refined map of the second region map, the second response message includes the second difference image and the image identifier of the second region map, and the second difference image is a difference image between the refined map of the second region map and the second region map; in a case that a data amount of the second difference image is less than a data amount of the refined map of the second region map, the second response message includes the refined map of the second region map.

[0038] In an implementation manner, in a case that the refined mode of the image is a full-image refined mode, the region map of the image includes image content in all regions of the image; in a case that the refined mode of the second image is the full-image refined mode, the second image and the second region map are the same, and the refined map of the second region map is the refined map of the second image.

[0039] In an implementation manner, in a case that the refined mode of the image is a main-body refined mode, the region map of the image includes a target main body in the image, and a size of the region map of the image is less than a size of the image; in a case that the refined mode of the second image is the main-body refined mode, before the electronic device sends the first difference image to the server through the second request message, the method further includes: the electronic device acquires a position of the second region map in the second image, and crops the second region map from the second image; and the electronic device determines the refined map of the second image based on the refined map of the second region map, including: based on the position of the second region map in the second image, the electronic device replaces the second region map in the second image with the refined map of the second region map, to acquire the refined map of the second image.

[0040] In an implementation manner, the first request message includes a first map refining parameter, the first map refining parameter is used to indicate at least one image processing mode adopted by the second image; and the first request message is used to request the server to perform image processing on the second image according to the at least one image processing mode indicated by the first map refining parameter, to acquire the refined map of the second region map.

[0041] In an implementation manner, the electronic device pre-stores a correspondence relationship between a plurality of image processing modes and refined modes, the refined modes include a full-image refined mode and a main-body refined mode; the plurality of image processing modes include a plurality of shooting modes, the plurality of shooting modes include a portrait mode, and the refined mode corresponding to the portrait mode is the main-body refined mode; and the image processing mode adopted by the image is used to determine the refined mode of the image.

[0042] In an implementation manner, the first request message further comprises an image identifier of the first region map; the second request message further comprises a reference image identifier and a source image identifier of the first difference image, the reference image of the first difference image being the first region map, and the source image of the first difference image being the second region map; and the second response message comprises an image identifier of the second region map.

[0043] In an implementation manner, the second request message further comprises an algorithm identifier, the algorithm identifier being used to indicate that a restoration algorithm used by the server to restore the second region map is the first restoration algorithm.

[0044] In a third aspect, an image processing method based on end-cloud cooperation is provided, and the method is applied to a server. The method comprises the following steps: the server receives a first region map of a first image sent by an electronic device through a first request message; the first region map comprises images in part or all regions of the first image; the server receives a first difference image sent by the electronic device through a second request message, the first difference image being a difference image of a second region map and the first region map; the second region map comprises images in part or all regions of a second image; the server restores the second region map according to the first difference image and the first region map; the server performs image processing on the second region map to obtain a refined map of the second region map; and the server sends a second response message to the electronic device, the second response message being used to indicate the refined map of the second region map, and the refined map of the second region map being used to determine a refined map of the second image.

[0045] By implementing the embodiments of the present application, for a second image to be processed, the end side can upload the image and a difference image of a first image, the first image being uploaded to the cloud side before the second image or being uploaded to the cloud side at the same time as the second image; the cloud side (i.e., the server) restores the second image according to the difference image and refines the second image. By cooperating with the cloud side to refine the second image, the requirement for the computing power and the load of the end side can be reduced; compared with uploading the second image, uploading the difference image can reduce the data volume of the image transmitted between the end side and the cloud side, thereby effectively reducing the data flow of image transmission and reducing the processing time delay, and the end side can view the image refined by the cloud side in time, thereby effectively improving the user experience.

[0046] In an implementation manner, the second response message comprises a second difference image, the second difference image being a difference image of the refined map of the second region map and the second region map; and the second response message is used to instruct the electronic device to restore the refined map of the second region map according to the second difference image and the second region map.

[0047] In an implementation manner, the method further includes: performing image processing on the first region map by the server to obtain a refined map of the first region map; and sending, by the server, a first response message to the electronic device, the first response message being used to indicate the refined map of the first region map; and the refined map of the first region map is used to determine a refined map of the first image.

[0048] In an implementation manner, the method further includes: receiving, by the server, a third difference image sent by the electronic device through a third request message, the third difference image being a difference image of a third region map and the first region map; the third region map including images in part or all regions of a third image; restoring, by the server, the third region map according to the third difference image and the first region map; performing image processing on the third region map by the server to obtain a refined map of the third region map; sending, by the server, a third response message to the electronic device, the third response message being used to indicate the refined map of the third region map; and the refined map of the third region map being used to determine a refined map of the third image.

[0049] In an implementation manner, the method further includes: receiving, by the server, a third difference image sent by the electronic device through a third request message, the third difference image being a difference image of a third region map and the second region map; the third region map including images in part or all regions of a third image; restoring, by the server, the third region map according to the third difference image and the second region map; performing image processing on the third region map by the server to obtain a refined map of the third region map; sending, by the server, a third response message to the electronic device, the third response message being used to indicate the refined map of the third region map; and the refined map of the third region map being used to determine a refined map of the third image.

[0050] In an implementation manner, when a data amount of the second difference image is less than a data amount of the refined map of the second region map, the second response message includes the second difference image and an image identifier of the second region map, and the second difference image is a difference image of the refined map of the second region map and the second region map; and when the data amount of the second difference image is less than the data amount of the refined map of the second region map, the second response message includes the refined map of the second region map.

[0051] In an implementation manner, the first request message includes a first retouching parameter, and the first retouching parameter is used to indicate at least one image processing manner adopted by the second image; and the image processing performed by the server on the second region map of the second image to obtain the refined map of the second region map includes: performing image processing on the second image according to the at least one image processing manner indicated by the first retouching parameter to obtain the refined map of the second region map.

[0052] In an implementation manner, the first request message further comprises an image identifier of the first region map; the second request message further comprises a reference image identifier of the first difference image and a source image identifier of the first difference image, the reference image of the first difference image being the first region map, and the source image of the first difference image being the second region map; and the server restores the second region map according to the first difference image and the first region map, comprising: the server restores the second region map according to the first difference image and the first region map indicated by the reference image identifier, determines that an image identifier of the restored second region map is the source image identifier; and the second response message comprises the image identifier of the second region map.

[0053] In an implementation manner, the second request message further comprises an algorithm identifier, and the algorithm identifier indicates the first restoration algorithm; and the server restores the second region map according to the first difference image and the first region map, comprising: the server restores the second region map according to the first difference image, the first region map and the first restoration algorithm.

[0054] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, the memory being coupled with the processor, the memory being configured to store computer program codes, the computer program codes comprising computer instructions, and the processor being configured to read the computer instructions from the memory, so that the electronic device executes the image processing method of the end-cloud cooperation according to the second aspect.

[0055] In a fifth aspect, an embodiment of the present application provides a server, comprising: a processor and a memory, the memory being coupled with the processor, the memory being configured to store computer program codes, the computer program codes comprising computer instructions, and the processor being configured to read the computer instructions from the memory, so that the electronic device executes the image processing method of the end-cloud cooperation according to the third aspect.

[0056] In a sixth aspect, an embodiment of the present application provides a computer readable storage medium, comprising computer instructions, and the computer instructions are configured to cause a device (for example, the electronic device according to the fourth aspect or the server according to the fifth aspect) to execute the image processing method of the end-cloud cooperation according to any implementation manner of any aspect. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 A system architecture schematic diagram of a communication system is provided for an embodiment of the present application;

[0058] Figures 2A-2C An image processing flowchart in a continuous shooting scene is provided for an embodiment of the present application;

[0059] Figures 3A-3L A related user interface in a continuous shooting scene is provided for an embodiment of the present application.

[0060] Figures 4A-4F A related user interface of a continuous shooting scene provided for an embodiment of the present application;

[0061] Figures 5A-5H A related user interface of a continuous shooting scene provided for an embodiment of the present application;

[0062] Figures 6A-6D A related user interface of a continuous shooting scene provided for an embodiment of the present application;

[0063] Figure 7A A flowchart of an image processing method of end-cloud cooperation provided for an embodiment of the present application;

[0064] Figure 7B A flowchart of a method for judging a non-first image of a continuous shooting scene provided for an embodiment of the present application;

[0065] Figure 8 A device architecture diagram provided for an embodiment of the present application;

[0066] Figure 9A A flowchart of an image processing method of end-cloud cooperation provided for an embodiment of the present application;

[0067] Figure 9B An image schematic diagram involved in an image processing method of end-cloud cooperation provided for an embodiment of the present application;

[0068] Figure 10A A flowchart of an image processing method of end-cloud cooperation provided for an embodiment of the present application;

[0069] Figure 10B An image schematic diagram involved in an image processing method of end-cloud cooperation provided for an embodiment of the present application;

[0070] Figure 11 A structural schematic diagram of a terminal device provided for an embodiment of the present application;

[0071] Figure 12 A structural schematic diagram of a server provided for an embodiment of the present application. DETAILED DESCRIPTION

[0072] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " represents the meaning of or, for example, A / B can represent A or B; the "and / or" in the text only represents a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0073] Hereinafter, the terms "first", "second" are used only for the purpose of description, and cannot be understood as implying or suggesting relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, unless otherwise specified.

[0074] The term "user interface (UI)" in the embodiments of the present application is a medium interface for interaction and information exchange between an application program or an operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is source code written in a specific computer language such as Java, extensible markup language (XML), etc. The interface source code is parsed, rendered and finally presented as content recognizable by the user on the electronic device. The commonly used form of user interface is graphic user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icon, button, menu, tab, text box, dialog box, status bar, navigation bar, Widget, etc. displayed in the display screen of the electronic device.

[0075] The communication system 10 related to the image processing method provided by the embodiments of the present application is introduced as follows.

[0076] Figure 1 The system architecture of the communication system 10 provided by the embodiments of the present application is exemplarily shown. As shown in the figure, Figure 1 The communication system 10 includes a terminal device 100 and a server 200. The terminal device 100 can communicate with the server 200 through a communication network.

[0077] The communication network can include local area networks (LANs) and / or wide area networks (WANs). The communication network can be implemented using any known network communications protocol, which can be various wired or wireless communication protocols, such as Ethernet, universal serial bus (USB), FIREWIRE, global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), Bluetooth, wireless fidelity (Wi-Fi), NFC, voice over Internet protocol (VoIP), a communication protocol supporting a network slicing architecture, or any other suitable communication protocol.

[0078] The terminal device 100 can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device (a smart bracelet), an in-vehicle device, a smart home device (a smart television, a smart screen, a large-screen device, etc.), and / or a smart city device. The embodiments of the present application do not specially limit the specific type of the terminal device 100. The terminal device 100 can also be referred to as an electronic device.

[0079] The server 200 can be a server, or a server cluster composed of multiple servers, or a cloud computing center. The server 200 involved in the embodiments of the present application can also be referred to as a cloud server.

[0080] It should be understood that Figure 1 The system structure diagram of the communication system provided in the embodiments of the present application is only a schematic diagram of the system structure of the communication system 10, and does not constitute a specific limitation on the communication system 10. The communication system 10 can include more or fewer devices than shown in the diagram, for example, it can also include wireless relay devices and wireless backhaul devices (not shown in the diagram), which are not limited herein. Figure 1 The system structure diagram of the communication system provided in the embodiments of the present application is only a schematic diagram of the system structure of the communication system 10, and does not constitute a specific limitation on the communication system 10. The communication system 10 can include more or fewer devices than shown in the diagram, for example, it can also include wireless relay devices and wireless backhaul devices (not shown in the diagram), which are not limited herein.

[0081] In the image processing method of terminal-cloud cooperation provided in the embodiments of the present application, the terminal device 100 can upload an image a to the cloud for fine processing, and the cloud returns the fine image of the image a to the terminal side.

[0082] In some embodiments of the present application, the image a described above can be a photo a taken in real time by the terminal device 100. When the user takes a photo using the camera of the terminal device 100, the terminal device 100 can automatically upload the taken photo a to the cloud for fine processing, and the cloud returns the fine image of the photo a to the terminal side. When the user views the photo a, the terminal device displays the fine image of the photo a returned by the cloud to the user. In the embodiments of the present application, the camera supporting automatic transmission of the photo to the cloud for fine processing when taking a photo at the terminal side can be referred to as a cloud camera or a cloud photo taking mode.

[0083] In the image processing method of terminal-cloud cooperation described above, the time-consuming steps include uploading the photo, image processing (i.e. fine processing) at the cloud, and returning the fine image of the photo. Among them, the time consumption of uploading the photo and returning the fine image is limited by the network speed and the size of the photo. When the image data volume of the photo is large and / or the network speed is generally slow, the time consumption is long. In particular, due to the limitation of the uplink bandwidth of the mobile network (for example, the 5th Generation Mobile Communication Technology (5G)), the time consumption of uploading the photo is longer. For example, the 5G uplink bandwidth of 90% of users is 21Mbps, and it takes 2.28S to transfer a 6MB photo to the cloud, and it takes 0.53s to return the fine processed photo to the terminal side.

[0084] The image processing method described above has the following problems: the data volume of the image uploaded at the terminal side and the image returned at the cloud is high, which will cause high data flow consumption and high image processing delay at the terminal side, and further cause the user to be unable to view the fine image in time. Especially in the real-time cloud photo taking mode, the requirement for image processing delay is higher, and the above problems will cause poor user experience.

[0085] The above problems are more serious for a continuous shooting scene. The continuous shooting scene can be a scene of shooting photos at a high frequency. In the scene, the terminal device 100 continuously shoots multiple photos, for example, the first photo to the nth photo in the continuous shooting, and refines the multiple photos. For example, refer to the end-cloud cooperation image processing flow in the continuous shooting scene shown in FIG. 9. In the continuous shooting scene, the image processing time delays of the multiple photos are superimposed, the total processing time delay is long, and the user experience is poor. Figure 2A

[0086] In another end-cloud cooperation image processing method provided in the embodiments of the present application, the terminal side uploads multiple images to the cloud side. For an image a in a non-continuous shooting scene or a first image a (for example, the first photo in the continuous shooting shown in FIG. 9) in a continuous shooting scene, the terminal side sends the complete image of the image a to the cloud side. The cloud side returns a difference image of the image a before and after refinement to the terminal side. According to the difference image and the image a, the terminal side can restore the refined image of the image a. For any non-first image b (for example, the nth photo in the continuous shooting shown in FIG. 9, n>1) in the continuous shooting scene, the terminal side sends a difference image of the image b and an image c (for example, the ith photo in the continuous shooting shown in FIG. 9, i Figure 2B Figure 2B Figure 2B The cloud side restores the image b according to the difference image and the image c, and returns a difference image of the image b before and after refinement to the terminal side. According to the difference image and the image b, the terminal side can restore the refined image of the image b. The image c is a reference image for the difference of the image b. The image c is an image that has been uploaded to the cloud side before the image b, or an image that is uploaded to the cloud side at the same time as the image b.

[0087] In an implementation manner, the image c can be any image that has been uploaded to the cloud side. For example, the terminal device 100 uploads the complete image of the first photo to the cloud side for refinement. The terminal device 100 uploads the difference image of the second photo and the first photo to the cloud side. The cloud side restores the second photo according to the difference image and the first photo, and then refines the second photo. In this way, the cloud side has obtained the first photo and the second photo. After shooting the third photo, the terminal device 100 can upload the difference image of the third photo and the first photo to the cloud side. The cloud side restores the third photo according to the difference image and the first photo. Alternatively, the terminal device 100 can upload the difference image of the third photo and the second photo to the cloud side. The cloud side restores the third photo according to the difference image and the second photo. Similarly, when the nth photo is uploaded, the cloud side has obtained any photo before the nth photo, for example, the ith photo. The terminal device 100 can upload the difference image of the nth photo and the ith photo to the cloud side. The cloud side restores the nth photo according to the difference image and the ith photo, and then refines the nth photo.

[0088] ​​​In an implementation, the image c can be the image a, i.e., the image uploaded to the cloud is the complete image.

[0089] In an implementation, referring to the image processing flowchart shown in FIG. 2, the image c is the previous image of the image b, i.e., i equals n-1. Specifically, the terminal device 100 uploads the complete image of the first photo to the cloud for refinement; the terminal device 100 uploads the differential image of the second photo and the first photo to the cloud, and the cloud can restore the second photo according to the differential image and the first photo, and then refines the second photo; similarly, the terminal device 100 uploads the differential image of the n (for example, 3) photo and the n-1 (for example, 2) photo to the cloud, and the cloud can restore the n photo according to the differential image and the n-1 photo, and then refines the n photo. Figure 2C

[0090] Implementing the above image processing method, the amount of differential image data uploaded by the terminal side is small, effectively reducing the network traffic and latency of uploading photos; the amount of differential image data returned by the cloud is also small, effectively reducing the network traffic and latency of returning photos; in the burst shooting scenario, the user can view the refined image in time, effectively improving the user experience.

[0091] The uploading image c referred to in the embodiments of the present application refers to uploading the complete image / differential image of the image c to the cloud; similarly, according to the foregoing scheme, in the case of uploading the differential image corresponding to the image c to the cloud, the cloud can restore the complete image c according to the differential image and the reference image corresponding to the image c.

[0092] The "refinement" referred to in the embodiments of the present application refers to image processing of an image by using a specific image processing method / image processing algorithm / model, and the specific image processing method is not limited in the embodiments of the present application, which will be described in detail in subsequent embodiments, and will not be described here.

[0093] The application scenarios referred to in the embodiments of the present application will be introduced below, and the burst shooting scenario and the refinement mode will be exemplarily described in combination with the application scenarios.

[0094] It should be noted that the terminal-cloud cooperative image processing method provided by the embodiments of the present application can be applied to two types of application scenarios. In the first type of application scenario, the image to be processed is an image captured by a camera in real time in a cloud shooting mode, and the terminal device 100 automatically uploads the image to the cloud for refinement. The first type of application scenario includes the following application scenarios one to three. In the second type of application scenario, the image to be processed is a non-real-time captured image stored locally, and the user manually triggers the terminal device 100 to upload the image to the cloud for refinement. The second type of application scenario includes the following application scenarios four and five.

[0095] ​The local stored image can be a photo previously taken by the terminal device 100, or an image acquired by the terminal device 100 in other manners, such as an image sent by another device, an image acquired by screenshot, an image downloaded from a webpage, and the like, which is not limited here.

[0096] Scenario 1: Taking multiple photos by using the continuous shooting function and refining in the cloud shooting mode

[0097] In some embodiments, after the terminal device 100 installs the APP1 (for example, a camera APP), the APP1 can automatically start the cloud shooting mode. In some embodiments, after the terminal device 100 installs the APP1, the user needs to manually start the cloud shooting mode. An exemplary process is shown in FIG. 1. Figures 3A-3B An exemplary process of manually starting the cloud shooting mode is shown in FIG. 2.

[0098] An exemplary process of manually starting the cloud shooting mode is shown in FIG. 2. Figure 3A The main interface 11 is used to display installed applications (APPs). The terminal device 100 installs a camera APP, and the main interface 11 can display an application icon 101 of the camera APP. As shown in FIG. 3, Figure 3A and Figure 3B As shown in FIG. 4, after detecting that the user clicks the application icon 101, the terminal device 100 displays a shooting interface 12 of the camera APP.

[0099] The shooting interface 12 includes a preview frame 102, a shooting mode bar 103, a camera switching control 104, a shooting control 105, an album control 106, and a cloud shooting control 107.

[0100] The preview frame 102 is used to display a preview image collected by the camera of the terminal device 100 in a specific shooting mode. The shooting mode bar 103 is used to select the shooting mode. The camera switching control 104 is used to switch the camera for collecting the image. The shooting control 105 can receive an input operation (for example, a touch operation) of the user. In response to the input operation, the camera APP can shoot and save an image based on the current shooting mode. The album control 106 is used to view the shot photos.

[0101] The cloud shooting control 107 is used to start / stop the cloud shooting mode. The cloud shooting control 107 includes two states of starting and stopping. The user can control the cloud shooting control 107 to switch between the two states, Figure 3B As shown in FIG. 5, the cloud shooting control 107 is in the stopping state; as shown in FIG. 6, Figure 3B and Figure 3C As shown in FIG. 7, after detecting an input operation (for example, a click operation) on the cloud shooting control 107, in response to the input operation, the terminal device 100 starts the cloud shooting mode and switches the cloud shooting control 107 to the starting state.

[0102] In some embodiments, after the cloud photographing mode is enabled, if the terminal device 100 has a network, the photographed photo is uploaded to the cloud for fine-tuning; if the terminal device 100 has no network, the photographed photo is fine-tuned locally.

[0103] The embodiments of the present application do not make specific limitation on the manner of enabling the cloud photographing mode, for example, the cloud photographing mode can also be enabled through application setting of the camera APP.

[0104] As shown in the example of FIG. 1, the photographing mode bar 103 can include a night scene mode, an aperture mode, a professional mode, a photographing mode 103A, a video recording mode, a portrait mode 103B, and the like. Figure 3B As shown in the example of FIG. 1, the photographing mode bar 103 can include a night scene mode, an aperture mode, a professional mode, a photographing mode 103A, a video recording mode, a portrait mode 103B, and the like. Figure 3B As shown in the example of FIG. 1, the photographing mode bar 103 can include a night scene mode, an aperture mode, a professional mode, a photographing mode 103A, a video recording mode, a portrait mode 103B, and the like.

[0105] Optionally, in a specific photographing mode, a filter effect, a beauty effect, a face feature, a makeup effect, a body beautifying effect, and the like can also be superimposed. As an example, in the portrait mode 103B, a makeup effect is added. As an example, Figure 3C and Figure 3D As shown in the example of FIG. 1, in the photographing mode 103A, the photographing interface 12 further includes a filter control 108; after detecting an input operation (for example, a click operation) acting on the filter control 108, the camera APP displays a plurality of filter options (for example, a landscape filter 109), and one filter option is displayed with a filter effect picture and a filter name. Taking the landscape filter 109 as an example, after detecting an input operation of the user selecting the landscape filter 109, the camera APP can add the landscape filter to the image captured by the camera.

[0106] It can be understood that in the embodiments of the present application, the camera APP can fine-tune the image captured by the camera according to the retouching parameters used during photographing. The above-mentioned retouching parameters indicate one or more image processing modes selected by the user from the photographing mode, the filter option, the beauty option, the makeup option, the body beautifying option, and the face feature option. It can be understood that one photographing mode can correspond to one image processing mode, and one filter option can also correspond to one image processing mode.

[0107] In the embodiment, the camera APP of the terminal device 100 is provided with a button or operation supporting the continuous shooting function, and the photos taken by the continuous shooting function usually have high similarity; in the cloud shooting mode, when the continuous shooting function is started, the terminal device 100 can continuously shoot multiple photos and automatically upload the multiple photos to the cloud for fine editing.

[0108] As shown in Figure 3E and Figure 3F , the terminal device 100 detects the input operation (for example, long-pressing the shooting control 105) of the user starting the continuous shooting function; in response to the input operation, the terminal device 100 starts continuously shooting photos and displays the number of continuously shot photos. Figure 3F As shown in Figure 3G , the number of continuously shot photos is 2. Figure 3H and Figure 3I As shown in , when the number of continuously shot photos is 6, the terminal device 100 detects the input operation (for example, stopping long-pressing the shooting control 105) of the user stopping the continuous shooting; in response to the input operation, the terminal device 100 stops continuously shooting photos and saves the fine edited photos of the 6 continuously shot photos in the cloud for collaborative fine editing, and displays the thumbnail of the cover photo of the continuously shot photos on the album control 106. The cover photo can be the first photo of the continuous shooting, or the best image in the continuously shot photos determined by the terminal device 100 using a preset selection model.

[0109] Figure 3I and Figure 3J As shown in Figure 3J , when the user clicks the album control 106, the terminal device 100 displays the image display interface 13. The image display interface 13 includes an image display box 201 and a continuous shooting viewing control 202, and the image display box 201 currently displays the cover photo of the continuously shot photos after the cloud collaborative fine editing, for example, the fine edited photo of the first image of the continuous shooting. Figure 3K As shown in Figure 3K , when the user clicks the continuous shooting viewing control 202, the terminal device 100 displays the user interface 14, and the user interface 14 includes thumbnails 203 of the images of the continuous shooting, for example, the thumbnail 203A of the second image. Figure 3L As shown in , when the user clicks the thumbnail 203A of the second image, the terminal device 100 displays the second image after the cloud collaborative fine editing in the image display box 201.

[0110] In some embodiments, the terminal device 100 determines the first photo a under the one-shot burst function as the first image under the burst scene, and uploads the first photo to the cloud for retouching; the terminal device 100 determines a non-first photo (e.g., photo b) under the one-shot burst function as a non-first image under the burst scene, and uploads the difference image between photo b and photo c to the cloud for retouching photo b. In an implementation, photo c is the last photo of photo b under the one-shot burst function. In an implementation, photo c is any photo (e.g., the first photo a) taken before photo b under the one-shot burst function.

[0111] In some embodiments, the camera APP of the terminal device 100 can provide multiple image processing modes (e.g., multiple shooting modes, multiple makeup options, multiple filter options, multiple beautification options, multiple light effect options, multiple face special effect options, etc.) for the user to select in the shooting interface 12; the cloud stores image processing algorithms / models corresponding to each image processing mode. The camera APP of the terminal device 100 takes a photo, and uploads the photo to the cloud, and also uploads the retouching parameters corresponding to the photo, which are used to indicate one or more image processing modes used when taking the photo and the required image processing parameters; the cloud refines the photo according to the image processing algorithms / models corresponding to the image processing modes indicated by the retouching parameters to meet the user's shooting requirements on the terminal side.

[0112] The embodiments of the present application do not make specific limitations on the image processing modes provided by the camera APP of the terminal device 100, the process of the user selecting the image processing mode, and the image processing algorithms / models corresponding to the image processing mode.

[0113] For example, the user selects the shooting mode as the portrait mode; the retouching parameters corresponding to the taken photo indicate the above-mentioned portrait mode; the cloud can refine the photo according to the image processing algorithms corresponding to the portrait mode.

[0114] For example, the user selects the shooting mode as the photo mode, and selects the landscape filter under the photo mode through the filter control 108; the retouching parameters corresponding to the taken photo indicate the landscape filter under the photo mode; the cloud can refine the photo according to the image processing algorithms corresponding to the landscape filter under the photo mode.

[0115] For example, the user selects the shooting mode as the portrait mode, and adjusts the image processing parameters (e.g., skin smoothing parameters, face slimming parameters, etc.) under the portrait mode; the retouching parameters corresponding to the taken photo indicate the above-mentioned portrait mode and the above-mentioned image processing parameters under the portrait mode; the cloud refines the photo according to the image processing algorithms corresponding to the portrait mode and the above-mentioned image processing parameters.

[0116] Application scenario two: taking a single photo through the non-burst function under the cloud photo mode and refining

[0117] In the cloud photographing mode, the terminal device 100 photographs a single photo through the photographing control 105 of the photographing interface 12, and automatically uploads the photo to the cloud for fine-tuning each time a photo is photographed.

[0118] For example, the user controls the terminal device 100 to photograph photo c and photo b in sequence. The image processing modes (for example, photographing modes or filter options) selected for photographing photo c and photo b can be the same or different. Figure 4A Figure 4B As shown in FIG. 1C, the terminal device 100 detects an input operation (for example, a click operation) of the user on the portrait mode 103B, and switches the photographing mode to the portrait mode. Figure 4B As shown in FIG. 1D, in the portrait mode, when the terminal device 100 detects a photographing operation (for example, a click operation on the photographing control 105) of the user, the camera APP photographs photo c, and obtains and saves the fine-tuned image of photo c in the cloud. Figure 4C As shown in FIG. 1E, when the terminal device 100 detects a photographing operation of the user, the terminal device 100 photographs photo b, and obtains and saves the fine-tuned image of photo b in the cloud. Figure 4D Figure 4E As shown in FIG. 1F, when the terminal device 100 detects a click operation of the user on the album control 106, the terminal device 100 displays the fine-tuned image of photo b on the image display interface 13. Figure 4E Figure 4F As shown in FIG. 1G, when the user swipes photo b to the left, the user can view the fine-tuned image of photo c.

[0119] In some embodiments, when the terminal device 100 photographs photo b, if the terminal device 100 confirms that photo b meets at least one of the following conditions: “the photographing time interval between photo b and photo c is less than a time threshold (for example, 0.5s); and the similarity between photo b and photo c is greater than a similarity threshold (for example, 80%)”, the terminal device 100 determines that photo b is a non-first image in a continuous shooting scene, uploads the difference image between photo b and photo c to the cloud for fine-tuning of photo b; otherwise, the terminal device 100 determines that photo b is an image in a non-continuous shooting scene, and uploads the complete image of photo b to the cloud for fine-tuning.

[0120] In some embodiments, when the terminal device 100 photographs photo b, if the terminal device 100 confirms that photo b meets “the photographing time interval between photo b and photo c is less than a time threshold, and the similarity between photo b and photo c is greater than a similarity threshold”, the terminal device 100 determines that photo b is a non-first image in a continuous shooting scene.

[0121] ​​​In an implementation, the photo c is the most recently uploaded photo to the cloud taken by the terminal device 100 before the photo b. In an implementation, the photo c can be any photo uploaded to the cloud taken by the terminal device 100 within a preset time length (e.g., 1 minute) before the photo b. In an implementation, according to the time of taking the photos from near to far, the terminal device 100 traverses the similarity of the photos uploaded to the cloud and the photo b, and the photo c is the first photo with a similarity greater than the similarity threshold.

[0122] Similarly, the image processing manner and retouching parameters of application scenario two can refer to the related description of application scenario one, which will not be described here.

[0123] In some embodiments, in the aforementioned application scenario one and application scenario two, the terminal device 100 performs local retouching on the preview image captured by the camera, and displays the locally retouched preview image in the preview frame 102; uploads the photo taken in response to the user's shooting operation to the cloud for cloud retouching, and displays the cloud retouched photo in the image display interface 13. In an implementation, due to the limited load capacity of the terminal device 100, for the same image processing manner, the terminal device 100 uses an image processing algorithm / model with lower required load compared to the cloud; using the same image processing manner to retouch the photo, the image quality of the cloud retouched photo is higher than that of the locally retouched photo. For example, referring to Figures 3A-3K , Figure 3D The preview frame 102 of FIG. 7 shows a preview image without adding a landscape filter; Figure 3E The preview frame 102 of FIG. 8 shows a locally retouched preview image after adding a landscape filter, and the image quality of the preview image is higher than that of the unretouched preview image; Figure 3J The image display frame 201 of FIG. 9 shows a cloud retouched photo under a landscape filter taken by the camera, and the image quality of the photo is higher than that of the locally retouched preview image.

[0124] Application scenario three: cloud shooting mode for retouching each preview image captured by the camera

[0125] In some embodiments, the terminal device 100 can also upload the preview image captured by the camera to the cloud for real-time retouching, and display the cloud cooperatively retouched preview image in the preview frame 102, so that the user can preview the cloud retouching effect of the image through the preview frame 102 of the shooting interface 12. It can be understood that the retouching of the preview image requires higher image processing time delay, and the user can timely preview the retouched image through the preview frame 102 only when the image processing time delay is within a lower time delay range.

[0126] In some embodiments, after the terminal device 100 confirms that the first preview image a collected by the camera after starting the shooting interface 12 is the first image in the continuous shooting scene, the terminal device 100 uploads the first preview image to the cloud for fine processing; determines that the preview image b after the first preview image collected by the camera is the non-first image in the continuous shooting scene, and uploads the difference image between the preview image b and the preview image c to the cloud for fine processing of the preview image b.

[0127] In an implementation manner, the preview image c is the last preview image before the preview image b after starting the shooting interface 12. In an implementation manner, the preview image c is any preview image collected before the preview image b after starting the shooting interface 12, for example, the first preview image a.

[0128] Similarly, when uploading the preview image, the corresponding retouching parameter of the preview image is also uploaded, and the retouching parameter indicates the image processing manner used by the camera APP when collecting the preview image. Specifically, the image processing manner and the retouching parameter can refer to the related description of application scenario one, which will not be described here. It should be noted that the image processing manner (for example, the shooting mode, the filter option) used by the camera APP when collecting the preview image a, the preview image b and the preview image c can be the same or different.

[0129] It should be noted that, Figures 3A-4F Only the related user interface of the camera APP on the terminal device 100 is exemplarily shown, and should not constitute a limitation on the embodiments of the present application. The above interface can include more or fewer controls. The APP1 can be a system camera application of the terminal device 100, or other system applications or third-party applications with a shooting function, for example, an instant messaging application with a shooting function, which will not be specifically limited here.

[0130] Application scenario four: one-key fine processing of multiple images in the local

[0131] In some embodiments, the APP2 (for example, the album APP) of the terminal device 100 provides a one-key fine processing function of multiple images. After detecting the input operation of the user for one-key fine processing of multiple images, the terminal device 100 uploads the multiple images to the cloud for fine processing.

[0132] Exemplarily, Figures 5A-5H A related user interface for one-key fine processing of multiple images is shown.

[0133] Figure 5A An image display interface 15 of the album APP is shown, and the image display interface 15 displays multiple images stored in the local. As Figure 5A and Figure 5BAs shown in FIG. 13, after detecting that the user long-presses an image (e.g., image d) in the image display interface 15, the terminal device 100 displays a selection control on each image, and the selection control 301 of the image d is in a selected state, and the selection controls of other images are in unselected states. The selection control can be switched between the selected state and the unselected state by clicking the selection control. After detecting that the user long-presses an image in the image display interface 15, a menu bar 302 is also displayed, and the menu bar 302 displays a share control, a collection control, a delete control, a creation control 302A, and a more control. As shown in FIG. 14, after the user selects multiple images by using the selection control, the terminal device 100 detects that the user clicks the creation control 302A; in response to the click operation, the terminal device 100 displays a creation menu bar 303, and the creation menu bar 303 includes a one-key photo editing control 303A. Figure 5C and Figure 5D As shown in FIG. 13, after detecting that the user long-presses an image (e.g., image d) in the image display interface 15, the terminal device 100 displays a selection control on each image, and the selection control 301 of the image d is in a selected state, and the selection controls of other images are in unselected states. The selection control can be switched between the selected state and the unselected state by clicking the selection control. After detecting that the user long-presses an image in the image display interface 15, a menu bar 302 is also displayed, and the menu bar 302 displays a share control, a collection control, a delete control, a creation control 302A, and a more control. As shown in FIG. 14, after the user selects multiple images by using the selection control, the terminal device 100 detects that the user clicks the creation control 302A; in response to the click operation, the terminal device 100 displays a creation menu bar 303, and the creation menu bar 303 includes a one-key photo editing control 303A. Figure 5D and Figure 5E As shown in FIG. 13, after detecting that the user long-presses an image (e.g., image d) in the image display interface 15, the terminal device 100 displays a selection control on each image, and the selection control 301 of the image d is in a selected state, and the selection controls of other images are in unselected states. The selection control can be switched between the selected state and the unselected state by clicking the selection control. After detecting that the user long-presses an image in the image display interface 15, a menu bar 302 is also displayed, and the menu bar 302 displays a share control, a collection control, a delete control, a creation control 302A, and a more control. As shown in FIG. 14, after the user selects multiple images by using the selection control, the terminal device 100 detects that the user clicks the creation control 302A; in response to the click operation, the terminal device 100 displays a creation menu bar 303, and the creation menu bar 303 includes a one-key photo editing control 303A.

[0134] As shown in FIG. 13, after detecting that the user long-presses an image (e.g., image d) in the image display interface 15, the terminal device 100 displays a selection control on each image, and the selection control 301 of the image d is in a selected state, and the selection controls of other images are in unselected states. The selection control can be switched between the selected state and the unselected state by clicking the selection control. After detecting that the user long-presses an image in the image display interface 15, a menu bar 302 is also displayed, and the menu bar 302 displays a share control, a collection control, a delete control, a creation control 302A, and a more control. As shown in FIG. 14, after the user selects multiple images by using the selection control, the terminal device 100 detects that the user clicks the creation control 302A; in response to the click operation, the terminal device 100 displays a creation menu bar 303, and the creation menu bar 303 includes a one-key photo editing control 303A. Figure 5E and Figure 5F As shown in FIG. 13, after detecting that the user long-presses an image (e.g., image d) in the image display interface 15, the terminal device 100 displays a selection control on each image, and the selection control 301 of the image d is in a selected state, and the selection controls of other images are in unselected states. The selection control can be switched between the selected state and the unselected state by clicking the selection control. After detecting that the user long-presses an image in the image display interface 15, a menu bar 302 is also displayed, and the menu bar 302 displays a share control, a collection control, a delete control, a creation control 302A, and a more control. As shown in FIG. 14, after the user selects multiple images by using the selection control, the terminal device 100 detects that the user clicks the creation control 302A; in response to the click operation, the terminal device 100 displays a creation menu bar 303, and the creation menu bar 303 includes a one-key photo editing control 303A. Figure 5F and Figure 5G As shown in FIG. 13, after detecting that the user long-presses an image (e.g., image d) in the image display interface 15, the terminal device 100 displays a selection control on each image, and the selection control 301 of the image d is in a selected state, and the selection controls of other images are in unselected states. The selection control can be switched between the selected state and the unselected state by clicking the selection control. After detecting that the user long-presses an image in the image display interface 15, a menu bar 302 is also displayed, and the menu bar 302 displays a share control, a collection control, a delete control, a creation control 302A, and a more control. As shown in FIG. 14, after the user selects multiple images by using the selection control, the terminal device 100 detects that the user clicks the creation control 302A; in response to the click operation, the terminal device 100 displays a creation menu bar 303, and the creation menu bar 303 includes a one-key photo editing control 303A. Figure 5G and Figure 5H As shown in FIG. 13, after detecting that the user long-presses an image (e.g., image d) in the image display interface 15, the terminal device 100 displays a selection control on each image, and the selection control 301 of the image d is in a selected state, and the selection controls of other images are in unselected states. The selection control can be switched between the selected state and the unselected state by clicking the selection control. After detecting that the user long-presses an image in the image display interface 15, a menu bar 302 is also displayed, and the menu bar 302 displays a share control, a collection control, a delete control, a creation control 302A, and a more control. As shown in FIG. 14, after the user selects multiple images by using the selection control, the terminal device 100 detects that the user clicks the creation control 302A; in response to the click operation, the terminal device 100 displays a creation menu bar 303, and the creation menu bar 303 includes a one-key photo editing control 303A.

[0135] In some embodiments, the attribute parameter of the images in the album includes file time, the plurality of images are sorted according to the file time, and the terminal device 100 sequentially determines whether each image is the first image of the continuous shooting scene. In some embodiments, the plurality of images are sorted according to the order in which the user selects the images, and each image is sequentially determined whether it is the first image of the continuous shooting scene.

[0136] In some embodiments, the first image of the plurality of images is determined as the first image of the continuous shooting scene, and for the non-first image (for example, image b) of the plurality of images, if the terminal device 100 confirms that the image b satisfies any one of the following conditions: "the similarity between the image b and the image c is greater than the similarity threshold, and the shooting time interval between the image b and the image c is less than the time threshold", it is determined that the image b is a non-first image in the continuous shooting scene, and the difference image between the image b and the image c is uploaded to the cloud for refining the image b; otherwise, it is determined that the image b is an image in the non-continuous shooting scene, and the image b is uploaded to the cloud for refining.

[0137] In some embodiments, if the terminal device 100 confirms that the image b satisfies "the similarity between the image b and the image c is greater than the similarity threshold, and the shooting time interval between the image b and the image c is less than the time threshold", it is determined that the image b is a non-first image in the continuous shooting scene.

[0138] In an implementation manner, the image c is the image before the image b in the plurality of images.

[0139] In an implementation manner, the attribute parameter of the image b stored locally can indicate the source of the image b. For example, the image name of the image b obtained by the screenshot includes "screenshot", and the image name of the photographed photo includes "IMG". It can be understood that if the image b is a photo taken by the camera, the file time of the image b can include the shooting time of the image b.

[0140] Application scenario five: refining a single image locally

[0141] In the embodiments of the application, the APP2 (for example, the album APP) of the terminal device 100 provides a function of refining a single image, detects the input operation of the user on the image b, and uploads the image b to the cloud for refining.

[0142] For example, the terminal device 100 refines the image c and the image b in sequence. Figures 6A-6D A related user interface for refining a single image is shown. The image processing mode of the image c and the image b can be the same or different.

[0143] For example, the terminal device 100 refines the image c and the image b in sequence. Figure 6AAs shown, the image display interface 15 of the album APP displays the locally stored image c and image b. After the user triggers the terminal device 100 to perform fine-tuning on the image c, the image display interface 15 displays the fine-tuned image of the image c. As shown in Figure 6A and Figure 6B As shown, after detecting that the user clicks the image b, the terminal device 100 displays the image display interface 13 of the image b, and the image display interface 13 displays the image b and the editing control 401; as shown in Figure 6B and Figure 6C As shown, after detecting that the user clicks the editing control 401, the terminal device 100 displays the editing bar 402 in the image editing interface 17, and the editing controls in the editing bar 402 are used to perform image processing on the image b, such as beautifying controls, filter controls, high-definition portrait controls 402A, character special effect controls, and the like; the high-definition portrait control 402A is used to repair the character in the image to improve the character definition and visual effect. As shown in Figure 6C and Figure 6D As shown, taking the high-definition portrait control 402A as an example, after detecting that the user clicks the high-definition portrait control 402A, the terminal device 100 uploads the image b to the cloud for fine-tuning, and displays the fine-tuned image of the image b after fine-tuning, and also displays the confirmation control 403 and the cancel control 404; the confirmation control 403 is used to confirm the repair of the image, and after confirming the repair, the above fine-tuned image can be saved to the album; the cancel control 404 is used to cancel the repair of the image.

[0144] In some embodiments, in application scenario five, the specific implementation of judging whether the image b is a non-first image of a continuous shooting scene can refer to the related description of the image b in application scenario four, which will not be described here. The above specific implementation involves the image c. In an implementation manner, the image c is the last image uploaded to the cloud for fine-tuning. In an implementation manner, the image c is any image uploaded to the cloud for fine-tuning. In an implementation manner, the image c is any image uploaded to the cloud for fine-tuning within a preset time period (for example, 10 minutes).

[0145] Similarly, in application scenario four and application scenario five, the album APP provides a plurality of image processing modes (for example, a plurality of filter options, a plurality of beautifying options, a plurality of makeup options, a plurality of character special effect options, a high-definition repair option, and the like) of the local image for the user to select, and the cloud stores the image processing algorithms / models corresponding to each image processing mode. When the album APP of the terminal device 100 uploads the local image to the cloud for fine-tuning, the local image corresponding fine-tuning parameter is also uploaded, and the fine-tuning parameter indicates the above image processing mode. Specifically, the fine-tuning parameter and the image processing mode can refer to the related description of application scenario one.

[0146] It should be noted that, Figures 5A-6DThe user interface of the photo album APP on the terminal device 100 is merely illustrative and should not be construed as limiting the embodiments of the present application. The above interface can include more or fewer controls. The APP 2 can be a system photo album application of the terminal device 100, or other system applications or third-party applications that have the function of refining local images.

[0147] Without being limited to the above five application scenarios, the image processing method provided by the embodiments of the present application can also be applied to other application scenarios that require uploading images to the cloud, which is not limited here.

[0148] In some embodiments, the refinement mode can be divided into full-image refinement and subject refinement. In the full-image refinement mode, the refinement region of the image includes the entire region of the image. In the subject refinement mode, the refinement region of the image includes the display region of the target subject in the image, and the size of the refinement region is smaller than the size of the image. Before uploading the image to the cloud, the terminal side can identify the refinement mode of the image to be processed. In the full-image refinement mode, the complete image of the image to be processed / difference image of the complete image is uploaded to the cloud. In the subject refinement mode, the terminal side identifies the target subject in the image to be processed, cuts out the target subject image from the image to be processed, and only sends the target subject image / difference image of the target subject image to the cloud. The cloud returns the difference image before and after the refinement of the target subject image to the terminal side. Based on the difference image and the target subject image, the terminal side can restore the refined image of the target subject image. Replacing the target subject image in the image to be processed with the refined image of the target subject image can obtain the refined image of the image to be processed. In this way, for the subject refinement mode, the data volume of the uploaded image and the returned image is further reduced, and the required network traffic and latency of the uploaded image and the returned image are further reduced, effectively improving the user experience.

[0149] In combination with the foregoing application scenarios, the refinement mode is illustratively described.

[0150] In some embodiments, the terminal side pre-stores the correspondence between the image processing mode and the refinement mode. The image processing mode of the image to be processed can include one or more of the following: shooting mode, filter, character special effect, high-definition repair, beautifying face, beautifying body, high-definition portrait, etc. The refinement modes corresponding to different image processing modes can be the same or different.

[0151] For example, the shooting mode can include one or more of a portrait mode, an aperture mode, a night mode, a landscape mode, a professional mode, etc., the portrait mode and the aperture mode correspond to the subject refinement mode, the night mode, the landscape mode, and the professional mode correspond to the full-image refinement mode; various filter options correspond to the full-image refinement mode; various beautification options correspond to the full-image refinement mode; various character special effect options correspond to the subject refinement mode; high-definition repair corresponds to the full-image refinement mode; high-definition portrait corresponds to the subject refinement mode. Specifically, the application developer can set the correspondence between the image processing mode and the refinement mode according to actual needs, which is not limited here. If the to-be-processed image has multiple image processing modes, and the refinement modes corresponding to the multiple image processing modes include both the full-image refinement and the subject refinement, the image is processed in the full-image refinement mode. For example, the image processing mode of the to-be-processed image includes a portrait mode and a filter 1, the portrait mode corresponds to the subject refinement mode, and the filter 1 corresponds to the full-image refinement mode, so the refinement mode of the image is determined to be the full-image refinement mode, and the target subject image does not need to be cut.

[0152] For example, as shown in Figure 3C and Figure 3D , the user selects a landscape filter in the shooting mode to take a photo, and the photo taken under the landscape filter is processed in the full-image refinement mode. As shown in Figure 3C and Figures 3J-3L , the preview image before selecting the landscape filter and the refined image of the photo taken under the landscape filter show that the photo is processed in the full-image refinement mode.

[0153] For example, as shown in Figures 4A-4E , the terminal device 100 takes a photo in a portrait mode, and the target subject of the photo is a person. As shown in Figure 4A and Figure 4E , the preview image before selecting the portrait mode and the refined image of the photo taken under the portrait mode show that the photo is processed in the subject refinement mode, and only the person in the photo is refined.

[0154] In some embodiments, a correspondence between a subject type and a refinement mode is pre-stored, and the subject type can include landscape, person, animal, building, city, etc. The terminal side can intelligently identify the subject type of the target subject in the to-be-processed image. For example, the refinement mode corresponding to the landscape, building, and city is the full-image refinement mode; the refinement mode corresponding to the person and the animal is the subject refinement mode.

[0155] In some embodiments, the subject type of the target subject in the to-be-processed image corresponds to the subject refinement mode, and the display area of the target subject in the to-be-processed image accounts for less than a preset proportion (for example, 80%), and the terminal device 100 determines that the refinement mode of the to-be-processed image is the subject refinement mode; otherwise, it is the full-image refinement mode.

[0156] In the embodiments of the present application, the image in the retouching region of the to-be-processed image can be referred to as a region image; in the full-image retouching mode, the region image of the to-be-processed image is the to-be-processed image; in the subject retouching mode, the region image of the to-be-processed image is the target subject image cut from the to-be-processed image.

[0157] In the image processing method of the end-cloud cooperation provided in the embodiments of the present application, for the image a in the non-burst scene or the first image a in the burst scene, the terminal device 100 sends the complete image of the region image of the image a to the cloud; receives the difference image before and after the region image is retouched sent by the server 200, and restores the retouched image of the image a based on the difference image. For the non-first image b in the burst scene, the terminal device 100 sends the difference image of the region image of the image b and the region image of the reference image (i.e., the image c) to the cloud, and the difference image is used to restore the region image of the image b; receives the difference image before and after the region image is retouched sent by the cloud, and restores the retouched image of the image b based on the difference image.

[0158] The image c is the reference image for the difference of the image b, and the image c is an image uploaded to the cloud or an image uploaded to the cloud together with the image b, for example, the image a. In this way, on the basis of effectively reducing the data flow of image transmission and reducing the processing delay, the cloud is cooperated to realize efficient image processing, so that the user can view the image retouched by the cloud in time on the end side, and the user experience is effectively improved.

[0159] Exemplarily, Figure 7A A flowchart of an image processing method of end-cloud cooperation is shown, and the method includes steps S101 to S124. The method includes three stages: an end-side uploading processing stage, a cloud processing stage and an end-side returning processing stage.

[0160] Stage 1: End-side uploading processing stage

[0161] S101, the terminal device 100 acquires the current to-be-processed image.

[0162] The current to-be-processed image can be the image a, the image b or the image c in any of the application scenarios.

[0163] For example, referring to the descriptions of the aforementioned application scenarios 1 and 2, the to-be-processed image can be a photo captured by a camera APP of the terminal device 100; when detecting a user's shooting operation, the camera APP acquires the to-be-processed image captured by the camera. For example, referring to the descriptions of the aforementioned application scenario 3, the to-be-processed image can be a preview image captured by the camera APP of the terminal device 100; after detecting that the shooting interface 12 for displaying the preview image is running, the camera APP acquires the to-be-processed image captured by the camera. For example, referring to the descriptions of the aforementioned application scenarios 4 and 5, the to-be-processed image is a locally stored image of the terminal device 100; when detecting a user operation of triggering the fine-tuning of the local image, the album APP takes the image as the to-be-processed image.

[0164] In S102, the terminal device 100 identifies whether the fine-tuning mode of the to-be-processed image is the subject fine-tuning mode; if not, S103 is performed; if yes, S104 is performed.

[0165] In S103, the terminal device 100 determines that the fine-tuning mode of the to-be-processed image is the full-image fine-tuning mode, and the fine-tuning region of the to-be-processed image is the to-be-processed image itself. Figure 1

[0166] In S104, the terminal device 100 determines that the fine-tuning mode of the to-be-processed image is the subject fine-tuning mode, and the fine-tuning region of the to-be-processed image is a target subject image in the to-be-processed image. Figure 1 Figure 1 In S104, the terminal device 100 determines that the fine-tuning mode of the to-be-processed image is the subject fine-tuning mode, and the fine-tuning region of the to-be-processed image is a target subject image in the to-be-processed image. Figure 1 In some embodiments, the terminal device 100 identifies a target subject in the to-be-processed image, the determination of the fine-tuning region includes displaying the subject, and the fine-tuning region is greater than or equal to the display region of the target subject.

[0167] The implementation manner of the position of the indicated region in the to-be-processed image is not specifically limited.

[0168] Figure 1 In some embodiments, in the subject fine-tuning mode, the region (i.e., the target subject image) is a rectangle. The position of the indicated region in the to-be-processed image is indicated by the upper-left corner coordinate and the lower-right corner coordinate of the region in the to-be-processed image; or the position of the indicated region in the to-be-processed image is indicated by the upper-left corner coordinate of the region in the to-be-processed image, and the height and width of the region.

[0169] In some embodiments, in the subject fine-tuning mode, the region (i.e., the target subject image) is a rectangle. The position of the indicated region in the to-be-processed image is indicated by the upper-left corner coordinate and the lower-right corner coordinate of the region in the to-be-processed image; or the position of the indicated region in the to-be-processed image is indicated by the upper-left corner coordinate of the region in the to-be-processed image, and the height and width of the region. Figure 1 Figure 1 In some embodiments, in the subject fine-tuning mode, the region (i.e., the target subject image) is a rectangle. The position of the indicated region in the to-be-processed image is indicated by the upper-left corner coordinate and the lower-right corner coordinate of the region in the to-be-processed image; or the position of the indicated region in the to-be-processed image is indicated by the upper-left corner coordinate of the region in the to-be-processed image, and the height and width of the region. Figure 1 Figure 1 In some embodiments, in the subject fine-tuning mode, the region (i.e., the target subject image) is a rectangle. The position of the indicated region in the to-be-processed image is indicated by the upper-left corner coordinate and the lower-right corner coordinate of the region in the to-be-processed image; or the position of the indicated region in the to-be-processed image is indicated by the upper-left corner coordinate of the region in the to-be-processed image, and the height and width of the region. Figure 1 Figure 1 In some embodiments, in the subject fine-tuning mode, the region (i.e., the target subject image) is a rectangle. The position of the indicated region in the to-be-processed image is indicated by the upper-left corner coordinate and the lower-right corner coordinate of the region in the to-be-processed image; or the position of the indicated region in the to-be-processed image is indicated by the upper-left corner coordinate of the region in the to-be-processed image, and the height and width of the region.

[0170] In some embodiments, in the subject fine-tuning mode, the region (i.e., the target subject image) is a rectangle. The position of the indicated region in the to-be-processed image is indicated by the upper-left corner coordinate and the lower-right corner coordinate of the region in the to-be-processed image; or the position of the indicated region in the to-be-processed image is indicated by the upper-left corner coordinate of the region in the to-be-processed image, and the height and width of the region. Figure 1 ​​​​​​(i.e. the target subject image) is composed of the edge lines of the target subject in the to-be-processed image, i.e. the region Figure 1 The shape of the region is the shape of the target subject. The position of the region in the to-be-processed image is indicated by the coordinates of each pixel point on the edge lines of the target subject. Figure 1

[0171] In some embodiments, steps S102 to S104 need not be performed, i.e. the fine-tuning mode need not be determined, and the complete image of the to-be-processed image / difference image of the complete image is uploaded to the cloud for any to-be-processed image.

[0172] S105, the terminal device 100 identifies whether the to-be-processed image is a non-first image in a continuous shooting scene; if not, S106 is performed; if yes, S107 is performed.

[0173] In some embodiments, the to-be-processed image is a non-first image in a continuous shooting scene when it satisfies any one of the following conditions 1 to 3; otherwise, it is not a non-first image in a continuous shooting scene.

[0174] Condition 1: the to-be-processed image is a non-first image in a continuous shooting function.

[0175] Condition 2: the time interval between the to-be-processed image and the image c is less than a time threshold (e.g. 0.5s).

[0176] Condition 3: the similarity between the to-be-processed image and the image c is greater than a similarity threshold (e.g. 80%).

[0177] In some embodiments, the to-be-processed image adopts the subject fine-tuning mode, and condition 3 specifically includes that the similarity between the region map (i.e. the target subject image) of the to-be-processed image and the region map of the image c is greater than the similarity threshold.

[0178] Wherein, the image c and the to-be-processed image adopt the same fine-tuning mode; the image c is an image that has been uploaded to the server 200 before the to-be-processed image or is an image that is uploaded to the server 200 at the same time as the to-be-processed image. The image c can be a non-first image in a continuous shooting scene (e.g. image a), and uploading the image c to the server 200 means uploading the difference image corresponding to the region map of the image c to the server 200, and the cloud can restore the complete image of the region map of the image c according to the difference image. The image c can also not be a non-first image in a continuous shooting scene, and uploading the image c to the server 200 means uploading the complete image of the region map of the image c to the server 200.

[0179] For example, the to-be-processed image is the aforementioned image a, and the image a does not satisfy any of the above conditions, and the image a is not a non-first image in a continuous shooting scene. The to-be-processed image is the aforementioned image b, and the image b satisfies at least one of the above conditions, and the image b is a non-first image in a continuous shooting scene. ​

[0180] In an implementation, in condition 2 and / or condition 3, the image c is the last image uploaded to the server 200. In an implementation, in condition 3, the image c can be any image uploaded to the server 200 within a preset time length (e.g., 10 minutes); specifically, the images uploaded to the server 200 within the preset time length are traversed in the order of uploading time from recent to remote, to determine whether there is an image c satisfying condition 3. In an implementation, in condition 3, the image c can be any image uploaded to the server 200 at the same time as the image to be processed; specifically, the images uploaded to the server 200 at the same time are traversed to determine whether there is an image c satisfying condition 3.

[0181] For example, taking the image to be processed as image b, the foregoing application scenarios one to five detail how to determine whether the image b is the non-first image in the continuous shooting scenario, which will not be repeated here.

[0182] In an implementation, the image c is the last image to be processed. For example, in the foregoing application scenarios one and two, the image c can be the last photo taken by the camera; in the foregoing application scenario three, the image c can be the last preview image captured by the camera; in the foregoing application scenarios four and five, the image c is the last image to be processed according to the file time or the time when the image is selected by the user; in the foregoing application scenario five, the image c can be the last image uploaded to the cloud.

[0183] In some embodiments, as shown in FIG. 1, step S105 can specifically include steps D1 to D3. Figure 7B

[0184] D1, determine whether the image to be processed is a non-first photo taken in the continuous shooting function; if yes, the image to be processed is a non-first image in the continuous shooting scenario; if no, i.e., the image to be processed is a first photo taken in the continuous shooting function or a photo taken in the non-continuous shooting function, the next step is performed.

[0185] D2, determine whether the interval of the shooting time of the image to be processed and the last image to be processed is less than a time threshold; if yes, the image to be processed is a non-first image in the continuous shooting scenario; otherwise, the next step is performed.

[0186] D3, determine whether the similarity of the image to be processed and the last image to be processed is greater than a similarity threshold; if yes, the image to be processed is a non-first image in the continuous shooting scenario; otherwise, it is determined that the image to be processed is not a non-first image in the continuous shooting scenario.

[0187] The execution order of steps S102 and S105 is not specifically limited in the embodiments of the present application. In some embodiments, S105 can be executed first to determine the non-first image in the continuous shooting scenario, and then S102 can be executed to identify the retouch mode.​

[0188] S106. If the image to be processed is not the first image in a continuous shooting scenario, the terminal device 100 uploads a request message to the server 200. The request message includes the region... Figure 1 ,area Figure 1 Image ID and region Figure 1 Image type, image type indicates region Figure 1 It is a non-difference image.

[0189] In step S106, in full-image retouching mode, terminal device 100 uploads the image to be processed to server 200; in subject retouching mode, terminal device 100 uploads the target subject image from the image to be processed to server 200.

[0190] S107, Terminal device 100 acquires the area of ​​the image to be processed. Figure 1 The difference image 1 between the reference image and the reference image.

[0191] Differential Image 1 = Source Image (i.e., the region of the image to be processed) Figure 1 - Reference image. In this embodiment, the terminal device 100 selects a reference image from the images already uploaded to the server 200 or uploaded to the cloud together, according to preset rules, as the reference image to be processed. The reference image and the image to be processed are usually quite similar. The retouching mode used for the image to be processed and the reference image is usually the same.

[0192] In full-map refinement mode, the region Figure 1 The image to be processed is defined as follows: If the image to be processed satisfies condition 1 above, it is a photo taken by the burst shooting function. The reference image for the image to be processed can be the previous photo taken in the burst, the first photo taken in the burst, or any photo taken before the image to be processed in the burst. The previous photo taken in the burst will be used as an example for the following explanation. If the image to be processed satisfies conditions 2 and / or 3 above, the reference image for the image to be processed is the aforementioned image c. For example, in application scenario 2, image c is the previous photo taken by the camera.

[0193] In subject retouching mode, the area of ​​the image to be processed Figure 1 The reference image (i.e., the target subject image) is the region map (i.e., the target subject image) of the reference image of the image to be processed.

[0194] S108, Terminal device 100 determines whether the data volume of differential image 1 is less than that of the region. Figure 1 The amount of data; if yes, then execute S109; otherwise, execute S106.

[0195] Step S108 is optional. In some embodiments, step S107 is directly followed by S109 without performing S108.

[0196] S109, the terminal device 100 uploads a request message to the server 200, including the difference image 1, the image ID of the difference image 1, the source image ID, the reference image ID and the image type, the image type indicating that the difference image 1 is a difference image.

[0197] The source image corresponding to the difference image 1 is the region Figure 1 of the source image, the source image ID is the image ID of the source image, and the reference image ID is the image ID of the reference image.

[0198] It can be understood that if the data amount of the difference image 1 is still larger than that of the region Figure 1 , the region Figure 1 is still uploaded to the server 200; and when the data amount of the difference image 1 is smaller than that of the region Figure 1 , the difference image 1 is uploaded to the server 200.

[0199] Phase two: cloud processing phase

[0200] S110, the server 200 receives the request message.

[0201] S111, the server 200 determines whether the image type in the request message is a difference image; if so, S112 is performed; otherwise, S113 is performed.

[0202] S112, when the image type is a difference image, the server 200 restores the region Figure 1 of the to-be-processed image according to the reference image ID and the difference image 1 in the request message; and the server 200 determines that the image ID of the restored region Figure 1 is the source image ID in the request message.

[0203] In the embodiments of the present application, if the image type is a difference image, the difference image carried by the request message needs to be restored to the region image of the to-be-processed image; if the image type is a non-difference image, it indicates that the request message carries the region image of the to-be-processed image, and no difference restoration is needed.

[0204] In some embodiments, when the image type is a difference image, the request message also carries a difference algorithm identifier, and the terminal device 100 obtains the difference image 1 of the two images by using the difference algorithm indicated by the difference algorithm identifier. The server 200 determines the restoration algorithm corresponding to the difference algorithm indicated by the difference algorithm identifier, obtains the reference image indicated by the reference image ID, and then uses the restoration algorithm, the reference image and the difference image 1 to restore the region Figure 1 of the to-be-processed image; and the source image (i.e. the regionFigure 1 ) = diff image 1 + reference image.

[0205] In some embodiments, the terminal device 100 and the server 200 pre-negotiate to use a specific difference algorithm, and the difference algorithm identifier does not need to be carried in the request message.

[0206] The application embodiments do not make specific limitations on the difference algorithm involved, for example, the above difference algorithm can be an absolute value difference method (absDiff), a relative difference method, a threshold difference method, a normalized difference method, or a Gaussian difference method, etc.

[0207] S113, the server 200 records the image ID of the region Figure 1 , and obtains the retouched image of the region Figure 1 . Figure 1

[0208] In some embodiments, the request message carries the retouching parameter of the to-be-processed image, and the server 200 performs retouching on the region Figure 1 of the to-be-processed image according to the retouching parameter to obtain the retouched image of the region Figure 1 . Specifically, the above retouching parameter can refer to the related description of the foregoing application scenarios, which will not be described here.

[0209] S114, the server 200 takes the region Figure 1 of the to-be-processed image as a reference image to obtain the difference image 2 before and after retouching of the region Figure 1 . Wherein, the difference image 2 = source image (i.e., the retouched image of the region Figure 1 ) - reference image (i.e., the retouched image of the region Figure 1 ).

[0210] S115, the server 200 determines whether the data amount of the difference image 2 is less than the data amount of the retouched image of the region Figure 1 ; if yes, S116 is executed; otherwise, S117 is executed.

[0211] Step S115 is optional. In some embodiments, after step S114, S116 is directly executed without executing S115.

[0212] S116, the server 200 returns a response message to the terminal device 100, and the response message includes the difference image 2, the image ID of the difference image 2, the reference image ID, and the image type, wherein the image type indicates that the difference image 2 is a difference image.

[0213] S117, the server 200 returns a response message to the terminal device 100, and the response message includes the retouched image of the region Figure 1 , the image ID of the region Figure 1 , and the image type, wherein the image type indicates that the retouched image is a non-difference image.​

[0214] It can be understood that if the differential image 2 is less than the data amount of the refined image, the differential image 2 is returned to the terminal device 100; if the differential image 2 is greater than or equal to the data amount of the refined image, the refined image is directly returned.

[0215] Stage three: end-side return processing stage

[0216] S118, the terminal device 100 receives the response message.

[0217] S119, the terminal device 100 determines whether the image type in the response message is a differential image; if yes, S120 is executed; otherwise, S121 is executed.

[0218] S120, the refined image of the region of the to-be-processed image is restored according to the reference image ID and the differential image 2. Figure 1 The source image (i.e., the refined image of the region Figure 1 ) = the differential image 2 + the reference image (i.e., the region Figure 1 ).

[0219] In the embodiment of the present application, if the image type is a differential image, the differential image carried in the response message needs to be restored to a refined image; the reference image ID in the response message indicates the image ID of the region Figure 1 of the to-be-processed image. If the image type is a non-differential image, it indicates that the response message carries a refined image, and no differential restoration is needed; the image ID in the response message indicates the image ID of the region Figure 1 of the to-be-processed image.

[0220] Similarly, when the image type is a differential image, the response message can also carry a differential algorithm identifier, and the terminal device 100 performs differential restoration according to the restoration algorithm corresponding to the differential algorithm indicated by the differential algorithm identifier.

[0221] S121, the terminal device 100 determines whether the to-be-processed image corresponding to the region Figure 1 adopts a main body refinement mode; if not, S122 is executed; if yes, S123 is executed.

[0222] According to the related description of steps S119 and S120, the response message indicates the image ID of the region Figure 8 , and the terminal device 100 can query the refinement mode of the to-be-processed image according to the image ID of the region Figure 8 . It can be understood that in the full-image refinement mode, the image ID of the region Figure 1 is the image ID of the to-be-processed image; in the main body refinement mode, the image ID of the region Figure 1 is the image ID of the target main body image of the to-be-processed image.

[0223] In some embodiments, under the subject refinement mode, the terminal device 100 records the correspondence between the image ID of the image to be processed and the image ID of the target subject image; in step S121, the image to be processed and its refinement mode can be queried based on this correspondence. In some embodiments, the terminal device 100 records the image ID of the region image of the image to be processed using the subject refinement mode in file 1. If file 1 includes a region... Figure 1 The image ID, then the region Figure 1 The corresponding retouching mode for the image to be processed is subject retouching. In one implementation, the terminal device 100 records the image ID of the image to be processed using the full-image retouching mode in file 2. If file 2 does not include a region... Figure 1 The image ID, then the region Figure 1 The corresponding retouching mode for the image to be processed is subject retouching.

[0224] S122, Terminal equipment 100 defines the area Figure 1 The retouched image is the retouched version of the image to be processed.

[0225] S123, Terminal equipment 100 according to region Figure 1 Location and region in the image to be processed Figure 1 The retouched image is obtained from the image to be processed.

[0226] In the main body refinement mode, according to the area Figure 1 The terminal device 100 replaces the target subject image in the image to be processed with the refined image of the target subject image (i.e., the position of the target subject image) in the image to be processed.

[0227] It is understandable that in the subject retouching mode, the area of ​​the image to be processed... Figure 1 To obtain the target subject image after segmentation, the region needs to be... Figure 1 The refined image and the image to be processed are stitched together to obtain the refined image of the image to be processed.

[0228] S124. Terminal device 100 displays the refined image of the image to be processed.

[0229] For example, referring to the relevant descriptions of application scenarios one to five above, after the terminal device 100 obtains the refined image from cloud-based collaborative retouching, the terminal device 100 can display the refined image for the user to view.

[0230] In conjunction with the foregoing embodiments, Figure 1 A device architecture diagram provided in an embodiment of this application is shown. For example... Figure 1 As shown, the terminal device 100 may include an acquisition module, a cutting module, a continuous shooting recognition module, an end-side differential module, and a display module. The server 200 may include a cloud-based differential module and a retouching module.

[0231] an end-side acquisition module configured to acquire an image to be processed. The image to be processed can be the image a, the image b or the image c.

[0232] an end-side cutting module configured to, for the image to be processed in the subject refinement mode, cut a target subject graph from the image to be processed, and determine a region Figure 9A of the image to be processed as the target subject graph. For the image to be processed in the full-image refinement mode, the region Figure 9B of the image to be processed is determined as the image to be processed itself.

[0233] an end-side burst recognition module configured to recognize whether the image to be processed is a non-first image in a burst scene. If the image to be processed is not a non-first image in a burst scene (e.g., the image a), the end-side difference module is triggered to acquire a region Figure 7A of the image to be processed. If the image to be processed is a non-first image in a burst scene (e.g., the image b), the end-side difference module is instructed to acquire a difference image of the region Figure 9B .

[0234] an end-side difference module configured to, for the non-first image in a burst scene (e.g., the image b), acquire a difference image 1 of a region graph of the image to be processed and a reference image (e.g., the region graph of the image c), and trigger the terminal device 100 to upload the difference image 1 and a reference image ID to the server 200. The reference image has been uploaded to the cloud or is uploaded to the cloud at the same time as the difference image 1.

[0235] a cloud-side difference module configured to restore a region Figures 3C-3L of the image to be processed according to the received difference image 1 and the reference image ID.

[0236] a cloud-side refinement module configured to refine the region Figure 9B of the image to be processed, and acquire a refined graph of the region Figure 9B .

[0237] the cloud-side difference module is further configured to acquire a difference image 2 before and after refinement of the region Figure 9B , and trigger the terminal device 100 to upload the difference image 2 and the reference image ID (i.e., the image ID of the region Figure 9B of the image to be processed) to the server 200.

[0238] the end-side difference module is further configured to restore the refined graph of the region Figures 3C-3L of the image to be processed based on the difference image 2 and the region Figure 9B of the image to be processed.

[0239] the end-side cutting module is further configured to, for the image to be processed in the subject refinement mode, utilize the refined graph of the region Figure 9Brefined image of the region in the to-be-processed image Figure 9B , thereby obtaining a refined image of the to-be-processed image. It can be understood that, for the to-be-processed image in the full-image refinement mode, the refined image of the region Figure 9B is the refined image of the to-be-processed image.

[0240] the display module on the end side, for displaying the refined image of the to-be-processed image.

[0241] The implementation of the image ID in the embodiments of the present application is not specifically limited. For example, the image IDs of the to-be-processed images are numbered as 1, 2, 3, and so on respectively. The image with the image ID x can be simply referred to as image x.

[0242] The image processing flow is exemplarily illustrated below for the full-image refinement mode and the main-body refinement mode respectively.

[0243] Exemplarily, Figure 9A an image processing flow of a to-be-processed image in a full-image refinement mode is shown, Figure 9B schematic diagrams of the images involved in the processing flow are shown, and the processing flow includes steps A1 to A24. The specific implementation of each step can be referred to the related description of the corresponding step in Figure 10A .

[0244] The to-be-processed image includes image 1 and image 4; image 1 is not the first image in the continuous shooting scene; and image 4 is the first image in the continuous shooting scene. Image 1 and image 4 can be images in any of the foregoing application scenarios.

[0245] A1, the acquisition module of the terminal device 100 acquires the to-be-processed image 1.

[0246] Exemplarily, image 1 is the first photo taken by the camera APP of the terminal device 100 in the full-image refinement mode, Figure 10B a schematic diagram of image 1 is shown.

[0247] In an embodiment, the acquisition module can further acquire the image parameters of image 1, including part or all of the image name, the refinement mode, the file time (for example, the shooting time), the image ID, and the retouching parameters. Optionally, if image 1 is a photo taken by the continuous shooting function, the image parameters further include a continuous shooting identifier, which can indicate that image 1 is a photo taken by the continuous shooting function and whether it is the first photo in the continuous shooting. Optionally, the continuous shooting identifier can be an independent identifier or can be carried in the image name or the image ID. Exemplarily, the image name of the continuous shooting photo includes the continuous shooting identifier and the continuous shooting number, and the continuous shooting identifier is “Burst”. The continuous shooting number is used to indicate which photo in the continuous shooting it is; for example, the image name of the first photo in the continuous shooting includes Burst01, and the image name of the third photo in the continuous shooting includes Burst03.

[0248] A2, the cutting module of the terminal device 100 determines that the target subject image of the image 1 does not need to be cut according to the full-image refining mode adopted by the image 1, and the region image in the refining region of the image 1 is the image 1 itself.

[0249] For example, referring to Figure 7A According to the related description, the image 1 is a photo taken by the terminal device 100 using a landscape filter, and the refining mode corresponding to the landscape filter is full-image refining.

[0250] A3, the continuous shooting recognition module of the terminal device 100 determines that the image 1 is not a non-first image in a continuous shooting scene.

[0251] For example, since the image 1 is the first photo taken by the continuous shooting function, the terminal device 100 determines that the image 1 is not a non-first image in a continuous shooting scene.

[0252] A4, the terminal-side difference module determines that the image 1 does not need to be differentiated, and records the image 1 and the image ID of the image 1.

[0253] The interaction between the modules in the terminal device 100 is not limited in the embodiments of the present application.

[0254] In an embodiment, the acquisition module sends the image 1, the image ID of the image 1, and the refining mode of the image 1 to the cutting module, and calls the cutting module to execute step A2; the acquisition module sends the image 1, the image ID of the image 1, and the continuous shooting identifier of the image 1 to the continuous shooting recognition module, and calls the continuous shooting recognition module to execute step A3; the acquisition module sends the execution results of steps A2 and A3, and the image ID of the image 1 to the terminal-side difference module, and calls the terminal-side difference module to execute step A4. The execution order of steps A2 and A3 is not limited here.

[0255] In an embodiment, the acquisition module sends the image 1 and the image parameters of the image 1 to the cutting module, and calls the cutting module to execute step A2; in the case where the image 1 does not need to be cut, the cutting module sends the image 1 and the image parameters of the image 1 to the continuous shooting recognition module, and calls the continuous shooting recognition module to execute step A3; the continuous shooting recognition module sends the image 1, the image ID of the image 1, and the execution result of step A3 to the terminal-side difference module, and calls the terminal-side difference module to execute step A4.

[0256] A5, the terminal-side difference module triggers the terminal device 100 to send a request message 1 to the server 200, the request message 1 including the image 1, the image ID of the image 1, the image type of the image 1, and the retouching parameters, and the image type indicates that the image 1 is a non-difference image.

[0257] It can be understood that the image 1 adopts the full image retouching mode, and the image 1 is not the first image of the continuous shooting scene; therefore, the terminal device 100 does not need to perform difference on the image 1, and directly uploads the complete image of the image 1 to the server 200.

[0258] Optionally, the request message 1 can further include an image name. For example, the image name, the image type and the image ID carried by the request message 1 are as follows:

[0259] {

[0260] "imageName":"image1", / / image name

[0261] "imagetype":"srcImage", / / image type, srcImage indicates a non-difference image

[0262] "imageId":"1" / / image ID

[0263] }

[0264] A6, according to the non-difference image indicated by the image type in the request message 1, the cloud difference module determines that the image 1 does not need to be restored by difference, and records the image 1 and the image ID of the image 1.

[0265] A7, the retouching module of the server 200 obtains the retouched image (i.e. image 2) of the image 1.

[0266] The retouching module of the server 200 performs retouching on the image 1 according to the image processing algorithm indicated by the retouching parameter in the request message 1, obtains the retouched image 2, and the retouching module of the server 200 sends the image 2 to the cloud difference module. For example, Figure 10B A schematic diagram of a retouched image 2 is shown.

[0267] A8, the cloud difference module obtains the difference image (i.e. image 3) between the retouched image (i.e. image 2) of the image 1 and the image 1, and the image 1 is the reference image of the difference image.

[0268] For example, Figures 4A-4F A schematic diagram of an image 3 is shown, image 3=image 2-image 1.

[0269] A9, the cloud difference module triggers the server 200 to send a response message 1 to the terminal device 100, and the response message 1 includes the image 3, the image ID of the image 3, the reference image ID, the image type and the difference algorithm identifier, and the image type indicates that the image 3 is a difference image.

[0270] In some embodiments, the cloud differential module determines that the differential image (i.e., image 3) is smaller in data volume than the refined image (i.e., image 2), and returns the differential image to the terminal device 100 through a response message 1.

[0271] For example, the image name, image type, differential algorithm identifier, image ID, and reference image ID carried by the response message 1 are as follows:

[0272]

[0273] A10. The terminal-side differential module determines that the differential image indicated by the image type in the response message 1 needs to be restored, and restores the image 2 according to the image 3, the image 1 indicated by the reference image ID, and the restoration algorithm indicated by the differential algorithm identifier.

[0274] For example, as shown in Figure 10B Image 2 (i.e., the refined image of image 1) = image 3 (i.e., the differential image) + image 1.

[0275] A11. If the image 1 indicated by the reference image ID in the response message 1 is determined not to be the target subject image, the cutting module of the terminal device 100 determines that the image 2 is the refined image of the image 1.

[0276] In some embodiments, the cutting module of the terminal device 100 records the image ID of the cut target subject image; if the cutting module determines that the recorded image ID of the target subject image does not include the reference image ID (i.e., the image ID of the image 1), it is determined that the image 1 is not the target subject image, but a complete image to be processed.

[0277] In some embodiments, the terminal device 100 records the image ID of the image to be processed and the refinement mode; if the cutting module queries the image ID of the image to be processed and determines that it includes the reference image ID (i.e., the image ID of the image 1), it is determined that the image 1 is not the target subject image, but a complete image to be processed.

[0278] A12. The display module of the terminal device 100 displays the refined image of the image 1 (i.e., the image 2).

[0279] Step A12 is optional. The present embodiment does not specifically limit the time when the terminal device 100 displays the image 1.

[0280] A13. The acquisition module of the terminal device 100 acquires an image 4 to be processed.

[0281] For example, the image 4 is the second photo taken by the camera APP of the terminal device 100 in the full-image refinement mode, Figure 10B which shows a schematic diagram of the image 4.

[0282] In an embodiment, the acquisition module can further acquire image parameters of the image 4, including some or all of the image name, the retouching mode, the retouching parameters, the image ID, the burst identification, and the retouching parameters. For details, refer to the related description of the image 1.

[0283] A14, the cutting module of the terminal device 100 determines that the image 4 does not need to be cut according to the full-image retouching mode adopted by the image 4, and the region image in the retouching region of the image 4 is the image 4 itself.

[0284] A15, the burst identification module of the terminal device 100 determines that the image 4 is a non-first image in a burst scene.

[0285] For example, referring to Figure 10B , the image 4 is the second photo b taken by the burst function, and the terminal device 100 determines that the image 4 is not a non-first image in a burst scene. For example, the image 4 is a photo taken by a non-burst function, and the terminal device 100 determines that the interval between the photo and the previous photo (i.e., the image 1) is less than a time threshold, or the similarity between the photo and the previous photo is greater than a similarity threshold; therefore, the terminal device 100 determines that the image 4 is a non-first image in a burst scene.

[0286] A16, the terminal side difference module acquires a difference image (i.e., the image 5) of the image 4 and the image 1, and the image 1 is the reference image of the difference image.

[0287] The terminal device 100 selects a reference image for the image 4 to be different, for example, the reference image is the previous photo (i.e., the image 1). Figure 10B A schematic diagram of a difference image (i.e., the image 5) of the image 4 and the image 1 is shown, and the image 5 = image 4 - image 1.

[0288] A17, the terminal side difference module triggers the terminal device 100 to send a request message 2 to the server 200, including the image 5, the image ID of the image 5, the reference image ID, the source image ID, the image type, and the difference algorithm identification, and the image type indicates that the image 5 is a difference image.

[0289] In some embodiments, the request message 1 of step A5 and the request message 2 of step A17 can also be uploaded to the server 200 at the same time, that is, the image 1 (i.e., the first photo) and the image 5 (i.e., the difference image of the second photo and the first photo) can be uploaded to the server 200 at the same time.

[0290] For example, the request message 2 carries the image name, the image type, the difference algorithm identification, the image ID, and the reference image ID and the source image ID (i.e., the image ID of the image 4), as follows:

[0291]

[0292] A18、Cloud difference module determines that differential restoration is needed according to the differential image indicated by the image type in the request message 2, restores the image 4 according to the image 5, the image 1 indicated by the reference image ID, and the restoration algorithm indicated by the differential algorithm identification, records the image 4 and the image ID of the image 4, and the image ID of the image 4 is the source image ID.

[0293] A19、The refining module of the server 200 obtains the refined image of the image 4 (i.e., the image 6).

[0294] The refining module of the server 200 sends the image 6 to the cloud difference module. Exemplarily, Figure 10B A schematic diagram of the image 6 is shown.

[0295] The execution order of steps A7 and A19 is not specifically limited in the embodiments. In some embodiments, steps A7 and A19 can be executed simultaneously, that is, the image 1 and the image 4 are refined simultaneously. In an implementation manner, the refining module is provided with multiple containers for refining, and the terminal device 100 allocates the image 1 and the image 4 to different containers for refining, and the refining task execution modules in the two containers perform refining on the two images respectively and in parallel.

[0296] A20、The cloud difference module obtains the refined image of the image 4 (i.e., the image 6) and the differential image of the image 4 (i.e., the image 7).

[0297] Exemplarily, Figure 10B A schematic diagram of the differential image (i.e., the image 7) of the image 4 before and after refining is shown, and the image 7 = image 6 - image 4.

[0298] A21、The cloud difference module triggers the server 200 to send a response message 2 to the terminal device 100, including the image 7, the image ID of the image 7, the reference image ID, the image type, and the differential algorithm identification, and the image type indicates that the image 7 is a differential image.

[0299] In some embodiments, the cloud difference module determines that the differential image (i.e., the image 7) has a smaller data amount than the refined image (i.e., the image 6), and returns the differential image to the terminal device 100 through the response message 2.

[0300] Exemplarily, the image name, the image type, the differential algorithm identification, the image ID, and the reference image ID carried by the response message 2 are as follows:

[0301]

[0302] A22, the end-side difference module determines that the difference image indicated by the image type indication in the response message 1 needs to be restored, and restores the image 6 according to the image 7, the image 4 indicated by the reference image ID, and the restoration algorithm indicated by the difference algorithm ID.

[0303] As shown in the example, Figures 4A-4F image 6 (i.e., the retouched image of image 4) = image 7 (i.e., the difference image) + image 4.

[0304] A23, in the case where it is determined that the image 4 indicated by the reference image ID in the response message 2 is not the target subject image, the cutting module of the terminal device 100 determines that the image 6 is the retouched image of the image 4.

[0305] For details, please refer to the related description of step A11, which will not be repeated here.

[0306] A24, the display module of the terminal device 100 displays the retouched image (i.e., image 6) of the image 4.

[0307] The method greatly reduces the amount of data transmitted in the network by uploading and returning photo data in a difference manner, and maximizes the reduction of the overall total time consumption.

[0308] As an example, after step A24, the end-side acquisition module also acquires the third photo taken by the camera APP in the full-image retouching mode. The end-side cutting module determines that there is no need to cut the target subject according to the full-image retouching mode adopted by the photo. After the end-side burst recognition module recognizes that the photo is a non-first image in the burst scene, the end-side difference module acquires the difference image corresponding to the third photo, and triggers the device to send the difference image to the server 200. The difference image can be the difference image between the third photo and the second photo, or the difference image between the third photo and the first photo, i.e., the reference image of the difference image can be the second photo or the first photo. Since the cloud-side difference module has acquired the first photo and the second photo, the third photo can be restored according to the above difference image and reference image. The subsequent cloud-side retouching and end-side return processing of the third photo can refer to the related description of the second photo (i.e., image 4) in Figure 10B and Figure 10B , which will not be repeated here.

[0309] As an example, Figure 10B shows an image processing flow of a to-be-processed image in a subject retouching mode, Figure 10A shows a schematic diagram of each image involved in the processing flow, and the processing flow includes steps A1 to A24. For details of the implementation of each step, please refer to the related description of the corresponding step in Figure 10B .

[0310] The image to be processed includes image 11 and image 16; image 11 is not the first image in the continuous shooting scene; image 16 is the non-first image in the continuous shooting scene. Image 11 and image 16 can be images in any of the foregoing application scenarios.

[0311] B1, the acquisition module of the terminal device 100 acquires the image 11 to be processed.

[0312] For example, image 11 is the first photo taken by the camera APP of the terminal device 100 in the subject retouching mode, Figure 7A A schematic diagram of an image 11 is shown.

[0313] In an embodiment, the acquisition module can also acquire image parameters of image 11, including part or all of image name, retouching mode, file time (such as shooting time), retouching parameter, image ID, continuous shooting identifier, and retouching parameter. The image parameters of image 11 can refer to the related description of image 1, which will not be repeated here.

[0314] B2, the cutting module of the terminal device 100 determines to cut the target subject image (i.e. image 12) from image 11 according to the subject retouching mode adopted by image 11; records the position of image 12 in image 11.

[0315] For example, referring to Figure 1 , image 11 is the first photo taken by the terminal device 100 in the portrait mode, and the retouching mode corresponding to the portrait mode is the subject retouching. Figure 9A An example is shown in the schematic diagram of cutting the target subject image (i.e. image 12) from image 11.

[0316] The cutting module of the terminal device 100 records the image parameters of the target subject image (i.e. image 12), including the image ID of the target subject image, the image ID of the corresponding full image (i.e. image 11), and the position in image 11 (i.e. the upper left corner coordinates, width and height). For example, the image parameters 2 of image 12 are as follows:

[0317]

[0318] B3, the continuous shooting recognition module of the terminal device 100 determines that image 11 is not the non-first image in the continuous shooting scene.

[0319] For example, image 11 is the first photo taken by the continuous shooting function, and the terminal device 100 determines that image 11 is not the non-first image in the continuous shooting scene.

[0320] B4, the terminal side difference module determines that the target subject image (i.e. image 12) of image 11 does not need to be differentiated, and records image 12 and the image ID of image 12.

[0321] B5, the end side difference module triggers the terminal device 100 to send a request message 3 to the server 200, including the image 12, the image ID and the image type, and the image type indicates that the image 12 is a non-difference image.

[0322] The embodiments of the present application do not make specific limitation on the interaction between the modules in the terminal device 100. Specifically, reference can be made to the related description of step A5, which will not be repeated here.

[0323] It can be understood that the image 11 adopts the subject retouching mode, and the image 11 is not the first image of the continuous shooting scene; therefore, the terminal device 100 does not need to difference the target subject image of the image 11, and directly uploads the complete image of the target subject image of the image 11 to the server 200.

[0324] Optionally, the request message 3 can also include the image name of the target subject image. Exemplarily, the image name, the image type and the image ID carried by the request message 3 are as follows:

[0325]

[0326] B6, according to the non-difference image indicated by the image type in the request message 3, the cloud difference module determines that there is no need for difference restoration, and records the image 12 and the image ID of the image 12.

[0327] B7, the retouching module of the server 200 obtains the retouched image (i.e. image 13) of the image 12.

[0328] The retouching module of the server 200 retouches the image 12 according to the image processing algorithm indicated by the retouching parameter in the request message 3, obtains the retouched image 13, and the retouching module of the server 200 sends the image 13 to the cloud difference module. Exemplarily, Figure 9B A schematic diagram of a retouched image 13 is shown.

[0329] B8, the cloud difference module obtains the retouched image (i.e. image 13) of the image 12 and the difference image (i.e. image 14) of the image 12, and the image 12 is the reference image of the difference image.

[0330] Exemplarily, Figure 10A A schematic diagram of a difference image (i.e. image 14) before and after retouching of the image 11 is shown, and image 14 = image 13 - image 12.

[0331] B9, the cloud difference module triggers the server 200 to send a response message 3 to the terminal device 100, and the response message 3 includes the image 14, the image ID of the image 14, the reference image ID, the image type and the difference algorithm identifier, and the image type indicates that the image 14 is a difference image.

[0332] In some embodiments, the cloud difference module determines that the data amount of the difference image (i.e., image 14) is smaller than that of the retouched image (i.e., image 13), and returns the difference image to the terminal device 100 through a response message 3.

[0333] For example, the image name, image type, difference algorithm identifier, image ID, and reference image ID carried by the response message 3 are as follows, respectively:

[0334]

[0335]

[0336] B10. The terminal-side difference module determines that the difference image indicated by the image type in the response message 1 needs to be restored, and restores the image 13 according to the image 14, the image 12 indicated by the reference image ID, and the restoration algorithm indicated by the difference algorithm identifier.

[0337] For example, as shown in Figure 10B , the image 13 (i.e., the retouched image of the image 12) = the image 14 (i.e., the difference image) + the image 12.

[0338] B11. The cutting module of the terminal device 100 determines that the image 12 indicated by the reference image ID is the target subject image, replaces the image 12 in the image 11 with the image 13 to obtain an image 15, and determines that the image 15 is the retouched image of the image 11.

[0339] Specifically, the specific implementation of step B11 can refer to the related description of step A11, which will not be described here.

[0340] B12. The display module of the terminal device 100 displays the retouched image (i.e., the image 15) of the image 11.

[0341] Step B12 is optional. The application embodiment does not make specific limits on the time when the terminal device 100 displays the image 1.

[0342] B13. The acquisition module of the terminal device 100 acquires a to-be-processed image 16.

[0343] For example, the image 16 is a second photo taken by the camera APP of the terminal device 100 in the subject retouch mode, Figures 3E-3I which shows a schematic diagram of an image 16.

[0344] In an embodiment, the acquisition module can also acquire the image parameters of the image 16, including part or all of the image name, retouch mode, file time, image ID, continuous shooting identifier, and retouch parameters. Specifically, it can refer to the related description of the image 1.

[0345] B14. The cutting module of the terminal device 100 cuts the target subject image (i.e., image 17) from the image 16 according to the subject refinement mode adopted by the image 16; and records the position of the image 17 in the image 16.

[0346] Exemplarily, Figure 4B Exemplarily, a schematic diagram of cutting the target subject image (i.e., image 17) from the image 16 is shown.

[0347] The cutting module of the terminal device 100 records the image parameters of the target subject image, including the image ID of the target subject image (i.e., image 17), the image ID of the corresponding full image (i.e., image 16), and the position in the image 16 (i.e., the upper left corner coordinates, width, and height). Exemplarily, the image parameters of the image 17 are as follows:

[0348]

[0349] B15. The burst recognition module of the terminal device 100 determines that the image 16 is a non-first image in a burst scene.

[0350] Exemplarily, the image 16 is the second photo taken by the burst function, and the terminal device 100 determines that the image 16 is not a non-first image in a burst scene. Exemplarily, referring to Figure 4C , the image 16 is a photo taken by the non-burst function, and the terminal device 100 determines that the shooting time interval between the photo and the previous photo (i.e., image 11) is less than a time threshold, or the similarity between the target subject image of the photo and the target subject image of the previous photo is greater than a similarity threshold; therefore, it is determined that the image 16 is a non-first image in a burst scene.

[0351] B16. The terminal side difference module obtains a difference image (i.e., image 18) of the image 17 and the image 12.

[0352] Under the subject refinement mode, the terminal device 100 selects a reference image for the target subject image (i.e., image 17) of the image 16 for difference, for example, the reference image is the target subject image (i.e., image 12) of the previous photo. Figure 2B Exemplarily, a schematic diagram of the difference image (i.e., image 18) of the image 17 and the image 12 is shown, image 18 = image 17 - image 12.

[0353] B17. The terminal side difference module triggers the terminal device 100 to send a request message 4 to the server 200, including the image 18, the image ID of the image 18, the reference image ID, the source image ID, the image type, and the difference algorithm identifier, the image type indicating the difference image.

[0354] In some embodiments, when the cloud difference module determines that the data amount of the difference image (i.e., image 18) is smaller than that of the original image (i.e., image 17), the cloud difference module uploads the difference image to the server 200 through the request message 4.

[0355] In some embodiments, the request message 2 of step B5 and the request message 4 of step B17 can also be uploaded to the server 200 at the same time, i.e., image 12 (i.e., the target subject image of the first photo) and image 18 (i.e., the difference image between the target subject image of the second photo and the target subject image of the first photo) can be uploaded to the server 200 at the same time.

[0356] For example, the image name, image type, difference algorithm identifier, image ID, reference image ID, and source image ID carried by the request message 4 are as follows:

[0357]

[0358] B18, the end-side difference module determines that the difference image indicated by the image type in the request message 4 needs to be restored, restores the image 17 according to the image 18, the image 12 indicated by the reference image ID, and the difference algorithm identifier, records the image 17 and the image ID of the image 17, and the image ID of the image 17 is the above-mentioned source image ID.

[0359] B19, the retouching module of the server 200 obtains a retouched image (i.e., image 19) of the image 17.

[0360] The retouching module of the server 200 performs retouching on the image 17 according to the image processing algorithm indicated by the retouching parameter in the request message 4, obtains the retouched image 19, and the retouching module of the server 200 sends the image 19 to the cloud difference module. For example, Figure 2C A schematic diagram of a retouched image 19 is shown.

[0361] The execution order of steps B7 and B19 is not specifically limited in the embodiments of the present application. In some embodiments, steps B7 and B19 can be executed at the same time, i.e., the image 12 and the image 17 are retouched at the same time.

[0362] B20, the cloud difference module obtains a difference image (i.e., image 20) between the retouched image (i.e., image 19) of the image 17 and the image 17, and the image 17 is the reference image of the difference image.

[0363] For example, Figure 9A A schematic diagram of a difference image (i.e., image 20) before and after retouching of the image 17 is shown, and image 20 = image 19 - image 17.

[0364] B21. The terminal differential module triggers the server 200 to send a response message 4 to the terminal device 100, including the image 20, the image ID of the image 20, the reference image ID, the image type, and the differential algorithm identifier. The image type indicates that the image 20 is a differential image.

[0365] In some embodiments, the cloud differential module determines that the differential image (i.e., the image 20) has a smaller data volume than the refined image (i.e., the image 19), and only then returns the differential image to the terminal device 100 through the response message 4.

[0366] For example, the response message 4 carries the image name, the image type, the differential algorithm identifier, the image ID, and the reference image ID, as shown below:

[0367]

[0368] B22. The terminal differential module determines that the differential image indicated by the image type in the response message 1 needs to be restored, and restores the image 19 according to the image 20, the image 17 indicated by the reference image ID, and the restoration algorithm indicated by the differential algorithm identifier.

[0369] B23. The cutting module of the terminal device 100 determines that the image 17 indicated by the reference image ID is the target subject image, replaces the image 17 in the image 16 with the image 19 to obtain the image 21, and determines that the image 21 is the refined image of the image 16.

[0370] B24. The display module of the terminal device 100 displays the refined image (i.e., the image 21) of the image 16.

[0371] Step B24 is optional. The present embodiment does not specifically limit the time when the terminal device 100 displays the image 16.

[0372] For example, after step B24, the end-side acquisition module further acquires the third image captured by the camera APP in the subject refinement mode. The end-side cutting module determines the target subject image of the third image according to the subject refinement mode used by the third image. After the end-side continuous shooting recognition module recognizes that the third image is a non-first image in the continuous shooting scene, the end-side difference module acquires the difference image corresponding to the target subject image of the third image, and triggers the device to send the difference image to the server 200. The difference image can be the difference image between the target subject image of the third image and the target subject image of the second image, or the difference image between the target subject image of the third image and the target subject image of the first image, that is, the reference image of the difference image is the target subject image of the second image or the target subject image of the first image. Since the cloud-side difference module has acquired the target subject image of the first image and the target subject image of the second image, the target subject image of the third image can be restored according to the above difference image and reference image. The cloud-side refinement and end-side return processing of the third image can refer to the related description of the second image (i.e., image 16) in Figure 9B and Figure 10A , and details are not repeated here.

[0373] In combination with the foregoing embodiments, the embodiments of the present application provide an end-cloud collaborative image processing method, which includes steps S201 to S208.

[0374] S201, an electronic device acquires a first image to be processed.

[0375] S202, the electronic device sends a first region image of the first image to a server through a first request message; the first region image includes an image in part or all regions of the first image.

[0376] S203, the electronic device acquires a second image to be processed, and a second region image includes an image in part or all regions of the second image.

[0377] S204, the electronic device sends a first difference image to the server through a second request message, the first difference image being a difference image between a second region image of the second image and the first region image of the first image.

[0378] S205, the server restores the second region image according to the first difference image and the first region image.

[0379] S206, the server performs image processing on the second region image to acquire a refined image of the second region image.

[0380] S207, the server sends a second response message to the electronic device, the second response message being used to indicate the refined image of the second region image.

[0381] S208, the electronic device determines the refined map of the second image based on the refined map of the second region map.

[0382] In the embodiments of the present application, the to-be-processed image involved in the foregoing embodiments can be the first image, the second image, or the third image. The first image can also be the image a involved in the foregoing embodiments, which is not the first non-photography image in the continuous shooting scene; the second image / third image can be the image b involved in the foregoing embodiments, which is the non-photography image in the continuous shooting scene.

[0383] In an implementation manner, the second response message includes a second difference image, and the second difference image is a difference between the refined map of the second region map and the difference image of the second region map; and the method further includes: restoring, by the electronic device, the refined map of the second region map according to the second difference image and the second region map.

[0384] For example, referring to the related descriptions of Figure 10B , when the to-be-processed image is the second image, the second region map can be the region map 4, and the first difference image can be the difference image 5. Figure 11 , the first difference image can be the difference image 1, and the second difference image can be the difference image 2.

[0385] In an implementation manner, the method further includes: performing, by the server, image processing on the first region map to obtain a refined map of the first region map; sending, by the server, a first response message to the electronic device, the first response message being used to indicate the refined map of the first region map; and determining, by the electronic device, a refined map of the first image based on the refined map of the first region map.

[0386] For example, referring to the embodiments described in Figure 12 and Figure 12 , in the full-image refinement mode, the first image and the first region map of the first image can be the image 1, the first request message can be the request message 1, and the first response message can be the response message 1. The second image and the second region map of the second image can be the image 4, and the first difference image can be the image 5; the second request message can be the request message 2, and the second response message can be the response message 2.

[0387] For example, referring to the embodiments described in Figure 12 and Figure 12 , in the subject refinement mode, the first image can be the image 11, and the first region map of the first image can be the image 12; the first request message can be the request message 3, and the first response message can be the response message 3. The second image can be the image 16, the second region map of the second image can be the image 17, and the first difference image can be the image 18; the second request message can be the request message 4, and the second response message can be the response message 4.

[0388] In an implementation, the electronic device is provided with a burst shooting function, and the first image and the second image are two images captured in the burst shooting function; the method further includes: the electronic device detecting a first input operation for starting the burst shooting function; the electronic device obtaining the first image to be processed, and the electronic device obtaining the second image to be processed, including: in response to the first input operation, the electronic device obtaining the first image and the second image in the burst shooting; the first image being a first image in the burst shooting, and the second image being a non-first image in the burst shooting.

[0389] For example, referring to the description of the application scenario one, as shown in ​ The first input operation can include a long-press operation on the shooting control 105.

[0390] In an implementation, the electronic device obtaining the first image to be processed includes: in response to a detected first shooting instruction, the camera of the electronic device capturing the first image; the electronic device obtaining the second image to be processed includes: in response to a detected second shooting instruction, the camera of the electronic device capturing the second image; the shooting time interval between the second image and the first image is less than a time threshold, and / or the similarity between the second image and the first image is greater than a similarity threshold. For example, the time threshold is equal to 0.5s, and the similarity threshold is equal to 80%.

[0391] For example, referring to the description of the application scenario two, as shown in ​ and ​ The first shooting instruction and the second shooting instruction can both include a click operation on the shooting control 105.

[0392] In an implementation, the method further includes: the electronic device obtaining a third image to be processed; the electronic device sending, to the server, a third difference image through a third request message, the third difference image being a difference image between a third region image of the third image and a first region image of the first image; the third region image including an image in part or all of the third image; the server restoring the third region image according to the third difference image and the first region image; the server performing image processing on the third region image to obtain a retouched image of the third region image; the server sending, to the electronic device, a third response message, the third response message being used to indicate the retouched image of the third region image; and the electronic device determining a retouched image of the third image based on the retouched image of the third region image.

[0393] For example, referring to the description of ​ The first image can be a first photo, the second image can be a second photo, and the third image can be an n-th photo (for example, a third photo).

[0394] In an implementation, the method further comprises: the electronic device obtaining a third image to be processed; the electronic device sending a third difference image to the server through a third request message, the third difference image being a difference image of a third region image of the third image and the second region image of the second image; the third region image comprising image content in part or all regions of the third image; the server restoring the third region image according to the third difference image and the second region image; the server performing image processing on the third region image to obtain a refined image of the third region image; the server sending a third response message to the electronic device, the third response message being used to indicate the refined image of the third region image; and the electronic device determining a refined image of the third image based on the refined image of the third region image.

[0395] For example, referring to the related descriptions of ​ , the first image can be a first photo, the second image can be a second photo, and the third image can be a third photo.

[0396] In an implementation, for the third image to be processed, the terminal side can upload the third image and a difference image of any image uploaded to the cloud to the cloud side.

[0397] In an implementation, the electronic device sends the first difference image to the server through the second request message, comprising: in a case where a data amount of the first difference image is less than a data amount of the second region image, the electronic device sends the first difference image and an image ID of the first region image to the server through the first request message; in a case where a data amount of the second difference image is less than a data amount of the refined image of the second region image, the second response message comprises the second difference image and the image ID of the second region image, the second difference image being a difference image of the refined image of the second region image and the second region image; and in a case where the data amount of the second difference image is less than the data amount of the refined image of the second region image, the second response message comprises the refined image of the second region image.

[0398] In an implementation, when the refined mode of the image is a full-image refined mode, the region image of the image comprises image content in all regions of the image; when the refined mode of the second image is the full-image refined mode, the second image and the second region image are the same, and the refined image of the second region image is the refined image of the second image. For example, referring to the embodiments described in ​ and ​ , under the full-image refined mode, the first image and the first region image of the first image are both image 1, and the second image and the second region image of the second image are both image 4.

[0399] In an implementation, when the refining mode of the image is the subject refining mode, the region map of the image includes the target subject in the image, and the size of the region map of the image is smaller than the size of the image; when the refining mode of the second image is the subject refining mode, before the electronic device sends the first difference image to the server through the second request message, the method further includes: the electronic device acquires the position of the second region map in the second image, and crops the second region map from the second image; and the electronic device determines the refined image of the second image based on the refined image of the second region map, including: based on the position of the second region map in the second image, the electronic device replaces the second region map in the second image with the refined image of the second region map, to acquire the refined image of the second image. For example, refer to ​ and ​ In the described embodiment, in the subject refining mode, the first image can be the aforementioned image 11, the first region map of the first image can be the aforementioned image 12, and the refined image of the first region map can be the aforementioned image 13. By replacing the image 12 in the image 11 with the image 13, the refined image of the image 11 can be obtained.

[0400] In an implementation, the first request message includes first retouching parameters, and the first retouching parameters are used to indicate at least one image processing manner adopted by the second image; and the server performs image processing on the second region map of the second image to acquire the refined image of the second region map, including: the server performs image processing on the second image according to the at least one image processing manner indicated by the first retouching parameters, to acquire the refined image of the second region map.

[0401] In an implementation, the electronic device pre-stores a correspondence relationship between a plurality of image processing manners and refining modes, and the refining modes include a full-image refining mode and a subject refining mode; the plurality of image processing manners include a plurality of shooting modes, the plurality of shooting modes include a portrait mode, and the refining mode corresponding to the portrait mode is the subject refining mode; and the image processing manner adopted by the image is used to determine the refining mode of the image.

[0402] In an implementation, the first request message further includes an image ID of the first region map; the second request message further includes a reference image ID and a source image ID of the first difference image, the reference image of the first difference image is the first region map, and the source image of the first difference image is the second region map; and the server restores the second region map according to the first difference image and the first region map, including: the server restores the second region map according to the first difference image and the first region map indicated by the reference image ID, determines that the image ID of the restored second region map is the source image ID; and the second response message includes the image ID of the second region map.

[0403] In an implementation manner, the second request message further comprises an algorithm identifier, the algorithm identifier indicating the first restoration algorithm; and the server restores the second region map according to the first difference image and the first region map, comprising: the server restores the second region map according to the first difference image, the first region map and the first restoration algorithm. The algorithm identifier can be the difference algorithm identifier.

[0404] The structure of the terminal device 100 provided by the embodiment of the application is introduced below. ​ The structure schematic diagram of the terminal device 100 is shown.

[0405] The terminal device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a loudspeaker 170A, a receiver 170B, a microphone 170C, a headset interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0406] It can be understood that the structure shown in the embodiment of the application does not constitute a specific limitation on the terminal device 100. In another embodiment of the application, the terminal device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0407] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices or integrated in one or more processors.

[0408] The controller can generate operation control signals according to the instruction operation code and the timing signal, and complete the control of fetching and executing instructions.

[0409] The processor 110 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can save instructions or data that have just been used or are used repeatedly by the processor 110. If the processor 110 needs to use the instructions or data again, it can directly call from the memory. This avoids repeated access and reduces the waiting time of the processor 110, thereby improving the efficiency of the system.

[0410] In some embodiments, the processor 110 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0411] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 can contain multiple sets of I2C bus. The processor 110 can be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces respectively. For example, the processor 110 can be coupled to the touch sensor 180K through an I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface, and the touch function of the terminal device 100 is realized.

[0412] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can contain multiple sets of I2S bus. The processor 110 can be coupled to the audio module 170 through the I2S bus, and communication between the processor 110 and the audio module 170 is realized. In some embodiments, the audio module 170 can deliver audio signals to the wireless communication module 160 through the I2S interface, and the function of answering a phone through a Bluetooth headset is realized.

[0413] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled through the PCM bus interface. In some embodiments, the audio module 170 can also deliver audio signals to the wireless communication module 160 through the PCM interface, and the function of answering a phone through a Bluetooth headset is realized. Both the I2S interface and the PCM interface can be used for audio communication.

[0414] The UART interface is a universal serial data bus, which is used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is usually used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface, and the Bluetooth function is realized. In some embodiments, the audio module 170 can deliver audio signals to the wireless communication module 160 through the UART interface, and the function of playing music through a Bluetooth headset is realized.

[0415] The MIPI interface can be used to connect the processor 110 and the display screen 194, the camera 193 and other peripheral devices. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), and the like. In some embodiments, the processor 110 and the camera 193 communicate through the CSI interface to implement the photographing function of the terminal device 100. The processor 110 and the display screen 194 communicate through the DSI interface to implement the display function of the terminal device 100.

[0416] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 and the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, and the like.

[0417] The USB interface 130 is an interface that conforms to the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like. The USB interface 130 can be used to connect a charger to charge the terminal device 100, and can also be used to transmit data between the terminal device 100 and a peripheral device. It can also be used to connect a headset to play audio through the headset. The interface can also be used to connect other electronic devices, such as AR devices and the like.

[0418] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a structural limitation of the terminal device 100. In other embodiments of the present application, the terminal device 100 can also use different interface connection methods or combinations of multiple interface connection methods in the above embodiments.

[0419] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input through the wireless charging coil of the terminal device 100. The charging management module 140 can charge the battery 142 while also providing power to the electronic device through the power management module 141.

[0420] The power management module 141 is configured to connect the battery 142 and the charging management module 140 to the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the display 194, the camera 193, the wireless communication module 160, and the like. The power management module 141 can also be configured to monitor parameters such as battery capacity, battery cycle count, battery health status (leakage, impedance), and the like. In some embodiments, the power management module 141 can also be disposed in the processor 110. In some other embodiments, the power management module 141 and the charging management module 140 can be disposed in the same device.

[0421] The wireless communication function of the terminal device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, and the like.

[0422] The antenna 1 and the antenna 2 are configured to transmit and receive electromagnetic wave signals. Each antenna in the terminal device 100 can be configured to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example, the antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in combination with a tuning switch.

[0423] The mobile communication module 150 can provide a solution for wireless communication including 2G / 3G / 4G / 5G and the like applied to the terminal device 100. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter, amplify, and the like the received electromagnetic waves, and transmit the processed signals to the modem processor for demodulation. The mobile communication module 150 can also amplify signals modulated by the modem processor and radiate the signals as electromagnetic waves through the antenna 1. In some embodiments, at least part of the functional modules of the mobile communication module 150 can be disposed in the processor 110. In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 can be disposed in the same device.

[0424] The modem processor can include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal processed by the baseband processor is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a microphone 170B, etc.), or displays an image or a video through a display screen 194. In some embodiments, the modem processor can be a separate device. In other embodiments, the modem processor can be independent of the processor 110 and disposed in the same device as the mobile communication module 150 or other functional modules.

[0425] The wireless communication module 160 can provide a wireless communication solution applied to the terminal device 100, including wireless local area networks (WLAN) (such as a wireless fidelity (Wi-Fi) network), Bluetooth (BT), a global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, and the like. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via an antenna 2, demodulates and filters the electromagnetic wave signal, and transmits the processed signal to the processor 110. The wireless communication module 160 can also receive a signal to be transmitted from the processor 110, modulate it, amplify it, and radiate it as an electromagnetic wave via the antenna 2.

[0426] In some embodiments, the antenna 1 and the mobile communication module 150 of the terminal device 100 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the terminal device 100 can communicate with a network and other devices through wireless communication technology. The wireless communication technology can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS can include global positioning system (GPS), global navigation satellite system (GLONASS), beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS), and / or satellite based augmentation systems (SBAS).

[0427] The terminal device 100 implements a display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information.

[0428] The display screen 194 is configured to display images, videos, and the like. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diode (QLED), or the like. In some embodiments, the terminal device 100 can include one or N display screens 194, where N is a positive integer greater than 1.

[0429] The terminal device 100 can implement the photographing function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor.

[0430] The ISP is configured to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing to convert it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise and brightness of the image. The ISP can also optimize the exposure, color temperature, and other parameters of the shooting scene. In some embodiments, the ISP can be arranged in the camera 193.

[0431] The camera 193 is configured to capture still images or videos. An object generates an optical image through a lens and projects it onto a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV, or the like format. In some embodiments, the terminal device 100 can include one or N cameras 193, where N is a positive integer greater than 1.

[0432] The digital signal processor is used to process digital signals, in addition to being able to process digital image signals, it can also process other digital signals. For example, when the terminal device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0433] The video codec is used to compress or decompress digital video. The terminal device 100 can support one or more video codecs. In this way, the terminal device 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0434] The NPU is a neural-network (NN) calculation processor, which processes input information quickly by drawing on the structure of a biological neural network, such as drawing on the transmission mode between human brain neurons, and can also constantly self-learn. Through the NPU, intelligent cognitive applications of the terminal device 100 can be realized, such as: image recognition, face recognition, voice recognition, text understanding, etc.

[0435] The internal memory 121 can include one or more random access memories (RAMs) and one or more non-volatile memories (NVMs).

[0436] The random access memory can include a static random-access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation of DDR SDRAM is generally referred to as DDR5 SDRAM), etc.; the non-volatile memory can include a magnetic disk storage device, a flash memory.

[0437] According to the operation principle, the flash memory can include NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. According to the potential order of the storage unit, the flash memory can include single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. According to the storage specification, the flash memory can include universal flash storage (UFS), embedded multi media Card (eMMC), etc.

[0438] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of an operating system or other programs running, and can also be used to store data of a user and an application program, etc.

[0439] The non-volatile memory can also store executable programs and data of a user and an application program, etc., and can be loaded in advance into the random access memory for direct reading and writing by the processor 110.

[0440] The external memory interface 120 can be used to connect an external non-volatile memory, so as to expand the storage capacity of the terminal device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120, so as to realize a data storage function. For example, music, video, etc. files are saved in the external non-volatile memory.

[0441] The terminal device 100 can realize an audio function through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, etc. For example, music playing, recording, etc.

[0442] The audio module 170 is used to convert digital audio information into an analog audio signal, and is also used to convert an analog audio input into a digital audio signal. The audio module 170 can also be used to encode and decode an audio signal. In some embodiments, the audio module 170 can be arranged in the processor 110, or part of the function modules of the audio module 170 can be arranged in the processor 110.

[0443] The speaker 170A, also called a “loudspeaker”, is used to convert an audio electrical signal into a sound signal. The terminal device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0444] The receiver 170B, also called "earpiece", is used to convert audio electrical signals into sound signals. When the terminal device 100 answers a phone call or a voice message, the user can listen to the voice by holding the receiver 170B close to the ear.

[0445] The microphone 170C, also called "microphone", "sound collector", is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can make a sound by holding the mouth close to the microphone 170C, and input the sound signal into the microphone 170C. The terminal device 100 can be provided with at least one microphone 170C. In some other embodiments, the terminal device 100 can be provided with two microphones 170C, which can realize the noise reduction function in addition to collecting sound signals. In some other embodiments, the terminal device 100 can be provided with three, four or more microphones 170C, which can realize the functions of collecting sound signals, noise reduction, identifying sound sources, realizing directional recording, etc.

[0446] The earphone interface 170D is used to connect a wired earphone. The earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0447] The pressure sensor 180A is used to sense pressure signals, and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A can be provided on the display screen 194. There are many types of pressure sensors 180A, such as resistance type pressure sensors, inductance type pressure sensors, and capacitance type pressure sensors.

[0448] The gyroscope sensor 180B can be used to determine the motion posture of the terminal device 100. In some embodiments, the angular velocity of the terminal device 100 around three axes (i.e., x, y and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for shooting anti-shake.

[0449] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal device 100 calculates the altitude, assists in positioning and navigation by measuring the air pressure value through the barometric pressure sensor 180C.

[0450] The magnetic sensor 180D includes a Hall sensor. The terminal device 100 can detect the opening and closing of a flip leather cover by using the magnetic sensor 180D.

[0451] The acceleration sensor 180E can detect the magnitude of acceleration of the terminal device 100 in various directions (typically, three axes).

[0452] The distance sensor 180F is used to measure distance. The terminal device 100 can measure distance by infrared or laser.

[0453] The proximity light sensor 180G can include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The light-emitting diode can be an infrared light-emitting diode.

[0454] The ambient light sensor 180L is used to sense ambient light brightness. The terminal device 100 can adaptively adjust the brightness of the display screen 194 according to the sensed ambient light brightness.

[0455] The fingerprint sensor 180H is used to collect fingerprints. The terminal device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locking, fingerprint photographing, fingerprint answering a call, and the like.

[0456] The temperature sensor 180J is used to detect temperature. In some embodiments, the terminal device 100 uses the temperature detected by the temperature sensor 180J to implement a temperature processing strategy.

[0457] The touch sensor 180K, also referred to as a "touch device". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 together form a touch screen, also referred to as a "touch screen". The touch sensor 180K is used to detect a touch operation acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the touch event type. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the terminal device 100, which is different from the position where the display screen 194 is located.

[0458] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a human vocal part vibration bone block.

[0459] The keys 190 include a power-on key, a volume key, and the like. The keys 190 can be mechanical keys. They can also be touch keys. The terminal device 100 can receive key input and generate key signal input related to user settings and function control of the terminal device 100.

[0460] The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompt, and can also be used for touch vibration feedback.

[0461] The indicator 192 can be an indicator light, which can be used to indicate the charging status, the power change, and can also be used to indicate messages, missed calls, notifications, etc.

[0462] The SIM card interface 195 is used to connect the SIM card.

[0463] The structure of a server 200 provided by an embodiment of the present application is described below. ​ The structure of a server 200 provided by an embodiment of the present application is described below.

[0464] As shown in ​ The server 200 can include one or more processors 1001, a memory 1002, a communication interface 1003, a transmitter 1005, a receiver 1006, a coupler 1007, and an antenna 1008. These components can be connected by a bus 1004 or other means, ​ For example, the bus connection. Among them:

[0465] The communication interface 1003 can be used for the server 200 to communicate with other communication devices, such as the terminal device 100. Specifically, the communication interface 1003 can be a 3G communication interface, a 4G communication interface, a 5G communication interface, or a future new radio communication interface, etc. Not limited to wireless communication interfaces, the server 200 can also be configured with a wired communication interface 1003, such as a local access network (local access network, LAN) interface. The transmitter 1005 can be used for transmitting the signal output by the processor 1001. The receiver 1006 can be used for receiving the mobile communication signal received by the antenna 1008.

[0466] In some embodiments of the present application, the transmitter 1005 and the receiver 1006 can be regarded as a wireless modem. In the server 200, the number of transmitters 1005 and receivers 1006 can be one or more. The antenna 1008 can be used to convert electromagnetic energy in the transmission line into electromagnetic waves in free space, or convert electromagnetic waves in free space into electromagnetic energy in the transmission line. The coupler 1007 is used to divide the mobile communication signal received by the antenna 1008 into multiple paths and distribute it to multiple receivers 1006.

[0467] The memory 1002 is coupled to the processor 1001 and is used to store various software programs and / or groups of instructions. Specifically, the memory 1002 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 1002 can store a network communication program, which can be used to communicate with one or more additional devices, one or more terminal devices, and one or more network devices.

[0468] In some embodiments of the present application, the memory 1002 can be configured to store the implementation program of the application distribution method provided by one or more embodiments of the present application on the server 200 side. For the implementation of the application distribution method provided by one or more embodiments of the present application, please refer to the above embodiments.

[0469] The processor 1001 can be configured to read and execute computer readable instructions. Specifically, the processor 1001 can be configured to call the program stored in the memory 1002, for example, the implementation program of the application distribution method provided by one or more embodiments of the present application on the server 200 side, and execute the instructions contained in the program.

[0470] It should be noted that, ​ The server 200 shown is only one implementation of the embodiments of the present application, and in actual applications, the server 200 can also include more or fewer components, which are not limited here.

[0471] For more details about the functions and working principles of the server 200, please refer to the related contents in the above embodiments, which will not be repeated here.

[0472] The embodiments of the present application can be combined in any way to achieve different technical effects.

[0473] In the above embodiments, all or part of them can be realized by software, hardware, firmware or any combination thereof. When realized by software, all or part of them can be realized in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (for example, floppy disk, hard disk, magnetic tape), optical media (for example, DVD), or semiconductor media (for example, solid state disk (SSD)) and the like.

[0474] Those skilled in the art can understand that all or part of the processes in the above-mentioned method embodiments can be implemented by a computer program instructing relevant hardware to complete, and the program can be stored in a computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned method embodiments. The aforementioned storage medium includes ROM, random access memory (RAM), magnetic disk or optical disk, and various storage media that can store program codes.

[0475] In summary, the above only describes the embodiments of the technical scheme of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of the present application shall be included in the protection scope of the present application.

Claims

1. An image processing method of end-cloud collaboration, characterized in that, The method is applied to an image processing system, the system comprising an electronic device and a server, and the method comprises: The electronic device acquires a first image to be processed; The electronic device sends a first region map of the first image to the server through a first request message; the first region map comprises images in part or all regions of the first image; The electronic device acquires a second image to be processed, and a second region map comprises images in part or all regions of the second image; The electronic device sends a first difference image to the server through a second request message, the first difference image being a difference image between the second region map of the second image and the first region map of the first image; The server restores the second region map according to the first difference image and the first region map; The server performs image processing on the second region map to acquire a refined map of the second region map; The server sends a second response message to the electronic device, the second response message being used for indicating the refined map of the second region map; The electronic device determines a refined map of the second image based on the refined map of the second region map.

2. The method of claim 1, wherein, The second response message comprises a second difference image, the second difference image being a difference image between the refined map of the second region map and the second region map; the method further comprises: The electronic device restores the refined map of the second region map according to the second difference image and the second region map.

3. The method of claim 1, wherein, The method further comprises: The server performs image processing on the first region map to acquire a refined map of the first region map; The server sends a first response message to the electronic device, the first response message being used for indicating the refined map of the first region map; The electronic device determines a refined map of the first image based on the refined map of the first region map.

4. The method according to any one of claims 1 to 3, characterized in that, The electronic device is provided with a continuous shooting function, the first image and the second image being two images shot under the continuous shooting function; the method further comprises: The electronic device detects a first input operation for starting the continuous shooting function; The electronic device acquires a first image to be processed, and the electronic device acquires a second image to be processed, comprising: In response to the first input operation, the electronic device acquires the first image and the second image of the continuous shooting; the first image being a first image of the continuous shooting, and the second image being a non-first image of the continuous shooting.

5. The method of any one of claims 1 to 3, wherein: The electronic device acquires a first image to be processed, comprising: In response to a detected first shooting instruction, a camera of the electronic device shoots a first image; The electronic device acquires a second image to be processed, comprising: In response to a detected second shooting instruction, a camera of the electronic device shoots a second image; The time interval between the shooting of the second image and the first image is less than a time threshold, and / or the similarity between the second image and the first image is greater than a similarity threshold.

6. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: The electronic device acquires a third image to be processed; The electronic device sends a third difference image to the server through a third request message, the third difference image being a difference image between a third region map of the third image and the first region map of the first image; the third region map including images within part or all regions of the third image; The server restores the third region map according to the third difference image and the first region map; The server performs image processing on the third region map to obtain a refined map of the third region map; The server sends a third response message to the electronic device, the third response message being used to indicate the refined map of the third region map; The electronic device determines a refined map of the third image based on the refined map of the third region map.

7. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: The electronic device obtains a third image to be processed; The electronic device sends a third difference image to the server through a third request message, the third difference image being a difference image between a third region map of the third image and the second region map of the second image; the third region map including images within part or all regions of the third image; The server restores the third region map according to the third difference image and the second region map; The server performs image processing on the third region map to obtain a refined map of the third region map; The server sends a third response message to the electronic device, the third response message being used to indicate the refined map of the third region map; The electronic device determines a refined map of the third image based on the refined map of the third region map.

8. The method according to any one of claims 1 to 7, characterized in that, The electronic device sends a first difference image to the server through a second request message, comprising: In a case where a data amount of the first difference image is less than a data amount of the second region map, the electronic device sends the first difference image and an image identifier of the first region map to the server through the second request message; In a case where a data amount of a second difference image is less than a data amount of a refined map of the second region map, the second response message includes the second difference image and an image identifier of the second region map, the second difference image being a difference image between the refined map of the second region map and the second region map; In a case where the data amount of the second difference image is greater than or equal to the data amount of the refined map of the second region map, the second response message includes the refined map of the second region map.

9. The method of any one of claims 1 to 8, wherein In a case where the refinement mode of the image is a full-image refinement mode, the region map of the image includes image contents within all regions of the image; In a case where the refinement mode of the second image is the full-image refinement mode, the second image and the second region map are the same, and the refined map of the second region map is the refined map of the second image.

10. The method of any one of claims 1 to 9, wherein In a case where the refinement mode of the image is a main-body refinement mode, the region map of the image includes a target main body in the image, and a size of the region map of the image is less than a size of the image. When the refining mode of the second image is the main body refining mode, before the electronic device sends the first differential image to the server through the second request message, the method further includes: The electronic device acquires the position of the second region map in the second image, and crops the second region map from the second image; The electronic device determines the refined map of the second image based on the refined map of the second region map, including: Based on the position of the second region map in the second image, the electronic device replaces the second region map in the second image with the refined map of the second region map to acquire the refined map of the second image.

11. The method according to any one of claims 1 to 10, characterized in that, The first request message includes first retouching parameters, and the first retouching parameters are used to indicate at least one image processing mode adopted by the second image; the server performs image processing on the second region map of the second image to acquire the refined map of the second region map, including: The server performs image processing on the second image according to the at least one image processing mode indicated by the first retouching parameters to acquire the refined map of the second region map.

12. The method according to any one of claims 9 to 11, characterized in that, The electronic device pre-stores a correspondence relationship between a plurality of image processing modes and refining modes, and the refining modes include a full-image refining mode and a main body refining mode; The plurality of image processing modes include a plurality of shooting modes, and the plurality of shooting modes include a portrait mode, and the refining mode corresponding to the portrait mode is the main body refining mode; An image processing mode adopted by an image is used to determine a refining mode of the image.

13. The method according to any one of claims 1 to 12, characterized in that, The first request message further includes an image identifier of the first region map; the second request message further includes a reference image identifier and a source image identifier of the first differential image, the reference image of the first differential image is the first region map, and the source image of the first differential image is the second region map; The server restores the second region map according to the first differential image and the first region map, including: The server restores the second region map according to the first differential image and the first region map indicated by the reference image identifier, determines that the image identifier of the restored second region map is the source image identifier, and the second response message includes the image identifier of the second region map.

14. The method according to any one of claims 1 to 13, characterized in that, The second request message further includes an algorithm identifier, and the algorithm identifier indicates a first restoration algorithm; The server restores the second region map according to the first differential image and the first region map, including: The server restores the second region map according to the first differential image, the first region map and the first restoration algorithm.

15. An end-to-cloud collaborative image processing method, comprising: The method includes: An electronic device acquires a first image to be processed; The electronic device sends a first region map of the first image to a server through a first request message; the first region map includes images in part or all regions of the first image; The electronic device acquires a second image to be processed, and a second region map includes images in part or all regions of the second image; The electronic device acquires a second image to be processed, and a second region map includes images in part or all regions of the second image; The electronic device sends a first difference image to the server through a second request message, the first difference image being a difference image of a second region map of the second image and the first region map of the first image; the second request message is used to instruct to restore the second region map according to the first difference image, and to obtain a refined map of the second region map; The electronic device receives a second response message sent by the server, and the second response message is used to instruct the refined map of the second region map; The electronic device determines the refined map of the first image based on the refined map of the second region map.

16. The method of claim 15, wherein, The second response message includes a second difference image, and the second difference image is a difference image of the refined map of the second region map and the second region map; the method further comprises: The electronic device restores the refined map of the second region map according to the second difference image and the second region map.

17. The method of claim 15, wherein, The first request message is used to instruct the server to obtain the refined map of the first region map; the method further comprises: The electronic device receives a first response message sent by the server, and the first response message is used to instruct the refined map of the first region map; The electronic device determines the refined map of the first image based on the refined map of the first region map.

18. The method according to any one of claims 15 to 17, characterized in that, The electronic device is provided with a continuous shooting function, and the first image and the second image are two images shot under the continuous shooting function; the method further comprises: The electronic device detects a first input operation for starting the continuous shooting function; The electronic device obtains a first image to be processed, and the electronic device obtains a second image to be processed, comprising: In response to the first input operation, the electronic device obtains the first image and the second image of the continuous shooting; the first image is the first image of the continuous shooting, and the second image is the non-first image of the continuous shooting.

19. The method of any one of claims 15 to 17, wherein The electronic device obtains a first image to be processed, comprising: In response to the detected first shooting instruction, the camera of the electronic device shoots the first image; The electronic device obtains a second image to be processed, comprising: In response to the detected second shooting instruction, the camera of the electronic device shoots the second image; The time interval between the shooting of the second image and the first image is less than a time threshold, and / or the similarity between the second image and the first image is greater than a similarity threshold.

20. The method according to any one of claims 15 to 19, characterized in that, The method further comprises: The electronic device obtains a third image to be processed; The electronic device sends a third difference image to the server through a third request message, the third difference image being a difference image of a third region map of the third image and the first region map of the first image; the third region map includes images in part or all regions of the third image; the third request message is used to instruct to restore the third region map according to the third difference image, and to obtain a refined map of the third region map; The electronic device receives a third response message sent by the server, and the third response message is used to instruct the refined map of the third region map; The electronic device determines a refined map of the third image based on the refined map of the third region map.

21. The method according to any one of claims 15 to 19, characterized in that, The method further comprises: The electronic device acquires a third image to be processed; The electronic device sends a third difference image to the server through a third request message, the third difference image being a difference image between a third region map of the third image and the second region map of the second image; the third region map comprising image in part or all regions of the third image; the third request message being used to instruct to restore the third region map according to the third difference image and acquire a refined map of the third region map; The electronic device receives a third response message sent by the server, the third response message being used to instruct a refined map of the third region map; The electronic device determines a refined map of the third image based on the refined map of the third region map.

22. The method according to any one of claims 15 to 21, characterized in that, The electronic device sends a first difference image to the server through a second request message, comprising: In the case that the data amount of the first difference image is less than that of the second region map, the electronic device sends the first difference image and the image identifier of the first region map to the server through the second request message; In the case that the data amount of the second difference image is less than that of the refined map of the second region map, the second response message comprises a second difference image and the image identifier of the second region map, the second difference image being a difference image between the refined map of the second region map and the second region map; In the case that the data amount of the second difference image is greater than or equal to that of the refined map of the second region map, the second response message comprises the refined map of the second region map.

23. The method of any one of claims 15-22, wherein: In the case that the refined mode of the image is a full-image refined mode, the region map of the image comprises image content in all regions of the image; In the case that the refined mode of the second image is a full-image refined mode, the second image and the second region map are the same, and the refined map of the second region map is the refined map of the second image.

24. The method of any one of claims 15-23, wherein: In the case that the refined mode of the image is a main-body refined mode, the region map of the image comprises a target main body in the image, and the size of the region map of the image is less than that of the image; In the case that the refined mode of the second image is a main-body refined mode, before the electronic device sends a first difference image to the server through a second request message, the method further comprises: The electronic device acquires a position of the second region map in the second image and crops the second region map from the second image; The electronic device determines a refined map of the second image based on the refined map of the second region map, comprising: Based on the position of the second region map in the second image, the electronic device replaces the second region map in the second image with the refined map of the second region map to acquire the refined map of the second image.

25. The method according to any one of claims 15 to 24, characterized in that, The first request message includes first retouching parameters, and the first retouching parameters are used to indicate at least one image processing mode adopted by a second image; the first request message is used to request the server to perform image processing on the second image according to the at least one image processing mode indicated by the first retouching parameters, so as to obtain a retouched image of the second region image.

26. The method of any one of claims 23 to 25, wherein, The electronic device pre-stores a correspondence relationship between a plurality of image processing modes and retouching modes, and the retouching modes include a full-image retouching mode and a subject retouching mode. The plurality of image processing modes include a plurality of shooting modes, and the plurality of shooting modes include a portrait mode, and the retouching mode corresponding to the portrait mode is the subject retouching mode. An image processing mode adopted by an image is used to determine a retouching mode of the image.

27. The method of any one of claims 15 to 26, wherein, The first request message further includes an image identifier of the first region image. The second request message further includes a reference image identifier and a source image identifier of the first difference image, the reference image of the first difference image is the first region image, and the source image of the first difference image is the second region image. The second response message includes an image identifier of the second region image.

28. The method of any one of claims 15 to 27, wherein, The second request message further includes an algorithm identifier, and the algorithm identifier is used to indicate that a restoration algorithm used by the server to restore the second region image is a first restoration algorithm.

29. An end-to-cloud collaborative image processing method, comprising: The method comprises: A server receives a first region image of a first image sent by an electronic device through a first request message; the first region image includes images in part or all regions of the first image; The server receives a first difference image sent by the electronic device through a second request message, and the first difference image is a difference image between a second region image and the first region image; the second region image includes images in part or all regions of a second image; The server restores the second region image according to the first difference image and the first region image; The server performs image processing on the second region image to obtain a retouched image of the second region image; The server sends a second response message to the electronic device, and the second response message is used to indicate the retouched image of the second region image; and the retouched image of the second region image is used to determine a retouched image of the second image.

30. The method of claim 29, wherein, The second response message includes a second difference image, and the second difference image is a difference image between the retouched image of the second region image and the second region image; and the second response message is used to instruct the electronic device to restore the retouched image of the second region image according to the second difference image and the second region image.

31. The method of claim 29, wherein, The method further comprises: The server performs image processing on the first region image to obtain a retouched image of the first region image; The server sends a first response message to the electronic device, and the first response message is used to indicate the retouched image of the first region image; and the retouched image of the first region image is used to determine a retouched image of the first image.

32. The method of any one of claims 29 to 31, wherein, The method further comprises: The server receives a third difference image sent by the electronic device through a third request message, the third difference image being a difference image of a third region map and the first region map; the third region map including images in part or all regions of a third image; The server restores the third region map according to the third difference image and the first region map; The server performs image processing on the third region map to obtain a retouched map of the third region map; The server sends a third response message to the electronic device, the third response message being used to indicate the retouched map of the third region map; the retouched map of the third region map being used to determine a retouched map of the third image.

33. The method of any one of claims 29 to 31, wherein, The method further comprises: The server receives a third difference image sent by the electronic device through a third request message, the third difference image being a difference image of a third region map and the second region map; the third region map including images in part or all regions of a third image; The server restores the third region map according to the third difference image and the second region map; The server performs image processing on the third region map to obtain a retouched map of the third region map; The server sends a third response message to the electronic device, the third response message being used to indicate the retouched map of the third region map; the retouched map of the third region map being used to determine a retouched map of the third image.

34. The method of any one of claims 29-33, wherein, in a case where a data amount of the second difference image is less than a data amount of the retouched map of the second region map, the second response message includes the second difference image and an image identifier of the second region map, the second difference image being a difference image of the retouched map of the second region map and the second region map; in a case where the data amount of the second difference image is greater than or equal to the data amount of the retouched map of the second region map, the second response message includes the retouched map of the second region map.

35. The method of any one of claims 29 to 34, wherein, The first request message includes a first retouching parameter, the first retouching parameter being used to indicate at least one image processing manner adopted by the second image; the server performs image processing on a second region map of the second image to obtain a retouched map of the second region map, comprising: The server performs image processing on the second image according to the at least one image processing manner indicated by the first retouching parameter to obtain the retouched map of the second region map.

36. The method of any one of claims 29 to 35, wherein, The first request message further includes an image identifier of the first region map; the second request message further includes a reference image identifier and a source image identifier of the first difference image, the reference image of the first difference image being the first region map, and the source image of the first difference image being the second region map; The server restores the second region map according to the first difference image and the first region map, comprising: The server restores the second region image according to the first differential image and the first region image, and determines an image identifier of the restored second region image as the source image identifier; and the second response message comprises the image identifier of the second region image.

37. The method of any one of claims 29 to 36, wherein, The second request message further comprises an algorithm identifier, and the algorithm identifier indicates a first restoration algorithm; The server restores the second region image according to the first differential image and the first region image, and comprises: The server restores the second region image according to the first differential image, the first region image and the first restoration algorithm.

38. An electronic device, comprising: Comprise: A processor and a memory, the memory is coupled with the processor, the memory is used for storing computer program code, the computer program code comprises computer instructions, when the processor reads the computer instructions from the memory, so that the electronic equipment executes the image processing method of end cloud cooperation as any one of claims 15-28 or claims 29-37.

39. A computer-readable storage medium, characterized in that, Comprise computer instructions, when the computer instructions run on the server, so that the server executes the image processing method of end cloud cooperation as any one of claims 15-28 or claims 29-37.

Images