Image processing method and device, terminal equipment and storage medium

Abstract

The invention relates to an image processing method and device, terminal equipment and a storage medium. The image processing method comprises the following steps: acquiring a preview image in a preview interface of a camera; determining a reference image according to the preview image; wherein the content in the reference image is at least partially the same as or similar to the content in the preview image; and displaying the reference image on the preview interface. By displaying the reference image on the preview interface, a reference sample for image acquisition is provided for a user, so that the user can conveniently view, adjust the angle and composition of image acquisition and the like by referring to the reference image during image acquisition, thereby obtaining a required image, reducing the difficulty of image acquisition, and improving the user experience. And the time for the user to carry out various adjustment operations in the image acquisition process is reduced, and the user experience is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of information processing technologies, and particularly to an image processing method, apparatus, terminal device, and storage medium. Background Art

[0002] With the development of imaging technologies, more and more electronic devices have imaging functions. An electronic device with an imaging function usually has an image acquisition module, through which image acquisition can be achieved, and video recording, etc. can also be realized. Summary of the Invention

[0003] The present disclosure provides an image processing method, apparatus, terminal device, and storage medium.

[0004] In a first aspect of an embodiment of the present disclosure, an image processing method is provided, including: obtaining a preview image in a preview interface of a camera; determining a reference image according to the preview image; wherein, the content in the reference image is at least partially the same as or similar to the content in the preview image; and displaying the reference image in the preview interface.

[0005] In one embodiment, the displaying the reference image in the preview interface includes: displaying the reference image in a preset area of the preview interface; wherein, the area of the preset area is smaller than a preset area, and / or, the preset area is located at a corner or an edge of the preview interface.

[0006] In one embodiment, the method further includes: detecting a touch operation acting on the reference image; and adjusting a display state and / or a display effect of the reference image according to the touch operation.

[0007] In one embodiment, the adjusting the display state and / or the display effect of the reference image according to the touch operation includes at least one or more of the following: zooming the reference image based on the touch operation; adjusting a position of the reference image in the preview interface based on the touch operation; adjusting a transparency of the reference image in the preview interface based on the touch operation; adjusting a number of the reference images displayed in the preview interface based on the touch operation; and switching the reference image displayed in the preview interface when there are multiple reference images based on the touch operation.

[0008] In one embodiment, a display window is provided in the preview interface for displaying the reference image.

[0009] In one embodiment, a target control is included in the preview interface, where the target control is used to control whether to display the reference image in the preview interface; displaying the reference image in the preview interface includes: according to an opening operation acting on the target control, the preview interface displays the reference image; according to a closing operation acting on the target control, the preview interface hides the reference image.

[0010] In one embodiment, the terminal device is a foldable device, and the foldable device has a connected first display part and a second display part; displaying the reference image in the preview interface includes: displaying the reference image on the first display part and displaying the preview image on the second display part.

[0011] In one embodiment, the foreground, middle ground, and / or background included in the reference image and the preview image are the same.

[0012] In one embodiment, determining the reference image according to the preview image includes: using an image with a similarity greater than a preset similarity to the preview image as the reference image.

[0013] In one embodiment, determining the reference image according to the preview image includes: obtaining the location information of the terminal device and / or weather information matching the location information; determining the reference image according to the location information and / or the weather information.

[0014] In one embodiment, determining the reference image according to the preview image includes: using a reference image recognition model to determine the reference image based on the preview image.

[0015] In one embodiment, determining the reference image according to the preview image includes: when it is determined that the change amplitude of the preview image within a preset duration is less than a preset value, determining the reference image according to the preview image.

[0016] In one embodiment, the preview interface is a preview interface in different image capture modes; where the image capture modes include at least one of the following: photo mode; portrait mode; night mode; video mode; panoramic mode; slow motion mode; time-lapse photography mode; professional mode.

[0017] In a second aspect of the embodiments of the present disclosure, there is provided an image processing apparatus, including: an acquisition module 1, configured to acquire a preview image in a preview interface of a camera; a determination module 2, configured to determine a reference image according to the preview image; where the content in the reference image is at least partially the same or similar to the content in the preview image; a display module 3, configured to display the reference image in the preview interface.

[0018] In a third aspect of the embodiments of the present disclosure, a terminal device is provided, including:

[0019] a processor and a memory for storing executable instructions that can run on the processor, where: when the processor is used to run the executable instructions, the executable instructions execute the method described in any of the above embodiments.

[0020] In a fourth aspect of the embodiments of the present disclosure, a non-transitory computer-readable storage medium is provided, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the method described in any of the above embodiments is implemented.

[0021] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

[0022] The solution of the embodiments of the present disclosure provides a reference example for image acquisition for the user by displaying a reference image on the preview interface, facilitating the user to frame, adjust the angle of image acquisition, and compose the picture, etc. with reference to the reference image when performing image acquisition, so as to obtain the required image, reducing the difficulty of image acquisition and also reducing the time for the user to perform various adjustment operations during the image acquisition process, improving the user experience.

[0023] In the case where the user has little experience in using the terminal device for image acquisition, or when the user does not know how to compose the picture, adjust the image acquisition angle, etc., the reference image can provide reference information for the user, facilitating the user to perform image acquisition according to the effects shown in the reference image, thereby improving the user experience.

[0024] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure and used together with the specification to explain the principles of the present disclosure.

[0026] Figure 1 is a schematic diagram of an image processing method shown according to an exemplary embodiment;

[0027] Figure 2 is a schematic diagram of displaying a reference image on a preview interface shown according to an exemplary embodiment;

[0028] Figure 3 is a schematic diagram of another image processing method shown according to an exemplary embodiment;

[0029] Figure 4It is a schematic diagram of an information processing device shown according to an exemplary embodiment;

[0030] Figure 5 It is a block diagram of a terminal device shown according to an exemplary embodiment. Detailed implementation manners

[0031] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present disclosure. On the contrary, they are merely examples of devices consistent with some aspects of the present disclosure as detailed in the appended claims.

[0032] Reference Figure 1 , is a schematic diagram of an image processing method, and the information processing method includes:

[0033] S100: Obtain a preview image in the preview interface of the camera.

[0034] S200: Determine a reference image according to the preview image; wherein, the content in the reference image is at least partially the same as or similar to the content in the preview image.

[0035] S300: Display the reference image in the preview interface.

[0036] This method can be executed at least in a terminal device with an image acquisition function. The terminal device may include a mobile terminal device and a fixed terminal device, etc., that is, the execution subject of this method may include at least a mobile terminal device and a fixed terminal device. The mobile terminal device may include a mobile phone, a tablet computer, an in-vehicle central control device, a wearable device, a smart device, and an aircraft, etc., and the smart device may further include a smart office device, a smart home device, and a robot, etc.

[0037] The terminal device may include an image acquisition module, and the image acquisition module may include one or more image acquisition units. The image acquisition unit may be a camera, and the models and parameters of different image acquisition units may be different. For example, the image acquisition unit may include at least one of the following: a wide-angle camera, a telephoto camera, an ultra-wide-angle camera, a black-and-white camera, a macro camera, a depth camera (Time-of-Flight, ToF), or other cameras with different parameters.

[0038] The terminal device has a touch display screen, and various operations acting on the touch display screen can be detected through the touch display screen.

[0039] The terminal device is installed with a camera application program, and at least the image acquisition module can be called and controlled through the camera application program to perform image acquisition.

[0040] Preview can be performed through a camera application which has a preview interface. The preview interface can be displayed after the camera application is launched. The style, size, and display control information of the preview interface can be adjusted according to the system configuration of the terminal device or the configuration of the camera application, which is not limited here.

[0041] The information of the image to be captured can be displayed through the preview interface. The image displayed in the preview interface can be used as a reference for the image to be captured, and the image displayed in the preview interface is the preview image. After the preview image is determined, image capture can be performed.

[0042] After the preview image is displayed in the preview interface, the preview image can be obtained. The obtaining method is not limited. It can be obtained by taking a screenshot or by other means.

[0043] The preview image includes image information, such as the shooting object. One preview image can include at least one shooting object. After the preview image is obtained, a reference image can be determined according to the preview image, and the reference image can be determined according to the content in the preview image.

[0044] The content in the reference image is at least partially the same or similar to the content in the preview image. The more the same or similar parts between the content in the reference image and the content in the preview image, the higher the similarity between the reference image and the preview image, and the higher the reference value of the reference image.

[0045] The content in the image can include the shooting object in the image, and it can be that the shooting object in the reference image is the same or similar to the shooting object in the preview image.

[0046] For example, the preset image includes shooting object 1. According to shooting object 1, a reference image including shooting object 2 can be determined. Shooting object 2 can be the same shooting object as shooting object 1 or a shooting object with a similarity to shooting object 1 reaching a preset similarity.

[0047] For another example, the preset image includes shooting object 1 and shooting object 2. According to shooting object 1 and shooting object 2, a reference image including shooting object 2 and / or shooting object 1 can be determined. Shooting object 2 can be the same shooting object as shooting object 1 or a shooting object with a similarity to shooting object 1 reaching a preset similarity.

[0048] The content included in the preview image can be determined through a detection algorithm or an identification algorithm.

[0049] Exemplarily, the content in the reference image is at least partially the same as or similar to the content in the preview image, and may further include: the image acquisition angle of the subject in the reference image is the same as or similar to the image acquisition angle of the subject in the preview image. It may further include: the composition method of the subject in the reference image is the same as or similar to the composition method of the subject in the preview image. It may further include: the magnification of the subject in the reference image is the same as the magnification of the subject in the preview image.

[0050] Exemplarily, the composition method may include magnification and image acquisition angle, etc.

[0051] When determining the reference image, the reference image may be obtained from a preset image library according to the preview image or from the network through a search engine. For example, according to the content in the preview image, an image including content matching the content in the preview image may be determined from the image library, and this image is used as the reference image. The reference image may also be determined by other means. In short, any method that can determine a reference image whose similarity or included content matches the preview image or meets the preset conditions is within the protection scope of this embodiment.

[0052] After determining the reference image, the reference image may be displayed on the preview interface, so that both the preview image and the reference image can be displayed on the preview interface. The reference image can be used to adjust the preview image, that is, to adjust the preview image according to the reference image for image acquisition.

[0053] It is sufficient to be able to display the reference image on the preview interface, and any solution for displaying the reference image on the preview interface is within the protection scope of this embodiment.

[0054] Reference Figure 2 , is a schematic diagram of displaying a reference image on the preview interface. Figure 2 The image displayed in the upper right corner in is the reference image, and the image displayed in the middle is the preview image. In this case, the reference image can provide reference information for image acquisition to the user, and image preview and image acquisition are performed according to the image acquisition angle, composition method or other image acquisition parameters of the reference image.

[0055] By displaying the reference image on the preview interface, a reference example for image acquisition is provided to the user, which is convenient for the user to frame, adjust the image acquisition angle and compose the picture, etc. with reference to the reference image when performing image acquisition, so as to obtain the required image, reduce the difficulty of image acquisition, and also reduce the time for the user to perform various adjustment operations during the image acquisition process, improving the user experience.

[0056] When the user has little experience in using terminal devices to capture images, or the user does not know how to compose or adjust the image capture angle, reference images can provide reference information for the user, making it easier for the user to capture images based on the effects shown in the reference images, thereby improving the user experience.

[0057] In one embodiment, the position of the reference image can be determined according to actual usage requirements and is not limited. It can be displayed in a partial area of ​​the preview interface, and can partially overlap with the preview image or be staggered with the preview image.

[0058] The preview interface may include the area where the preview image is located, and may also include an area outside the preview image. When the preview image and the reference image are staggered, the reference image may be located outside the preview image. This can reduce the area where the reference image blocks the preview image, thereby reducing the impact of the reference image on the preview image, facilitating the display of the preview image, and reducing the impact on the user's acquisition of the preview image, thereby improving the user's experience.

[0059] In one embodiment, S300, displaying a reference image on a preview interface includes:

[0060] The reference image is displayed in a preset area of ​​the preview interface. The preset area may be pre-set, and the reference image may be displayed in the preset area when displaying the reference image.

[0061] Exemplarily, the number of the preset areas may be one or more, that is, the reference image may be displayed in at least one preset area.

[0062] The reference image can be displayed in the preset area, which is convenient for the user to obtain the reference image, so as to collect images according to the reference image and improve the user experience.

[0063] Exemplarily, the preset area can be configured according to the configuration information.

[0064] Exemplarily, the preset area may also be determined according to a first configuration interface, wherein the first configuration interface includes a configuration control for the preset area, and the preset area may be determined through the configuration control.

[0065] Exemplarily, the location and / or size of the preset area may be determined by configuring the control.

[0066] Exemplarily, the first configuration interface includes multiple candidate areas, and different candidate areas have different positions and / or sizes. Each candidate area has a selection prompt box, and the preset area is determined from the candidate areas according to the selection operation detected on the candidate area. After the candidate area is selected, the selection prompt box of the candidate area has a check mark.

[0067] Exemplarily, the method may further include:

[0068] Display a first configuration interface according to an input operation acting on a first configuration control. The first configuration control may be located in a preview interface, may also be located in a settings interface of a camera application, or may also be located in a system settings interface of a terminal device.

[0069] Exemplarily, there is a first configuration control in a preview interface or a settings interface of a camera application, and a plurality of alternative areas are included on the first configuration interface. Detect a first input operation acting on the first configuration control, and display the first configuration interface according to the first input operation. Detect a second input operation acting on the first configuration interface, and determine a preset area from the plurality of alternative areas in the first configuration interface. The second input operation may include an operation of selecting a preset area. Both the first input operation and the second input operation belong to configuration operations.

[0070] Through the above input operations related to determining the preset area, the preset area can be configured according to the user's usage requirements, so as to determine a preset area that matches the user's usage requirements, improving the user's usage experience. In addition, through the configuration of the preset area, the user's selectivity is increased, so that the preset area can be adjusted according to the user's configuration operation, increasing the flexibility of displaying the preset area, thereby also improving the user's usage experience.

[0071] In one embodiment, the area of the preset area is smaller than a preset area, and / or the preset area is located at a corner or an edge of the preview interface.

[0072] The preset area can be determined according to actual usage requirements, and is not limited here.

[0073] Since the size of the preview interface is limited, when the area occupied by the reference image in the preview interface is too large, it will affect the display of the preview image, thereby affecting the user's acquisition of the preview image. By limiting the area of the preset area, the area of the preset area for displaying the reference image can be limited within a certain range, so that when the reference image is displayed, the area of the reference image covering the preset image is also reduced, thereby reducing the influence of the reference image on the preview image, facilitating increasing the display area of the preview image, and improving the user's usage experience of acquiring the preview image.

[0074] Exemplarily, the preset area can be configured through a second configuration interface. There is a second configuration control in the preview interface, or the settings interface of the camera application, or the system settings interface of the terminal device, that is, the second configuration control is located in the preview interface, or the settings interface of the camera application, or the system settings interface of the terminal device. According to a third input operation acting on the second configuration control, the second configuration interface is displayed, and the preset area is determined according to the third input operation used on the second configuration interface.

[0075] Exemplarily, there is an adjustment control for the preset area on the second configuration interface, and the preset area is determined according to the adjustment operation used on the adjustment control.

[0076] By adjusting the preset area, the size of the reference area can be displayed according to the user's needs, thereby improving the user experience.

[0077] Exemplarily, the position of the preset area can be at the corner or edge of the preview interface. Since most of the relatively important image content in the preview image is located in the area close to the middle of the preview image, the areas of the preview image close to the corner or edge of the preview interface are basically relatively unimportant image content in the preview image. Therefore, this can reduce the impact on the preview image after displaying the reference image, that is, reduce the impact on the user's acquisition of the preview image and improve the user experience.

[0078] In one embodiment, refer to Figure 3 , which is a schematic diagram of another image processing method. This method further includes:

[0079] S10, detecting a touch operation acting on the reference image.

[0080] S20, adjusting the display state and / or display effect of the reference image according to the touch operation.

[0081] This embodiment provides a solution for adjusting the reference image after displaying the reference image. After displaying the reference image, the terminal device can also detect the touch operation acting on the reference image. After detecting the touch operation acting on the reference image, the display state and / or display effect of the reference image is adjusted according to the touch operation. Here, the display state and / or display effect can include area scaling (scaling of size), position movement, and hiding and showing states, etc., of the reference image. The touch operation can be an input operation matching the adjusted display state and / or display effect.

[0082] Adjusting the reference image through a touch operation on the reference image can improve the adjustment efficiency of the reference image. Without exiting the preview interface or switching to other interfaces, the adjustment operation can be directly applied to the reference image on the preview interface, without entering the configuration interface through other operations for configuration operations, reducing some unnecessary cumbersome operations of the user and simplifying the operation process of adjusting the reference image, thereby improving the user experience.

[0083] In addition, adjusting the reference image can increase the diversity of the reference image display, meet more usage needs of users, and can be displayed differently according to actual usage needs, improving the convenience of displaying the reference image.

[0084] In one embodiment, S20, according to the touch operation, adjust the display state and / or display effect of the reference image, including at least one or more of the following:

[0085] Zoom the reference image based on the touch operation;

[0086] Adjust the position of the reference image in the preview interface based on the touch operation;

[0087] Adjust the transparency of the reference image in the preview interface based on the touch operation;

[0088] Adjust the number of reference images displayed in the preview interface based on the touch operation;

[0089] Based on the touch operation, when there are multiple reference images, switch the reference image displayed in the preview interface.

[0090] Adjusting the display state and / or display effect of the reference image may include:

[0091] In one embodiment, based on the touch operation, zoom the reference image displayed in the preview interface. In this case, the touch operation is a zoom operation. When it is detected that the touch operation is a touch operation for zooming the reference image, reduce or enlarge the reference image displayed in the preview interface.

[0092] The size of the reference image can be adjusted through the touch operation on the reference image, thereby improving the convenience of adjusting the size of the reference image. The user can adjust the size of the reference image according to their own usage needs. For example, the size of the reference image can be adjusted according to the content in the preview image. When the shooting object in the preview image is small or occupies a small proportion of the preview image, the size of the reference image can be increased, making it easier for the user to obtain the content in the reference image. For another example, when the shooting object in the preview image is large or occupies a high proportion of the preview image, the size of the reference image can be reduced, thereby reducing the occlusion of the preview image by the reference image.

[0093] Exemplarily, the touch operation may include a stretching operation and a pinching operation. The pinching operation is a shrinking operation. After detecting the pinching operation acting on the reference image, the reference image is shrunk according to the pinching operation. The stretching operation is a magnifying operation. After detecting the stretching operation acting on the reference image, the reference image is magnified according to the stretching operation.

[0094] Exemplarily, the touch operation may further include a click operation, such as a double-click operation. The reference image can be scaled according to the click operation. The reference image is shrunk after detecting a single click operation, and the reference image is magnified after detecting the same click operation the next time. It is also possible to magnify the reference image after detecting a single click operation and shrink the reference image after detecting the same click operation the next time.

[0095] In one embodiment, based on the touch operation, the position of the reference image in the preview interface is adjusted. In this case, the touch operation is a moving operation for adjusting the position of the reference image. When it is detected that the touch operation is a moving operation for moving the reference image, the position of the reference image in the preview interface can be changed.

[0096] Exemplarily, the touch operation may include an up-moving operation, a down-moving operation, a left-moving operation, and / or a right-moving operation. After detecting the up-moving operation acting on the reference image, the reference image is moved up. After detecting the down-moving operation acting on the reference image, the reference image is moved down. After detecting the left-moving operation acting on the reference image, the reference image is moved left. After detecting the right-moving operation acting on the reference image, the reference image is moved right.

[0097] By moving the reference image, the position of the reference image can be adjusted according to the content of the preview image, and the reference image is moved to a position in the preview interface that has less impact on the display of the preview image, thereby improving the display effect of the preview image and enhancing the user experience.

[0098] Exemplarily, the up-moving operation, the down-moving operation, the left-moving operation, and the right-moving operation may all include a dragging operation. For example, the up-moving operation may include an operation of dragging upward, the down-moving operation may include an operation of dragging downward, the left-moving operation may include an operation of dragging leftward, and the right-moving operation may all include an operation of dragging rightward.

[0099] In one embodiment, based on the touch operation, the transparency of the reference image in the preview interface is adjusted. In this case, the touch operation is a transparency adjustment operation for adjusting the transparency of the reference image. When it is detected that the touch operation is a touch operation for adjusting the transparency, the display transparency of the reference image is adjusted, including reducing the display transparency of the reference image and increasing the display transparency of the reference image, etc.

[0100] By adjusting the transparency, the influence of the reference image on the preview image can also be reduced.

[0101] Exemplarily, adjusting the transparency of the reference image may include:

[0102] Based on a touch operation, a window for adjusting the transparency of the reference image is displayed. An adjustment bar for adjusting the transparency is displayed in this window. Through this adjustment bar, the transparency of the reference image is adjusted. It can also be by sliding the window left - right or up - down to adjust the transparency, etc.

[0103] Exemplarily, the transparency may include opaque, semi - transparent, etc., and can also be other percentage - form transparencies. The higher the transparency of the reference image, the smaller the influence of the reference image on the preview image, thereby reducing the influence on the display effect of the preview image.

[0104] In one embodiment, based on a touch operation, the number of reference images displayed in the preview interface is adjusted. In this case, the touch operation is an operation for adjusting the display number of the reference images. This adjustment operation of the display number can be preset, such as a click operation, and this click operation can be different from the click operations for other functions.

[0105] When it is detected that the touch operation is an operation for adjusting the number of reference images, the display number of the reference images is adjusted, including increasing the display number and decreasing the display number.

[0106] By adjusting the display number, when there are multiple reference images, multiple reference images can be displayed, or one reference image can be displayed. When multiple reference images are displayed, multiple reference images can be provided for the user as references, facilitating the user to select favorite reference images from multiple reference images for reference in image acquisition, and also reducing the operation of the user to switch the displayed reference images, improving the user experience.

[0107] When the user does not need multiple reference images for reference, the display number can be decreased, which can reduce the area occupied by the reference images in the preview interface, thereby reducing the influence of the reference images on the display effect of the preview image.

[0108] In one embodiment, based on a touch operation, when there are multiple reference images, the reference images displayed in the preview interface are switched. In this case, the touch operation is a switching operation of the reference images. When the switching operation is detected, the reference images displayed in the preview interface are switched, and the currently displayed reference image is switched to other reference images. In this way, when there are multiple reference images, the user - favorite reference images can be switched and displayed in the preview interface according to the switching requirements.

[0109] Exemplarily, the switching operation may include a sliding operation, including an upward sliding operation, a downward sliding operation, a leftward sliding operation, a rightward sliding operation, etc., which can be determined according to the user's preference. The switching operation may be different from the moving operation for adjusting the position of the reference image, that is, the sliding operation is different from the dragging operation.

[0110] In one embodiment, the preview interface has a display window for displaying a reference image.

[0111] The reference image may be displayed in the display window, and the display window may be dedicated to displaying the reference image.

[0112] Exemplarily, the size of the display window matches the size of the reference image.

[0113] Exemplarily, through a touch operation, the size of the display window can be scaled, so as to scale the reference image according to the size of the display window.

[0114] Exemplarily, based on a touch operation, the position of the display window in the preview interface is adjusted, so as to adjust the position of the reference image in the preview interface according to the adjusted position of the display window.

[0115] Exemplarily, based on a touch operation, the number of display windows in the preview interface is adjusted, and the number of displayed reference images is adjusted according to the number of display windows. For example, one display window can display one reference image.

[0116] In one embodiment, the preview interface includes a target control, wherein the target control is used to control whether to display a reference image in the preview interface. The style, size and position of the target control in the preview interface can be determined according to actual usage requirements and are not limited. All ways of having the target control in the preview interface are within the protection scope of this embodiment.

[0117] For example, the target control may be a switch control that can display the current switch state. In the on state, the reference image is displayed in the preview interface; in the off state, the reference image is not displayed in the preview interface.

[0118] S300: Displaying a reference image in the preview interface includes:

[0119] According to the opening operation acting on the target control, the preview interface displays the reference image.

[0120] S300: Displaying a reference image in the preview interface includes:

[0121] According to the closing operation acting on the target control, the preview interface hides the reference image.

[0122] By operating on the target control, the control of whether the reference image is displayed on the preview interface can be achieved. When the user needs to display the reference image on the preview interface, the reference image can be displayed on the preview interface by performing an opening operation. When the user does not need to display the reference image on the preview interface, the display of the reference image can be closed on the preview interface by performing a closing operation.

[0123] In one embodiment, the terminal device is a foldable device, and the foldable device has a first display part and a second display part connected to each other. The folding method of the foldable device is not limited, including any form of foldable device, such as an inward-fold device and an outward-fold device.

[0124] S300, displaying the reference image on the preview interface includes:

[0125] Displaying the reference image on the first display part and displaying the preview image on the second display part.

[0126] A folding screen usually has two connected screens. When the unfolding angle is greater than a preset angle, the first display part and the second display part can be used as a single display screen. The preview image can be displayed through the first display part, and the reference image can be displayed through the second display part. In this way, the mutual influence between the reference image and the preview image can be reduced, and the user experience can be improved.

[0127] For example, if the foldable device is an inward-fold device, in the folded state, the first display part and the second display part are attached to each other and do not display any content. The first display part and the second display part serve as the inner screen. In the unfolded state, the first display part and the second display part are no longer attached to each other and are in a flat unfolded state, and content is displayed as a single display screen. In the unfolded state, the inner screen displays content. Since the area of the display screen of the foldable device is relatively large in the unfolded state, the preview image is displayed through the first display part, and the reference image is displayed through the second display part. In this way, neither the preview of the preview image is affected, nor the size of the reference image can be increased, thereby improving the user experience.

[0128] When the foldable device is an outward-fold device, the preview image can also be displayed through the first display part, and the reference image can be displayed through the second display part. In this way, neither the preview of the preview image is affected, nor the size of the reference image can be increased, thereby improving the user experience.

[0129] In one embodiment, the foreground, middle ground, and / or background included in the reference image and the preview image are the same.

[0130] The image can have a foreground, a middle ground, and / or a background. Similarly, the reference image and the preview image can also have a foreground, a middle ground, and / or a background. The more identical parts there are between the foreground, middle ground, and / or background in the reference image and those in the preview image, the higher the similarity between the reference image and the preview image, the more valuable the reference image is, and the more it can provide reference information for image acquisition for the user, facilitating the user to acquire images with reference to the reference image.

[0131] Exemplarily, the foreground, middle ground, and / or background included in the reference image and the preview image are similar, and the similarity is greater than a preset threshold.

[0132] In one embodiment, S300, determining a reference image according to the preview image includes:

[0133] Taking an image whose similarity to the preview image is greater than a preset similarity as the reference image, that is, the similarity between the reference image and the preview image is greater than the preset similarity. An image whose similarity to the reference image is greater than the preset similarity can be determined from a gallery or through a search engine from the network as the reference image. For example, it can be determined through an algorithm related to similarity.

[0134] The gallery can be a pre-stored gallery, which includes multiple images. The gallery has an update cycle, and when updating, images can be added to the gallery.

[0135] Exemplarily, there is a reference image in the gallery or on the network that is the same as or similar to the preview image, or there is at least one reference image in the gallery or on the network whose similarity to the preview image is greater than the preset similarity.

[0136] In one embodiment, determining a reference image according to the preview image includes:

[0137] Obtaining the location information of the terminal device and / or the weather information matching the location information, and determining the reference image according to the location information and / or the weather information.

[0138] The terminal device has a sensor or a functional module for obtaining location information and can determine the location information of the terminal device. When determining the location information, the terminal device can also obtain the weather information of the current location through the network.

[0139] Since some shooting objects are related to geographical locations and the location information of these shooting objects does not change, such as buildings, mountains, rivers, or some trees, etc., the images of these shooting objects collected by other devices are more likely to appear at the locations where these shooting objects are located. For example, after other devices collect the images of these shooting objects, they upload them to the network or application. Therefore, when the user's terminal device collects images at the locations where these shooting objects are located, the reference image can be determined by combining the location information of the terminal device and the weather information matching the location information, thereby improving the accuracy of determining the reference image and making the matching degree between the reference image and the preview image higher.

[0140] Exemplarily, the reference images in the image library or network can have location information or weather information. The location information of the terminal device and / or the weather information matching the location information can be combined, and then the reference images in the image library or network can have location information or weather information. In this way, the efficiency and matching degree of determining the reference image can be improved when determining the reference image.

[0141] In one embodiment, S300, determining the reference image according to the preview image includes:

[0142] Determining the reference image based on the preview image through a reference image recognition model.

[0143] When determining the reference image, the reference image can also be determined based on the preview image through a pre-trained reference image recognition model. The preview image and the images in the image library can be used as the input of the reference image recognition model, and the output of the reference image recognition model is whether the images in the image library can be used as the reference image. The preview image and the images in the network can be used as the input of the reference image recognition model, and the output of the reference image recognition model is whether the images in the network can be used as the reference image.

[0144] The reference image recognition model can at least extract the image features of the preview image, can also extract the image features of the images in the network, can also extract the image features of the images in the image library, and can also perform convolution or other processing on the image features to determine the reference image.

[0145] Exemplarily, the reference image recognition model can include a Siamese network.

[0146] In one embodiment, the image processing method further includes:

[0147] When it is determined that the change amplitude of the preview image within a preset time duration is less than a preset value, determining the reference image according to the preview image.

[0148] Multiple preview images can be obtained within a preset duration. According to the obtained multiple preview images, the change amplitude of the multiple preview images is determined. When the change amplitude of the preview images within the preset duration is less than a preset value, a reference image is determined based on the preview images.

[0149] In such a case, it indicates that the probability of the preview image being the image to be captured is relatively high, and the probability of the user performing image capture is relatively high. After determining the reference image and displaying it, it is convenient for the user to perform image capture. When the change amplitude is large, it indicates that the user has not determined the image to be captured, and the probability of performing image capture is low. The process of determining the reference image when the change amplitude is large is reduced, thereby reducing the computational load of the terminal device and the power consumption and power consumption of the device terminal.

[0150] Exemplarily, to determine the change amplitude of the preview images within the preset duration, it can be determined according to the similarity of the multiple preview images or the feature information.

[0151] In one embodiment, the preview interface is the preview interface in different image capture modes, and the image capture modes at least include one of the following:

[0152] Photo mode;

[0153] Portrait mode;

[0154] Night scene mode;

[0155] Video mode;

[0156] Panorama mode;

[0157] Slow motion mode;

[0158] Time-lapse photography mode;

[0159] Professional mode.

[0160] Of course, it can also be other modes, as long as it can perform image preview on the preview interface.

[0161] Exemplarily, when the image capture mode is the video mode, the reference image is multiple frames of images, or a reference video is displayed on the preview interface.

[0162] In one embodiment, the reference Figure 4 , is a schematic diagram of an information processing device, and the device includes:

[0163] An acquisition module 1, configured to acquire preview images in the preview interface of the camera;

[0164] A determination module 2, configured to determine a reference image according to the preview images; wherein, the content in the reference image is at least partially the same or similar to the content in the preview images;

[0165] A display module 3, configured to display the reference image on the preview interface.

[0166] In one embodiment, the display module 3 is further configured to:

[0167] Display the reference image in a preset area of the preview interface; wherein, the area of the preset area is smaller than a preset area, and / or, the preset area is located at a corner or an edge of the preview interface.

[0168] In one embodiment, the apparatus further includes:

[0169] A first detection module, configured to detect a touch operation acting on the reference image;

[0170] An adjustment module, configured to adjust the display state and / or display effect of the reference image according to the touch operation.

[0171] In one embodiment, the adjustment module is at least configured to perform one or more of the following:

[0172] Zoom the reference image based on the touch operation;

[0173] Adjust the position of the reference image in the preview interface based on the touch operation;

[0174] Adjust the transparency of the reference image in the preview interface based on the touch operation;

[0175] Adjust the number of the reference images displayed in the preview interface based on the touch operation;

[0176] Based on the touch operation, when there are multiple reference images, switch the reference image displayed in the preview interface.

[0177] In one embodiment, a display window is provided in the preview interface for displaying the reference image.

[0178] In one embodiment, a target control is included in the preview interface, wherein the target control is configured to control whether to display the reference image on the preview interface;

[0179] The display module 3 is configured to:

[0180] According to an opening operation acting on the target control, the reference image is displayed on the preview interface;

[0181] According to a closing operation acting on the target control, the reference image is hidden on the preview interface.

[0182] In one embodiment, the terminal device is a foldable device, and the foldable device has a first display part and a second display part connected to each other;

[0183] A display module 3, configured to display the reference image on the first display part and display the preview image on the second display part.

[0184] In one embodiment, the foreground, middle ground, and / or background included in the reference image and the preview image are the same.

[0185] In one embodiment, the determination module 2 is configured to:

[0186] Use an image with a similarity greater than a preset similarity to the preview image as the reference image.

[0187] In one embodiment, the determination module 2 is configured to:

[0188] An acquisition unit, configured to acquire the location information of the terminal device and / or weather information matching the location information;

[0189] A determination unit, configured to determine the reference image according to the location information and / or the weather information.

[0190] In one embodiment, the determination module 2 is configured to:

[0191] Determine the reference image based on the preview image through a reference image recognition model.

[0192] In one embodiment, the determination module 2 is further configured to:

[0193] When it is determined that the change amplitude of the preview image within a preset duration is less than a preset value, determine the reference image according to the preview image.

[0194] In one embodiment, the preview interface is a preview interface in different image acquisition modes; wherein, the image acquisition modes at least include one of the following:

[0195] Photo mode;

[0196] Portrait mode;

[0197] Night scene mode;

[0198] Video mode;

[0199] Panorama mode;

[0200] Slow motion mode;

[0201] Time-lapse photography mode;

[0202] Professional mode.

[0203] In another embodiment, an embodiment of image processing is also provided.

[0204] A considerable number of mobile phone users are novice photographers. When using the camera for image capture, they don't understand composition, which is a major focus of photography. When capturing images, auxiliary lines can be displayed on the terminal device to assist users in composition. Users can also learn composition by referring to photos taken by other devices, but the effect is not intuitive and effective enough.

[0205] Therefore, this embodiment proposes an AI photo-taking assisted composition method: when the user uses the camera APP, by detecting the current geographical location and preview image (such as the content in the preview frame) of the terminal device, a deep learning method is used to determine a reference image (for example, the picture with the highest similarity to the preview image) in the database and display it on the preview interface. The displayed content can be placed in a corner of the preview interface (including the preview frame) without affecting the user's preview of the preview image.

[0206] Through this embodiment, the user can see excellent photos taken by others in the same place or under the same scenario as a reference while taking pictures, which helps the user improve their photo-taking ability and makes the user more enthusiastic about taking pictures.

[0207] For example, for the current popular tourist check-in photos, the user can see excellent works of others as a reference while opening the camera, enhancing the user's photo-taking experience.

[0208] The detailed process is as follows:

[0209] 1. Open the camera APP according to the user's operation. This function can be manually turned on or off through a button. The APP will obtain some device information in the background, including but not limited to auxiliary information such as geographical location information and local weather information.

[0210] 2. According to the known information, call the trained model to perform real-time detection and matching on the preview image, perform matching in the database or network picture content, and match the reference image that is most similar to the preview image in the current preview frame, and display it in the upper right corner of the preview frame.

[0211] 3. The picture displayed in the upper right corner of the screen can be a single photo or a collection of multiple photos. The reference image, such as the recommended photos, can be switched by swiping up and down inside the window.

[0212] 4. The position of the recommended photos can be anywhere on the screen, and the pictures can also be in a semi-transparent state to reduce the impact on the user when taking pictures.

[0213] 5. In particular, for foldable screen mobile phones, when the user uses the inner screen, they can preview the camera content on one side of the screen and display the reference image, such as the recommended picture, on the other side.

[0214] 6. When determining the reference image, it is possible to determine whether to determine the reference image according to the change range of multiple preview images within a preset duration. For example, after the user holds the camera steadily for 1 - 2 s, the image analysis can be started to complete photo recommendation; mainly to reduce the computational amount.

[0215] It should be noted that the "first" and "second" in the embodiments of the present disclosure are only for convenience of expression and distinction, and have no other specific meanings.

[0216] Figure 5 It is a block diagram of a terminal device shown according to an exemplary embodiment. For example, the terminal device may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0217] Refer to Figure 5 , the terminal device may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0218] The processing component 802 generally controls the overall operation of the terminal device, such as operations associated with display, telephone call, data communication, camera operation, and recording operation. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above - mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0219] The memory 804 is configured to store various types of data to support the operation of the terminal device. Examples of these data include instructions for any application or method operating on the terminal device, contact data, phone book data, messages, pictures, videos, etc. The memory 804 may be implemented by any type of volatile or non - volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read - only memory (EEPROM), erasable programmable read - only memory (EPROM), programmable read - only memory (PROM), read - only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0220] The power component 806 provides power for various components of the terminal device. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the terminal device.

[0221] The multimedia component 808 includes a screen that provides an output interface between the terminal device and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operations. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the terminal device is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0222] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when the terminal device is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting audio signals.

[0223] The I / O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which can be a keyboard, a click wheel, buttons, etc. These buttons can include but are not limited to: a home button, a volume button, a power button, and a lock button.

[0224] The sensor component 814 includes one or more sensors for providing status assessments of various aspects of the terminal device. For example, the sensor component 814 can detect the on / off state of the terminal device, the relative positioning of components, such as the displayer and the keypad of the terminal device. The sensor component 814 can also detect a change in the position of the terminal device or a component of the terminal device, the presence or absence of user contact with the terminal device, the orientation or acceleration / deceleration of the terminal device, and the temperature change of the terminal device. The sensor component 814 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 814 can also include a light sensor, such as a CMOS or a CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0225] The communication component 816 is configured to facilitate communication between the terminal device and other devices in a wired or wireless manner. The terminal device can access a wireless network based on a communication standard, such as Wi-Fi, 4G, or 5G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0226] In an exemplary embodiment, the terminal device can be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0227] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not disclosed herein. The specification and embodiments are only to be considered as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0228] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. An image processing method, characterized in that, comprising: obtaining a preview image in the preview interface of a camera; determining a reference image according to the preview image; wherein, the content in the reference image is at least partially the same as or similar to the content in the preview image; displaying the reference image in the preview interface.

2. The method according to claim 1, characterized in that, the displaying the reference image in the preview interface includes: displaying the reference image in a preset area of the preview interface; wherein, the area of the preset area is smaller than a preset area, and / or, the preset area is located at a corner or an edge of the preview interface.

3. The method according to claim 1, characterized in that, the method further includes: detecting a touch operation acting on the reference image; adjusting the display state and / or display effect of the reference image according to the touch operation.

4. The method according to claim 3, characterized in that, the adjusting the display state and / or display effect of the reference image according to the touch operation includes at least one or more of the following: zooming in or out the reference image based on the touch operation; adjusting the position of the reference image in the preview interface based on the touch operation; adjusting the transparency of the reference image in the preview interface based on the touch operation; adjusting the number of the reference images displayed in the preview interface based on the touch operation; switching the reference image displayed in the preview interface when there are multiple reference images based on the touch operation.

5. The method according to claim 1, characterized in that, a display window is provided in the preview interface for displaying the reference image.

6. The method according to claim 1, characterized in that, a target control is included in the preview interface, wherein the target control is used to control whether to display the reference image in the preview interface; the displaying the reference image in the preview interface includes: displaying the reference image in the preview interface according to an opening operation acting on the target control; hiding the reference image in the preview interface according to a closing operation acting on the target control.

7. The method according to claim 1, characterized in that, the terminal device is a foldable device, and the foldable device has a first display part and a second display part connected to each other; the displaying the reference image in the preview interface includes: displaying the reference image in the first display part and displaying the preview image in the second display part.

8. The method according to claim 1, characterized in that, the foreground, middle ground and / or background included in the reference image and the preview image are the same.

9. The method according to claim 1, characterized in that, the determining a reference image according to the preview image includes: using an image with a similarity greater than a preset similarity to the preview image as the reference image.

10. The method according to claim 1, characterized in that, the determining a reference image according to the preview image includes: Obtain the location information of the terminal device and / or weather information matching the location information; Determine the reference image according to the location information and / or the weather information.

11. The method according to claim 1 or 9 or 10, wherein, The determining the reference image according to the preview image includes: Determine the reference image based on the preview image through a reference image recognition model.

12. The method according to claim 1, wherein, The determining the reference image according to the preview image includes: When it is determined that the change amplitude of the preview image within a preset time duration is less than a preset value, determine the reference image according to the preview image.

13. The method according to claim 1, wherein, The preview interface is a preview interface in different image capture modes; wherein, the image capture modes include at least one of the following: Photo mode; Portrait mode; Night scene mode; Video mode; Panorama mode; Slow motion mode; Time-lapse photography mode; Professional mode.

14. An image processing apparatus, wherein, comprises: An acquisition module, configured to acquire a preview image in a preview interface of a camera; A determination module, configured to determine a reference image according to the preview image; wherein, the content in the reference image is at least partially the same as or similar to the content in the preview image; A display module, configured to display the reference image in the preview interface.

15. A terminal device, wherein, comprises: A processor and a memory for storing executable instructions that can run on the processor, wherein: When the processor is used to run the executable instructions, the executable instructions execute the method according to any one of claims 1 to 13 above.

16. A non-transitory computer-readable storage medium, wherein, The non-transitory computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the method according to any one of claims 1 to 13 above is implemented.

Images