Image processing method and device and electronic equipment

Abstract

The invention provides an image processing method and apparatus, and an electronic device. The method comprises the steps of cutting a foreground image meeting a target condition from a first target image; processing the foreground image according to a first processing strategy to obtain a first image; processing the first target image according to a second processing strategy to obtain a second image; performing fusion processing on the first image and the second image to obtain a second target image corresponding to the first target image; wherein the processing effects of the first processing strategy and the second processing strategy are different.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of image technology, and in particular to an image processing method, device and electronic device. Background Art

[0002] In the related art, an image can be beautified in any of the following ways: (a) first modifying the overall filter of the image, and then performing local beautification in the area of ​​the face; (b) locally modifying the background area of ​​the image; (c) locally beautifying the area of ​​the face. However, in the above methods, when locally modifying the background area or the face area, the other area may be distorted, thereby affecting the display effect of the other area, and there are problems such as low accuracy and poor flexibility. Summary of the invention

[0003] In view of this, the embodiments of the present application at least provide an image processing method, device and electronic device.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides an image processing method, the method comprising:

[0006] Cut out a foreground image that meets the target condition from the first target image;

[0007] Processing the foreground image with a first processing strategy to obtain a first image;

[0008] Processing the first target image with a second processing strategy to obtain a second image;

[0009] The first image and the second image are fused to obtain a second target image corresponding to the first target image;

[0010] The first processing strategy and the second processing strategy have different processing effects.

[0011] The present application provides an image processing device, the device comprising:

[0012] A cropping unit, used for cropping a foreground image that meets the target condition from the first target image;

[0013] A processing unit, configured to process the foreground image with a first processing strategy to obtain a first image; and process the first target image with a second processing strategy to obtain a second image;

[0014] A fusion unit, used for fusing the first image and the second image to obtain a second target image corresponding to the first target image;

[0015] The first processing strategy and the second processing strategy have different processing effects.

[0016] An embodiment of the present application provides an electronic device, including:

[0017] A processing unit is used to crop a foreground image that meets the target conditions from a first target image; process the foreground image with a first processing strategy to obtain a first image; process the first target image with a second processing strategy to obtain a second image; and fuse the first image and the second image to obtain a second target image corresponding to the first target image; wherein the processing effects of the first processing strategy and the second processing strategy are different.

[0018] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0020] Figure 1 A schematic diagram of an implementation process of an image processing method provided in an embodiment of the present application Figure 1 ;

[0021] Figure 2 A schematic diagram of cropping a foreground image provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of an implementation process of an image processing method provided in an embodiment of the present application Figure 2 ;

[0023] Figure 4 A schematic diagram of image processing provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of processing a foreground image provided in an embodiment of the present application;

[0025] Figure 6 A schematic diagram of processing a first target image provided in an embodiment of the present application;

[0026] Figure 7 A schematic diagram of a fusion process provided in an embodiment of the present application;

[0027] Figure 8 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0028] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0029] Fig.10A hardware entity schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0031] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0032] It should be pointed out that the terms "first\second\third" involved in the embodiments of the present application are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0033] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those skilled in the art in the field to which the embodiments of the present application belong. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0034] Imaging devices can be used to frame biological objects (e.g., people) to produce images. However, when the lighting conditions are poor or there is shaking during shooting, the display effect of the image produced by the imaging device is poor. For example, when taking a group photo of multiple people in certain scenes (e.g., group photos at a dinner party, outdoor group photos), the imaging device is far away from the people and the background. In the captured image, the face of the person will be relatively small and not clear enough, the background will also be blurred, and the image display effect will be poor. Especially when the lighting conditions are poor or there is shaking during shooting, the display effect of the image will be further affected.

[0035] In the related art, the following methods can be used to beautify an image: (a) first modify the overall filter of the image, and then perform local beautification in the area of ​​the face; (b) locally modify the background area of ​​the image; (c) locally beautify the area of ​​the face. However, in the above methods, when locally modifying the background area or the face area, the local modification of the background area or the local modification of the face area will affect the processing effect of the other area, which may cause the other area to be distorted, thereby affecting the display effect of the image, and there are problems such as low accuracy and poor flexibility.

[0036] The embodiment of the present application provides an image processing method, first, a foreground image that meets the target condition is cropped out from a first target image, secondly, the foreground image is processed by a first processing strategy to obtain a first image, and the first target image is processed by a second processing strategy to obtain a second image, wherein the processing effects of the first processing strategy and the second processing strategy are different, and finally, the first image and the second image are fused to obtain a second target image corresponding to the first target image. In this way, on the one hand, since the foreground image is cropped out first, and then the foreground image and the first target image are processed respectively by different processing strategies, the foreground image and the first target image can be processed independently and in parallel, which reduces the demand for the processing strategy capability compared to the processing of the entire image separately, improves the image processing efficiency, and enhances the intelligence of the image processing method; on the other hand, since the processing effects of the first processing strategy and the second processing strategy are different, it is possible to select a suitable processing strategy for different areas of the image, thereby improving the flexibility of the image processing method; on the other hand, the first image and the second image are fused to obtain the second target image, which reduces the possibility of the display effects of different areas of the image affecting each other, thereby improving the accuracy of the image processing method.

[0037] The method provided in the embodiment of the present application can be executed by an electronic device, wherein the electronic device can be a laptop, a tablet computer, a desktop computer, a set-top box, a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable gaming device) and other types of terminals, and can also be implemented as a server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0038] Figure 1 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 1 ,like Figure 1 As shown, the method includes steps S101 to S104, wherein:

[0039] Step S101, cutting out a foreground image that meets a target condition from a first target image.

[0040] Here, the first target image refers to an image to be processed. In some embodiments, the format of the first target image may include but is not limited to: JPEG (Joint Photographic Experts Group), TIFF (Tag Image File Format), BMP (Bitmap Image File), etc.

[0041] In some embodiments, the execution device of the present application may capture the first target image through an imaging device, or may receive the first target image sent by other devices. The present application does not limit the source of the first target image.

[0042] The foreground image is a partial image cropped from the first target image. In some implementations, the number of the foreground image is at least one.

[0043] In some implementations, the format of the foreground image may include, but is not limited to, JPEG, TIFF, BMP, etc. The format of the first target image and the format of the foreground image may be the same.

[0044] In some embodiments, the foreground image may include, but is not limited to, biological objects, non-biological objects, etc. For example, a biological object refers to a living organism with life functions, and is also a collection of objects. Biological objects may include, but are not limited to, humans, animals, etc. For another example, non-biological objects refer to substances and entities without life characteristics. Non-biological objects may include, but are not limited to, buildings, electronic products, etc.

[0045] The target condition is a condition used to determine whether to crop the foreground image. The target condition may include but is not limited to: the image area needs to optimize details, the image area has distortion, etc.

[0046] In some embodiments, a foreground image that meets the target condition indicates that the foreground image contains a biological object. Correspondingly, a target area containing the biological object in the first target image can be obtained. If the biological parameters of the biological object contained in the target area meet the biological parameter conditions, the target area is cropped to obtain the foreground image. The biological parameters are used to characterize the distance parameters between biological key feature points. The biological parameters of the biological object contained in the target area meet the biological parameter conditions to indicate that the distance parameters between biological key feature points do not meet the distance threshold.

[0047] The method of cropping the foreground image may include but is not limited to: cropping the foreground image in the first target image through a detection frame, cropping the foreground image in the first target image through deep learning, etc. For example, the detection frame can surround the object, so that the target object can be located and cropped. The foreground image can be cropped out of the first target image according to the detection frame by calling the preset function in the OpenCV library. The preset function may include but is not limited to: cv2.waitKey(), cv2.imwrite(), etc. OpenCV is an open source computer vision and machine learning software library. It provides the functions of reading, displaying and processing images. For another example, deep learning uses a multi-layer neural network to simulate the complex decision-making ability of the human brain, and can perform foreground extraction on the input target image, thereby cropping the foreground image out of the first target image.

[0048] Step S102: Process the foreground image using a first processing strategy to obtain a first image.

[0049] Here, the first processing strategy is a strategy for optimizing image details. The first image is an image processed by the first processing strategy. In some embodiments, the number of the first images is at least one. The number of the first images is the same as the number of the foreground images.

[0050] In some embodiments, the image size refers to the total number of pixels contained in the entire image, which is expressed as the product of the pixels in the width direction and the pixels in the height direction. The image size of the first image is larger than the image size of the foreground image. For example, if the image size of a foreground image is 1000x2000, the image size of the first image corresponding to the foreground image may be 3000x6000.

[0051] In some implementations, the format of the first image may include, but is not limited to: JPEG, TIFF, BMP, etc. The format of the first target image, the format of the foreground image, and the format of the first image may be the same or different.

[0052] In some embodiments, when the foreground image includes a biological object, the first processing strategy may include but is not limited to at least one of the following: beauty algorithm, skin color adjustment, posture correction, etc. When the foreground image includes a non-biological object, the first processing strategy may include but is not limited to at least one of the following: region growing, brightness contrast adjustment, sharpening, denoising, etc.

[0053] In some embodiments, the first processing strategy may be in any form including but not limited to at least one of the following: an algorithm, a model, a neural network, etc. For example, the first processing strategy may be a neural network equipped with a portrait optimization algorithm, or a model. Any object that can achieve the function of optimizing image details may be used as the first processing strategy, and the present application does not limit the form of the first processing strategy.

[0054] In some implementations, facial features may be extracted from the foreground image, and then standard features corresponding to the facial features may be matched, and then the foreground image may be processed based on the standard features to obtain the first image.

[0055] In some embodiments, a target parameter representing a biological posture of a biological object contained in a foreground image may be obtained. If the target parameter does not meet the biological posture condition, the foreground image is processed with a first sub-processing strategy to obtain the first image, and if the target parameter meets the biological posture condition, the foreground image is processed with a second sub-processing strategy to obtain the first image. The first sub-processing strategy includes a first strategy and a second strategy, the second sub-processing strategy includes the first strategy, and the second strategy is used for posture correction.

[0056] Step S103: Process the first target image with a second processing strategy to obtain a second image.

[0057] Here, the second processing strategy is a strategy for performing high-definition processing on an image. The second image is an image processed by the second processing strategy. In some embodiments, the number of the second image is at least one.

[0058] The number of the second images is the same as the number of the first target images.

[0059] In some embodiments, the image size of the second image is larger than the image size of the first target image. For example, if the image size of a first target image is 3000x4000, the image size of the second image corresponding to the first target image may be 6000x8000.

[0060] In some implementations, the format of the second image may include, but is not limited to, JPEG, TIFF, BMP, etc. The format of the first target image, the format of the first image, and the format of the second image may be the same.

[0061] In some embodiments, the second processing strategy may be in any form including but not limited to at least one of the following: an algorithm, a model, a neural network, etc. For example, the second processing strategy may be a model equipped with an image high-definition algorithm, or may be an image high-definition algorithm. As long as an object that can achieve high-definition processing of optimized images can be used as the second processing strategy, and this application does not limit the form of the second processing strategy.

[0062] In some embodiments, the image high-definition algorithm is used to denoise and enhance the image, which can improve the resolution and quality of the image. The image high-definition algorithm may include, but is not limited to: Generative Adversarial Network (GAN), Fourier transform, high-order degradation operation, etc. GAN uses adversarial training to generate realistic high-resolution images. Fourier transform improves image clarity by enhancing high-frequency components in the frequency domain. High-order degradation operations simulate image degradation problems, remove unknown noise and complex artifacts, and restore noisy and blurred images.

[0063] In some embodiments, the second processing strategy can be integrated into a neural network, and the neural network can be obtained by training an untrained neural network with a training set. When in use, the first target image is input to the neural network, and the neural network can output a second image.

[0064] In some embodiments, the first target image can be processed in blocks to obtain at least one first sub-image, and then each first sub-image can be processed with a third processing strategy to obtain a second sub-image corresponding to the first sub-image. Finally, based on the position of each first sub-image in the first image, the second sub-images can be spliced ​​to obtain a second image.

[0065] Step S104, fusing the first image and the second image to obtain a second target image corresponding to the first target image; wherein the first processing strategy and the second processing strategy have different processing effects.

[0066] Here, the second target image refers to a processed image. In some embodiments, the format of the second target image may include but is not limited to: JPEG, TIFF, BMP, etc. The format of the first target image may be the same as the format of the first target image.

[0067] In some embodiments, the image size of the second target image is larger than the image size of the first target image. For example, if the image size of a first target image is 4000x6000, the image size of the second target image corresponding to the first target image may be 8000x12000.

[0068] Fusion processing can process and combine different images appropriately to achieve information complementarity and enhancement. The fusion processing method of the first image and the second image may include but is not limited to: fusing the first image and the second image based on pixels, fusing the first image and the second image based on features, fusing the first image and the second image based on transform domain, etc. For example, based on pixels, the pixels of the image can be processed and combined one by one to achieve fusion processing of the first image and the second image. During implementation, the pixels can be fused by weighted average, maximum value or minimum value. For another example, the first image and the second image are extracted and matched based on their features, and then fused according to the matching results to achieve fusion processing of the first image and the second image. For another example, based on the transform domain, the first image and the second image are fused in the frequency domain or the wavelet domain by means of Fourier transform or wavelet transform.

[0069] In some embodiments, based on the position of the biological object contained in the second image in the first image, the pixel fusion weight information corresponding to each pixel of the first image can be obtained, and then based on the weight information corresponding to at least one pixel in the first image, the pixel points of the biological object contained in the second image are fused with the pixel points of the biological object in the first image to obtain a second target image corresponding to the first target image.

[0070] In some embodiments, first size difference information between the first image and the foreground image and second size difference information between the second image and the first target image may be determined, and then based on the first size difference information and the second size difference information, the size ratio between the first image and the second image is adjusted to the first size ratio, and image fusion processing is performed on the adjusted first image and second image to obtain the second target image corresponding to a target image. The first size ratio is the same as the second size ratio, and the second size ratio is the size ratio between the first target image and the foreground image.

[0071] It should be noted that if only one processing strategy is used to process the first target image to obtain the second target image corresponding to the first target image, the following situation may occur: the capability requirements for the processing strategy are very high, and the second target image corresponding to the first target image cannot be obtained by processing the first target image with only one processing strategy. For example, a very large model or neural network is required, or there is no algorithm that can implement the process of this application. Therefore, this application first crops the foreground image from the first target image, and then two processing strategies with different processing effects process the foreground image and the first target image respectively, and finally fuses the first image and the second image to obtain the second target image corresponding to the first target image. On the one hand, it is possible to select appropriate processing strategies for different areas of the image and then fuse them, reducing the possibility of the display effects of different areas of the image affecting each other; on the other hand, it reduces the demand for the capabilities of the processing strategy (for example, reducing the amount of data of the model), and when the processing strategy is a model, the model can be deployed in the terminal device.

[0072] In the embodiment of the present application, first, a foreground image that meets the target condition is cropped out from the first target image, and then the foreground image is processed by the first processing strategy to obtain the first image, and the first target image is processed by the second processing strategy to obtain the second image, wherein the processing effects of the first processing strategy and the second processing strategy are different, and finally, the first image and the second image are fused to obtain the second target image corresponding to the first target image. In this way, on the one hand, since the foreground image is cropped out first, and then the foreground image and the first target image are processed respectively by different processing strategies, the foreground image and the first target image can be processed independently and in parallel, which reduces the demand for the processing strategy capability compared to the whole image being processed separately, improves the image processing efficiency, and enhances the intelligence of the image processing method; on the other hand, since the processing effects of the first processing strategy and the second processing strategy are different, it is possible to select a suitable processing strategy for different areas of the image, thereby improving the flexibility of the image processing method; on the other hand, the first image and the second image are fused to obtain the second target image, which reduces the possibility of the display effects of different areas of the image affecting each other, thereby improving the accuracy of the image processing method.

[0073] In some embodiments, a foreground image meeting a target condition indicates that the foreground image includes a biological object.

[0074] Here, the biological object may include but is not limited to: humans, animals, etc. In some embodiments, the foreground image including the biological object means that the foreground image may include a background in addition to at least including the biological object.

[0075] Figure 2 A schematic diagram of cutting a foreground image provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, a person is detected in the first target image 21 , and therefore a foreground image 22 is cropped out from the first target image 21 through a detection frame 22 , wherein the foreground image 22 contains the person.

[0076] In the implementation manner of the present application, by cropping the foreground image containing the biological object, the image processing method can be applied to portrait processing, thereby improving the compatibility of the image processing method.

[0077] In some embodiments, step S101 includes steps S11 to S12, wherein:

[0078] Step S11, acquiring a target area containing a biological object in a first target image;

[0079] Here, the target region refers to a region containing a biological object. The target region may be in any suitable shape, such as a rectangle, a circle, etc. In some embodiments, the shape of the target region is related to the shape of the detection frame.

[0080] In some embodiments, the target area containing the biological object in the first target image can be obtained by object detection. The object detection method may include but is not limited to: implementing object detection through a single-stage multi-layer target detection model (SingleShot MultiBox Detector, SSD), implementing object detection through YOLO, etc. SDD uses VGG16 as the basic model, and adds a convolutional layer to obtain more feature maps for object detection, which has faster speed and accuracy. YOLO is a deep learning framework for object detection. It can detect the objects and their positions in the image with only one image viewing, and can be used to implement object detection.

[0081] Step S12, if the biological parameters of the biological objects contained in the target area meet the biological parameter conditions, the target area is cropped to obtain a foreground image; wherein the biological parameters are used to characterize the distance parameters between biological key feature points, and the biological parameters of the biological objects contained in the target area meet the biological parameter conditions to characterize that the distance parameters between biological key feature points do not meet the distance threshold.

[0082] Here, the biological parameter of the biological object is used to characterize the distance parameter between biological key feature points. Biological key feature points refer to the significant feature points of the organism. The distance parameter between biological key feature points refers to the distance between two biological key feature points. The distance parameter between biological key feature points can be any suitable size, for example, 4 cm (centimeter), 20 cm, etc. The distance threshold can be any suitable size, for example, 10 cm, 5 cm, etc.

[0083] In some embodiments, there are differences between different biological parameters. Therefore, different distance thresholds can be set for biological key feature points at different positions. For example, when the biological parameter is face length, the corresponding distance threshold can be 20 cm, and when the biological parameter is eye distance, the corresponding distance threshold can be 8 cm.

[0084] In some embodiments, biological parameters may include, but are not limited to: eye distance, face length, face width, etc.

[0085] In some embodiments, the position of the key biological feature points may include but is not limited to one of the following: eyes, eyebrows, nose, mouth, cheeks, chin, facial contour, etc. Among them, the position of the eyes may include but is not limited to one of the following: corners of the eyes, eyelids, pupils, etc. The position of the eyebrows may include but is not limited to one of the following: eyebrows, eyebrow tails, etc. The position of the nose may include but is not limited to one of the following: nose tip, nose wings, nostrils, etc. The position of the mouth may include but is not limited to one of the following: corners of the mouth, lip contours, etc. The position of the cheek may include but is not limited to one of the following: the highest point of the cheek, contour lines, etc. The position of the chin may include but is not limited to one of the following: chin tip, contour lines, etc. The position of the facial contour may include but is not limited to one of the following: cheekbones, jaw lines, etc.

[0086] In some embodiments, when the biological object is a person, since a severely distorted human face is not suitable for processing by the first processing strategy, it is necessary to screen out human faces that are not severely distorted. If the biological parameters of the biological object meet the biological parameter conditions to indicate that the human face is not severely distorted, the target area of ​​the biological object can be cropped. Correspondingly, if the biological parameters of the biological object do not meet the biological parameter conditions to indicate that the human face is severely distorted, the target area of ​​the biological object is not cropped.

[0087] In some embodiments, a severely distorted face may include, but is not limited to, at least one of the following: a small face, a side face, a lowered head, a deformed face, etc.

[0088] In some embodiments, the distance parameter between biological key feature points not meeting the distance threshold may include but is not limited to: the distance parameter between biological key feature points is greater than the distance threshold, the distance parameter between biological key feature points is less than the distance threshold, etc.

[0089] In the implementation manner of the present application, a target area containing a biological object in a first target image is first acquired. When the biological parameters of the biological object contained in the target area meet the biological parameter conditions, the target area obtains a foreground image, thereby improving the accuracy of the foreground image.

[0090] In some embodiments, step S102 includes steps S121 to S123, wherein:

[0091] Step S121, obtaining target parameters representing the biological posture of the biological object contained in the foreground image.

[0092] Here, the target parameters of the biological gesture are used to determine the processing strategy required for processing the foreground image. In some embodiments, the target parameters of the biological gesture may include but are not limited to: eye distance, face length, face width, etc.

[0093] In some embodiments, the distance between the eyes can be any suitable size, for example, 6 cm, 7.6 cm, etc. The length of the face can be any suitable size, for example, 20 cm, 23.6 cm, etc. The width of the face can be any suitable size, for example, 16.6 cm, 17.3 cm, etc.

[0094] In some embodiments, the target parameters of the biological posture can be obtained by posture estimation. In implementation, posture estimation can be achieved through OpenPose. OpenPose is a popular open source library that can be used for real-time multi-person posture estimation from images. OpenPose can detect 18 key points on the human body, including joints such as the head, elbows, wrists, knees and ankles. When performing head posture estimation, the biological posture of the head can be calculated by the coordinates of the facial key points. The coordinates of the facial key points may include but are not limited to: the pitch angle of the head relative to the horizontal plane (also known as pitch), the roll angle of the head relative to the vertical plane (also known as roll), and the yaw angle of the head relative to the vertical plane (also known as yaw).

[0095] In some embodiments, the target parameters of the biological posture can be obtained by means of facial detection. The implementation method of facial detection may include but is not limited to: implementing facial detection by using OpenCV, implementing facial detection by using MediaPipe, etc. For example, OpenCV comes with some pre-trained Haar cascade models that can be used to implement facial detection. For another example, MediaPipe provides a pre-built graph to implement facial detection, which is called FaceDetection. FaceDetection is based on the BlazeFace model, which is a lightweight and efficient face detector.

[0096] In some embodiments, the target parameter of the biological posture can be obtained by the distance parameter between the biological key feature points. In implementation, the distance parameter between the biological key feature points, the weighted / logarithmic / index of the distance parameter between the biological key feature points, etc. can be used as the target parameter of the biological posture.

[0097] It should be noted that the target parameters of the biological posture and the content of the biological parameters can be the same, but the two have different purposes. The biological parameters are used to crop a foreground image that meets the target conditions from the first target image. The target parameters of the biological posture are used to determine the processing strategy required for the cropped foreground image. The specific reason is that the foreground images cropped by the biological parameters have different degrees of distortion. For example, for a foreground image that is not distorted, it is not necessary to process it through the second strategy for posture correction, while for a foreground image that is distorted, it is necessary to process it through the second strategy for posture correction.

[0098] Step S122: if the target parameter does not satisfy the biological posture condition, the foreground image is processed with the first sub-processing strategy to obtain a first image.

[0099] Here, the biological posture condition is a condition for determining whether the biological object is distorted. The first sub-processing strategy is a strategy for achieving posture correction and face beautification.

[0100] In some implementations, the first sub-processing strategy includes a first strategy and a second strategy, wherein the first strategy is used for face beautification and the second strategy is used for posture correction.

[0101] In some embodiments, the first strategy may be implemented by algorithms including but not limited to one of the following: beautification algorithm, skin color adjustment, texture enhancement, detail enhancement, etc. Among them, the beautification algorithm can achieve at least one of the following functions including but not limited to: skin smoothing, whitening, face thinning, enlarging eyes, thinning nose, bright eyes, removing dark circles, etc. Skin color adjustment can achieve at least one of the following functions including but not limited to: color balance, saturation adjustment, etc. Texture enhancement can achieve at least one of the following functions including but not limited to: skin texture preservation, hair texture enhancement, etc. Detail enhancement can achieve at least one of the following functions including but not limited to: sharpening, contrast adjustment, etc.

[0102] The second strategy may be implemented by algorithms including but not limited to one of the following: face reshaping, head posture correction, etc. For example, face reshaping generally refers to changing or optimizing certain features of a person's face through image editing techniques. For another example, head posture correction is to adjust the head posture of a person in an image to conform to a specific angle or direction.

[0103] In some embodiments, the first strategy or the second strategy may be in any form including but not limited to at least one of the following: algorithm, model, neural network, etc. Any object that can achieve the function of face beautification can be used as the first strategy, and any object that can achieve the function of posture correction can be used as the second strategy. This application does not limit the form of the first strategy and the form of the second strategy.

[0104] In some embodiments, the posture information of the biological object can be obtained according to the target parameter, and then it is determined whether the posture information meets the biological posture condition. When the posture information of the biological object does not meet the biological posture condition, the foreground image is processed with the first sub-processing strategy to obtain the first image. The posture information of the biological object may include but is not limited to: eye distance ratio, face length ratio, aspect ratio, etc. The posture information of the biological object not meeting the biological posture condition may be that the posture information of the biological object does not meet the posture threshold range.

[0105] In some embodiments, the gesture threshold range may be any suitable range, for example, 0.4-1.2, 0.35-1.5, etc. The gesture information of the biological object not meeting the gesture threshold range may include but is not limited to: the gesture information of the biological object is greater than the gesture threshold range, the gesture information of the biological object is less than the gesture threshold range, etc.

[0106] Step S123, if the target parameters meet the biological posture conditions, the foreground image is processed with the second sub-processing strategy to obtain the first image; wherein the first sub-processing strategy includes the first strategy and the second strategy; the second sub-processing strategy includes the first strategy; and the second strategy is used for posture correction.

[0107] Here, the second sub-processing strategy is a strategy for achieving posture correction.

[0108] In some embodiments, the posture information of the biological object can be obtained according to the target parameter, and then it is determined whether the posture information meets the biological posture condition. When the posture information of the biological object meets the biological posture condition, the foreground image is processed with the second sub-processing strategy to obtain the first image. The posture information of the biological object meets the biological posture condition, which can be that the posture information of the biological object meets the posture threshold range.

[0109] In some embodiments, the posture information of the biological object satisfies the posture threshold range, which may include but is not limited to: the posture information of the biological object is within the posture threshold range, etc.

[0110] In some embodiments, the first strategy and the second strategy may be integrated together or separately in a neural network, and the neural network may be obtained by training an untrained neural network through a training set. When in use, the foreground image is input to the neural network, and the neural network may output the first image.

[0111] In an embodiment of the present application, by obtaining target parameters that characterize the biological posture of the biological object contained in the foreground image, the foreground image is processed by a first sub-processing strategy when the target parameters do not meet the biological posture conditions, and the foreground image is processed by a second sub-processing strategy when the target parameters meet the biological posture conditions. On the one hand, compared with the first sub-processing strategy, the second sub-processing strategy includes fewer strategies, further reducing the demand for the capabilities of the processing strategies and improving the image processing efficiency; on the other hand, it is possible to select appropriate processing strategies according to different situations of the foreground image, thereby improving the accuracy of the foreground image and the flexibility of the image processing method.

[0112] In some implementations, step S104 includes steps S104a to S104b, wherein:

[0113] Step S104a, based on the position of the biological object contained in the second image in the first image, obtaining pixel fusion weight information corresponding to each pixel point of the first image.

[0114] Here, the second image is an image obtained by processing the first target image, and since the first target image contains biological objects, the second image also contains biological objects. At the same time, since the first image is an image obtained by processing the foreground image cropped from the first target image, the biological objects contained in the second image have a corresponding relationship with the biological objects in the first image.

[0115] The pixel fusion weight information refers to the weight value occupied by a certain pixel when performing fusion processing. The pixel fusion weight information can be any suitable size, for example, 0.8, 0.9, etc. In some embodiments, the pixel fusion weight information is not greater than a preset value. The preset value can be any suitable size, for example, 1, 0.5, etc.

[0116] In some embodiments, the pixel fusion weight information is related to the position of the pixel point, and the pixel point closer to the face position has a larger pixel fusion weight information, and the pixel point farther from the face position has a smaller pixel fusion weight information. For example, the pixel point located at the human eye may have a pixel fusion weight information of 1, and the pixel point located at the edge of the face may have a pixel fusion weight information of 0.7.

[0117] Methods for obtaining pixel fusion weight information may include but are not limited to: obtaining pixel fusion weight information corresponding to pixels through semantic segmentation, obtaining pixel fusion weight information corresponding to pixels through attention mechanism, etc. For example, semantic segmentation is a computer vision task that uses a deep learning algorithm to assign pixel fusion weight information to pixels. For another example, the attention mechanism is a data processing method in machine learning. The attention mechanism can reflect the importance of different information through weights, and then obtain the pixel fusion weight information corresponding to the pixel.

[0118] Step S104b, based on weight information corresponding to at least one pixel in the first image, the pixel points of the biological object contained in the second image are fused with the pixel points of the biological object in the first image to obtain a second target image corresponding to the first target image.

[0119] Here, based on the weight information corresponding to at least one pixel in the first image, the weight information of the pixel corresponding to each pixel in the first image in the second image is determined, and then the pixel points of the biological object contained in the second image are fused with the pixel points of the biological object in the first image to obtain a second target image corresponding to the first target image.

[0120] In some embodiments, the sum of the weight information corresponding to each pixel in the first image and the weight information of the pixel corresponding to the pixel in the second image is a fixed value, which can be any suitable value, for example, 1, 2, etc.

[0121] In some implementations, the fused pixel point f(x, y) may be expressed by the following formula (1-1), namely:

[0122]

[0123] Among them, p is the weight information corresponding to the pixel point, and the above formula satisfies and Wherein, R is the first image, E is the second image, x is any pixel point on R, and y is the pixel point in E corresponding to the position of x.

[0124] In an embodiment of the present application, pixel points of the biological object contained in the second image are fused with pixel points of the biological object in the first image through weight information corresponding to at least one pixel point in the first image to obtain a second target image. On the one hand, the pixel points are fused through weight information, which improves the accuracy of the fused image and the display effect of the second target image. On the other hand, since the fused pixels are at the same position, the transition area of ​​the image is more natural, which reduces the possibility of the display effects of different areas of the image affecting each other.

[0125] In some embodiments, the method further includes steps S105 to S106, wherein:

[0126] Step S105, inputting the first target image into the target intelligent engine to obtain the position of the biological object contained in the first target image in the first target image.

[0127] Here, the target intelligent engine is an intelligent engine with the function of analyzing images. The target intelligent engine can analyze the input image and obtain the position of the biological object in the image.

[0128] In some embodiments, the intelligent engine is a software component or system that combines artificial intelligence technology. The target intelligent engine may include, but is not limited to, a machine learning engine, a GAN engine, a semantic segmentation network, etc. Machine learning can reorganize existing knowledge structures to continuously improve their performance. The GAN engine consists of a generator and a discriminator, and can process input data through adversarial training.

[0129] Step S106: obtaining the position of the biological object contained in the second image in the first target image based on the position of the biological object contained in the first target image in the first target image.

[0130] Here, since the image content of the first target image is the same as that of the second image, the position of the biological object contained in the first target image in the first target image can be used as the position of the biological object contained in the second image in the first target image.

[0131] In some embodiments, the position of the biological object in the first target image may be represented by any suitable form, such as Cartesian coordinates, polar coordinates, pixel coordinates, etc.

[0132] In some embodiments, the second image may be input into the target intelligence engine to obtain the position of the biological object contained in the second image in the first target image. However, the image size of the second image is larger than the image size of the first target image, so it is more efficient to input the first target image into the target intelligence engine to obtain the position of the biological object contained in the second image in the first target image.

[0133] In an embodiment of the present application, by inputting the first target image into the target intelligent engine, the position of the biological object contained in the first target image is obtained, and then the position of the biological object contained in the second image is obtained in the first target image, so that the first image and the second image can be accurately aligned when they are fused, thereby improving the accuracy of the fusion processing.

[0134] In some implementations, step S104 includes steps S104c to S104d, wherein:

[0135] Step S104c: determining first size difference information between the first image and the foreground image, and second size difference information between the second image and the first target image.

[0136] Here, the first size difference information refers to the difference information between the image size of the first image and the image size of the foreground image. The first size difference information can be any suitable size. For example, in some embodiments, the first size difference information is greater than a preset value. The preset value can be 1.

[0137] The method for determining the first size difference information may include but is not limited to: a first ratio, a weighted / logarithmic / indexed value of the first ratio, etc. The first ratio refers to the ratio between the image size of the first image and the image size of the foreground image. The first ratio may be any suitable value, such as 2, 2.5, etc. For example, the first ratio is used as the first size difference information.

[0138] In some embodiments, when the foreground image is processed by the neural network to obtain the first image, the first size difference information is related to the processing capability of the neural network. The processing capability of the neural network is a fixed property of the neural network. The stronger the processing capability of the neural network, the larger the first size difference information, and the larger the image size of the first image obtained by processing the foreground image of the same image size.

[0139] The second size difference information refers to the difference information between the image size of the second image and the image size of the first target image. In some embodiments, the second size difference information is greater than a preset value. The preset value may be 1.

[0140] The method for determining the second size difference information may include, but is not limited to: a second ratio, a weighted / logarithmic / indexed value of the second ratio, etc. The second ratio refers to the ratio between the image size of the second image and the image size of the first target image. The second ratio may be any suitable value, such as 2, 1.5, etc. For example, the second ratio is used as the second size difference information.

[0141] In some implementations, the first size difference information is not smaller than the second size difference information.

[0142] In some embodiments, when the foreground image is processed by the neural network to obtain the second image, the first size difference information is related to the processing capability of the neural network. The stronger the processing capability of the neural network, the larger the second size difference information, and the larger the image size of the second image obtained by processing the first target image of the same image size.

[0143] Step S104d, based on the first size difference information and the second size difference information, adjusting the size ratio between the first image and the second image to the first size ratio, and performing image fusion processing on the adjusted first image and second image to obtain a second target image corresponding to the first target image; wherein the first size ratio is the same as the second size ratio, and the second size ratio is the size ratio between the first target image and the foreground image.

[0144] Here, since the first image and the second image are to be fused, in order to realize the fusion process, the size ratio between the first image and the second image is equal to the size ratio between the first target image and the foreground image.

[0145] The first size ratio is the size ratio between the first image and the second image. The first size ratio can be any appropriate size, for example, 1.5, 2, etc.

[0146] The second size ratio is the size ratio between the first target image and the foreground image. The second size ratio can be any suitable size, for example, 1, 2, etc. The method for determining the second size ratio can include but is not limited to: a third ratio, a weighted / logarithmic / indexed value of the third ratio, etc. The third ratio refers to the ratio between the image size of the first target image and the image size of the foreground image. The third ratio can be any suitable size, for example, 2, 1.5, etc. For example, the third ratio is used as the second size ratio.

[0147] In some embodiments, an image processing library may be used to adjust the size ratio between the first image and the second image to the first size ratio by interpolation. The image processing library may include, but is not limited to, GIMP and OpenCV. The interpolation method may include, but is not limited to, nearest neighbor interpolation, bilinear interpolation, regional interpolation, bicubic interpolation, etc.

[0148] The specific process of performing image fusion processing on the adjusted first image and second image to obtain the second target image corresponding to the first target image can refer to the specific implementation of step S104.

[0149] In an implementation manner of the present application, the size ratio between the first image and the second image is adjusted to the first size ratio through the first size difference information and the second size difference information, and the adjusted first image and second image are subjected to image fusion processing, so that the fusion of the first image and the second image is more natural, and the accuracy of the fusion processing is improved, thereby improving the display effect of the second target image.

[0150] In some embodiments, step S103 includes steps S131 to S133, wherein:

[0151] Step S131, performing block processing on the first target image to obtain at least one first sub-image.

[0152] Here, since the first target image is directly processed by the second processing strategy, the processing strategy has higher requirements. For example, a larger model or neural network or a more complex algorithm is required. Therefore, the first target image can be processed in blocks to obtain at least one first sub-image, and then each first sub-image is processed separately.

[0153] The first sub-image is a partial image of the first target image. The number of the first sub-image is at least one. In some embodiments, the image size of the first sub-image may be any suitable size, for example, 300x600, 1000x2000, etc. The image size of the first sub-image is smaller than the image size of the first target image.

[0154] In some embodiments, the format of the first sub-image may include, but is not limited to: JPEG, TIFF, BMP, etc. The format of the first target image and the format of the first sub-image may be the same.

[0155] Block processing is the process of dividing an image into several smaller blocks or regions. The implementation methods of block processing may include but are not limited to: using Python and NumPy for block processing, using OpenCV for block processing, etc. For example, when using Python and NumPy for block processing, first determine the image size of the first sub-image, then calculate the total number of rows and columns of the first sub-image, and finally use a loop to traverse the first target image to obtain at least one first sub-image. For another example, when using OpenCV for block processing, cv2.imshow() or other functions can be called to perform block processing on the first target image.

[0156] Step S132 , processing each first sub-image with the third processing strategy to obtain a second sub-image corresponding to the first sub-image.

[0157] Here, the third processing strategy is a strategy for performing high-definition processing on the image. The second sub-image is an image processed by the third processing strategy. The number of the second sub-image is at least one. The image size of the second sub-image can be any suitable size, for example, 2000x4000, 1500x3000, etc.

[0158] In some embodiments, the format of the second sub-image may include, but is not limited to: JPEG, TIFF, BMP, etc. The format of the second sub-image may be the same as the format of the first sub-image.

[0159] In some embodiments, the image size of the first sub-image is smaller than the image size of the second sub-image corresponding to the first sub-image.

[0160] In some embodiments, the third processing strategy may be in any form including but not limited to at least one of the following: an algorithm, a model, a neural network, etc. For example, the third processing strategy may be a model equipped with an image high-definition algorithm, or may be an image high-definition algorithm. As long as an object that can achieve high-definition processing of optimized images can be used as the third processing strategy, the present application does not limit the form of the third processing strategy.

[0161] In some embodiments, the third processing strategy can be integrated into a neural network, and the neural network can be obtained by training an untrained neural network through a training set. When in use, the first sub-image is input to the neural network, and the neural network can output a second sub-image corresponding to the first sub-image.

[0162] Step S133: based on the position of each first sub-image in the first image, stitching processing is performed on each second sub-image to obtain a second image.

[0163] Here, based on the position of each first sub-image in the first image, the second sub-image can be stitched to the position of the first sub-image corresponding to the second sub-image in the first image, so that the image content of the second image is the same as the image content of the first target image.

[0164] In some embodiments, the image size of the second sub-image is smaller than the image size of the second image.

[0165] In some embodiments, the position of the first sub-image in the first image may be represented by any suitable form, such as Cartesian coordinates, polar coordinates, pixel coordinates, etc.

[0166] In some embodiments, the first sub-images and the first image may be input into a target intelligence engine to obtain the position of each first sub-image in the first image. The target intelligence engine may include but is not limited to: a machine learning engine, a GAN engine, a semantic segmentation network, etc.

[0167] Stitching is the process of stitching several smaller images into a larger image. The implementation method of stitching may include, but is not limited to: stitching by using Python and NumPy, stitching by using OpenCV, etc. For example, when stitching using Python and NumPy, first determine the image size of the first image, then create a new blank canvas, the size of which is the same as the image size of the first image, and finally loop through each first sub-image, and according to the position of each first sub-image in the first image, put each first sub-image into the corresponding position in the canvas to stitch the first sub-images to obtain the second image. For another example, when stitching using OpenCV, you can call concatenated_image() or other functions to stitch the first sub-images to obtain the second image.

[0168] In some embodiments, when the first target image is processed in blocks, at least one third sub-image is first obtained, and then each side of the at least one third sub-image is edge-added to obtain at least one first sub-image, and then each first sub-image is processed with the third processing strategy to obtain a second sub-image corresponding to the first sub-image, and finally each second sub-image is spliced ​​to obtain a second image. Since adjacent first sub-images have overlapping areas, adjacent second sub-images also have overlapping areas, and when the second sub-images are spliced, pixel information in the image will not be lost, thereby achieving complete image processing.

[0169] In an embodiment of the present application, the first target image is processed in blocks to obtain at least one first sub-image, and then each first sub-image is processed with a third processing strategy to obtain a second sub-image corresponding to the first sub-image. Finally, based on the position of each first sub-image in the first image, each second sub-image is spliced ​​to obtain a second image. On the one hand, the larger first target image can be divided into smaller first sub-images for parallel processing, thereby improving the efficiency of processing the first target image; on the other hand, since the image size of the first sub-image obtained by the block processing becomes smaller, the requirements for the processing strategy can be further reduced, thereby improving the intelligence of the image processing method while also improving the compatibility of the image processing method.

[0170] The following describes the application of the image processing method provided in the embodiments of the present application in actual scenarios, taking the processing of portrait images as an example.

[0171] Imaging devices can be used to frame biological objects (e.g., people) to produce images. However, when the lighting conditions are poor or there is shaking during shooting, the display effect of the image produced by the imaging device is poor. For example, when taking a group photo of multiple people in certain scenes (e.g., group photos at a dinner party, outdoor group photos), the imaging device is far away from the people and the background. In the captured image, the face of the person will be relatively small and not clear enough, the background will also be blurred, and the image display effect will be poor. Especially when the lighting conditions are poor or there is shaking during shooting, the display effect of the image will be further affected.

[0172] In the related art, the following methods can be used to beautify an image: (a) first modify the overall filter of the image, and then perform local beautification in the area of ​​the face; (b) locally modify the background area of ​​the image; (c) locally beautify the area of ​​the face. However, in the above methods, when locally modifying the background area or the face area, the local modification of the background area or the local modification of the face area will affect the processing effect of the other area, which may cause the other area to be distorted, thereby affecting the display effect of the image, and there are problems such as low accuracy and poor flexibility.

[0173] The embodiment of the present application provides an image processing method, first, a foreground image that meets the target condition is cropped out from a first target image, secondly, the foreground image is processed by a first processing strategy to obtain a first image, and the first target image is processed by a second processing strategy to obtain a second image, wherein the processing effects of the first processing strategy and the second processing strategy are different, and finally, the first image and the second image are fused to obtain a second target image corresponding to the first target image. In this way, on the one hand, since the foreground image is cropped out first, and then the foreground image and the first target image are processed respectively by different processing strategies, the foreground image and the first target image can be processed independently and in parallel, which reduces the demand for the processing strategy capability compared to the processing of the entire image separately, improves the image processing efficiency, and enhances the intelligence of the image processing method; on the other hand, since the processing effects of the first processing strategy and the second processing strategy are different, it is possible to select a suitable processing strategy for different areas of the image, thereby improving the flexibility of the image processing method; on the other hand, the first image and the second image are fused to obtain the second target image, which reduces the possibility of the display effects of different areas of the image affecting each other, thereby improving the accuracy of the image processing method.

[0174] Figure 3 A schematic diagram of an implementation process of an image processing method provided in an embodiment of the present application Figure 2 ,like Figure 3 As shown, the method includes steps S201 to S208, wherein:

[0175] Step S201, obtaining a target area containing a face in a first target image;

[0176] Step S202, if the distance parameter between the biological key feature points of the human face in the target area does not meet the distance threshold, the target area is cropped to obtain a human face image (corresponding to the foreground image mentioned above);

[0177] Step S203, processing the face image by a first neural network (corresponding to the aforementioned first processing strategy) to obtain a first image;

[0178] Step S204, processing the first target image by a second neural network (corresponding to the aforementioned second processing strategy) to obtain a second image;

[0179] Step S205, inputting the first target image into a semantic segmentation network to obtain the position of the face contained in the first target image in the first target image;

[0180] Step S206, obtaining the position of the face contained in the second image in the first target image based on the position of the face contained in the first target image in the first target image;

[0181] Step S207, obtaining pixel fusion weight information corresponding to each pixel point of the first image based on the position of the face contained in the second image in the first image;

[0182] Step S208, based on weight information corresponding to at least one pixel in the first image, the pixel points of the face contained in the second image are merged with the pixel points of the face in the first image to obtain a second target image corresponding to the first target image.

[0183] Figure 4 A schematic diagram of image processing provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, where:

[0184] The first target image 41 is subjected to face detection and face cropping to obtain a face image, and the face image is subjected to face beautification and posture correction to obtain a first image. The first target image 41 is subjected to picture high definition to obtain a second image. The first image and the second image are subjected to face fusion to obtain a second target image 42, and different regions of the first target image are subjected to different processing to improve the display effect of the second target image.

[0185] Figure 5 A schematic diagram of processing a foreground image provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, where:

[0186] The foreground image 51 is input into the first neural network 52 to obtain the first image 53. In the first neural network 52, the facial features of the foreground image 51 are firstly extracted, and then the target features corresponding to the facial features are matched, and finally the face is restored based on the target features to obtain the first image 53, which can beautify the foreground image while processing the foreground image quickly.

[0187] Figure 6 A schematic diagram of processing a first target image provided in an embodiment of the present application is as follows: Figure 6 As shown, where:

[0188] The first target image 61 is input into the second neural network 62 to obtain the second image 63. In the second neural network 62, the image features of the first target image 61 are firstly extracted, and then the image features are subjected to a high-order degradation operation, and finally the image features after the high-order degradation operation at the GAN generator are subjected to the second image 63, which can improve the display effect of the second image while processing the second image quickly.

[0189] Figure 7 A schematic diagram of a fusion process provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, where:

[0190] The face area in the first image 71 is analyzed based on the semantic segmentation network to obtain weight information of the pixel points in the face area in the first image 71, and then the face is fused based on the weight information. Specifically, the first image 71 and the second image 72 are fused based on the weight information to obtain the second target image 73, which can accurately fuse the images.

[0191] Based on the above embodiments, the present application also provides an image processing device. Figure 8 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the data processing device 800 includes a cropping unit 801, a processing unit 802 and a fusion unit 803, wherein:

[0192] A cropping unit 801 is used to crop a foreground image that meets a target condition from a first target image;

[0193] The processing unit 802 is used to process the foreground image with a first processing strategy to obtain a first image; and process the first target image with a second processing strategy to obtain a second image;

[0194] A fusion unit 803 is used to fuse the first image and the second image to obtain a second target image corresponding to the first target image;

[0195] The first processing strategy and the second processing strategy have different processing effects.

[0196] In some embodiments, a foreground image meeting a target condition indicates that the foreground image includes a biological object.

[0197] In some embodiments, the cropping unit 801 is also used to obtain a target area in the first target image that contains a biological object; if the biological parameters of the biological object contained in the target area meet the biological parameter conditions, the target area is cropped to obtain a foreground image; wherein the biological parameters are used to characterize the distance parameters between biological key feature points, and the biological parameters of the biological object contained in the target area meet the biological parameter conditions to characterize that the distance parameters between the biological key feature points do not meet the distance threshold.

[0198] In some embodiments, the processing unit 802 is also used to obtain target parameters that characterize the biological posture of the biological object contained in the foreground image; if the target parameters do not meet the biological posture conditions, the foreground image is processed with a first sub-processing strategy to obtain a first image; if the target parameters meet the biological posture conditions, the foreground image is processed with a second sub-processing strategy to obtain the first image; wherein the first sub-processing strategy includes the first strategy and the second strategy; the second sub-processing strategy includes the first strategy; and the second strategy is used for posture correction.

[0199] In some embodiments, the fusion unit 803 is further used to obtain pixel fusion weight information corresponding to each pixel of the first image based on the position of the biological object contained in the second image in the first image; based on the weight information corresponding to at least one pixel in the first image, the pixel points of the biological object contained in the second image are fused with the pixel points of the biological object in the first image to obtain a second target image corresponding to the first target image.

[0200] In some embodiments, the processing unit 802 is further used to input the first target image into the target intelligence engine to obtain the position of the biological object contained in the first target image in the first target image; based on the position of the biological object contained in the first target image in the first target image, obtain the position of the biological object contained in the second image in the first target image.

[0201] In some embodiments, the fusion unit 803 is further used to determine first size difference information between the first image and the foreground image, and second size difference information between the second image and the first target image; based on the first size difference information and the second size difference information, the size ratio between the first image and the second image is adjusted to the first size ratio, and the adjusted first image and second image are subjected to image fusion processing to obtain a second target image corresponding to the first target image; wherein the first size ratio is the same as the second size ratio, and the second size ratio is the size ratio between the first target image and the foreground image.

[0202] In some embodiments, the processing unit 802 is further used to perform block processing on the first target image to obtain at least one first sub-image; process each first sub-image with a third processing strategy to obtain a second sub-image corresponding to the first sub-image; and splice each second sub-image based on the position of each first sub-image in the first image to obtain a second image.

[0203] Based on the above embodiments, the present application also provides an electronic device, Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Fig. 9 As shown, the electronic device 900 includes a processing unit 901, wherein:

[0204] The processing unit 901 is used to crop a foreground image that meets the target condition from a first target image; process the foreground image with a first processing strategy to obtain a first image; process the first target image with a second processing strategy to obtain a second image; and fuse the first image and the second image to obtain a second target image corresponding to the first target image; wherein the processing effects of the first processing strategy and the second processing strategy are different.

[0205] Here, the first target image refers to an image that needs to be processed. In some implementations, the format of the first target image may include but is not limited to: JPEG, TIFF, BMP, etc.

[0206] The foreground image is a partial image cropped from the first target image. In some embodiments, the number of the foreground image is at least one. In some embodiments, the format of the foreground image may include but is not limited to: JPEG, TIFF, BMP, etc. The format of the first target image and the format of the foreground image may be the same.

[0207] The target condition is a condition used to determine whether to crop the foreground image. The target condition may include but is not limited to: the image area needs to optimize details, the image area has distortion, etc.

[0208] The method of cropping the foreground image may include but is not limited to: cropping the foreground image in the first target image through a detection frame, cropping the foreground image in the first target image through deep learning, etc.

[0209] The first processing strategy is a strategy for optimizing image details. The first image is an image processed by the first processing strategy. In some embodiments, the number of the first images is at least one. The number of the first images is the same as the number of the foreground images.

[0210] The second processing strategy is a strategy for performing high-definition processing on an image. The second image is an image processed by the second processing strategy. In some embodiments, the number of the second image is at least one. The number of the second images is the same as the number of the first target images.

[0211] The second target image refers to a processed image. In some embodiments, the format of the second target image may include but is not limited to: JPEG, TIFF, BMP, etc. The format of the first target image may be the same as the format of the first target image.

[0212] The fusion processing method of the first image and the second image may include but is not limited to: fusing the first image and the second image based on pixels, fusing the first image and the second image based on features, fusing the first image and the second image based on a transform domain, etc.

[0213] In the embodiment of the present application, the electronic device firstly cuts out the foreground image that meets the target condition from the first target image, then processes the foreground image with the first processing strategy to obtain the first image, and processes the first target image with the second processing strategy to obtain the second image, wherein the processing effects of the first processing strategy and the second processing strategy are different, and finally, the first image and the second image are fused to obtain the second target image corresponding to the first target image. In this way, on the one hand, since the foreground image is firstly cut out, and then the foreground image and the first target image are respectively processed by different processing strategies, the foreground image and the first target image can be processed independently and in parallel, which reduces the demand for the processing strategy capability compared to the whole image being processed separately, improves the efficiency of the electronic device in processing the image, and enhances the intelligence of the electronic device; on the other hand, since the processing effects of the first processing strategy and the second processing strategy are different, it is possible to select a suitable processing strategy for different areas of the image, thereby improving the flexibility of the electronic device; on the other hand, the first image and the second image are fused to obtain the second target image, which reduces the possibility of the display effects of different areas of the image affecting each other, thereby improving the accuracy of the second target image.

[0214] The description of the electronic device embodiment above is similar to the description of the method embodiment above, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the electronic device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0215] It should be noted here that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. The description of the above device, storage medium, computer program and computer program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiments of the device, storage medium, computer program and computer program product of this application, please refer to the description of the method embodiment of this application for understanding.

[0216] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0217] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor implements the above method when executing the computer program.

[0218] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and the computer program implements the above method when executed by a processor. The computer-readable storage medium may be transient or non-transient.

[0219] The embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be implemented specifically by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.

[0220] It should be noted that Fig.10A schematic diagram of a hardware entity of an electronic device provided in an embodiment of the present application is shown in the figure. As shown in the figure, the hardware entity of the electronic device 1000 includes: a processor 1001, a communication interface 1002 and a memory 1003, wherein:

[0221] The processor 1001 generally controls the overall operation of the electronic device 1000 .

[0222] The communication interface 1002 enables the electronic device to communicate with other terminals or servers through a network.

[0223] The memory 1003 is configured to store instructions and applications executable by the processor 1001, and can also cache data to be processed or processed by the processor 1001 and each module in the electronic device 1000 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented by flash memory (FLASH) or random access memory (Random Access Memory, RAM). Data transmission can be performed between the processor 1001, the communication interface 1002, and the memory 1003 through the bus 1004.

[0224] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0225] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the serial number of each step / process mentioned above does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The serial numbers of the embodiments of the present application mentioned above are for description only and do not represent the advantages and disadvantages of the embodiments.

[0226] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0227] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0228] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0229] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0230] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, read-only memories, magnetic disks or optical disks.

[0231] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0232] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. An image processing method, the method comprising: Cut out a foreground image that meets the target condition from the first target image; Processing the foreground image with a first processing strategy to obtain a first image; Processing the first target image with a second processing strategy to obtain a second image; Fusing the first image and the second image to obtain a second target image corresponding to the first target image; The first processing strategy and the second processing strategy have different processing effects.

2. The method according to claim 1, A foreground image that meets the target condition indicates that the foreground image contains a biological object.

3. The method according to claim 2, wherein the step of cutting out a foreground image that meets the target condition from the first target image comprises: Acquire a target area containing a biological object in the first target image; If the biological parameters of the biological object contained in the target area meet the biological parameter conditions, cropping the target area to obtain the foreground image; The biological parameter is used to characterize the distance parameter between biological key feature points, and the biological parameter of the biological object contained in the target area satisfies the biological parameter condition, which indicates that the distance parameter between the biological key feature points does not meet the distance threshold.

4. The method according to claim 2, wherein the step of processing the foreground image with a first processing strategy to obtain a first image comprises: Obtaining target parameters representing the biological posture of the biological object contained in the foreground image; If the target parameter does not satisfy the biological posture condition, processing the foreground image with a first sub-processing strategy to obtain the first image; If the target parameter satisfies the biological posture condition, processing the foreground image with a second sub-processing strategy to obtain the first image; The first sub-processing strategy includes a first strategy and a second strategy; the second sub-processing strategy includes the first strategy; and the second strategy is used for posture correction.

5. The method according to claim 2, wherein the fusing the first image and the second image to obtain a second target image corresponding to the first target image comprises: Based on the position of the biological object contained in the second image in the first image, obtaining pixel fusion weight information corresponding to each pixel point of the first image; Based on weight information corresponding to at least one pixel in the first image, the pixel of the biological object contained in the second image is fused with the pixel of the biological object in the first image to obtain the second target image corresponding to the first target image.

6. The method according to claim 5, further comprising: Inputting the first target image into a target intelligent engine to obtain a position of a biological object contained in the first target image in the first target image; Based on the position of the biological object contained in the first target image, the position of the biological object contained in the second image in the first target image is obtained.

7. The method according to claim 1, wherein the fusing the first image and the second image to obtain a second target image corresponding to the first target image comprises: Determine first size difference information between the first image and the foreground image, and second size difference information between the second image and the first target image; Based on the first size difference information and the second size difference information, adjusting the size ratio between the first image and the second image to a first size ratio, and performing image fusion processing on the adjusted first image and second image to obtain the second target image corresponding to the first target image; The first size ratio is the same as the second size ratio, and the second size ratio is the size ratio between the first target image and the foreground image.

8. The method according to any one of claims 1 to 7, wherein the step of processing the first target image with a second processing strategy to obtain a second image comprises: Performing block processing on the first target image to obtain at least one first sub-image; Processing each of the first sub-images using a third processing strategy to obtain a second sub-image corresponding to the first sub-image; Based on the position of each of the first sub-images in the first image, each of the second sub-images is spliced ​​to obtain the second image.

9. An image processing device, comprising: A cropping unit, used for cropping a foreground image that meets the target condition from the first target image; A processing unit, configured to process the foreground image using a first processing strategy to obtain a first image; Processing the first target image with a second processing strategy to obtain a second image; a fusion unit, configured to fuse the first image and the second image to obtain a second target image corresponding to the first target image; The first processing strategy and the second processing strategy have different processing effects.

10. An electronic device comprising: A processing unit, used for cutting out a foreground image that meets the target condition from the first target image; Processing the foreground image with a first processing strategy to obtain a first image; The first target image is processed with a second processing strategy to obtain a second image; the first image and the second image are fused to obtain a second target image corresponding to the first target image; wherein the processing effects of the first processing strategy and the second processing strategy are different.

Images