Method and electronic device for eliminating reflections in images

Abstract

The present application relates to the field of terminal equipment technology, and specifically to a method and electronic device for eliminating reflections in images. The method for eliminating reflections in images includes: obtaining a basic image; wherein the basic image includes a first basic image and a second basic image, the first basic image is an image in a first direction, and the second basic image is an image in a second direction; the first direction and the second direction are opposite directions; extracting a first image from the first basic image, the grayscale value of the first image is higher than the first grayscale value, and the first grayscale value is calculated and determined based on the grayscale value of the first basic image; extracting a second image from the second basic image, the grayscale value of the second image is higher than the second grayscale value, and the second grayscale value is calculated and determined based on the grayscale value of the second basic image; determining the same image contained in the first image and the second image as a third image; clearing the third image in the first image to obtain a first target image.

Description

Technical Field

[0001] The present application relates to the technical field of terminal devices, and in particular to a method and electronic device for eliminating reflections in images. Background Art

[0002] At present, more and more users use mobile phones and other electronic devices to take pictures of the beautiful scenery along the way. When riding in a train or car, if you want to take pictures of the beautiful scenery outside the window, you need to open the window and use a mobile phone or other electronic device to take pictures. Otherwise, the pictures taken will reflect the scenery inside the vehicle, making the pictures unclear and the scenery that users want to take pictures cannot be perfectly presented. However, if the windows of the vehicle cannot be opened, such as high-speed trains, it is impossible to solve the problem of glass reflection affecting photography by opening the windows.

[0003] In order to reduce the impact of glass reflection on the photos taken, in the related technology, a sunshade is installed on the camera of electronic devices such as mobile phones to reduce the light inside the vehicle from being transmitted to the window and then reflected to the camera of electronic devices such as mobile phones, thereby reducing the problem of glass reflection when directly shooting the scenery outside the window without opening the window.

[0004] In the process of implementing relevant technologies, at least the following technical problems are found:

[0005] If relevant technology is used to reduce the glass reflection problem when directly photographing the scenery outside the window without opening the window, it is necessary to carry a lens hood with you, and put the lens hood on the camera of electronic devices such as mobile phones before taking pictures. The process is rather cumbersome. Summary of the invention

[0006] The embodiments of the present application provide a method and an electronic device for eliminating reflections in an image. The method determines a reflection image in the first basic image by combining a first basic image and a second basic image acquired in opposite directions, and then eliminates the reflection image in the first basic image, thereby eliminating reflections in the image, reducing the problem of glass reflections when a user uses an electronic device to directly shoot the scenery outside the window without opening the window, and does not require other external structures for shading, thereby improving the user experience.

[0007] In some embodiments, a method for eliminating reflections in an image is provided, comprising: acquiring a base image; wherein the base image comprises a first base image and a second base image, the first base image is an image in a first direction, and the second base image is an image in a second direction; the first direction and the second direction are opposite directions; extracting a first image from the first base image, wherein a grayscale value of the first image is higher than a first grayscale value, and the first grayscale value is determined by calculation based on the grayscale value of the first base image; extracting a second image from the second base image, wherein a grayscale value of the second image is higher than a second grayscale value, and the second grayscale value is determined by calculation based on the grayscale value of the second base image; determining the same image contained in the first image and the second image as a third image; clearing the third image in the first image to obtain a first target image.

[0008] By adopting the method for eliminating reflections in images provided in the embodiment of the present application, reflections in the first basic image taken by the first camera are eliminated by combining the images taken by the first camera and the second camera, thereby reducing the impact of reflections on the photos taken by users and improving the user experience.

[0009] Optionally, acquiring the basic image includes: acquiring the basic image in response to an instruction to start the elimination function.

[0010] In this embodiment, through the elimination function start instruction, it is determined that the user needs to enter the function of eliminating reflections in the image taken by the first camera, and then the method provided in the embodiment of the present application is used to eliminate the reflections in the image taken by the first camera, thereby improving the user experience.

[0011] Optionally, extracting the first image from the first basic image includes: grayscale processing the first basic image to obtain the grayscale first basic image; dividing the grayscale first basic image into multiple regions according to a first preset rule; calculating the grayscale average value corresponding to each region as multiple first grayscale values; and determining the first image based on the grayscale first basic image and the first grayscale value.

[0012] In this embodiment, by calculating the grayscale average of each area as the first grayscale value, the influence of uneven grayscale of the captured image (first basic image) caused by uneven light inside and outside the vehicle compartment on the extraction of the first image is reduced, thereby reducing the influence on the elimination effect. Specifically, the portion with higher brightness in the first basic image is the portion formed by the reflection of the vehicle window glass.

[0013] Optionally, based on the first basic image after grayscale and the first grayscale value, the first image is determined, including: in the first basic image after grayscale, each pixel point is taken as the first target pixel point in turn; the difference between the grayscale value of the first target pixel point and the first target grayscale value is calculated to obtain the first image; wherein the first target grayscale value is the first grayscale value corresponding to the area where the first target pixel point is located.

[0014] In this embodiment, the area with higher grayscale value (higher brightness value) is highlighted, so as to determine the reflective image formed by the reflection of the window glass according to the first image. Specifically, in the image, the larger the grayscale value of a point is, the higher the brightness of the point is.

[0015] Optionally, extracting the second image from the first basic image includes: grayscale processing the second basic image to obtain a grayscale second basic image; dividing the grayscale second basic image into multiple regions according to a second preset rule; calculating the grayscale average value corresponding to each region as multiple second grayscale values; and determining the second image based on the grayscale second basic image and the second grayscale value.

[0016] In this embodiment, by calculating the grayscale average of each area as the second grayscale value, the influence of uneven grayscale of the captured image (second basic image) caused by uneven light inside and outside the vehicle cabin on the extraction of the second image is reduced, thereby reducing the influence on the elimination effect. Specifically, the brighter the area in the second basic image, the more likely it is to be reflected by the window glass and become a reflective feature in the first basic image.

[0017] Optionally, based on the grayscaled second basic image and the second grayscale value, a second image is determined, including: in the grayscaled second basic image, each pixel is taken as a second target pixel in turn; the difference between the grayscale value of the second target pixel and the second target grayscale value is calculated to obtain a second image; wherein the second target grayscale value is the second grayscale value corresponding to the area where the second target pixel is located.

[0018] In this embodiment, the area with higher grayscale value (higher brightness value) is highlighted, so as to determine the area image with higher brightness in the vehicle compartment according to the second image, that is, the area image with higher probability of being a reflective image.

[0019] Optionally, determining the same image contained in the first image and the second image as the third image includes: extracting an image of a first sub-region in the first image; wherein the grayscale value of the image of the first sub-region is greater than that of images of other regions in the first image; extracting an image of a second sub-region in the second image; wherein the grayscale value of the image of the second sub-region is greater than that of images of other regions in the second image; and determining the same image in the image of the first sub-region and the image of the second sub-region as the third image.

[0020] In this embodiment, the third image is determined by respectively extracting the image of the first sub-region and the image of the second sub-region, thereby reducing the amount of calculation in determining the third image and improving the operation speed.

[0021] Optionally, extracting an image of a first sub-region in a first image includes: extracting a set of pixel points of the first sub-region in the first image as a first pixel point set; wherein each pixel point in the first pixel point set constitutes the image of the first sub-region.

[0022] In this embodiment, the shape of the reflective feature in the first image (such as the interior of the vehicle or the figure of the user, etc.) is determined by extracting the image of the first sub-area. Specifically, the first pixel point set includes pixel points of the reflective feature.

[0023] Optionally, extracting a pixel point set of a first sub-area in a first image includes: in the first image, taking each pixel point as a third target pixel point in turn; calculating a grayscale value difference between the third target pixel point and a fourth target pixel point to obtain a plurality of first grayscale differences; wherein the fourth target pixel point is a pixel point within a preset range of the third target pixel point; in the event that the first grayscale difference is greater than a grayscale difference threshold, adding the third target pixel point to a first edge pixel point set and a pixel point set of the first sub-area; and adding all pixels in an area formed by connecting all pixels in the first edge pixel point set to the pixel point set of the first sub-area.

[0024] In this embodiment, the edge of the first sub-area is first defined, and then the pixel points within the area enclosed by the edge are determined to obtain all the pixel points of the first sub-area, so that the same image in the first sub-area and the second sub-area can be determined through the pixel points of the grayscale value of the pixel points.

[0025] Optionally, extracting the image of the second sub-region in the second image includes: extracting a set of pixel points of the second sub-region in the second image as a second pixel point set; wherein each pixel point in the second pixel point set constitutes the image of the second sub-region.

[0026] In this embodiment, the shape of the highlight area in the second image (eg, the interior of the vehicle or the figure of the user, etc.) is determined by extracting the image of the second sub-area. Specifically, the second pixel point set includes pixel points with reflective features.

[0027] Optionally, extracting a pixel point set of the second sub-area in the second image includes: in the second image, taking each pixel point as the fifth target pixel point in turn; calculating the grayscale value difference between the fifth target pixel point and the sixth target pixel point to obtain multiple second grayscale differences; wherein the sixth target pixel point is a pixel point within a preset range of the fifth target pixel point; in the event that the second grayscale difference is greater than a grayscale difference threshold, adding the fifth target pixel point to the second edge pixel point set and the pixel point set of the second sub-area; and adding all the pixels in the area formed by connecting all the pixels in the second edge pixel point set to the pixel point set of the second sub-area.

[0028] In this embodiment, the edge of the second sub-area is first delineated, and then the pixel points within the area enclosed by the edge are determined to obtain all the pixel points of the second sub-area, so that the same image in the first sub-area and the second sub-area can be determined through the pixel points of the grayscale value of the pixel points.

[0029] Optionally, the first pixel point set includes all pixel points of the image of the first sub-area, and the second pixel point set includes all pixel points of the image of the second sub-area; determining the same image in the image of the first sub-area and the image of the second sub-area as the third image includes: determining pixel points with similar grayscale values ​​between the first pixel point set and the second pixel point set, and adding pixel points with similar grayscale values ​​to the similar pixel point set; wherein the similar pixel point set includes all pixel points of the third image.

[0030] In this embodiment, pixels with similar grayscale values ​​are pixels with similar brightness. Since the first pixel set only includes pixels in the reflective area, and the second pixel set only includes pixels in the area with greater brightness, the grayscale values ​​of pixels with similar corresponding positions indicate that the image represented by the pixel at that position in the first sub-area is the same as the image represented by the pixel at that position in the image of the second sub-area. Thus, all pixels with similar grayscale values ​​(all pixels in the similar pixel set) constitute the third image.

[0031] Optionally, pixels with similar grayscale values ​​between the first pixel point set and the second pixel point set are determined, and the pixels with similar grayscale values ​​are added to the similar pixel point set, including: calculating the similarity between the first grayscale value and the second grayscale value; wherein the points in the first pixel point set include the first pixel point, and the grayscale value of the first pixel point is the first grayscale value; the second pixel point set includes the second pixel point, and the grayscale value of the second pixel point corresponding to the position of the first pixel point is the second grayscale value; and the second pixel point with a similarity greater than or equal to a similarity threshold is regarded as a similar pixel point.

[0032] In this embodiment, the second pixel point whose similarity is greater than or equal to the similarity threshold, that is, the second pixel point and the first pixel point corresponding to the position of the second pixel point represent the same image, so as to obtain a complete identical image, thereby facilitating the elimination of image reflection.

[0033] Optionally, calculating the similarity between the first grayscale value and the second grayscale value includes: calculating a first ratio between the first grayscale value and the second grayscale value, and a second ratio between the second grayscale value and the first grayscale value; and taking the ratio of the first ratio and the second ratio that is less than or equal to 1 as the similarity between the first grayscale value and the second grayscale value.

[0034] In this embodiment, since the similarity ratio is less than or equal to 1, when the magnitude relationship between the first grayscale value and the second grayscale value is uncertain, respectively calculating the first ratio and the second ratio can determine a similarity less than or equal to 1.

[0035] Optionally, eliminating the third image in the first image to obtain the first target image includes: performing grayscale processing on the first image to obtain the grayscale first image; performing grayscale processing on the third image to obtain the grayscale third image; eliminating the grayscale third image in the grayscale first image to obtain the grayscale first target image; converting the grayscale first target image into a color image to obtain the first target image.

[0036] In this embodiment, the third image in the first image is eliminated based on the grayscale value of each pixel of the image, thereby obtaining the first target image, that is, the image after the reflection in the first basic image is eliminated.

[0037] Optionally, eliminating the grayscale third image in the grayscale first image to obtain the grayscale first target image, including: extracting the grayscale matrix of the grayscale first image as the first matrix; extracting the grayscale matrix of the grayscale third image as the second matrix; calculating the difference between the first matrix and the second matrix to obtain a first difference matrix; in the first difference matrix, the grayscale value of each pixel point is added to the target grayscale average value to obtain the grayscale matrix corresponding to the first target image; wherein the target grayscale average value is the first grayscale value corresponding to the area where the pixel point is located; and converting the grayscale matrix corresponding to the first target image into the grayscale first target image.

[0038] In this embodiment, the first difference matrix obtained is the grayscale difference between each pixel in the first image and each pixel in the corresponding position in the third image, that is, the grayscale value of each pixel of the first target image is obtained, and then the first target image can be determined, that is, the image with the reflection of the first basic image eliminated.

[0039] Optionally, after obtaining the first target image, the method for eliminating reflections in the image also includes: obtaining the model of the electronic device; wherein the electronic device is configured to obtain a basic image; determining the electronic device image corresponding to the model of the electronic device; identifying whether the electronic device image exists in the first target image through AI technology; if the electronic device image exists in the first target image, eliminating the electronic device image in the first target image to obtain the second target image.

[0040] In this embodiment, the reflective features of the electronic device are eliminated to further reduce the reflective image in the first basic image, thereby reducing the impact of the reflection of the car window glass on the user's shooting of the scenery outside the window, thereby improving the user experience.

[0041] Optionally, determining the electronic device image corresponding to the model of the electronic device includes: determining a grayscale matrix corresponding to the electronic device image of the model of the electronic device as a third grayscale matrix.

[0042] In this embodiment, the third grayscale matrix is ​​determined to facilitate calculation of the grayscale value of the second target image, thereby facilitating determination of the second target image.

[0043] Optionally, eliminating the image of the electronic device in the first target image to obtain an image of the electronic device with reflections eliminated includes: calculating each pixel in the first target image minus the first grayscale value to obtain a second difference matrix; calculating the difference between the second difference matrix and the third grayscale matrix to obtain a grayscale matrix of the second target image; converting the grayscale matrix of the second target image into a grayscale image to obtain a grayscale second target image; and converting the grayscale second target image into a color image to obtain a second target image.

[0044] In this embodiment, the second difference matrix and the third grayscale matrix are used to determine whether the remaining reflected image in the first target image includes the electronic device image. In the case where the remaining reflected image in the first target image includes the electronic device image, the reflected image of the electronic device in the first target image is removed by subtracting the grayscale matrices to further eliminate the reflection in the image.

[0045] Optionally, eliminating the image of the electronic device in the first target image to obtain an image of the electronic device with reflections eliminated includes: calculating the difference between the grayscale matrix of the first target image and the third grayscale matrix to obtain the grayscale matrix of the second target image; converting the grayscale matrix of the second target image into a grayscale image to obtain a grayscale second target image; converting the grayscale second target image into a color image to obtain the second target image.

[0046] In this embodiment, the grayscale matrix of the second target image is determined by calculating the difference between the grayscale matrix of the first target image and the third grayscale matrix, and then the grayscale second target image is determined, thereby obtaining the second target image. This further eliminates the reflected image in the first basic image, reduces the impact of the reflection of the car window glass on the user's shooting of the scenery outside the window, and improves the user experience.

[0047] Optionally, before acquiring the basic image, the method for eliminating reflections in the image also includes: responding to a photography function start-up signal, identifying whether there is a reflective scene in the preview screen of the photography function; if there is a reflective scene, determining whether there is glass reflection in the captured image, and issuing a prompt whether to enter the elimination function.

[0048] In this embodiment, through scene recognition, it is determined whether there is a highlight area in the preview image, and then whether there is a reflective scene. In the case of a reflective scene, the user is prompted to eliminate the reflection, thereby improving the user's experience.

[0049] Optionally, identifying whether there is a reflective scene in the preview picture of the photography function includes: dividing the preview picture into multiple areas according to a third preset rule; and determining that there is a reflective scene when there is a highlight area among the multiple areas.

[0050] In this embodiment, the highlight area indicates that the brightness of the area is more prominent than the brightness of other areas, that is, the area is brighter and has a larger brightness difference than other areas. In addition, in the preview screen of the scenery outside the window taken by the user using the electronic device, the area with higher brightness is generally the area formed by the reflection of the car window glass, or there is a light source outside the car window. Therefore, when there is a highlight area, it is determined that there is a reflective scene.

[0051] Optionally, the presence of a highlight area in multiple areas is determined according to the following method, including: grayscale the preview image to obtain a grayscale preview image; calculate the grayscale mean of each area in the grayscale preview image; calculate the grayscale difference between the grayscale mean of each area and the grayscale mean of each remaining area; when there is a grayscale difference greater than a grayscale difference threshold, determine that a highlight area exists.

[0052] In this embodiment, by comparing the grayscale average of each area with the grayscale average of other areas, it is determined whether there is a highlighted area in the preview image, that is, whether the grayscale average of an area is higher, that is, the brightness of this area is significantly higher than that of other areas, thereby improving the accuracy of scene recognition.

[0053] In some embodiments, an electronic device is provided, including: a processor, a memory, and a touch screen; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes any one of the methods for eliminating reflections in images described in the above embodiments.

[0054] In some embodiments, a computer-readable storage medium is provided, including computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes any one of the methods for eliminating reflections in an image described in the above embodiments.

[0055] In some embodiments, a computer program product is provided. When the computer program product is run on a computer, the computer is caused to execute the method for eliminating reflections in an image described in any one of the above embodiments.

[0056] It can be understood that the beneficial effects that can be achieved by the electronic device, computer-readable storage medium and computer program product provided above can be referred to the beneficial effects of the method for eliminating reflections in an image and any optional implementation thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0058] Figure 1 is a schematic diagram of an electronic device provided by an embodiment of the present application from one viewing angle;

[0059] Figure 2 is a schematic diagram of an electronic device provided by an embodiment of the present application from another perspective;

[0060] Figure 3 is a schematic diagram of an implementation environment of a method for eliminating reflections in an image provided by an embodiment of the present application;

[0061] Figure 4 is a schematic diagram of a brief process of a method for eliminating reflections in an image provided by an embodiment of the present application;

[0062] Figure 5 is a schematic diagram of the system structure of an electronic device provided in an embodiment of the present application;

[0063] Figure 6 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0064] Figure 7 is a flow chart of a method for eliminating reflections in an image provided by an embodiment of the present application;

[0065] Figure 8 is a principle diagram of image reflection provided by an embodiment of the present application;

[0066] Fig. 9 is a schematic diagram of a first basic image provided in an embodiment of the present application;

[0067] Fig.10 is a schematic diagram of a second basic image provided in an embodiment of the present application;

[0068] Fig.11 is a schematic diagram of a process for extracting a first image provided in an embodiment of the present application;

[0069] Fig.12 is a schematic diagram of dividing a grayscale first basic image into multiple regions provided by an embodiment of the present application;

[0070] Fig.13 is a data flow diagram about a first basic image provided by an embodiment of the present application;

[0071] Fig.14 is a schematic diagram of a process for extracting a second image provided in an embodiment of the present application;

[0072] Fig.15 is a schematic diagram of dividing the grayscale second basic image into multiple regions provided by an embodiment of the present application;

[0073] Fig.16 is a data flow diagram of a second basic image provided in an embodiment of the present application;

[0074] Fig.17 is a schematic diagram of a process for determining the same images contained in a first image and a second image provided by an embodiment of the present application;

[0075] Fig.18 is a grayscale matrix schematic diagram of a third image provided in an embodiment of the present application;

[0076] Fig.19 is a schematic diagram of a process for obtaining a first target image provided by an embodiment of the present application;

[0077] Fig. 20 The embodiment of the present application provides a data flow diagram for calculating the difference between the first matrix and the second matrix to obtain the first difference matrix;

[0078] Fig.21 is a flow chart of another method for eliminating reflections in an image provided by an embodiment of the present application;

[0079] Fig. 22 is a data flow diagram for obtaining a grayscale matrix of a second target image provided by an embodiment of the present application;

[0080] Fig.23 is another data flow diagram for obtaining the grayscale matrix of the second target image provided by an embodiment of the present application;

[0081] Fig.24 is a flow chart of another method for eliminating reflections in an image provided by an embodiment of the present application;

[0082] Fig.25 is a schematic diagram of the preview screen area division provided in an embodiment of the present application;

[0083] Fig.26 It is a schematic diagram of the operation of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0084] The technical solutions in the embodiments of the present application will be described clearly below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments of the present application, other embodiments obtained by ordinary technicians in this field without making creative work all belong to the protection scope of the present application.

[0085] In the following, the terms "first", "second", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0086] In addition, in the present application, directional terms such as "upper", "lower", "inner" and "outer" are defined relative to the orientation of the components schematically placed in the drawings. It should be understood that these directional terms are relative concepts. They are used for relative description and clarification, and they can change accordingly according to the changes in the orientation of the components placed in the drawings.

[0087] Figure 1 It is a schematic diagram of an electronic device provided in an embodiment of the present application from one perspective.

[0088] Figure 2 It is a schematic diagram of the electronic device provided in an embodiment of the present application from another perspective.

[0089] Combination Figure 1 and Figure 2 As shown, the electronic device includes: a first camera 11 and a second camera 12. The first camera 11 is configured to capture a first direction ( Figure 1 and Figure 2The second camera 12 is configured to capture images in the second direction ( Figure 1 and Figure 2 The first direction and the second direction are opposite directions.

[0090] Specifically, the electronic device further includes: an electronic device body 13. A first camera 11 and a second camera 12 are respectively arranged on two opposite sides of the electronic device body 13. Exemplarily, the first camera 11 is a rear camera of the electronic device, and the second camera 12 is a front camera of the electronic device.

[0091] It should be understood that Figure 1 and Figure 2 The electronic device may be a mobile phone, a tablet computer, a wearable device, etc. As long as the electronic device can obtain images in the first direction and images in the second direction, the embodiment of the present application does not limit the specific form of the electronic device.

[0092] Figure 3 It is a schematic diagram of the implementation environment of the method for eliminating reflections in an image provided in an embodiment of the present application.

[0093] For example, Figure 3 In the implementation environment diagram in , user 70 holds electronic device 100 to take photos in a high-speed train compartment. User 70 wants to use the first camera (rear camera, not shown) of electronic device 100 to take photos of the scenery outside the window. However, due to the high brightness inside the compartment, user 70 and some compartment interior 80 inside the window are reflected by the window glass and are also captured by the first camera of electronic device 100. Therefore, the image captured by user 70 using the first camera of electronic device 100 includes not only the scenery outside the window, but also the reflected image of user 70 and compartment interior 80 inside the window.

[0094] Figure 4 It is a brief flowchart of a method for eliminating reflections in an image provided in an embodiment of the present application.

[0095] Combination Figure 4 As shown, the method for eliminating reflections in an image provided by an embodiment of the present application includes an image A1 captured by a first camera, which includes reflection features (image features formed by glass reflection, such as users in the window, interior decoration of a vehicle compartment, etc.) and target features (image features that a user wants to capture, such as scenery outside the window), and an image B1 including non-target features (users in the window and interior decoration of a vehicle compartment, etc.). By eliminating B1 in image A1, A2 is obtained, and the non-target features formed by the reflection of the vehicle window glass in image A1 are eliminated, thereby eliminating the reflections.

[0096] Furthermore, when taking photos, users often hold electronic devices, and the electronic devices will also be captured by the first camera due to reflections from the car window glass, so the reflective features in image A1 may also include the electronic devices. The present application can also obtain the model of the electronic device, determine the image B2 of the electronic device, and then identify whether there is an image of the electronic device in image A2, that is, determine whether the reflective features in image A1 include the electronic device. In the case where there is an image of the electronic device in image A2, image B2 in A2 is eliminated to obtain the final image A3 with the reflection eliminated.

[0097] Figure 5 It is a schematic diagram of the system structure of the electronic device provided in the embodiment of the present application.

[0098] Combination Figure 5 As shown, the layered architecture divides the system into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the system is divided into five layers, from top to bottom, namely, the application layer, the application framework layer, the hardware abstraction layer, the driver layer, and the hardware layer.

[0099] The application layer may include a series of application packages. In the embodiment of the present application, the application package may include a camera, a gallery, etc.

[0100] The application framework layer provides an application programming interface (API) and a programming framework for the application of the application layer. The application framework layer includes some predefined functions. In an embodiment of the present application, the application framework layer may include a camera access interface, wherein the camera access interface may include camera management and camera devices. The camera access interface is used to provide an application programming interface and a programming framework for camera applications.

[0101] The hardware abstraction layer is an interface layer between the application framework layer and the driver layer, and provides a virtual hardware platform for the operating system. In the embodiment of the present application, the hardware abstraction layer may include a camera hardware abstraction layer and a camera algorithm library.

[0102] The camera hardware abstraction layer may provide virtual hardware for camera device 1, camera device 2 or more camera devices. The camera algorithm library may include operating codes and data for implementing the method for eliminating reflections in images provided in the embodiment of the present application.

[0103] The driver layer is a layer between hardware and software. The driver layer includes drivers for various hardware. The driver layer may include camera device drivers, digital signal processor drivers, and image processor drivers.

[0104] The camera device driver is used to drive the camera sensor to collect images and drive the image signal processor to pre-process the images. The digital signal processor driver is used to drive the digital signal processor to process images. The image processor driver is used to drive the graphics processor to process images.

[0105] In combination with the above system structure, the method for eliminating reflections in an image in the embodiment of the present application is described in detail below:

[0106] In response to the user's operation of opening the camera application, such as clicking the camera application icon, the camera application calls the camera access interface of the application framework layer to start the camera application, and then sends an instruction to start the camera by calling the camera device (camera device 1 and camera device 2) in the camera hardware abstraction layer. The camera hardware abstraction layer sends the instruction to the camera device driver of the driver layer. The camera device driver can start the corresponding camera sensor (sensor 1 and sensor 2) and collect image light signals through the sensor. A camera device in the camera hardware abstraction layer corresponds to a camera sensor in the hardware layer.

[0107] Then, the camera sensor can transmit the collected image light signal to the image signal processor for preprocessing to obtain image electrical signals (first basic image and second basic image), and transmit the above first basic image and second basic image to the camera hardware abstraction layer through the camera device driver.

[0108] The camera hardware abstraction layer can send the basic image to the camera algorithm library. The camera algorithm library stores the program code for implementing the method for eliminating reflections in the image provided by the embodiment of the present application. Based on the digital signal processor and the image processor driver, the camera algorithm library executes the above code to achieve the above-mentioned capabilities such as eliminating image reflections.

[0109] The camera algorithm library may process the acquired first basic image and the second basic image comprehensively to obtain an image with the reflection eliminated. The camera algorithm library may send the image with the reflection eliminated to the camera hardware abstraction layer. Then, the camera hardware abstraction layer may send it for display.

[0110] Figure 6 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application.

[0111] like Figure 6 As shown, the electronic device may include a processor 110, an internal memory 121, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a camera 193, and a display screen 194, etc. The sensor module 180 may include a touch sensor 180K, etc.

[0112] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0113] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0114] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0115] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0116] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0117] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation of the electronic device. In other embodiments of the present application, the electronic device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0118] The electronic device implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0119] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0120] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals.

[0121] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to electronic devices. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1.

[0122] The wireless communication module 160 can provide wireless communication solutions for electronic devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and filters it, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0123] In some embodiments, antenna 1 of the electronic device is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that the electronic device can communicate with the network and other devices through wireless communication technology.

[0124] The electronic device implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0125] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD). The display panel can also be made of an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), miniled, microled, micro-oled, a quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0126] In the embodiment of the present application, the electronic device displays the image after the reflection is eliminated and the preview screen of the camera application, and Fig.26 (a), (b), (c), (d), and (e) rely on the display functions provided by the above-mentioned GPU, display screen 194, and application processor.

[0127] The electronic device can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0128] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0129] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device may include 2 or more than N cameras 193, where N is a positive integer greater than 2.

[0130] In the embodiment of the present application, the electronic device implements the method for eliminating reflections in an image provided in the embodiment of the present application, which firstly relies on the ISP and the image captured by the camera 193, and secondly relies on the image calculation and processing capabilities provided by the GPU, so as to capture the first basic image and the second basic image, and eliminate the reflections in the first basic image.

[0131] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0132] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.

[0133] Non-volatile memory can include disk storage devices and flash memory. Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle, single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the potential level of the storage cell, and universal flash storage (UFS), embedded multi media card (eMMC), etc. according to the storage specification.

[0134] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data, etc. The non-volatile memory can also store executable programs and store user and application data, etc., and can be loaded into the random access memory in advance for direct reading and writing by the processor 110.

[0135] In the embodiment of the present application, the code for implementing the method for eliminating reflections in an image described in the embodiment of the present application may be stored in a non-volatile memory. When running a camera application, the electronic device may load the executable code stored in the non-volatile memory into a random access memory. The electronic device may implement audio functions through the audio module 170. For example, voice-controlled photography, etc.

[0136] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device, which is different from the position of the display screen 194.

[0137] In the embodiment of the present application, the electronic device can use the touch sensor 180K to detect the user's click, slide and other operations on the display screen 194 to achieve Fig.26 The operations shown in (a), (d) and (e).

[0138] For ease of understanding, the method provided in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0139] Figure 7 It is a flowchart of a method for eliminating reflections in an image provided in an embodiment of the present application.

[0140] Combination Figure 7 As shown, the embodiment of the present application provides a method for eliminating reflections in an image. The execution subject of the method may be an electronic device. The method includes steps S100 to S500, which are specifically as follows:

[0141] Step S100, obtaining a basic image. The basic image includes a first basic image and a second basic image, the first basic image is an image in a first direction, and the second basic image is an image in a second direction. The first direction and the second direction are opposite directions.

[0142] In this step, the first basic image and the second basic image in opposite directions are obtained to facilitate determination of whether features in the second basic image exist in the first basic image, that is, to facilitate determination of whether a reflected image of a user or object in the car exists in the image of the scenery outside the window that the user wants to take.

[0143] Exemplarily, when a user is photographing the scenery outside the window in a high-speed rail carriage, he cannot open the window to shoot and can only shoot through the window glass. Since the areas with higher brightness in the carriage (including the user's figure and / or the carriage interior) are easily reflected by the window glass, when the user uses an electronic device to shoot an image of the scenery outside the window, the camera of the electronic device will simultaneously capture the scenery outside the window and the image formed by the reflection of the window glass. As a result, the first basic image (i.e., the image in the first direction taken by the user using the electronic device, the first direction is the direction toward the outside of the window) will contain the characteristics of the scenery outside the window and the reflection characteristics of the window glass (including the reflective characteristics of the user's figure and / or the reflective characteristics of the carriage interior). The second basic image (i.e., the image in the second direction taken by the user using the electronic device, the second direction is the direction toward the inside of the window) will contain the characteristics of the user's figure and the characteristics of the carriage interior.

[0144] Figure 8 It is a schematic diagram of the image reflection provided in the embodiment of the present application.

[0145] For example, in combination Figure 8 As shown, Figure 8The image includes a relatively bright area 40 in the vehicle cabin, an electronic device 100, a first camera 11, a second camera 12, a vehicle window glass 50, and a scenery outside the window 60. The relatively bright area 40 in the vehicle cabin is reflected by the vehicle window glass 50 along the light propagation arrows D1 and D3 to form a reflection. When the first camera 11 captures an image, the image captured by the first camera 11 includes a reflection image of the vehicle window glass 50 (formed along the light propagation arrows D2 and D4) and the scenery outside the window 60 (formed along the light propagation arrows D5 and D6). When the second camera 12 captures an image, the image captured by the second camera 12 includes an image of the relatively bright area 40 in the vehicle cabin (formed along the light propagation arrows E1 and E2). It can be seen that the same part in the first basic image as in the second basic image is the image formed by the reflection.

[0146] Fig. 9 It is a schematic diagram of the first basic image provided in an embodiment of the present application.

[0147] Fig.10 It is a schematic diagram of the second basic image provided in an embodiment of the present application.

[0148] For example, in combination Fig. 9 As shown, Fig. 9 In the figure, the solid black line is the view outside the window that the user wants to shoot, and the dotted black line is the image formed by the reflection of the car window glass (including the reflection characteristics of the interior of the car and the reflection characteristics of the user's figure). Fig.10 As shown, Fig.10 Including the interior of the car and the figure of the user.

[0149] Furthermore, since the first camera and the second camera of the electronic device have different shooting angles, the image taken by the second camera does not contain the image of the electronic device. However, the electronic device may also be reflected by the window glass as an image. Therefore, the present application adopts AI recognition means to identify whether there are electronic device features in the first basic image. If so, the electronic device features are eliminated. For the specific implementation method, please refer to the following steps S1100 to S1400.

[0150] Understandably, Fig. 9 and Fig.10 It is only shown schematically. Fig. 9 The reflective characteristics of the user's figure and the reflective characteristics of the interior of the car are given in Fig.10 The figure features of the user and the interior features of the vehicle cabin are completely consistent with each other. In the actual shooting process, since the figure of the user and the interior of the vehicle cabin may be deformed due to reflections from the window glass, the reflective features of the figure of the user and the interior of the vehicle cabin in the first basic image are not completely consistent with the figure features of the user and the interior features of the vehicle cabin in the second basic image, but this does not affect the effect of eliminating image reflections in the embodiment of the present application.

[0151] It should also be understood that due to the different brightness inside and outside the car, different reflection effects are often produced. For example, when it is bright outside the car, the window glass may only reflect the brighter position of the lights inside the car. When it is dark outside the car, the window glass may reflect both the interior of the car and the user's figure. Fig. 9 The reflection of users and interior decoration in the middle carriage is more comprehensive, which is just an example and does not limit the present application.

[0152] For example, in combination Figure 1 and Figure 2 As shown, Figure 1 and Figure 2 The first camera (rear camera) of the electronic device can obtain an image in the first direction F1, that is, a first basic image, and the second camera (front camera) of the electronic device can obtain an image in the second direction F2, that is, a second basic image. Specifically, if you want to use the front camera to shoot an image in the first direction and use the rear camera to shoot an image in the second direction, you only need to swap F1 and F2.

[0153] Optionally, acquiring the basic image includes: acquiring the basic image in response to an instruction to start the elimination function.

[0154] In this embodiment, through the elimination function start instruction, it is determined that the user needs to enter the function of eliminating reflections in the image taken by the first camera, and then the method provided in the embodiment of the present application is used to eliminate the reflections in the image taken by the first camera, thereby improving the user experience.

[0155] Specifically, the elimination function activation instruction is obtained after the user selects and confirms to enter the elimination function.

[0156] Step S200: extracting a first image from a first basic image, wherein a grayscale value of the first image is higher than a first grayscale value, and the first grayscale value is calculated and determined based on the grayscale value of the first basic image.

[0157] In this step, the first image is the part with higher brightness in the first basic image. Since the reflection of the window glass has higher brightness, the image with grayscale value higher than the first grayscale value is extracted to obtain the image with higher brightness in the first basic image.

[0158] Fig.11 It is a schematic diagram of the process of extracting the first image provided in an embodiment of the present application.

[0159] Optionally, combined Fig.11 As shown, extracting the first image from the first basic image includes steps S201 to S204, which are specifically as follows:

[0160] Step S201 , grayscale the first basic image to obtain a grayscaled first basic image.

[0161] In this step, grayscale is used to calculate the first grayscale value and to extract the first image with a higher grayscale value (brightness value) in the first basic image.

[0162] Specifically, the grayscale processing step includes: performing weighted summation on the RGB channels of each pixel. For example, the weighted summation is performed using the following formula:

[0163] The grayscale value of each pixel = 0.299*R+0.587*G+0.114*B; where “*” represents weighting, R represents the red channel, G represents the green channel, and B represents the blue channel.

[0164] Step S202: dividing the grayscale first basic image into a plurality of regions according to a first preset rule.

[0165] Specifically, the first preset rule is: the first basic image is divided into a plurality of regions according to the size of each region being M1×N1, wherein M1 and N1 may be equal or unequal, and the specific values ​​of M1 and N1 are determined based on the size of the first basic image.

[0166] Exemplarily, when the size of the first basic image is 150×100 (length×width), M1 is equal to 30, and N1 is equal to 20. That is, the first basic image is divided into 25 regions according to the size of 30×20.

[0167] Exemplarily, when the size of the first basic image is 150×150 (length×width), M1 is equal to 30, and N1 is equal to 30. That is, the first basic image is divided into 25 regions according to the size of 30×30.

[0168] Fig.12 It is a schematic diagram of dividing the grayscale first basic image into multiple regions provided in an embodiment of the present application.

[0169] For example, in combination Fig.12 As shown, the first basic image is divided into 15 areas according to the first preset rule.

[0170] Step S203, calculating the grayscale average value corresponding to each area as multiple first grayscale values.

[0171] Fig.13 It is a data flow diagram about the first basic image provided in an embodiment of the present application.

[0172] For example, in combination Fig.13 As shown, Fig.13 (a) represents the gray value of each pixel in the first basic image. Fig.13 (b) represents the grayscale mean value (i.e., the first grayscale value) of each region corresponding to the first basic image. The number of pixels of the first basic image is 9×12=108, and the first preset rule is to divide the first basic image into 3×3 sizes. Therefore, the process of calculating the grayscale mean value of the region in the upper left corner of the first basic image is: (210+210+255+200+235+240+245+255+220) / 9=230. The calculation process for the remaining regions is: calculate the sum of the 9 pixel values ​​in the region, and then divide it by the number of pixels in the region, 9. The specific values ​​are not repeated here.

[0173] Specifically, when the grayscale average value obtained has a decimal, the decimal may be removed by rounding down, or other rules may be used to remove the decimal, or no processing may be performed.

[0174] Exemplarily, the method of removing decimals used in the present application is to take the grayscale average value that is less than or equal to the next preset value as the previous preset value, where the previous preset value is less than the next preset value. For example, if the previous preset value is 175, the next preset value is 180, and the grayscale average value of a certain area is 176.9, then the grayscale average value of the area is taken as 175.

[0175] Step S204: determining a first image based on the grayscaled first basic image and the first grayscale value.

[0176] Optionally, based on the first basic image after grayscale and the first grayscale value, the first image is determined, including: in the first basic image after grayscale, each pixel point is taken as the first target pixel point in turn; the difference between the grayscale value of the first target pixel point and the first target grayscale value is calculated to obtain the first image; wherein the first target grayscale value is the first grayscale value corresponding to the area where the first target pixel point is located.

[0177] In this embodiment, the area with higher grayscale value (higher brightness value) is highlighted, so as to determine the reflective image formed by the reflection of the window glass according to the first image. Specifically, in the image, the larger the grayscale value of a point is, the higher the brightness of the point is.

[0178] For example, in combination Fig.13 As shown, Fig.13 (c) is the gray value of each pixel in the first image.

[0179] Exemplarily, taking the grayscale value of the pixel at position (1, 1) in the first image as an example, the calculation method is: 210-230=0. Since the grayscale value of the pixel in the first basic image is less than the grayscale average value of the area where the pixel is located, the difference is a negative number, and the grayscale value range is (0, 255). Therefore, 0 is used to represent the grayscale value of the pixel on the first image.

[0180] Taking the grayscale value of the pixel at position (1, 3) in the first image as an example, the average grayscale value of the area where the pixel is located is 230, and the calculation method is: 255-230=25. Therefore, 25 is used to represent the grayscale value of the pixel in the first image.

[0181] In this embodiment, by calculating the grayscale average of each area as the first grayscale value, the influence of uneven grayscale of the captured image (first basic image) caused by uneven light inside and outside the vehicle compartment on the extraction of the first image is reduced, thereby reducing the influence on the elimination effect. Specifically, the portion with higher brightness in the first basic image is the portion formed by the reflection of the vehicle window glass.

[0182] Specifically, the extracted first image is an image with higher brightness in the first basic image, and the first image contains a reflective feature image (reflective features of the vehicle interior and / or reflective features of the user's figure).

[0183] Step S300: extracting a second image from the second basic image, wherein the grayscale value of the second image is higher than the second grayscale value, and the second grayscale value is calculated and determined based on the grayscale value of the second basic image.

[0184] In this step, the second image is the part with higher brightness in the second basic image. Since the area with higher brightness in the car is more likely to be reflected by the window glass, the image with grayscale value higher than the second grayscale value is extracted to obtain the image with higher brightness in the second basic image, that is, to determine which image features are easy to reflect.

[0185] Specifically, the present application does not limit the order of steps S200 and S300. Step S200 may be performed first and then step S300, or step S300 may be performed first and then step S200. It is sufficient to complete steps S200 and S300 before step S400.

[0186] Fig.14 It is a schematic diagram of the process of extracting the second image provided in an embodiment of the present application.

[0187] Optionally, combined Fig.14 As shown, extracting the second image from the first basic image includes steps S205 to S208, which are specifically as follows:

[0188] Step S205 , grayscale the second basic image to obtain a grayscale second basic image.

[0189] In this step, grayscale is used to calculate the second grayscale value and extract the second image with a higher grayscale value (brightness value) in the second basic image.

[0190] Step S206: Divide the grayscale second basic image into a plurality of regions according to a second preset rule.

[0191] Specifically, the second preset rule is: the second basic image is divided into a plurality of regions according to the size of each region being M2×N2, wherein M2 and N2 may be equal or unequal, and the specific values ​​of M2 and N2 are determined based on the size of the second basic image.

[0192] Exemplarily, when the size of the second basic image is 160×120 (length×width), M2 is equal to 40, and N2 is equal to 20. That is, the second basic image is divided into 24 regions according to the size of 40×20.

[0193] Exemplarily, when the size of the second basic image is 160×160 (length×width), M2 is equal to 40, and N2 is equal to 40. That is, the second basic image is divided into 16 regions according to the size of 40×40.

[0194] Fig.15 It is a schematic diagram of dividing the grayscale second basic image into multiple areas provided by an embodiment of the present application.

[0195] For example, in combination Fig.15 As shown, the second basic image is divided into 15 areas according to the second preset rule.

[0196] Specifically, the second preset rule may be the same as or different from the first preset rule, and is specifically determined based on the size of the base image (the first base image or the second base image). A preferred selection scheme is that the length of the size of the base image (the first base image or the second base image) can be divided by the length of the size of the preset rule (M1 or M2), and the width of the size of the base image (the first base image or the second base image) can be divided by the width of the size of the preset rule (N1 or N2). In this way, each area of ​​the segmented base image can include complete pixels, so as to facilitate the calculation of the grayscale average value.

[0197] Step S207, calculating the grayscale average value corresponding to each area as multiple second grayscale values.

[0198] Fig.16 It is a data flow diagram of the second basic image provided in an embodiment of the present application.

[0199] For example, in combination Fig.16 As shown, Fig.16 (a) represents the gray value of each pixel in the second base image. Fig.16 (b) represents the grayscale mean value (i.e., the second grayscale value) of each area corresponding to the second basic image. The number of pixels of the second basic image is 9×12=108, and the first preset rule is to divide the second basic image into 3×3 sizes. Therefore, the process of calculating the grayscale mean value of the area in the upper left corner of the first basic image is: (185+185+235+180+215+235+225+235+195) / 9=210. The calculation process for the remaining areas is: calculate the sum of the 9 pixel values ​​in the area and divide it by 9. The specific values ​​are not repeated here.

[0200] Step S208: determining a second image based on the grayscaled second basic image and the second grayscale value.

[0201] Optionally, based on the grayscaled second basic image and the second grayscale value, a second image is determined, including: in the grayscaled second basic image, each pixel is taken as a second target pixel in turn; the difference between the grayscale value of the second target pixel and the second target grayscale value is calculated to obtain a second image; wherein the second target grayscale value is the second grayscale value corresponding to the area where the second target pixel is located.

[0202] In this embodiment, the area with higher grayscale value (higher brightness value) is highlighted, so as to determine the area image with higher brightness in the vehicle compartment according to the second image, that is, the area image with higher probability of being a reflective image.

[0203] For example, in combination Fig.16 As shown, Fig.16 (c) is the gray value of each pixel in the second image.

[0204] Exemplarily, taking the grayscale value of the pixel at position (1, 1) in the second image as an example, the calculation method is: 185-210=0. Since the grayscale value of the pixel in the second basic image is less than the grayscale average value of the area where the pixel is located, the difference is a negative number, and the grayscale value range is (0, 255). Therefore, 0 is used to represent the grayscale value of the pixel on the second image.

[0205] Taking the grayscale value of the pixel at position (1, 3) in the second image as an example, the calculation method is: 235-210=25, so 25 is used to represent the grayscale value of the pixel on the second image.

[0206] In this embodiment, by calculating the grayscale average of each area as the second grayscale value, the influence of uneven grayscale of the captured image (second basic image) caused by uneven light inside and outside the vehicle cabin on the extraction of the second image is reduced, thereby reducing the influence on the elimination effect. Specifically, the brighter the area in the second basic image, the more likely it is to be reflected by the window glass and become a reflective feature in the first basic image.

[0207] Specifically, the extracted second image is an image with higher brightness in the second basic image, and the second image contains a characteristic image of the vehicle cabin (vehicle cabin interior features and / or user's figure features).

[0208] Step S400: determining the same image contained in the first image and the second image as the third image.

[0209] Specifically, the area with higher brightness in the vehicle compartment is the part of the vehicle window glass reflection image, that is, the image with higher brightness in the first basic image. At the same time, the area with higher brightness in the vehicle compartment corresponds to the image with higher brightness in the second basic image. Therefore, it is only necessary to confirm that the second image is the same as the image in the first image, that is, the image formed by the vehicle window glass reflection, and eliminate the image formed by the vehicle window glass reflection to eliminate the reflection in the first basic image.

[0210] Fig.17 It is a schematic diagram of a process for determining the same images contained in a first image and a second image provided in an embodiment of the present application.

[0211] Optionally, combined Fig.17 As shown, determining the same image contained in the first image and the second image as the third image includes steps S401 to S403, which are specifically as follows:

[0212] Step S401: extracting an image of a first sub-region in a first image, wherein the grayscale value of the image of the first sub-region is greater than that of images of other regions in the first image.

[0213] Specifically, the first image may include some scattered noise points, and step S401 can reduce the noise of the first image to reduce the influence of the noise points on determining the reflective area image in the first basic image. The noise points are points with grayscale values, but the grayscale values ​​are low.

[0214] Optionally, extracting the image of the first sub-region in the first image includes: extracting a pixel point set of the first sub-region in the first image as a first pixel point set, wherein each pixel point in the first pixel point set constitutes the image of the first sub-region.

[0215] In this embodiment, the shape of the reflective feature in the first image (such as the interior of the vehicle or the figure of the user, etc.) is determined by extracting the image of the first sub-area. Specifically, the first pixel point set includes pixel points of the reflective feature.

[0216] Optionally, extracting a pixel point set of a first sub-region in the first image includes the following steps S4011 to S4012:

[0217] Step S4011: in the first image, each pixel point is taken as the third target pixel point in turn.

[0218] Step S4012, calculating the grayscale value difference between the third target pixel and the fourth target pixel to obtain a plurality of first grayscale differences, wherein the fourth target pixel is a pixel within a preset range of the third target pixel.

[0219] Specifically, in this step, the preset range can be determined according to the size of the image. For example, when the image size is large, a larger preset range is used, and when the image size is small, a smaller preset range is used.

[0220] Step S4013: when the first grayscale difference is greater than the grayscale difference threshold, add the third target pixel point to the first edge pixel point set and the pixel point set of the first sub-region.

[0221] Optionally, in this step, the grayscale difference threshold value ranges from [10, 30], and specific values ​​of the grayscale difference threshold value include 10, 20 or 30.

[0222] Specifically, steps S4011 to S4013 are used to extract edge pixel points of the first sub-region, and the pixel points in the first edge pixel point set are all edge pixel points of the first sub-region.

[0223] Step S4014: All pixels in the area formed by connecting all the pixels in the first edge pixel set are added to the pixel set of the first sub-area.

[0224] Specifically, step S4014 is used to determine all pixel points in the area enclosed by the edge pixel points of the first sub-area, that is, to obtain all pixel points of the first sub-area. Specifically, the enclosed area does not include the pixel points in the first edge pixel point set.

[0225] In this embodiment, the edge of the first sub-area is first defined, and then the pixel points within the area enclosed by the edge are determined to obtain all the pixel points of the first sub-area, so that the same image in the first sub-area and the second sub-area can be determined through the pixel points of the grayscale value of the pixel points.

[0226] For example, in combination Fig.13As shown in (c), the pixel points in the dotted box are the image corresponding to the first sub-area. The grayscale difference threshold value is 20, and the preset range is 1, that is, within the distance range of 1 pixel point of a certain pixel point, if the grayscale difference between the pixel point and other pixel points within the distance range of one pixel point is greater than or equal to 20, then the pixel point is added to the first edge pixel point set.

[0227] For example, in combination Fig.13 As shown in (c), the grayscale value of the pixel at position (1, 3) is 25, and the pixels within a distance of 1 pixel from the pixel include: (1, 2), (2, 3) and (1, 4). The first grayscale differences between the pixel and the aforementioned pixels within a distance of 1 pixel from the pixel are 25, 15 and 25 respectively, and there is a first grayscale difference greater than 20, so the pixel is added to the first edge pixel set.

[0228] It should be noted that Fig.13 The pixel point at position (4, 2) in (c) is not in the first edge pixel point set. However, in step S4014, the pixel point at position (4, 2) is located in the area formed by connecting and enclosing all the pixels in the first edge pixel point set. Therefore, the pixel point at position (4, 2) is in the pixel point set of the first sub-area.

[0229] Thus, the pixel points of the first sub-area include {(1, 3), 25}, {(1, 6), 25}, {(1, 9), 25}, {(1, 11), 50}, {(1, 12), 20}, {(2, 11), 30}, {(3, 2), 25}, {(3, 2), 25}, {(3, 5), 55}, {(3, 8), 25}, {(4, 1), 60}, {(4 , 2), 35}, {(4, 3), 30}, {(4, 4), 100}, {(4, 5), 80}, {(4, 6), 60}, {(4, 7), 50}, {(4, 8), 60}, {(4, 9), 40}, {(4, 10), 50}, {(4, 11), 60}, {(4, 12), 40}, {(5, 2), 80} and {(8, 3), 20}. Wherein, {(i, j), X} is used to represent the gray value of the pixel at the position (i, j) as X. This representation is only an example, and the embodiments of the present application do not limit the representation.

[0230] Step S402: extracting an image of a second sub-region in the second image, wherein the grayscale value of the image of the second sub-region is greater than that of images of other regions in the second image.

[0231] Specifically, the second image may include some scattered noise points, and step S402 can reduce the noise of the second image to reduce the influence of the noise points on determining the reflective area image in the second basic image. The noise points are points with grayscale values, but the grayscale values ​​are low.

[0232] Specifically, by performing noise reduction processing on the first image and the second image, the computational complexity in subsequent processing can be reduced. Furthermore, by performing noise reduction processing on the first image and the second image based on the first sub-region image and the second sub-region image, the reflection of the image can be eliminated, thereby reducing the problem that the grayscale value of the pixel point in the image formed by non-reflection is affected by using the entire image for calculation.

[0233] Optionally, extracting the image of the second sub-region in the second image includes: extracting a pixel point set of the second sub-region in the second image as a second pixel point set, wherein each pixel point in the second pixel point set constitutes the image of the second sub-region.

[0234] In this embodiment, the shape of the highlight area in the second image (eg, the interior of the vehicle or the figure of the user, etc.) is determined by extracting the image of the second sub-area. Specifically, the second pixel point set includes pixel points with reflective features.

[0235] Optionally, extracting a pixel point set of a second sub-region in the second image includes the following steps S4021 to S4024:

[0236] Step S4021: In the second image, each pixel point is sequentially used as the fifth target pixel point.

[0237] Step S4022, calculating the grayscale value difference between the fifth target pixel and the sixth target pixel to obtain a plurality of second grayscale differences, wherein the sixth target pixel is a pixel within a preset range of the fifth target pixel.

[0238] Step S4023: When the second grayscale difference is greater than the grayscale difference threshold, the fifth target pixel is added to the second edge pixel set and the pixel set of the second sub-region.

[0239] Specifically, steps S4021 to S4023 are used to extract edge pixel points of the second sub-region, and the pixel points in the second edge pixel point set are all edge pixel points of the second sub-region.

[0240] Step S4024: All pixels in the area formed by connecting all the pixels in the second edge pixel set are added to the pixel set of the second sub-area.

[0241] Specifically, step S4024 is used to determine all the pixels in the area enclosed by the edge pixels of the second sub-area, that is, to obtain all the pixels in the second sub-area. Specifically, the enclosed area does not include the pixels in the second edge pixel set.

[0242] In this embodiment, the edge of the second sub-area is first delineated, and then the pixel points within the area enclosed by the edge are determined to obtain all the pixel points of the second sub-area, so that the same image in the first sub-area and the second sub-area can be determined through the pixel points of the grayscale value of the pixel points.

[0243] For example, in combination Fig.16 As shown in (c), the pixel points in the dotted box are the image corresponding to the first sub-area. The grayscale difference threshold value is 20, and the preset range is 1, that is, within the distance range of 1 pixel point of a certain pixel point, if the grayscale difference between the pixel point and other pixel points within the distance range of one pixel point is greater than or equal to 20, then the pixel point is added to the second edge pixel point set.

[0244] For example, in combination Fig.16 As shown in (c), the grayscale value of the pixel at position (1, 3) is 25, and the pixels within a distance of 1 pixel from the pixel include: (1, 2), (2, 3) and (1, 4). The first grayscale differences between the pixel and the aforementioned pixels within a distance of 1 pixel from the pixel are 25, 0 and 25 respectively. There is a first grayscale difference greater than 20, so the pixel is added to the second edge pixel set.

[0245] It should be noted that Fig.16 The pixel point at position (4, 2) in (c) is not in the second edge pixel point set. However, in step S4024, the pixel point at position (4, 2) is located in the area formed by connecting and enclosing all the pixels in the second edge pixel point set. Therefore, the pixel point at position (4, 2) is in the pixel point set of the second sub-area.

[0246] Thus, the pixel points of the second sub-area include {(1, 3), 25}, {(1, 6), 25}, {(1, 9), 25}, {(1, 11), 50}, {(1, 12), 20}, {(2, 3), 25}, {(2, 11), 30}, {(3, 2), 25}, {(3, 5), 40}, {(3, 8), 25}, {(4, 1), 50}, {(4, 2), 30}, {(4, 3), 20}, {(4, 4), 90}, {(4, 5), 80}, {(4, 6), 60}, {(4, 7), 45}, {(4, 8), 60}, {(4, 9), 50}, {(4, 10), 40}, {(4, 11), 50}, {(4, 12), 40} and {(5, 2), 80}. Here, {(i, j), X} is used to indicate that the grayscale value of the pixel at the position (i, j) is X. This representation is only an example, and the embodiment of the present application does not limit the representation.

[0247] Step S403: determine the same image between the image of the first sub-region and the image of the second sub-region as the third image.

[0248] In this embodiment, the third image is determined by respectively extracting the image of the first sub-region and the image of the second sub-region, thereby reducing the amount of calculation in determining the third image and improving the operation speed.

[0249] Optionally, the first pixel point set includes all pixel points of the image of the first sub-area, and the second pixel point set includes all pixel points of the image of the second sub-area; determining the same image in the image of the first sub-area and the image of the second sub-area as the third image includes: determining pixel points with similar grayscale values ​​between the first pixel point set and the second pixel point set, and adding pixel points with similar grayscale values ​​to the similar pixel point set; wherein the similar pixel point set includes all pixel points of the third image.

[0250] In this embodiment, pixels with similar grayscale values ​​are pixels with similar brightness. Since the first pixel set only includes pixels in the reflective area, and the second pixel set only includes pixels in the area with greater brightness, the grayscale values ​​of pixels with similar corresponding positions indicate that the image represented by the pixel at that position in the first sub-area is the same as the image represented by the pixel at that position in the image of the second sub-area. Thus, all pixels with similar grayscale values ​​(all pixels in the similar pixel set) constitute the third image.

[0251] Optionally, determining pixels having similar grayscale values ​​between the first pixel set and the second pixel set, and adding the pixels having similar grayscale values ​​to the similar pixel set includes the following steps S4031 and S4032:

[0252] Step S4031, calculating the similarity between the first grayscale value and the second grayscale value, wherein the first pixel point set includes the first pixel point, and the grayscale value of the first pixel point is the first grayscale value. The second pixel point set includes the second pixel point, and the grayscale value of the second pixel point corresponding to the position of the first pixel point is the second grayscale value.

[0253] Optionally, calculating the similarity between the first grayscale value and the second grayscale value includes: calculating a first ratio between the first grayscale value and the second grayscale value, and a second ratio between the second grayscale value and the first grayscale value; and taking the ratio of the first ratio and the second ratio that is less than or equal to 1 as the similarity between the first grayscale value and the second grayscale value.

[0254] In this embodiment, since the similarity ratio is less than or equal to 1, when the magnitude relationship between the first grayscale value and the second grayscale value is uncertain, respectively calculating the first ratio and the second ratio can determine a similarity less than or equal to 1.

[0255] Step S4032: taking the second pixel point whose similarity is greater than or equal to the similarity threshold as a similar pixel point.

[0256] Optionally, the value range of the similarity threshold includes [70%, 100%], and the specific value of the similarity threshold includes 70%, 85% or 100%.

[0257] In this embodiment, the second pixel point whose similarity is greater than or equal to the similarity threshold, that is, the second pixel point and the first pixel point corresponding to the position of the second pixel point represent the same image, so as to obtain a complete identical image, thereby facilitating the elimination of image reflection.

[0258] Fig.18 It is a grayscale matrix schematic diagram of the third image provided in the embodiment of the present application.

[0259] Combination Fig.18 As shown, Fig.18 A grayscale matrix of a third image is given as an example. For example, taking the similarity threshold as 70% as an example, the process of obtaining the grayscale matrix of the third image is explained in combination with the pixel point set of the first sub-region and the pixel point set of the second sub-region, including:

[0260] In the first step, the similarity of the gray value of each pixel in the pixel point set of the first sub-region and the position of the pixel in the pixel point set of the second sub-region is calculated.

[0261] For example, the grayscale value of the pixel at position (1, 3) in the pixel point set of the first sub-region is 25, that is, the first grayscale value is 25. The grayscale value of the pixel at position (1, 3) in the pixel point set of the second sub-region is also 25, that is, the second grayscale value is 25. Therefore, the similarity of the pixel at position (1, 3) is 25 / 25=100%>70%. Therefore, the pixel at position (1, 3) in the first image is the pixel of the image generated by reflection, that is, it is located in the third image.

[0262] For another example, the grayscale value of the pixel at position (8, 3) in the pixel point set of the first sub-region is 20, that is, the first grayscale value is 20. However, there is no pixel at the same position (8, 3) in the pixel point set of the second sub-region. Therefore, the pixel at position (8, 3) is not a pixel of the image generated by reflection, that is, it is not located in the third image.

[0263] For another example, the grayscale value of the pixel at position (3, 5) in the first sub-region pixel concentration is 55, that is, the first grayscale value is 55. The grayscale value of the pixel at position (3, 5) in the second sub-region pixel concentration is 40, that is, the second grayscale value is 40. Therefore, the first ratio of the pixel at position (1, 3) is 55 / 40=137.5%>1, and the second ratio is 40 / 55=72.7%<1. The second ratio is selected for the similarity, that is, 72.7%>70%. Therefore, the pixel at position (3, 5) in the first image is the pixel of the image generated by reflection, that is, it is located in the third image.

[0264] For another example, the grayscale value of the pixel at the position (4, 3) in the pixel point concentration of the first sub-region is 30, that is, the first grayscale value is 30. The grayscale value of the pixel at the same position (4, 3) in the pixel point concentration of the second sub-region is 20, that is, the second grayscale value is 20. Therefore, the first ratio of the pixel at the position (4, 3) is 30 / 20=150%>1, and the second ratio is 20 / 30=66.7%<1. The second ratio is selected for the similarity, that is, 66.7%<70%. Therefore, the pixel at the position (4, 3) in the first image is the pixel of the image generated by non-reflection, that is, it is not located in the third image.

[0265] In the second step, the grayscale values ​​of the pixels other than the pixels whose similarity is greater than or equal to the similarity threshold are set to 0 to obtain a grayscale matrix of the third image, wherein the pixels with grayscale values ​​are similar images of the first image and the second image.

[0266] Step S500, clearing the third image in the first image to obtain a first target image.

[0267] In this step, the third image in the first image is cleared to eliminate the reflected image in the first image, and then the reflected image in the first basic image is cleared, so as to obtain the first target image with the reflection eliminated.

[0268] Fig.19 It is a schematic diagram of the process of obtaining the first target image provided in an embodiment of the present application.

[0269] Optionally, combined Fig.19 As shown, eliminating the third image in the first image to obtain the first target image includes the following steps S501 to S504:

[0270] Step S501, grayscale processing is performed on the first image to obtain a grayscale first image.

[0271] Step S502, grayscale processing is performed on the third image to obtain a grayscale third image.

[0272] Specifically, based on steps S501 and S502, the third image in the first image is eliminated according to the grayscale value of the first image and the grayscale value of the third image.

[0273] Step S503, eliminating the grayscale third image in the grayscale first image to obtain the grayscale first target image.

[0274] Optionally, eliminating the grayscale third image in the grayscale first image to obtain the grayscale first target image includes the following steps S5031 to S5035:

[0275] Step S5031, extracting the grayscale matrix of the first image after grayscale conversion as the first matrix.

[0276] Step S5032, extracting the grayscale matrix of the third image after grayscale conversion as the second matrix.

[0277] Step S5033, calculating the difference between the first matrix and the second matrix to obtain a first difference matrix.

[0278] Step S5034: In the first difference matrix, the grayscale value of each pixel is added to the target grayscale average value to obtain a grayscale matrix corresponding to the first target image, wherein the target grayscale average value is the first grayscale value corresponding to the area where the pixel is located.

[0279] Step S5035, converting the grayscale matrix corresponding to the first target image into a grayscale first target image.

[0280] In this embodiment, the first difference matrix obtained is the grayscale difference between each pixel in the first image and each pixel in the corresponding position in the third image, that is, the grayscale value of each pixel of the first target image is obtained, and then the first target image can be determined, that is, the image with the reflection of the first basic image eliminated.

[0281] Fig. 20 It is a data flow diagram for calculating the difference between the first matrix and the second matrix to obtain the first difference matrix provided in an embodiment of the present application.

[0282] For example, in combination Fig. 20 As shown, Fig. 20 Where (a) is the first matrix (same as Fig.13 In (c), Fig. 20 (b) is the second matrix (same Fig.18 ), Fig. 20 (c) is the first difference matrix. The value of (4, 7) in the first matrix is ​​50, and the value of (4, 7) in the second matrix is ​​45, so the value of (4, 7) in the first difference matrix should be 50-45=5. The value of (7, 2) in the first matrix is ​​10, and the value of (7, 2) in the second matrix is ​​0, so the value of (7, 2) in the first difference matrix should be 10-0=10.

[0283] Step S504, converting the grayscale first target image into a color image to obtain the first target image.

[0284] In this embodiment, the third image in the first image is eliminated based on the grayscale value of each pixel of the image, thereby obtaining the first target image, that is, the image after the reflection in the first basic image is eliminated.

[0285] Specifically, the grayscale first target image may be converted into a color image by manually defining a mapping table between grayscale values ​​and colors, thereby converting the grayscale first target image into a color image based on the mapping table.

[0286] By adopting the method for eliminating reflections in images provided in the embodiment of the present application, reflections in the first basic image taken by the first camera are eliminated by combining the images taken by the first camera and the second camera, thereby reducing the impact of reflections on the photos taken by users and improving the user experience.

[0287] Fig.21 It is a flowchart of another method for eliminating reflections in an image provided in an embodiment of the present application.

[0288] Combination Fig.21 As shown, the embodiment of the present application also provides another method for eliminating reflections in an image. The execution subject of the method may be an electronic device. The method includes the following steps S600 to S1400:

[0289] Step S600, obtaining a basic image, wherein the basic image includes a first basic image and a second basic image, the first basic image is an image in a first direction, and the second basic image is an image in a second direction, the first direction and the second direction being opposite directions.

[0290] Specifically, the specific implementation method of this step refers to the above-mentioned step S100, and this application will not repeat it here.

[0291] Step S700: extracting an image with a grayscale value higher than a first grayscale value from the first basic image as the first image, wherein the first grayscale value is calculated based on the grayscale value of the first basic image.

[0292] Specifically, the specific implementation method of this step refers to the above-mentioned step S200, and this application will not repeat it here.

[0293] Step S800: extracting an image with a grayscale value higher than a second grayscale value from the second basic image as the second image, wherein the second grayscale value is calculated based on the grayscale value of the second basic image.

[0294] Specifically, the specific implementation method of this step refers to the above step S300, and this application will not repeat it here.

[0295] Step S900: Determine the same image contained in the first image and the second image as the third image.

[0296] Specifically, the specific implementation method of this step refers to the above-mentioned step S400, and this application will not repeat it here.

[0297] Step S1000, clearing the third image in the first image to obtain a first target image.

[0298] Specifically, the specific implementation method of this step refers to the above-mentioned step S500, and this application will not repeat it here.

[0299] Step S1100, obtaining the model of the electronic device, wherein the electronic device is configured to obtain a basic image.

[0300] Exemplarily, the model of the electronic device is input by a user, or the camera application directly obtains the model of the device through the Internet.

[0301] Step S1200: determining an electronic device image corresponding to the model of the electronic device.

[0302] Exemplarily, the camera algorithm library stores images corresponding to various models of electronic devices.

[0303] Optionally, determining the electronic device image corresponding to the model of the electronic device includes: determining a grayscale matrix corresponding to the electronic device image of the model of the electronic device as a third grayscale matrix.

[0304] In this embodiment, the third grayscale matrix is ​​determined to facilitate calculation of the grayscale value of the second target image, thereby facilitating determination of the second target image.

[0305] Step S1300: using AI technology, identifying whether there is an electronic device image in the first target image.

[0306] This step is to determine whether the first target image still has the reflective features of the electronic device, so as to eliminate the reflective features of the electronic device if they exist, so as to further eliminate the image reflection in the first basic image.

[0307] Specifically, if there is an image of an electronic device in the first target image, AI can identify the location of the electronic device and reconstruct the grayscale matrix of the electronic device image based on the electronic device image and the location of the electronic device image, so that the size of the grayscale matrix of the electronic device image is the same as the size of the grayscale matrix of the first target image for easy calculation.

[0308] Step S1400: When there is an electronic device image in the first target image, the electronic device image in the first target image is eliminated to obtain a second target image.

[0309] In this step, the second target image is an image obtained by further eliminating reflections from the first basic image.

[0310] Optionally, eliminating the electronic device image in the first target image to obtain the image of the electronic device with reflections eliminated includes the following steps S1401 to S1404:

[0311] Step S1401, calculate each pixel in the first target image minus the first grayscale value to obtain a second difference matrix.

[0312] Step S1402, calculating the difference between the second difference matrix and the third gray matrix to obtain the gray matrix of the second target image.

[0313] Step S1403, converting the grayscale matrix of the second target image into a grayscale image to obtain a grayscale second target image.

[0314] Step S1404, converting the grayscale second target image into a color image to obtain a second target image.

[0315] In this embodiment, the second difference matrix and the third grayscale matrix are used to determine whether the remaining reflected image in the first target image includes the electronic device image. In the case where the remaining reflected image in the first target image includes the electronic device image, the reflected image of the electronic device in the first target image is removed by subtracting the grayscale matrices to further eliminate the reflection in the image.

[0316] Specifically, the second difference matrix is ​​the same as or different from the first difference matrix.

[0317] For example, when a user uses an electronic device to take a photo in a high-speed rail carriage, since the light brightness in the high-speed rail carriage is known, the grayscale matrix corresponding to the electronic device image is subtracted from the grayscale value corresponding to the known light brightness to form the third grayscale matrix. This can reduce the influence of the light in the carriage on the elimination effect.

[0318] Fig. 22 It is a data flow diagram for obtaining the grayscale matrix of the second target image provided in an embodiment of the present application.

[0319] For example, in combination Fig. 22 As shown, Fig. 22 Where (a) is the second difference matrix, Fig. 22 (b) is the third grayscale matrix, Fig. 22 (c) is the grayscale matrix of the second target image. The value of (4, 4) in the second difference matrix is ​​10, and the value of (4, 4) in the third grayscale matrix is ​​10, so the value of (4, 4) in the grayscale matrix of the second target image should be 10-10=0. The value of (4, 5) in the second difference matrix is ​​0, and the value of (4, 5) in the third grayscale matrix is ​​10, so the value of (4, 5) in the grayscale matrix of the second target image should be 0-10=-10, and 0 is used instead.

[0320] Optionally, eliminating the electronic device image in the first target image to obtain the image of the electronic device with reflections eliminated includes the following steps S1405 to S1407:

[0321] Step S1405, calculating the difference between the grayscale matrix of the first target image and the third grayscale matrix to obtain the grayscale matrix of the second target image.

[0322] Step S1406, converting the grayscale matrix of the second target image into a grayscale image to obtain a grayscale second target image.

[0323] Step S1407, converting the grayscale second target image into a color image to obtain a second target image.

[0324] In this embodiment, the grayscale matrix of the second target image is determined by calculating the difference between the grayscale matrix of the first target image and the third grayscale matrix, and then the grayscale second target image is determined, thereby obtaining the second target image. This further eliminates the reflected image in the first basic image, reduces the impact of the reflection of the car window glass on the user's shooting of the scenery outside the window, and improves the user experience.

[0325] Fig.23 It is another data flow diagram for obtaining the grayscale matrix of the second target image provided in an embodiment of the present application.

[0326] For example, in combination Fig.23 As shown, Fig.23 (a) is the grayscale matrix of the first target image. Fig.23 (b) is the third grayscale matrix, Fig.23 (c) is the grayscale matrix of the second target image. The value of (4, 4) in the grayscale matrix of the first target image is 110, and the value of (4, 4) in the third grayscale matrix is ​​10, so the value of (4, 4) in the grayscale matrix of the second target image should be 110-10=100. The value of (4, 5) in the grayscale matrix of the first target image is 100, and the value of (4, 5) in the third grayscale matrix is ​​10, so the value of (4, 4) in the grayscale matrix of the second target image should be 100-10=90.

[0327] In this embodiment, the reflective features of the electronic device are eliminated to further reduce the reflective image in the first basic image, thereby reducing the impact of the reflection of the car window glass on the user's shooting of the scenery outside the window, thereby improving the user experience.

[0328] Fig.24 It is a flowchart of another method for eliminating reflections in an image provided in an embodiment of the present application.

[0329] Combination Fig.24 As shown, the embodiment of the present application also provides another method for eliminating reflections in an image. The execution subject of the method may be an electronic device. The method includes the following steps S1500 to S2100:

[0330] Step S1500, in response to a photography function start signal, identifying whether there is a reflective scene in the preview image of the photography function.

[0331] Optionally, identifying whether there is a reflective scene in the preview picture of the photography function includes: dividing the preview picture into multiple areas according to a third preset rule; and determining that there is a reflective scene when there is a highlight area among the multiple areas.

[0332] In this embodiment, the highlight area indicates that the brightness of the area is more prominent than the brightness of other areas, that is, the area is brighter and has a larger brightness difference than other areas. In addition, in the preview screen of the scenery outside the window taken by the user using the electronic device, the area with higher brightness is generally the area formed by the reflection of the car window glass, or there is a light source outside the car window. Therefore, when there is a highlight area, it is determined that there is a reflective scene.

[0333] Specifically, the third preset rule is: the preview screen is divided into multiple areas according to the size of each area being M3×N3, wherein M3 and N3 may be equal or unequal, and the specific values ​​of M3 and N3 are determined based on the size of the preview screen.

[0334] Exemplarily, when the size of the second basic image is 160×120 (length×width), M3 is equal to 40, and N3 is equal to 20. That is, the preview screen is divided into 24 areas according to the size of 40×20.

[0335] Exemplarily, when the size of the second basic image is 160×160 (length×width), M2 is equal to 40, and N2 is equal to 40. That is, the preview screen is divided into 16 areas according to the size of 40×40.

[0336] Fig.25 It is a schematic diagram of the preview screen division areas provided in an embodiment of the present application.

[0337] For example, in combination Fig.25 As shown, the preview screen is divided into 15 areas according to the third preset rule.

[0338] Optionally, the presence of a highlight area in multiple areas is determined according to the following method, including: grayscale the preview image to obtain a grayscale preview image; calculate the grayscale mean of each area in the grayscale preview image; calculate the grayscale difference between the grayscale mean of each area and the grayscale mean of each remaining area; when there is a grayscale difference greater than a grayscale difference threshold, determine that a highlight area exists.

[0339] In this embodiment, by comparing the grayscale average of each area with the grayscale average of other areas, it is determined whether there is a highlighted area in the preview image, that is, whether the grayscale average of an area is higher, that is, the brightness of this area is significantly higher than that of other areas, thereby improving the accuracy of scene recognition.

[0340] Optionally, the value range of the grayscale difference threshold includes [40, 100], and the specific value range of the grayscale difference threshold includes 40, 70 or 100.

[0341] For example, in combination Fig.25As shown, the grayscale difference threshold is 70, the grayscale average value of region 111 is 210, and the grayscale average value of region 112 is 80. Therefore, the grayscale difference between region 111 and region 112 is 130>70, that is, the brightness of region 111 is much greater than the brightness of region 112. Therefore, it is determined that there is a highlight area in the preview image.

[0342] Step S1600: In the case of a reflective scene, it is determined that the captured image has glass reflections, and a prompt is issued to ask whether to enter the elimination function.

[0343] Specifically, after receiving a prompt for whether to enter the elimination function, if the user believes that there is no window glass reflection in the preview image, the user can click Cancel to not enter the elimination function. If the user believes that there is window glass reflection in the preview image, the user can click Enter to enter the elimination function. Specifically, after the user chooses to enter, the electronic device obtains an instruction to start the elimination function.

[0344] Fig.26 It is a schematic diagram of the operation of the electronic device provided in an embodiment of the present application.

[0345] Combination Fig.26 As shown, the user clicks the camera application icon (see Fig.26 After (a), the camera application opens and displays the preview screen of the camera application (see Fig.26 In (b)). When a reflective scene is detected, a pop-up window will pop up to ask "Do you want to enter the elimination function?" (See Fig.26 (c)). If the user clicks the "Enter" option (see Fig.26 In (d), the camera application calls the operating code and data in the camera algorithm library to implement the method for eliminating reflections in images provided in the embodiment of the present application, thereby eliminating reflections in the image taken by the first camera. If the user clicks the "Cancel" option (see Fig.26 In (e), the reflection elimination function is not performed.

[0346] Step S1700, obtaining a basic image, wherein the basic image includes a first basic image and a second basic image, the first basic image is an image in a first direction, and the second basic image is an image in a second direction, wherein the first direction and the second direction are opposite directions.

[0347] Specifically, the specific implementation method of this step refers to the above-mentioned step S100, and this application will not repeat it here.

[0348] Step S1800: extracting an image with a grayscale value higher than a first grayscale value from the first basic image as the first image, wherein the first grayscale value is calculated based on the grayscale value of the first basic image.

[0349] Specifically, the specific implementation method of this step refers to the above-mentioned step S200, and this application will not repeat it here.

[0350] Step S1900: extracting an image with a grayscale value higher than a second grayscale value from the second basic image as the second image, wherein the second grayscale value is calculated based on the grayscale value of the second basic image.

[0351] Specifically, the specific implementation method of this step refers to the above step S300, and this application will not repeat it here.

[0352] Step S2000: Determine the same image contained in the first image and the second image as the third image.

[0353] Specifically, the specific implementation method of this step refers to the above-mentioned step S400, and this application will not repeat it here.

[0354] Step S2100, clear the third image in the first image to obtain a first target image.

[0355] Specifically, the specific implementation method of this step refers to the above-mentioned step S500, and this application will not repeat it here.

[0356] In this embodiment, through scene recognition, it is determined whether there is a highlight area in the preview image, and then whether there is a reflective scene. In the case of a reflective scene, the user is prompted to eliminate the reflection, thereby improving the user's experience.

[0357] Some embodiments of the present application provide an electronic device, which may include: a display screen (such as a touch screen or a non-touch screen), a memory, and one or more processors. The display screen, the memory, and the processor are coupled. The memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device can perform the various functions or steps performed by the electronic device in the above method embodiment. The structure of the electronic device can refer to Figure 6 The structure of the electronic device shown.

[0358] An embodiment of the present application also provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device executes each function or step executed by the electronic device in the above-mentioned method embodiment.

[0359] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer executes each function or step executed by the electronic device in the above method embodiment.

[0360] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0361] It is easy to understand that those skilled in the art can combine, split, reorganize, etc. the embodiments of the present application to obtain other embodiments based on the several embodiments provided in the present application, and these embodiments do not exceed the protection scope of the present application.

[0362] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0363] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0364] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0365] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0366] It should be noted that those skilled in the art will easily think of other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application. The specification and examples are only regarded as exemplary, and the true scope of the present application is indicated by the claims.

[0367] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for eliminating reflections in an image, characterized in that: include: Acquire a basic image; wherein the basic image includes a first basic image and a second basic image, the first basic image is an image in a first direction, and the second basic image is an image in a second direction; the first direction and the second direction are opposite directions; Extracting a first image from the first base image, wherein a grayscale value of the first image is higher than a first grayscale value, and the first grayscale value is calculated and determined based on the grayscale value of the first base image; Extracting a second image from the second base image, the grayscale value of the second image being higher than a second grayscale value, and the second grayscale value being calculated and determined based on the grayscale value of the second base image; determining the same image contained in the first image and the second image as a third image; Clearing the third image from the first image to obtain a first target image; The determining the same image contained in the first image and the second image as the third image includes: Extracting an image of a first sub-region in the first image; wherein a grayscale value of the image of the first sub-region is greater than that of images of other regions in the first image; Extracting an image of a second sub-region in the second image; wherein the grayscale value of the image of the second sub-region is greater than that of images of other regions in the second image; determining an identical image between the image of the first sub-region and the image of the second sub-region as the third image; The step of extracting an image of a first sub-region in the first image comprises: Extracting a pixel point set of the first sub-region in the first image as a first pixel point set; Wherein, each pixel point in the first pixel point set constitutes an image of the first sub-region; The step of extracting an image of a second sub-region in the second image comprises: Extracting a pixel point set of the second sub-region in the second image as a second pixel point set; wherein each pixel point in the second pixel point set constitutes an image of the second sub-region; The first pixel point set includes all pixel points of the image of the first sub-area, and the second pixel point set includes all pixel points of the image of the second sub-area; and determining the same image between the image of the first sub-area and the image of the second sub-area as the third image includes: Determine pixels having similar grayscale values ​​between the first pixel point set and the second pixel point set, and add the pixels having similar grayscale values ​​to a similar pixel point set; wherein the similar pixel point set includes all pixels of the third image; The determining of pixels with similar grayscale values ​​between the first pixel point set and the second pixel point set, and adding the pixels with similar grayscale values ​​to the similar pixel point set includes: Calculating the similarity between a first grayscale value and a second grayscale value; wherein the first pixel point set includes a first pixel point, and the grayscale value of the first pixel point is the first grayscale value; the second pixel point set includes a second pixel point, and the grayscale value of the second pixel point corresponding to the position of the first pixel point is the second grayscale value; The second pixel points whose similarity is greater than or equal to the similarity threshold are taken as the similar pixel points.

2. The method for eliminating reflections in an image according to claim 1, characterized in that: The step of obtaining a basic image comprises: In response to the elimination function start instruction, the base image is acquired.

3. The method for eliminating reflections in an image according to claim 1, characterized in that: The step of extracting the first image from the first base image comprises: Grayscale the first basic image to obtain a grayscale first basic image; According to a first preset rule, dividing the grayscale first basic image into a plurality of regions; Calculating the grayscale average value corresponding to each of the regions as a plurality of first grayscale values; The first image is determined based on the grayscaled first basic image and the first grayscale value.

4. The method for eliminating reflections in an image according to claim 3, characterized in that: The step of determining the first image based on the grayscaled first basic image and the first grayscale value includes: In the grayscaled first basic image, each pixel is sequentially used as a first target pixel; The difference between the grayscale value of the first target pixel and the first target grayscale value is calculated to obtain the first image; wherein the first target grayscale value is a first grayscale value corresponding to the area where the first target pixel is located.

5. The method for eliminating reflections in an image according to claim 1, characterized in that: The step of extracting the second image from the first base image comprises: grayscale the second basic image to obtain a grayscale second basic image; According to a second preset rule, dividing the grayscale second basic image into a plurality of regions; Calculating the grayscale average value corresponding to each of the regions as a plurality of second grayscale values; The second image is determined based on the grayscaled second base image and the second grayscale value.

6. The method for eliminating reflections in an image according to claim 5, characterized in that: The step of determining the second image based on the grayscaled second basic image and the second grayscale value comprises: In the grayscaled second basic image, each pixel is sequentially used as a second target pixel; The difference between the grayscale value of the second target pixel and the second target grayscale value is calculated to obtain the second image; wherein the second target grayscale value is the second grayscale value corresponding to the area where the second target pixel is located.

7. The method for eliminating reflections in an image according to claim 1, characterized in that: The extracting a pixel point set of the first sub-region in the first image includes: In the first image, each pixel point is sequentially used as a third target pixel point; Calculating the grayscale value difference between the third target pixel and the fourth target pixel to obtain a plurality of first grayscale differences; wherein the fourth target pixel is a pixel within a preset range of the third target pixel; In a case where the first grayscale difference is greater than a grayscale difference threshold, adding the third target pixel point to the first edge pixel point set and the pixel point set of the first sub-region; In an area formed by connecting all the pixels in the first edge pixel point set, all the pixels are added to the pixel point set of the first sub-area.

8. The method for eliminating reflections in an image according to claim 1, characterized in that: The extracting the pixel point set of the second sub-area in the second image includes: In the second image, each pixel point is sequentially used as a fifth target pixel point; Calculating the grayscale value difference between the fifth target pixel and the sixth target pixel to obtain a plurality of second grayscale differences; wherein the sixth target pixel is a pixel within a preset range of the fifth target pixel; In a case where the second grayscale difference is greater than the grayscale difference threshold, adding the fifth target pixel point to the second edge pixel point set and the pixel point set of the second sub-area; In an area formed by connecting all the pixels in the second edge pixel point set, all the pixels are added to the pixel point set of the second sub-area.

9. The method for eliminating reflections in an image according to claim 1, characterized in that: The calculating the similarity between the first grayscale value and the second grayscale value comprises: Calculating a first ratio between the first grayscale value and the second grayscale value, and a second ratio between the second grayscale value and the first grayscale value; Of the first ratio and the second ratio, the ratio that is less than or equal to 1 is used as the similarity between the first grayscale value and the second grayscale value.

10. The method for eliminating reflections in an image according to claim 1, characterized in that: The clearing the third image in the first image to obtain a first target image includes: Performing grayscale processing on the first image to obtain a grayscaled first image; Performing grayscale processing on the third image to obtain a grayscaled third image; Eliminating the grayscale third image in the grayscale first image to obtain a grayscale first target image; The grayscaled first target image is converted into a color image to obtain the first target image.

11. The method for eliminating reflections in an image according to claim 10, characterized in that: The step of eliminating the grayscale third image in the grayscale first image to obtain the grayscale first target image includes: Extracting a grayscale matrix of the grayscaled first image as a first matrix; Extracting a grayscale matrix of the grayscaled third image as a second matrix; Calculate the difference between the first matrix and the second matrix to obtain a first difference matrix; In the first difference matrix, the grayscale value of each pixel is added to the target grayscale average value to obtain the grayscale matrix corresponding to the first target image; wherein the target grayscale average value is the first grayscale value corresponding to the area where the pixel is located; The grayscale matrix corresponding to the first target image is converted into a grayscale first target image.

12. The method for eliminating reflections in an image according to any one of claims 1 to 11, characterized in that: After obtaining the first target image, the method further includes: Acquire a model of an electronic device; wherein the electronic device is configured to acquire the basic image; Determine an electronic device image corresponding to the model of the electronic device; Using AI technology, identifying whether the electronic device image exists in the first target image; In the case that the electronic device image exists in the first target image, the electronic device image in the first target image is eliminated to obtain a second target image.

13. The method for eliminating reflections in an image according to claim 12, characterized in that: The determining the electronic device image corresponding to the model of the electronic device includes: A grayscale matrix corresponding to the electronic device image of the electronic device model is determined as a third grayscale matrix.

14. The method for eliminating reflections in an image according to claim 13, characterized in that: The step of eliminating the electronic device image in the first target image to obtain an image of the electronic device with reflections eliminated includes: Calculating the difference between the grayscale matrix of the first target image and the third grayscale matrix to obtain the grayscale matrix of the second target image; Converting the grayscale matrix of the second target image into a grayscale image to obtain a grayscale second target image; The grayscale second target image is converted into a color image to obtain a second target image.

15. The method for eliminating reflections in an image according to claim 13, characterized in that: The step of eliminating the electronic device image in the first target image to obtain an image of the electronic device with reflections eliminated includes: Calculate the first gray value of each pixel in the first target image and subtract the first gray value to obtain a second difference matrix; Calculating the difference between the second difference matrix and the third grayscale matrix to obtain a grayscale matrix of a second target image; Converting the grayscale matrix of the second target image into a grayscale image to obtain a grayscale second target image; The grayscale second target image is converted into a color image to obtain a second target image.

16. The method for eliminating reflections in an image according to any one of claims 1 to 11, characterized in that: Before obtaining the basic image, the method further includes: In response to a photography function activation signal, identifying whether there is a reflective scene in a preview image of the photography function; In the case of the reflective scene, it is determined that the captured image has glass reflections, and a prompt is issued to ask whether to enter the elimination function.

17. The method for eliminating reflections in an image according to claim 16, characterized in that: The identifying whether there is a reflective scene in the preview picture of the photography function includes: Dividing the preview screen into a plurality of regions according to a third preset rule; When there is a highlight area in the plurality of areas segmented according to the third preset rule, it is determined that there is the reflective scene.

18. The method for eliminating reflections in an image according to claim 17, characterized in that: Determining whether there is a highlight area among the multiple areas according to the following method includes: Gray-scale the preview image to obtain a gray-scaled preview image; Calculating the grayscale mean of each of the regions in the grayscaled preview image; Calculate the grayscale difference between the grayscale mean of each of the regions and the grayscale mean of each of the remaining regions; In the case that the grayscale difference is greater than the grayscale difference threshold, it is determined that the highlight area exists.

19. An electronic device, characterized in that: include: Processor, memory and touch screen; The memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the method for eliminating reflections in an image as claimed in any one of claims 1 to 18.

20. A computer-readable storage medium, characterized in that: The method comprises computer instructions, which, when executed on an electronic device, enable the electronic device to execute the method for eliminating glare in an image as claimed in any one of claims 1 to 18.

21. A computer program product, characterized in that When the computer program product is run on a computer, the computer is enabled to execute the method for eliminating glare in an image according to any one of claims 1 to 18.

Images