Image processing method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention discloses an image processing method and device, equipment, a storage medium and a program product, the method is applied to electronic equipment, the electronic equipment is provided with at least two cameras, and the method comprises the following steps: collecting an image through a target camera under the condition that access of a reflector is recognized; under the condition that a target recognition object in the image is shielded by the reflective mirror, recognizing a missing region corresponding to the target recognition object in the image; and performing completion processing on the missing region to obtain a completed image corresponding to the image. According to the embodiment of the invention, under the condition that the reflective mirror is connected to the electronic equipment, the image area, shielded by the reflective mirror, of the target recognition object in the image collected by the target camera can be complemented, so that the completeness of the image collected by the target camera is ensured.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and includes, but is not limited to, an image processing method, apparatus, device, storage medium, and program product. Background Technology

[0002] Some educational tablets and tutoring devices on the market use a reflector in front of their camera to capture images of the desktop or other designated areas, enabling functions such as acquiring textbook content and grading homework. To further enhance user experience and meet diverse usage scenarios, some devices also include an additional auxiliary camera. This auxiliary camera, while the main camera performs the aforementioned functions via the reflector, simultaneously captures the user's image, enabling functions such as attention monitoring and interactive teaching, thus providing a better user experience.

[0003] However, when a reflector is installed in the main camera's shooting area, the edge of the reflector can partially obstruct the auxiliary camera's view, resulting in incomplete images captured by the auxiliary camera. Typically, the cameras are spaced further apart to ensure the auxiliary camera isn't blocked. This causes the auxiliary camera to be too far from the main camera, resulting in off-center portraits in the auxiliary camera's view, negatively impacting the user experience. Summary of the Invention

[0004] In view of this, the image processing method, apparatus, device, storage medium, and program product provided in this application embodiment can, when a reflector is connected to an electronic device, complete the image area of ​​the target object in the image captured by the target camera that is obscured by the reflector, thus ensuring the completeness of the image captured by the target camera. The image processing method, apparatus, device, storage medium, and program product provided in this application embodiment are implemented as follows:

[0005] The image processing method provided in this application is applied to an electronic device equipped with at least two cameras. The method includes:

[0006] When a mirror is detected, an image is captured by the target camera.

[0007] When the target object in the image is obscured by the mirror, identify the missing region corresponding to the target object in the image;

[0008] The missing regions are filled in to obtain the filled image corresponding to the original image.

[0009] In some embodiments, the at least two cameras include a main camera and an auxiliary camera, the reflector is positioned facing the field of view of the main camera, and the acquisition of images through the target camera includes:

[0010] The images are captured by the auxiliary camera.

[0011] In some embodiments, identifying the missing region corresponding to the target object in the image includes:

[0012] Obtain the target location of the target object in the image;

[0013] Based on the target location and the preset image size, the image is selected by bounding to obtain the target image;

[0014] Identify the missing region corresponding to the target object in the target image.

[0015] In some embodiments, identifying the missing region corresponding to the target object in the image includes:

[0016] Based on a preset first artificial intelligence (AI) model, the missing region corresponding to the target object in the image is identified.

[0017] In some embodiments, the process of completing the missing region to obtain the completed image corresponding to the image includes:

[0018] Based on a preset second artificial intelligence (AI) model, the missing region is filled in to obtain the filled image corresponding to the original image.

[0019] In some embodiments, the target object is a human image, and the process of completing the missing region to obtain the completed image includes:

[0020] Based on the preset symmetry of the human figure and the human figure area in the image that is not obscured by the mirror, the missing area is filled in to obtain the filled image corresponding to the image.

[0021] In some embodiments, the step of completing the missing region based on a preset portrait symmetry relationship and the portrait area in the image not obscured by the mirror, to obtain a completed image corresponding to the image, includes:

[0022] Obtain the neighboring regions of the human figure region in the image that are within a preset distance from the missing region;

[0023] Based on the preset symmetry relationship of the human face and the pixel color of the neighboring area, the missing area is filled in to obtain the filled image corresponding to the image.

[0024] In some embodiments, the step of performing fill-in processing on the missing region based on the preset portrait symmetry relationship and the pixel color of the neighboring region to obtain the filled image corresponding to the image includes:

[0025] Obtain the color change trend of pixels in the neighboring region;

[0026] Based on the color change trend of the pixels in the neighboring areas and the preset symmetry relationship of the human figure, the pixel colors of the missing areas are filled in to obtain the filled image corresponding to the image.

[0027] In some embodiments, the step of performing fill-in processing on the missing region based on the preset portrait symmetry relationship and the pixel color of the neighboring region to obtain the filled image corresponding to the image includes:

[0028] Based on the preset portrait symmetry relationship and the pixel color of the adjacent area, the pixel color of the missing area is filled in;

[0029] The missing region and the adjacent region after completion are magnified to obtain the completed image corresponding to the image.

[0030] In some embodiments, the reflector is provided with a sensing element, the electronic device is provided with a sensing sensor, and the detection of reflector access includes:

[0031] When the sensing sensor detects the sensing element, the mirror is detected to be connected.

[0032] In some embodiments, after acquiring images via the target camera, the method further includes:

[0033] Determine whether the target object exists in the image captured by the target camera within a preset time period;

[0034] If not, turn off at least two cameras.

[0035] The image processing apparatus provided in this application embodiment includes at least two cameras, and the apparatus comprises:

[0036] The acquisition module is used to acquire images through the target camera when a mirror is detected in the field of view.

[0037] The completion module is used to identify the missing region corresponding to the target object in the image when the target object is obscured by the mirror; and to complete the missing region to obtain the completed image.

[0038] The computer device provided in this application includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the method described in this application.

[0039] The computer-readable storage medium provided in this application embodiment stores a computer program thereon, which, when executed by a processor, implements the methods described in this application embodiment.

[0040] The computer program product provided in this application includes a computer program that, when executed by a processor, implements the method described in this application.

[0041] The image processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product provided in this application embodiment, when applied to an electronic device equipped with at least two cameras, offer the following beneficial effects:

[0042] The electronic device is equipped with at least two cameras. When a mirror is detected, it can simultaneously acquire at least two images containing different content through the target camera and at least one other camera, providing more information. It is suitable for scenarios such as doing homework or taking online classes, where both desktop information and user image information need to be detected.

[0043] If the target object in the image captured by the target camera is obscured by a mirror, the system can identify the missing area corresponding to the target object in the image, and then perform targeted completion processing to obtain a complete image. This ensures the imaging quality and completeness of the image captured by the target camera, improving the user experience. Attached Figure Description

[0044] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0045] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0046] Figure 2 This is a flowchart illustrating an implementation of the image processing method provided in an embodiment of this application;

[0047] Figure 3This is a schematic diagram showing that the target device object in the image captured by the target camera is obscured by a reflector in the image processing method provided in the embodiments of this application;

[0048] Figure 4 This is a schematic diagram of a missing region in an image captured by a target camera in the image processing method provided in the embodiments of this application;

[0049] Figure 5 A flowchart illustrating another implementation of the image processing method provided in this application embodiment;

[0050] Figure 6 This is a schematic diagram of the adjacent area in the image captured by the target camera in the image processing method provided in the embodiments of this application;

[0051] Figure 7 This is a flowchart illustrating another implementation of the image processing method provided in the embodiments of this application;

[0052] Figure 8 This is a schematic diagram of the structure of the image processing apparatus provided in the embodiments of this application;

[0053] Figure 9 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0056] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0057] It should be noted that the terms "first, second, third" used in the embodiments of this application are used to distinguish similar or different objects and do not represent a specific order of objects. It can be understood that "first, second, third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0058] Learning tablets, tutoring devices, and other electronic devices on the market with reflective mirrors often have dual or even multiple cameras working simultaneously. When a reflective mirror is placed in front of the shooting area of ​​one of the electronic device's cameras, such as the main camera, the main camera can capture an image of the desktop through the mirror, enabling functions such as acquiring textbook content and grading homework. At the same time, other cameras, such as auxiliary cameras placed next to the main camera, can capture the user's image, enabling functions such as video calls, interactive teaching, or user attention detection.

[0059] In the aforementioned scenario, the inclusion of a reflective mirror can negatively impact the auxiliary camera's image capture. Specifically, the edges of the mirror may be captured by the auxiliary camera, affecting the complete display of people or other objects. A common solution in this field is to distance the main camera (covered by the mirror) from the auxiliary camera to ensure proper image capture by the auxiliary camera. However, this approach has drawbacks. The excessive distance between the cameras, coupled with the fact that the main camera with the reflective mirror is typically positioned in the center of the electronic device, results in the auxiliary camera being too far from the center. This causes objects like people to be out of center in the captured image, impacting the user experience and potentially leading to the omission of objects in some practical applications.

[0060] In view of this, embodiments of this application provide an image processing method applied to an electronic device equipped with at least two cameras. Please refer to... Figure 1 , Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. For example... Figure 1 The electronic device 100 shown is equipped with two cameras, including camera 101 and camera 102, wherein the framed area 103 is the mounting area for the reflector. It can be understood that after the reflector is mounted on the framed area 103 of the electronic device, camera 101 can capture the image content of the specified area through the reflector, such as textbooks or homework on a desktop, etc., without limitation.

[0061] It should be noted that, as Figure 1 The electronic device shown is for illustrative purposes only. Electronic devices applying the image processing method provided in this application may include, but are not limited to, mobile phones, tablets, and laptops. The functions implemented by this method can be achieved by a processor in the electronic device calling program code. Of course, the program code can be stored in a computer storage medium. Therefore, the electronic device includes at least a processor and a storage medium.

[0062] Please see Figure 2 , Figure 2 This is a flowchart illustrating one implementation of the image processing method provided in an embodiment of this application. Figure 2As shown, the method may include the following steps 201 to 203:

[0063] Step 201: If the mirror is detected to be connected, an image is captured by the target camera.

[0064] In this embodiment of the application, when a reflector is detected to be in contact with the object, an image is captured by a target camera.

[0065] In this application, the reflector can be connected to the electronic device by means of adhesive bonding, magnetic adsorption, or snap-fit ​​structure. The specific connection method is determined according to the structure of the reflector and the electronic device, and is not limited here.

[0066] In some embodiments, the reflector is provided with a sensing element, and the electronic device is provided with a sensing sensor to detect the reflector access, including:

[0067] When the sensing sensor detects the sensing element, it detects the mirror connection.

[0068] It should be noted that sensing elements include magnets, Near Field Communication (NFC) tags, etc., and sensing sensors include magnetic sensors, NFC chips, etc. The types of sensing elements on the reflector correspond to the types of sensing sensors on the electronic device. For example, if the electronic device has a magnetic sensor, the reflector will have a magnet; or if the electronic device has an NFC chip, the reflector will have an NFC tag.

[0069] By incorporating the aforementioned sensors and sensing elements into the electronic device and the reflector, the electronic device can determine the reflector's connection status and, upon detecting the reflector's connection, acquire an image through the target camera. This allows the user to freely connect or remove the reflector as needed, control the target camera's acquisition operation, and use the image processing method provided in this application to complete the image acquired by the target camera, improving image quality and ensuring the normal display of the target device object in the image.

[0070] In some embodiments, at least two cameras include a main camera and an auxiliary camera, a reflector is mounted at a position facing the main camera's field of view, and images are acquired through the target camera, including:

[0071] Images are captured using an auxiliary camera.

[0072] It should be noted that when an electronic device includes a main camera and an auxiliary camera, and the target camera is the auxiliary camera, one side of the target object in the image captured by the auxiliary camera is blocked by a reflector.

[0073] In some embodiments, the target device object is a human image. See also Figure 3 , Figure 3 This is a schematic diagram showing a target device object being obscured by a reflector in an image captured by a target camera in an image processing method provided in an embodiment of this application. For example... Figure 3 As shown, because the main camera and the secondary camera are close together, when a reflector is placed in the shooting area of ​​the main camera, the image captured by the secondary camera will also include the image content of the reflector, such as... Figure 3 The upper left area obstructs the target device object (human image), affecting its display integrity. It should be noted that the target device object can also be textbooks, scenery, or other objects; this is not a limitation.

[0074] In some embodiments, at least two cameras include a main camera and an auxiliary camera, a reflector is mounted at a position facing the main camera's field of view, and images are acquired through the target camera, including:

[0075] Images are captured using the main camera.

[0076] It is understandable that, to ensure the stability of the reflector after it is connected to electronic devices, the frame support area of ​​the reflector, excluding the mirror area, has a certain width. When adjusting the reflector angle, there may be instances where the reflector frame area appears in the image captured by the main camera.

[0077] Since the main camera is often used to capture images of a desktop using a reflector, in some embodiments, when the target camera is the main camera and the target device is a textbook, the image processing method provided in this application fills in the missing areas of the textbook obscured by the reflector in the image captured by the main camera, obtaining a completed image.

[0078] In some embodiments, the electronic device has two or more cameras, which can be categorized according to whether the reflector faces the camera's field of view. Cameras facing the reflector are designated as main cameras, and cameras outside the reflector are designated as auxiliary cameras. In the presence of multiple main cameras or multiple auxiliary cameras, images captured by the main cameras or auxiliary cameras are acquired using image fusion technology. The acquired images are then processed using the image processing method provided in this embodiment, as needed.

[0079] In some embodiments, although the electronic device is equipped with two or more cameras, not all cameras participate in image acquisition. Therefore, selecting the target camera to acquire images according to actual needs and performing image completion processing on the acquired images improves the applicability of the method.

[0080] Step 202: When the target object in the image is obscured by a mirror, identify the missing region corresponding to the target object in the image.

[0081] In this embodiment of the application, when the target object in the image is obscured by a mirror, the missing area corresponding to the target object in the image is identified.

[0082] In some embodiments, the target object is a human image; see [link to relevant documentation]. Figure 4 , Figure 4 This is a schematic diagram of the missing region in the image captured by the target camera in the image processing method provided in the embodiments of this application. Figure 4 The shaded area represents the missing area of ​​the image obscured by the reflector. Once the missing area is determined, it can be filled in using the method provided in this application. In practical applications, the structure of the reflector and the selection of the target device may differ; the above-mentioned appendix... Figure 4 For illustrative purposes only.

[0083] In some embodiments, identifying the missing region corresponding to the target object in the image includes:

[0084] Obtain the target location of the target object in the image;

[0085] Based on the target location and preset image size, the image is selected by bounding a box to obtain the target image;

[0086] Identify the missing regions corresponding to the target object in the target image.

[0087] It should be noted that, in addition to directly identifying the missing region in the image captured by the target camera, the image processing method provided in this application can, in some embodiments, first perform bounding processing on the captured image according to the target position of the target object to obtain a target image with a reduced size, and then identify the corresponding missing region in the target image.

[0088] The target location of the object in the image can be obtained through computer vision techniques, such as the corresponding recognition algorithms, including YOLO (You Only Look Once) and SSD (SingleShot Multi-Box Detector). The preset image size is determined based on the resolution of the image captured by the target camera.

[0089] The resulting target image is smaller than the original image captured by the target camera, while the target object can be magnified and displayed in the center area of ​​the target image, improving the user's viewing experience and usability.

[0090] In some embodiments, after performing the above-described bounding box processing on the image to obtain the target image, the missing regions are filled in to obtain the corresponding filled image, including:

[0091] The missing regions are filled in to obtain the filled image corresponding to the target image.

[0092] In some embodiments, identifying the missing region corresponding to the target object in the image includes:

[0093] Based on the preset first artificial intelligence (AI) model, identify the missing region corresponding to the target object in the image.

[0094] It should be noted that the first artificial intelligence AI model, hereinafter referred to as the first AI model, is used to obtain the missing region corresponding to the target object in the image captured by the target camera.

[0095] In some embodiments, the training process of the first AI model is as follows:

[0096] Acquire and label a series of image datasets that meet the preset order of magnitude requirements. These datasets contain mirrors of different types and sizes, as well as different types of target objects that they occlude in various scenarios. The different types of target objects include portraits, textbooks, school supplies, etc., which are not limited here.

[0097] The dataset was divided into training and validation sets, and the first AI model was trained using supervised learning. During training, the model learned how to identify missing parts from complete images. The first AI model used its deep network structure to extract key features from the images, including edges, textures, and color distribution.

[0098] After model training is complete, a series of optimization steps are required to improve the model's generalization ability and processing speed. This may include adjusting the network structure, using transfer learning techniques, or applying model pruning techniques. The training of the first AI model is considered complete when its recognition accuracy exceeds the preset expected accuracy.

[0099] It should be noted that the algorithm technology for recognizing human faces or specific objects is quite mature, and the shape of the parts that are obstructed by the mirror, such as the frame, is usually relatively fixed. After the mirror is connected to an electronic device, the area obstructed by the mirror is not easily changed.

[0100] Therefore, in some embodiments, identifying the missing region corresponding to the target object in the image includes:

[0101] Obtain the image region corresponding to the mirror in the image;

[0102] According to the preset recognition algorithm, the image contour corresponding to the target object is obtained;

[0103] Predict the complete contour of the target device object based on the image contour, wherein the complete contour includes the predicted contour of the area occluded by the mirror.

[0104] Based on the complete outline of the target device object, the image area corresponding to the reflector is cut to obtain the missing area.

[0105] Step 203: Complete the missing regions to obtain the completed image.

[0106] In this embodiment of the application, after obtaining the missing region corresponding to the target object in the image, the missing region is filled in to obtain the corresponding filled image. In the filled image, the original missing region of the target object is visually seamlessly connected with the unobstructed region, which enhances user satisfaction.

[0107] In some embodiments, the missing regions are filled in to obtain a filled image corresponding to the image, including:

[0108] Based on the preset second artificial intelligence (AI) model, the missing regions are filled in to obtain the corresponding filled image.

[0109] It should be noted that the second artificial intelligence (AI) model, hereinafter referred to as the second AI model, is used to complete the missing regions corresponding to the target recognition object.

[0110] In some embodiments, the training process of the second AI model is similar to that of the first AI model described above. The second AI model is trained using a large dataset of labeled images. This dataset should contain images with various missing parts and their completed versions, to train the second AI model to learn how to fill in the missing parts.

[0111] The second AI model learns the typical features and structure of the target object, such as key points of a face or the layout of a textbook, to predict the content of the missing region. In some embodiments, the second AI model is a generative model such as a generative adversarial network (GAN) or a variational autoencoder (VAE), which can generate image content of the missing region that blends naturally with the surrounding image.

[0112] It should be noted that in some embodiments, the first AI model and the second AI model are the same model, that is, a composite AI model that integrates recognition and completion functions. It adopts a joint training strategy, learning to recognize missing regions and complete images simultaneously. This design integrates two functions, improving processing efficiency.

[0113] In some embodiments, after completing the missing region corresponding to the target object, the method further includes: performing detail optimization processing on the completed image, wherein the detail optimization processing includes adjusting the color and brightness to match the original image, and using image sharpening and other techniques to improve the clarity of the completed region.

[0114] By implementing the above technical solution, electronic devices can intelligently identify the missing areas and perform targeted supplementation to address image loss caused by mirror obstruction, resulting in clearer and more accurate images. This design not only optimizes the efficiency of image acquisition and processing but also further enhances the user experience.

[0115] The image processing method provided in this application can also be performed through other steps. Please refer to [link / reference]. Figure 5 , Figure 5 This is a flowchart illustrating another implementation of the image processing method provided in an embodiment of this application. For example... Figure 5 As shown, the method may include the following steps 501 to 504:

[0116] Step 501: If the mirror is detected to be connected, an image is captured by the target camera.

[0117] Step 502: When the image of a person is obscured by a mirror, identify the missing area corresponding to the image of the person.

[0118] In some embodiments, the target object is a human image. When the target object is obscured by a mirror in the image, identifying the missing region corresponding to the target object in the image includes:

[0119] When a person's image is obscured by a mirror in an image, identify the missing area corresponding to the person's image in the image.

[0120] The method for identifying target objects, such as missing areas corresponding to human images, has been explained in the aforementioned instruction manual and will not be repeated here.

[0121] In some embodiments, the target object is a human image. Missing regions are filled in to obtain a completed image, including:

[0122] Based on the preset symmetry of the human figure and the human figure area in the image that is not obscured by the mirror, the missing area is filled in to obtain the corresponding filled image.

[0123] The pre-defined symmetry relationship in human face recognition refers to the general symmetry of facial features. Typically, the left and right sides of a person's face share structural similarities, particularly in the layout and shape of facial features. For example, the position, size, and shape of the left and right eyes are often symmetrical, as are the nose, mouth, and ears. When one side of a person's face in an image is not obscured by a mirror, the symmetry relationship can be used to infer the obscured area. Then, the unobscured areas of the face, especially those symmetrical to the obscured areas, are used as references to fill in the missing areas. For instance, if a mirror obscures the upper right side of a person's face, the missing area can be inferred and filled in based on the unobscured features on the upper left side.

[0124] The completion process can be based on various image processing techniques, such as interpolation and machine learning model prediction, and is not limited here. For example, a deep learning model can be used, which has been trained on a large amount of facial image data and can learn the symmetry and structural features of the face, thereby predicting and completing the missing parts based on the unoccluded parts.

[0125] After such completion processing, the resulting completed image will restore the human figure area in the image captured by the target camera as much as possible, making the image more complete and reducing the negative impact on users caused by the obstruction of their personal portrait. At the same time, electronic devices can perform interactive operations based on the completed portrait, such as attention detection and emotion recognition.

[0126] In some embodiments, the missing area includes not only the face area of ​​the portrait but also the clothing area. Although the clothing as a whole also possesses a certain degree of symmetry, the continuity of texture, color, and shape of the clothing area must also be considered. Key features, such as color, texture, and shape, can be extracted by analyzing the unoccluded clothing portions of the image. These features can then be used to infer the clothing features of the occluded portion.

[0127] Step 503: Obtain the neighboring regions of the portrait region in the image that are within a preset distance from the missing region.

[0128] In some embodiments, the neighboring regions of the portrait region within a preset distance from the missing region in the image are obtained.

[0129] It should be noted that the preset distance is set based on the actual application scenario and image resolution to ensure sufficient correlation between the acquired neighboring areas and the missing areas, while avoiding the introduction of too much irrelevant information. The preset distance can be adjusted through experimentation and experience to optimize the completion effect, and is not limited here.

[0130] Please see Figure 6 , Figure 6This is a schematic diagram of a neighboring region in an image captured by a target camera in an image processing method provided in an embodiment of this application. For example... Figure 6 As shown, in some embodiments, the target device object is a human image, and the shaded area in the figure is the neighboring area of ​​the human image area within a preset distance of the missing area.

[0131] Obtaining neighboring regions can provide richer information for subsequent missing region completion. For example, if the missing region is located on the face of a portrait, the neighboring regions may include areas adjacent to the missing region such as hair, face, and neck. These areas have certain similarities in color and texture with the missing region of the face, which helps to more accurately complete the missing part.

[0132] Compared to simply completing a portrait based on its symmetry, incorporating information from neighboring areas can further enrich the content and details of the completion, resulting in a more accurate and natural completion.

[0133] Step 504: Based on the preset symmetry relationship of the human face and the pixel color of the neighboring area, the missing area is filled in to obtain the corresponding filled image.

[0134] In some embodiments, based on a preset portrait symmetry relationship and the pixel colors of neighboring regions, the missing regions are filled in to obtain a filled image corresponding to the image, including:

[0135] Obtain the color change trend of pixels in the neighboring region;

[0136] Based on the color change trend of pixels in neighboring areas and the preset symmetry relationship of the human figure, the pixel colors of the missing areas are filled in to obtain the corresponding filled image.

[0137] It's important to note that obtaining the color variation trend of pixels in neighboring areas is crucial. This step aims to more accurately understand the distribution and changes in pixel colors in neighboring areas, providing a reliable basis for subsequent completion processing. By acquiring, analyzing, and statistically processing the pixel color information of neighboring areas, we can obtain the color variation trend, such as color gradations and transitions. This allows us to generate pixel colors that harmonize with those of neighboring areas when completing missing regions, based on these color variation trends. Furthermore, we can adjust and refine the completion result according to the symmetry of the human figure. This results in a more natural and coherent image in terms of color and texture, improving the accuracy and realism of the completion.

[0138] In some embodiments, based on a preset portrait symmetry relationship and the pixel colors of neighboring regions, the missing regions are filled in to obtain a filled image corresponding to the image, including:

[0139] Based on the preset symmetry of the human figure and the pixel colors of the adjacent areas, the pixel colors of the missing areas are filled in.

[0140] The missing and adjacent regions after completion are magnified to obtain the corresponding completed image.

[0141] It should be noted that after the missing area is filled in, there may be differences or abrupt changes in details such as color and texture at the splicing point between the filled missing area and the adjacent area, resulting in obvious splicing marks and affecting the overall visual effect of the image.

[0142] The missing and adjacent regions after completion are magnified. The pixels at the stitching point of the missing and adjacent regions are magnified, and the colors of the overlapping pixels after magnification are smoothed. For example, the RGB values ​​are averaged to make the completed region and the adjacent regions more visually harmonious and consistent.

[0143] The methods of the two embodiments described above can be implemented simultaneously or one of them can be selected, and no limitation is made here.

[0144] By implementing the above technical solution, the detail representation of the missing area after completion is enhanced, making it blend better with the surrounding neighboring areas. Through magnification processing, the pixel colors in the completed area can transition more smoothly, reducing completion artifacts and improving the overall image quality.

[0145] Please see Figure 7 , Figure 7 This is a flowchart illustrating another implementation of the image processing method provided in an embodiment of this application. For example... Figure 7 As shown, the method may include the following steps 701 to 704:

[0146] Step 701: If the mirror is detected to be connected, an image is captured by the target camera.

[0147] In some embodiments, when a mirror is detected to be in contact with the object, a target camera is activated to capture images.

[0148] In this way, users of electronic devices can easily start or stop the target camera's image acquisition function by controlling whether the reflector is connected. When the reflector is connected, the system automatically recognizes and activates the target camera, thus beginning image acquisition. This design not only simplifies the user's operation but also improves the system's intelligence.

[0149] In some embodiments, the user opens a specified application or process, such as an online homework grading application or a video call process. Afterward, the electronic device detects whether the rearview mirror is connected, and if the rearview mirror is connected, implements the method provided in this application.

[0150] Step 702: When the target object in the image is obscured by a mirror, identify the missing region corresponding to the target object in the image.

[0151] Step 703: Complete the missing regions to obtain the completed image.

[0152] Step 704: Determine whether there is a target object in the image captured by the target camera within a preset time; if not, turn off at least two cameras.

[0153] In some embodiments, it is determined whether a target object exists in the image captured by the target camera within a preset time period; if not, at least two cameras are turned off.

[0154] This saves energy and avoids unnecessary acquisition and processing of image data.

[0155] It should be understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0156] Based on the foregoing embodiments, this application provides an image processing device, which includes various modules and units included in each module, and can be implemented by a processor; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP) or field programmable gate array (FPGA), etc.

[0157] Figure 8 This is a schematic diagram of the structure of the image processing apparatus provided in the embodiments of this application, such as... Figure 8 As shown, the image processing device 800 includes an acquisition module 801 and a completion module 802, wherein:

[0158] The acquisition module 801 is used to acquire images through the target camera when a reflector is detected in the field of view;

[0159] The completion module 802 is used to identify the missing region corresponding to the target object in the image when the target object is obscured by the mirror; and to complete the missing region to obtain the completed image.

[0160] In some embodiments, the at least two cameras include a main camera and an auxiliary camera, the reflector is connected at a position facing the shooting range of the main camera, and the acquisition module 801 is also used to acquire the image through the auxiliary camera.

[0161] In some embodiments, the completion module 802 is further configured to obtain the target position of the target recognition object in the image; perform bounding selection processing on the image according to the target position and a preset image size to obtain a target image; and identify the missing region corresponding to the target recognition object in the target image.

[0162] In some embodiments, the completion module 802 is further configured to identify the missing region corresponding to the target object in the image based on a preset first artificial intelligence (AI) model.

[0163] In some embodiments, the completion module 802 is further configured to perform completion processing on the missing region according to a preset second artificial intelligence (AI) model to obtain the completed image corresponding to the image.

[0164] In some embodiments, the completion module 802 is further configured to complete the missing region according to a preset portrait symmetry relationship and the portrait area in the image that is not obscured by the reflector, so as to obtain a completed image corresponding to the image.

[0165] In some embodiments, the completion module 802 is further configured to obtain the neighboring regions of the portrait region within a preset distance from the missing region in the image; and to perform completion processing on the missing region according to the preset portrait symmetry relationship and the pixel color of the neighboring region to obtain the completed image corresponding to the image.

[0166] In some embodiments, the completion module 802 is further configured to obtain the color change trend of the pixels in the neighboring region; and to perform completion processing on the pixel colors of the missing region according to the color change trend of the pixels in the neighboring region and the preset portrait symmetry relationship, so as to obtain the completed image corresponding to the image.

[0167] In some embodiments, the completion module 802 is further configured to complete the pixel color of the missing region according to the preset portrait symmetry relationship and the pixel color of the neighboring region; and to enlarge the completed missing region and the neighboring region to obtain the completed image corresponding to the image.

[0168] In some embodiments, the reflector is provided with a sensing element, the electronic device is provided with a sensing sensor, and the acquisition module 801 is further configured to identify the reflector access when the sensing sensor identifies the sensing element.

[0169] In some embodiments, the acquisition module 801 is further configured to determine whether the target identification object exists in the image acquired by the target camera within a preset time period; if not, the at least two cameras are turned off.

[0170] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0171] It should be noted that, in the embodiments of this application... Figure 8 The module division of the image processing apparatus shown is illustrative and represents only one logical functional division; in actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, exist as separate physical units, or be integrated into one unit with two or more units. The integrated units described above can be implemented in hardware, as software functional units, or in a combination of software and hardware.

[0172] It should be noted that, in the embodiments of this application, if the above-described methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0173] This application provides a computer device, which may be a server, and its internal structure diagram may be as follows: Figure 9As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements the methods described above.

[0174] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method provided in the above embodiments.

[0175] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the method provided in the above-described method embodiments.

[0176] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0177] In one embodiment, the image processing apparatus provided in this application can be implemented as a computer program, and the computer program can be implemented as follows: Figure 9 The device operates on the computer device shown. The memory of the computer device can store the various program modules that make up the above-described apparatus. The computer program, composed of the various program modules, causes the processor to execute the steps of the methods in the various embodiments of this application described in this specification.

[0178] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium, storage medium, and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0179] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, they will not be repeated here.

[0180] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three kinds of relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0181] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0182] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or modules can be electrical, mechanical, or other forms.

[0183] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0184] In addition, each functional module in the various embodiments of this application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the integrated modules can be implemented in hardware or in the form of hardware plus software functional units.

[0185] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0186] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0187] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0188] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0189] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0190] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, The method is applied to an electronic device equipped with at least two cameras, and the method includes: When a mirror is detected, an image is captured by the target camera. When the target object in the image is obscured by the mirror, identify the missing region corresponding to the target object in the image; The missing regions are filled in to obtain the filled image corresponding to the original image.

2. The method according to claim 1, characterized in that, The at least two cameras include a main camera and an auxiliary camera, the reflector is positioned facing the main camera's field of view, and the image acquisition via the target camera includes: The images are captured using the auxiliary camera.

3. The method according to claim 1, characterized in that, The process of identifying the missing region corresponding to the target object in the image includes: Obtain the target location of the target object in the image; Based on the target location and the preset image size, the image is selected by bounding to obtain the target image; Identify the missing region corresponding to the target object in the target image.

4. The method according to claim 1, characterized in that, The process of identifying the missing region corresponding to the target object in the image includes: Based on a preset first artificial intelligence (AI) model, the missing region corresponding to the target object in the image is identified.

5. The method according to claim 1, characterized in that, The process of filling in the missing region to obtain the filled image corresponding to the image includes: Based on a preset second artificial intelligence (AI) model, the missing region is filled in to obtain the filled image corresponding to the original image.

6. The method according to claim 1, characterized in that, The target object to be identified is a human image, and the process of filling in the missing regions to obtain the filled image includes: Based on the preset symmetry of the human figure and the human figure area in the image that is not obscured by the mirror, the missing area is filled in to obtain the filled image corresponding to the image.

7. The method according to claim 6, characterized in that, The step of filling in the missing area based on a preset portrait symmetry relationship and the portrait area in the image not obscured by the mirror, to obtain the corresponding filled image, includes: Obtain the neighboring regions of the human figure region in the image that are within a preset distance from the missing region; Based on the preset symmetry relationship of the human face and the pixel color of the neighboring area, the missing area is filled in to obtain the filled image corresponding to the image.

8. The method according to claim 7, characterized in that, The step of performing a completion process on the missing region based on the preset portrait symmetry relationship and the pixel color of the neighboring region to obtain the completed image corresponding to the image includes: Obtain the color change trend of pixels in the neighboring region; Based on the color change trend of the pixels in the neighboring areas and the preset symmetry relationship of the human figure, the pixel colors of the missing areas are filled in to obtain the filled image corresponding to the image.

9. The method according to claim 7, characterized in that, The step of performing a completion process on the missing region based on the preset portrait symmetry relationship and the pixel color of the neighboring region to obtain the completed image corresponding to the image includes: Based on the preset portrait symmetry relationship and the pixel color of the adjacent area, the pixel color of the missing area is filled in; The missing region and the adjacent region after completion are magnified to obtain the completed image corresponding to the image.

10. The method according to any one of claims 1-9, characterized in that, The reflector is equipped with a sensing element, and the electronic device is equipped with a sensing sensor. The process of detecting the reflector connection includes: When the sensing sensor detects the sensing element, the mirror is detected to be connected.

11. The method according to any one of claims 1-9, characterized in that, After acquiring images via the target camera, the method further includes: Determine whether the target object exists in the image captured by the target camera within a preset time period; If not, turn off at least two cameras.

12. An image processing apparatus, characterized in that, The device is equipped with at least two cameras, and the device includes: The acquisition module is used to acquire images through the target camera when a mirror is detected in the field of view. The completion module is used to identify the missing region corresponding to the target object in the image when the target object is obscured by the mirror; and to complete the missing region to obtain the completed image.

13. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 11.

Images