Electronic device and method for protecting personal information included in an image

Abstract

The electronic device according to the embodiment includes: a camera including an image sensor; a communication module; a memory; and a processor operably connected to the camera, the communication module and the memory. The memory may store instructions that, when executed, instruct the processor to: acquire a first image including an external object by using the camera; acquire a second image including less information than the first image by using at least a portion of the first image; send the second image to an external electronic device through the communication module; receive security information corresponding to the second image from the external electronic device through the communication module; and protect the first image based at least in part on the security information. In addition, various other embodiments are also possible.

Description

Technical Field

[0001] One or more embodiments generally relate to electronic devices and methods for protecting personal information included in an image. Background Art

[0002] An electronic device, such as a smart phone, a tablet PC, a portable multimedia player (PMP), a personal digital assistant (PDA), a laptop personal computer (laptop PC), a watch, or a head-mounted display, may include one or more cameras and may capture an image by using the camera.

[0003] A user may upload an image captured by a camera of an electronic device to an external electronic device, such as an electronic device or a server of a social networking service (SNS).

[0004] The above information is presented as background information only to assist with an understanding of the present disclosure. No determination has been made, and no assertion is made, as to whether any of the above might be applicable as prior art with respect to the present disclosure. Summary of the invention

[0005] Technical issues

[0006] With the increasing popularity of high-speed and high-capacity wireless communications deployed on the above-mentioned electronic devices, high-quality original images (eg, raw images) captured by the above-mentioned cameras can be uploaded to SNS servers.

[0007] The server or the external electronic device may apply an image quality enhancement algorithm to the high-quality original image received from the electronic device to enhance the image quality, and may provide the image with the enhanced quality back to the electronic device.

[0008] If a user of an electronic device uploads an image to an external electronic device or shares an image with another user through a social networking service, personal information may be exposed or leaked to other people regardless of the user's intention. For example, one of the high-quality images may include the user's personal information (e.g., fingerprint information, iris information, or facial information).

[0009] Recently, illegal hacker attacks on servers have occurred frequently, and information illegally obtained through hacker attacks may cause great damage to users.

[0010] Solution to the problem

[0011] According to an embodiment, the electronic device may include: a camera including an image sensor; a communication module; a memory; and a processor operably connected to the camera, the communication module, and the memory. The memory may store instructions that, when executed, cause the processor to: acquire a first image including an external object by using the camera; acquire a second image including less information than the first image by using at least a portion of the first image; send the second image to an external electronic device through the communication module; receive security information corresponding to the second image from the external electronic device through the communication module; and protect the first image based at least in part on the security information.

[0012] According to an embodiment, a method for protecting personal information included in an image may include: acquiring a first image including an external object by using a camera including an image sensor; acquiring a second image including less information than the first image by using at least a portion of the first image; sending the second image to an external electronic device through a communication module; receiving security information corresponding to the second image from the external electronic device through the communication module; and protecting the first image at least partially based on the security information.

[0013] According to an embodiment, the electronic device may include: a communication module; a memory; and a processor operably connected to the communication module and the memory. The memory stores instructions, which, when executed, cause the processor to: receive an image from an external electronic device by using the communication module; identify at least one image region in the image; determine security information included in the image and associated with the at least one image region based at least in part on the result of the identification; and send the security information to the external electronic device by using the communication module.

[0014] Additional aspects will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the presented embodiments.

[0015] Advantageous Effects of the Invention

[0016] One aspect of the embodiments disclosed herein is to prevent unintended leakage of user personal information that may be included in an image.

[0017] According to certain embodiments, an electronic device and method for protecting personal information included in an image may prevent the personal information included in the image from being leaked to the public.

[0018] According to certain embodiments, in an electronic device and method for protecting personal information included in an image, only user personal information included in an image may be displayed on the user's electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] For a more thorough understanding of the present disclosure and its advantages, reference is now made to the following description in conjunction with the accompanying drawings, wherein like reference numerals represent like parts:

[0020] Figure 1 is a block diagram of an electronic device in a network environment according to an embodiment;

[0021] Figure 2 is a block diagram showing a camera module according to an embodiment;

[0022] Figure 3 is a block diagram illustrating operations of an electronic device and an external electronic device according to an embodiment;

[0023] Figure 4 is a block diagram of an electronic device according to an embodiment;

[0024] Figure 5 is a flowchart illustrating a method for protecting personal information included in an image in an electronic device according to an embodiment;

[0025] Figure 6 is a flow chart illustrating a method for protecting a first image in an electronic device according to an embodiment;

[0026] Figure 7 is a flow chart illustrating a method for protecting a first image in an electronic device according to an embodiment;

[0027] Figure 8 is a flowchart illustrating a method for protecting personal information included in an image in a system according to an embodiment; and

[0028] 9A illustrates an example of a method for protecting personal information according to an embodiment, FIG. 9B illustrates an example of a method for protecting personal information according to an embodiment, and FIG. 9C illustrates an example of a method for protecting personal information according to an embodiment. DETAILED DESCRIPTION

[0029] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0030] Figure 1 1 is a block diagram illustrating an electronic device 101 in a network environment 100 according to various embodiments. Figure 1, the electronic device 101 in the network environment 100 may communicate with the electronic device 102 via the first network 198 (e.g., a short-range wireless communication network), or communicate with the electronic device 104 or the server 108 via the second network 199 (e.g., a long-range wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 via the server 108. According to an embodiment, the electronic device 101 may include a processor 120, a memory 130, an input device 150, a sound output device 155, a display device 160, an audio module 170, a sensor module 176, an interface 177, a haptic module 179, a camera module 180, a power management module 188, a battery 189, a communication module 190, a user identification module (SIM) 196, or an antenna module 197. In some embodiments, at least one of the components (e.g., the display device 160 or the camera module 180) may be omitted from the electronic device 101, or one or more other components may be added to the electronic device 101. In some embodiments, some of the components may be implemented as a single integrated circuit.For example, the sensor module 176 (eg, a fingerprint sensor, an iris sensor, or an illumination sensor) may be implemented to be embedded in the display device 160 (eg, a display).

[0031] The processor 120 may run, for example, software (e.g., program 140) to control at least one other component (e.g., hardware component or software component) of the electronic device 101 connected to the processor 120, and may perform various data processing or calculations. According to an embodiment, as at least part of the data processing or calculation, the processor 120 may load a command or data received from another component (e.g., sensor module 176 or communication module 190) into the volatile memory 132, process the command or data stored in the volatile memory 132, and store the result data in the non-volatile memory 134. According to an embodiment, the processor 120 may include a main processor 121 (e.g., a central processing unit (CPU) or an application processor (AP)) and an auxiliary processor 123 (e.g., a graphics processing unit (GPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that is independent of or combined with the main processor 121 in operation. Additionally or alternatively, the auxiliary processor 123 may be adapted to consume less power than the main processor 121, or adapted to be specifically used for a specified function. The auxiliary processor 123 may be implemented separately from the main processor 121 , or as part of the main processor 121 .

[0032] When the main processor 121 is in an inactive (e.g., sleep) state, the auxiliary processor 123 may control at least some of the functions or states related to at least one component (e.g., display device 160, sensor module 176, or communication module 190) among the components of the electronic device 101 (not the main processor 121), or when the main processor 121 is in an active state (e.g., running an application), the auxiliary processor 123 may control at least some of the functions or states related to at least one component (e.g., display device 160, sensor module 176, or communication module 190) among the components of the electronic device 101 together with the main processor 121. According to an embodiment, the auxiliary processor 123 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., camera module 180 or communication module 190) that is functionally related to the auxiliary processor 123.

[0033] The memory 130 may store various data used by at least one component of the electronic device 101 (e.g., the processor 120 or the sensor module 176). The various data may include, for example, software (e.g., the program 140) and input data or output data for commands related thereto. The memory 130 may include a volatile memory 132 or a nonvolatile memory 134.

[0034] The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system (OS) 142 , middleware 144 , or applications 146 .

[0035] The input device 150 may receive commands or data from outside the electronic device 101 (e.g., a user) to be used by other components of the electronic device 101 (e.g., the processor 120). The input device 150 may include, for example, a microphone, a mouse, a keyboard, or a digital pen (e.g., a stylus).

[0036] The sound output device 155 can output a sound signal to the outside of the electronic device 101. The sound output device 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or playing records, and the receiver can be used for incoming calls. Depending on the embodiment, the receiver can be implemented as a separate part from the speaker, or as a part of the speaker.

[0037] The display device 160 may visually provide information to the outside of the electronic device 101 (e.g., a user). The display device 160 may include, for example, a display, a holographic device, or a projector, and a control circuit for controlling a corresponding one of the display, the holographic device, and the projector. According to an embodiment, the display device 160 may include a touch circuit adapted to detect a touch or a sensor circuit (e.g., a pressure sensor) adapted to measure the strength of a force caused by a touch.

[0038] The audio module 170 may convert sound into an electrical signal, or vice versa. According to an embodiment, the audio module 170 may obtain sound via the input device 150, or output sound via the sound output device 155 or an earphone of an external electronic device (e.g., electronic device 102) directly (e.g., wired) or wirelessly connected to the electronic device 101.

[0039] The sensor module 176 may detect an operating state (e.g., power or temperature) of the electronic device 101 or an environmental state (e.g., a user's state) outside the electronic device 101, and then generate an electrical signal or data value corresponding to the detected state. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illumination sensor.

[0040] The interface 177 may support one or more specific protocols to be used to connect the electronic device 101 directly (e.g., wired) or wirelessly with an external electronic device (e.g., the electronic device 102). According to an embodiment, the interface 177 may include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0041] The connection end 178 may include a connector, wherein the electronic device 101 can be physically connected to an external electronic device (e.g., the electronic device 102) via the connector. According to an embodiment, the connection end 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0042] The haptic module 179 may convert the electrical signal into mechanical stimulation (eg, vibration or motion) or electrical stimulation that can be recognized by the user via his sense of touch or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0043] The camera module 180 may capture still images or moving images. According to embodiments, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

[0044] The power management module 188 may manage power supply to the electronic device 101. According to an embodiment, the power management module 188 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0045] The battery 189 may power at least one component of the electronic device 101. According to an embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0046] The communication module 190 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. The communication module 190 may include one or more communication processors capable of operating independently from the processor 120 (e.g., an application processor (AP)) and supporting direct (e.g., wired) communication or wireless communication. According to an embodiment, the communication module 190 may include a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules can communicate with an external electronic device via a first network 198 (e.g., a short-range communication network such as Bluetooth, Wireless Fidelity (Wi-Fi) Direct, or Infrared Data Association (IrDA)) or a second network 199 (e.g., a long-range communication network such as a cellular network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be implemented as a single component (e.g., a single chip), or these various types of communication modules can be implemented as multiple components separated from each other (e.g., multiple chips). The wireless communication module 192 can identify and authenticate the electronic device 101 in a communication network (such as the first network 198 or the second network 199) using user information (e.g., an International Mobile Subscriber Identity (IMSI)) stored in the user identification module 196.

[0047] The antenna module 197 may transmit or receive a signal or power to or from the outside of the electronic device 101 (e.g., an external electronic device). According to an embodiment, the antenna module 197 may include an antenna including a radiating element formed of a conductive material or a conductive pattern formed in or on a substrate (e.g., a PCB). According to an embodiment, the antenna module 197 may include a plurality of antennas. In this case, at least one antenna suitable for a communication scheme used in a communication network (such as the first network 198 or the second network 199) may be selected from the plurality of antennas by, for example, the communication module 190 (e.g., the wireless communication module 192). A signal or power may then be transmitted or received between the communication module 190 and the external electronic device via the selected at least one antenna. According to an embodiment, another component (e.g., a radio frequency integrated circuit (RFIC)) other than the radiating element may be additionally formed as a part of the antenna module 197.

[0048] At least some of the above components may be connected to each other via an inter-peripheral communication scheme (e.g., a bus, a general purpose input output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)) and communicatively transmit signals (e.g., commands or data) therebetween.

[0049] According to an embodiment, a command or data may be sent or received between the electronic device 101 and the external electronic device 104 via the server 108 connected to the second network 199. Each of the electronic device 102 and the electronic device 104 may be a device of the same type as the electronic device 101, or a device of a different type from the electronic device 101. According to an embodiment, all or some operations to be executed in the electronic device 101 may be executed in one or more of the external electronic device 102, the external electronic device 104, or the server 108. For example, if the electronic device 101 should automatically execute a function or service or should execute a function or service in response to a request from a user or another device, the electronic device 101 may request the one or more external electronic devices to execute at least part of the function or service instead of executing the function or service, or the electronic device 101 may request the one or more external electronic devices to execute at least part of the function or service in addition to executing the function or service. The one or more external electronic devices that receive the request may execute the requested at least part of the function or service, or execute another function or another service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may provide the result as at least a partial reply to the request with or without further processing the result. To this end, for example, cloud computing technology, distributed computing technology, or client-server computing technology may be used.

[0050] Figure 2 2 is a block diagram showing a camera module 180 according to various embodiments. Figure 2 , the camera module 180 may include a lens assembly 210, a flash 220, an image sensor 230, an image stabilizer 240, a memory 250 (e.g., a buffer memory), or an image signal processor 260. The lens assembly 210 may collect light emitted or reflected from an object whose image is to be captured. The lens assembly 210 may include one or more lenses. According to an embodiment, the camera module 180 may include a plurality of lens assemblies 210. In this case, the camera module 180 may form, for example, a dual camera, a 360-degree camera, or a spherical camera. Some of the plurality of lens assemblies 210 may have the same lens properties (e.g., viewing angle, focal length, autofocus, f-number, or optical zoom), or at least one lens assembly may have one or more lens properties different from those of another lens assembly. The lens assembly 210 may include, for example, a wide-angle lens or a telephoto lens.

[0051] The flash 220 may emit light, wherein the emitted light is used to enhance light reflected from an object. According to an embodiment, the flash 220 may include one or more light emitting diodes (LEDs) (e.g., red, green, blue (RGB) LEDs, white LEDs, infrared (IR) LEDs, or ultraviolet (UV) LEDs) or xenon lamps. The image sensor 230 may acquire an image corresponding to an object by converting light emitted or reflected from an object and transmitted through the lens assembly 210 into an electrical signal. According to an embodiment, the image sensor 230 may include one image sensor selected from a plurality of image sensors having different properties (e.g., an RGB sensor, a black and white (BW) sensor, an IR sensor, or a UV sensor), a plurality of image sensors having the same properties, or a plurality of image sensors having different properties. Each image sensor included in the image sensor 230 may be implemented using, for example, a charge coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor.

[0052] The image stabilizer 240 may move the image sensor 230 or at least one lens included in the lens assembly 210 in a specific direction, or control an operational property of the image sensor 230 (e.g., adjust the readout timing) in response to the movement of the camera module 180 or the electronic device 101 including the camera module 180. In this way, at least a portion of the negative effects (e.g., image blur) generated due to the movement of the image being captured is allowed to be compensated. According to an embodiment, the image stabilizer 240 may sense such movement of the camera module 180 or the electronic device 101 using a gyro sensor (not shown) or an acceleration sensor (not shown) arranged inside or outside the camera module 180. According to an embodiment, the image stabilizer 240 may be implemented as, for example, an optical image stabilizer.

[0053] The memory 250 may at least temporarily store at least a portion of an image acquired via the image sensor 230 for subsequent image processing tasks. For example, if multiple images are captured quickly or image capture is delayed due to shutter lag, the acquired original image (e.g., Bayer pattern image, high-resolution image) may be stored in the memory 250, and its corresponding copy image (e.g., low-resolution image) may be previewed via the display device 160. Then, if a specified condition is met (e.g., by a user's input or a system command), at least a portion of the original image stored in the memory 250 may be acquired and processed by, for example, the image signal processor 260. According to an embodiment, the memory 250 may be configured as at least a portion of the memory 130, or the memory 250 may be configured as a separate memory that operates independently of the memory 130.

[0054] The image signal processor 260 may perform one or more image processing on an image acquired via the image sensor 230 or an image stored in the memory 250. The one or more image processing may include, for example, depth map generation, three-dimensional (3D) modeling, panorama generation, feature point extraction, image synthesis, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softening). Additionally or alternatively, the image signal processor 260 may perform control (e.g., exposure time control or readout timing control) on at least one of the components included in the camera module 180 (e.g., image sensor 230). The image processed by the image signal processor 260 may be stored back to the memory 250 for further processing, or the image may be provided to an external component outside the camera module 180 (e.g., memory 130, display device 160, electronic device 102, electronic device 104, or server 108). According to an embodiment, the image signal processor 260 may be configured as at least a part of the processor 120, or the image signal processor 260 may be configured as a separate processor that operates independently of the processor 120. If the image signal processor 260 is configured as a separate processor from the processor 120, at least one image processed by the image signal processor 260 may be displayed as it is by the processor 120 via the display device 160, or may be displayed after being further processed.

[0055] According to an embodiment, the electronic device 101 may include a plurality of camera modules 180 having different properties or functions. In this case, at least one camera module 180 of the plurality of camera modules 180 may form, for example, a wide-angle camera, and at least another camera module 180 of the plurality of camera modules 180 may form a telephoto camera. Similarly, at least one camera module 180 of the plurality of camera modules 180 may form, for example, a front camera, and at least another camera module 180 of the plurality of camera modules 180 may form a rear camera.

[0056] An electronic device and method for protecting personal information included in an image are disclosed herein. One aspect of the embodiments disclosed herein is to prevent accidental leakage of user personal information that may be included in an image.

[0057] According to certain embodiments, an electronic device and method for protecting personal information included in an image may prevent the personal information included in the image from being leaked to the public.

[0058] According to certain embodiments, in an electronic device and method for protecting personal information included in an image, user personal information included in an image may be displayed only on the user's electronic device.

[0059] Figure 3is a block diagram illustrating operations of an electronic device and an external electronic device according to an embodiment.

[0060] In an embodiment, the electronic device 101 may include an image sensor 321, an image signal processor (ISP) 323, and a memory 325. The external electronic device 300 (e.g., the electronic device 104 or the server 108) may include a recognition module 331, an ISP 333, and a memory 335. The recognition module 331 may be a logic module and may be implemented using a processor of the external electronic device 300 and software. The ISP 333 may also be implemented using a processor of the external electronic device 300. In this example, the processor of the external electronic device 300 may perform recognition and image processing. Although not shown, the electronic device 101 may include a communication module (e.g., Figure 1 The communication module 190 in the electronic device 101 may send data to or receive data from the external electronic device 300. The external electronic device 300 may include a communication module that may send data to or receive data from the electronic device 101. According to an embodiment, the electronic device 101 may include an identification module 327. The identification module 327 provided in the electronic device 101 may be configured to perform some of the same functions as the identification module 331. For example, the identification module 327 may be hardware or software configured to recognize a face in an image, and because the recognition is done locally rather than requiring the image to be sent over a network, the recognition module 327 is able to recognize a face more simply and quickly than the external electronic device 300 (e.g., a server).

[0061] In an embodiment, the image sensor 321 (e.g., Figure 2 The image sensor 230 in the image sensor 230 may acquire an image of an external object and may generate a raw image 322 corresponding thereto. The raw image 322 may be implemented in various formats, such as a Bayer format, a color filter array (CFA) pattern, a layer structure generated by sensing three colors of each pixel, a format generated by different parallax acquired by each pixel, etc. The image sensor 321 may transmit the raw image 322 to the ISP 323 (e.g., Figure 2 The image signal processor 260) and / or the recognition module 327 in.

[0062] In an embodiment, the image sensor 321 may generate a small original image 326. The image sensor 321 may generate the small original image 326 by, for example, reducing the resolution of the original image 322. For example, the image sensor 321 may generate the small original image 326 from the original image 322 by using various reduction methods or downsampling methods. Since the image sensor 321 adjusts the resolution of the original image 322, filters the original image 322 by one or more frequency bands, and / or selects one or more bit plane levels of the original image 322, the data amount of the small original image 326 may be less than the data amount of the original image 322. For example, the image sensor 321 may generate the small original image 326 by extracting a low frequency band from the original image 322. In another example, the image sensor 321 may generate the small original image 326 by selecting some from a plurality of bit plane levels of the original image 322. The image sensor 321 may transmit the small original image 326 to the external electronic device 300 through the communication module. The small original image 326 may include at least a portion of the original image 322, and may be a smaller data file than the original image 322. If the small original image 326 is sent to the external electronic device instead of the original image 322, the amount of data transmission is reduced, and thus the image may be sent to the external electronic device 300 more quickly. In another embodiment, the processor (e.g., processor 120) of the electronic device 101 may generate the small original image 326 instead of the image sensor 321, and the generated small original image 326 may be sent to the external electronic device 300 through the communication module. The processor 120 may include a microprocessor or any suitable type of processing circuit, such as one or more general-purpose processors (e.g., ARM-based processors), digital signal processors (DSPs), programmable logic devices (PLDs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), graphics processing units (GPUs), video card controllers, etc. In addition, it can be appreciated that when a general-purpose computer accesses code for implementing the processing shown herein, the execution of the code converts the general-purpose computer into a special-purpose computer for executing the processing shown herein. Certain functions and steps provided in the accompanying drawings may be implemented in hardware, software, or a combination of both, and may be performed in whole or in part within the programming instructions of a computer. Pursuant to 35 U.S.C. § 112(f), unless an element is explicitly stated using the phrase "means for...", this document does not explain the elements to be protected. In addition, the skilled person will understand and appreciate that a "processor" or "microprocessor" may be hardware in the disclosure claimed for protection. Under the broadest reasonable interpretation, the attached claims are statutory subject matter consistent with 35 U.S.C. § 101.

[0063] In an embodiment, the image sensor 321 may transmit the compressed raw image 322 to the ISP 323, the external electronic device 300, or the recognition module 327. The image sensor 321 may compress the raw image 322 and may store the compressed raw image 322 in an internal memory of the image sensor 321.

[0064] In an embodiment, the recognition module 331 of the external electronic device 300 can obtain the small original image 326 through the communication module, and can segment the small original image 326 into at least one image area. The recognition module 331 can identify at least one image area generated via segmentation. For example, the recognition module 331 can segment the small original image 326, and can identify at least one image area from the small original image 326 generated by the segmentation. The recognition module 331 can identify the image area by applying an object recognition algorithm or a text recognition algorithm to the image area. Therefore, the recognition module 331 of the external electronic device 300 can identify at least one image area by using various recognition algorithms (such as machine learning or deep learning algorithms). For example, the recognition module 331 of the external electronic device 300 can obtain information associated with the image area, wherein the pixel coordinates (100, 101), (100, 102), (102, 102) and (102, 103) represent "human teeth". The pixel coordinates may correspond to the pixel coordinates of the original image 322. The recognition module 331 may acquire, for example, classification information indicating that the small original image 326 is classified into a category of "people on the street". The recognition module 331 may acquire the classification information by using the recognition result, or may acquire the classification information in the small original image 326 by color distribution, etc. without using the recognition. The recognition module 331 may generate correction area information 332, which includes at least one of the classification information or information associated with at least one image area acquired through the above process. The recognition module 331 may send the correction area information 332 to the electronic device 101. The ISP 323 may correct the original image 322 by using the correction area information 332, and thus may generate a corrected image 324. The corrected image 324 may have, for example, a YUV format. The corrected image 324 may be stored in the memory 325. Alternatively, the corrected image 324 may be compressed by, for example, a JPEG method, and the compressed image may be stored in the memory 325. According to an embodiment, the correction area information may be generated by the identification module 327 of the electronic device 101 and may then be transmitted to the ISP 323 .

[0065] In an embodiment, the original image 322 provided from the image sensor 321 may be transmitted to the external electronic device 300 regardless of whether the small original image 326 is transmitted. The external electronic device 300 may generate another correction area information by using the original image 322. By using the original image 322 that is larger than the small original image 326, the external electronic device 300 (e.g., the ISP 333 of the external electronic device 300) may generate correction area information based on the original image 322 in addition to the correction area information generated from the small original image 326, and may designate the correction area information as extended correction area information. Since the original image 322 may include more information than the small original image 326, the external electronic device 300 may generate more detailed correction area information from the original image 322. In an embodiment, the external electronic device 300 (the ISP 333 of the external electronic device 300) may generate extended correction area information only from the original image 322. Alternatively, the external electronic device 300 (eg, the ISP 333 of the external electronic device 300 ) may generate the extended correction area information by using the original image 322 and the correction area information generated from the small original image 326 .

[0066] In an embodiment, since the original image 322 is larger than the small original image 326, the small original image 326 may be first transmitted to the external electronic device 300, and then the original image 322 may be transmitted to the external electronic device 300. For example, the original image 322 may be transmitted to the external electronic device 300, while the ISP 323 corrects the original image 322 using the correction region information generated from the small original image 326. The original image 322 may be uploaded to the external electronic device 300 as it is generated by the image sensor 321. Alternatively, a pre-processed version of the original image 322 that has undergone lens distortion correction and noise removal may be uploaded. The pre-processing may be performed by the external electronic device 300. The external electronic device 300 may perform a demosaicing process or an image format modification or other pre-processing to improve the image recognition performed by the recognition module 331. The ISP 333 of the external electronic device 300 may correct the received original image 322. The external electronic device 300 may correct the original image 322 by using the correction region information 332 and / or the extended correction region information. Since the original image 322 may have a higher quality than the small original image 326, the ISP 333 of the external device 300 may acquire more detailed region information from the higher quality image. The ISP 333 may generate extended correction region information by using the generated correction region information together with the original image 322. The ISP 333 may generate a high-quality image 334 by correcting the original image 322 using the extended correction region information. The high-quality image 334 may be stored in the memory 335 of the external electronic device 300 and may be downloaded to the electronic device 101.

[0067] Figure 4 is a block diagram of an electronic device according to an embodiment.

[0068] The first electronic device 470 (e.g., the electronic device 102 or the server 108) according to an embodiment may be implemented as a cloud server. The first electronic device 470 may perform network management for servers constituting the cloud system and for electronic devices (e.g., the second electronic device 400) connectable to the cloud system, cloud service management related to services and permissions to be provided, and storage management.

[0069] The first electronic device 470 according to an embodiment may include a processor 471, a database 478, a raw image storage 479, and an image storage 480. The processor 471 according to an embodiment may include a pre-processing module 474, an image recognition module 473, an ISP 476, an encoder 475, a monitoring module 472, a test target acquisition module 481, an image comparison module 482, and a correction target determination module 483. According to an embodiment, the ISP 476 may be implemented as separate hardware.

[0070] According to an embodiment, the preprocessing module 474, the image recognition module 473, the encoder 475, the monitoring module 472, the test target acquisition module 481, the image comparison module 482, and the correction target determination module 483 may be logic modules. Therefore, the operations performed thereby may be performed by the processor 471 or a processor (e.g., the first processor 430 or the second processor 412) of the second electronic device 400.

[0071] According to an embodiment, at least one of the preprocessing module 474, the image recognition module 473, the ISP 476, the encoder 475, the monitoring module 472, the test target acquisition module 481, the image comparison module 482, or the correction target determination module 483 can be implemented as a separate hardware module in the processor 471 of the first electronic device 470 or a separate hardware module in the processor of the second electronic device 400 (for example, the first processor 430 or the second processor 412).

[0072] The preprocessing module 474 according to an embodiment may preprocess an image (e.g., an original image or a small original image) received from the second electronic device 400, and may send the preprocessed image to the image recognition module 473, the original image storage 479, or the ISP 476. According to an embodiment, the preprocessing may include decoding an image file received from the second electronic device 400 to generate an original image.

[0073] According to an embodiment, the image recognition module 473 (eg, Figure 3The recognition module 331 in the embodiment may receive an image (eg, an original image or a small original image) from the second electronic device 400 through the communication module of the first electronic device 470 or via the pre-processing module 474 thereof.

[0074] The image recognition module 473 according to an embodiment may recognize the received image by using various algorithms, such as (1) an object recognition algorithm, (2) a text recognition algorithm, or (3) a recognition algorithm obtained by applying machine learning or deep learning to an image memory (e.g., the original image memory 479 and / or the image memory 480). The image recognition module 473 may store the recognized result in a database (e.g., the original image memory 479) associated with the received image, and / or may send the recognized result to the second electronic device 400. For example, the image recognition module 473 may recognize at least one object (e.g., a person, sky, cloud, cat, puppy, etc.) in the received image. The object may be recognized in an image region within the image. According to an embodiment, the image recognition module 473 may recognize an object in the image corresponding to user information. For example, user information may be obtained from an image stored in the database 478 (e.g., a photo of a family or acquaintance or a family photo). The recognized result may include location information (e.g., pixel coordinates) of the image region and identification information of the object in the image region. Furthermore, the identified results may be used when the ISP 476 processes images and when the test target acquisition module 481 selects test targets.

[0075] The image recognition module 473 according to an embodiment may store the identification information of the object (or multiple objects) recognized in the received image as the category information of the received image, and the identification information may be stored in a database (e.g., the original image memory 479) in association with the category information. For example, the image recognition module 473 may store the identification information as the tag information of the received image and store it in the database. Alternatively, the image recognition module 473 may send the identification information to the second electronic device 400. The category information may include a plurality of categories classified hierarchically. For example, if "cat" is recognized as an object in the image area, the image recognition module 473 may configure the identification information indicating "animal" as the upper category of the corresponding image, and may configure the identification information indicating "cat" as the lower category of the corresponding image. According to an embodiment, the first electronic device 470 may receive the category information corresponding to the image (e.g., the original image or the small original image) from the second electronic device 400, and may store the category information in association with the corresponding image in the database.

[0076] The ISP 476 according to an embodiment may perform image processing (e.g., correction) on the received image by adjusting various values ​​of the received image (e.g., brightness, hue, color, saturation, color temperature, sharpness, hue, vividness, contrast, or hue-saturation value (HSV)). Image processing may be performed based on recipe information (e.g., image area, layer, vector, scene category, etc.) acquired through image recognition. For example, the ISP 476 may identify an image area in which an object is placed as a correction target, and may correct the image area. The ISP 476 may receive additional information corresponding to the recipe information (e.g., a feature vector indicating a feature of an object or a part of an object (e.g., hair)) from the database 478, and may use the additional information to perform image processing. The processed image may be sent to the second electronic device 400 or the encoder 475, or may be stored in a database (e.g., an image memory 480). Image processing may include functions such as white balance, color adjustment, noise reduction, sharpness effects, and detail enhancement. These functions may be performed for each image area based on the recipe information.

[0077] The encoder 475 according to an embodiment may generate an image file (e.g., JPEG, MPEG, 360 degree panorama, etc.) by encoding the corrected image processed by the ISP 476 or the original image stored in the original image storage 479. The image file generated by the encoder 475 may be transmitted to the second electronic device 400 through the communication module of the first electronic device 470, or may be stored in a database (e.g., the image storage 480).

[0078] The monitoring module 472 according to an embodiment may determine that the ISP 476 configured to process an image in the first electronic device 470 has been changed from the first ISP 476a to the second ISP 476b. For example, the second ISP 476b may perform a correction method upgraded from the first ISP 476a.

[0079] In response to the determination made by the monitoring module 472, the test target acquisition module 481 according to an embodiment may acquire some images stored in a database (e.g., the original image memory 479 or the image memory 480) as test targets for image processing. According to an embodiment, if the test target is acquired from the original image memory 479, image processing may be performed by the correction method of the first ISP 476a to which no modification (e.g., replacement or upgrade) is applied and the correction method of the second ISP 476b to which the modification is applied. According to an embodiment, if the test target is acquired from the image memory 480, image processing may be performed on the second ISP 476b.

[0080] The test target acquisition module 481 according to an embodiment may acquire an equal number of test targets for one or more categories. For example, the test target acquisition module 481 may acquire the same number of images classified as the "puppy" category as the number of images classified as the "cat" category.

[0081] In an embodiment, each of the first ISP 476a and the second ISP 476b may correct the test target acquired by the test target acquisition module 481, i.e., by adjusting various values ​​(e.g., brightness, hue, color, saturation, color temperature, sharpness, hue, vividness, contrast, or HSV) of each test target. According to an embodiment, if the test target is a compressed image acquired from the image memory 480, the image may be first decompressed and then corrected.

[0082] The image comparison module 482 according to an embodiment may calculate the difference between the value obtained by the image correction of the first ISP 476a and the value obtained by the image correction of the second ISP 476b for each test target obtained from the database (e.g., the original image memory 479). For example, the image comparison module 482 may select at least one from brightness, hue, color, saturation, color temperature, sharpness, hue, vividness, contrast, or HSV as a comparison target based on the tuning strategy (e.g., effect) expected as the correction method changes. For example, if the expected effect of the change in the correction method is an increase in contrast, the image comparison module 482 may calculate the difference between the contrast values ​​obtained by the image correction of the first ISP 476a and the image correction of the second ISP 476b for each test target. If the expected effect is an increase in the vividness of an image area in which the HSV has about 30 to 60 degrees, the image comparison module 482 may calculate the difference between the color values ​​obtained by the image correction of the first ISP 476a and the image correction of the second ISP 476b for each of the test targets.

[0083] The image comparison module 482 according to an embodiment may calculate a difference between values ​​of images before and after correction by the second ISP 476 b for each test target acquired from a database (eg, the image memory 480 ).

[0084] The image comparison module 482 according to the embodiment can classify the test targets into two groups based on the comparison result. For example, the image comparison module 482 can classify the test targets into an upper group having a value difference exceeding a predetermined threshold and a lower group having a value difference below the threshold. Here, the value difference may refer to: 1) the difference between the values ​​in the image obtained by the image correction of the first ISP 476a and the image correction of the second ISP 476b, or 2) the difference between the values ​​in the image before and after correction by the second ISP 476b.

[0085] The correction target determination module 483 according to an embodiment may determine the correction target based on the comparison result acquired by the image comparison module 482. Here, the correction target may be selected from the images stored in the database (e.g., the original image memory 479 or the image memory 480). The correction target may be image processed by a correction method (e.g., the second ISP 476b) to which a modification (e.g., replacement or upgrade) has been applied, and then may be stored in the database (e.g., the image memory 480) and / or may be sent to the second electronic device 400. According to an embodiment, the correction target determination module 483 may identify the subordinate group as a group having a correction effect that is not as good as expected, and therefore, may identify the category of the image belonging to the subordinate group, and may determine that the image corresponding to the category of the subordinate group has not been corrected. The correction target determination module 483 may select at least one test target from the upper group, and may determine the image corresponding to the category of the selected test target as the correction target in the image stored in the database (e.g., the original image memory 479 or the image memory 480).

[0086] According to an embodiment, the category information used to determine the correction target may include information acquired by the image recognition module 473 and then stored in a database (e.g., the original image storage 479 or the image storage 480) in association with the corresponding image. In addition, the category information may include information received from an external device (e.g., the second electronic device 400) and stored in the database together with the corresponding image as metadata of the corresponding image.

[0087] In an embodiment, according to the comparison result obtained by the image comparison module 482, the category may be generally associated with the test target of the upper group or the test target of the lower group. For example, the first test target with "cat" may belong to the upper group, and the second test target also with "cat" may belong to the lower group. Therefore, images falling into the upper group and the lower group may have a smaller correction effect than expected. Therefore, the correction target determination module 483 may exclude images with "cat" from the correction targets.

[0088] As described above, according to an embodiment, the correction target determination module 483 may determine, as a correction target, a test target corresponding to category information corresponding to the first level and not corresponding to the second level. The category corresponding to the first level may be a category of a test target belonging to an upper group according to classification, and the category corresponding to the second level may be a category of a test target belonging to a lower group according to classification.

[0089] In an embodiment, according to the comparison result obtained by the image comparison module 482, some metadata may be generally associated with a test target belonging to an upper group and a test target belonging to a lower group. For example, despite having the same ISO sensitivity, a first test target may belong to an upper group, while a second test target may belong to a lower group. In addition, specific component information (e.g., scene category) may be generally associated with a test target belonging to an upper group and a test target belonging to a lower group. As described above, images corresponding to metadata or component information that are generally associated with an upper group and a lower group may have a smaller correction effect than expected. Therefore, the correction target determination module 483 may exclude images having metadata or component information associated with an upper group and a lower group among images stored in a database (e.g., the original image memory 479 or the image memory 480) from the correction targets.

[0090] According to an embodiment, the correction target determination module 483 may determine the correction target based on at least one of the metadata or the component information and based on the comparison result obtained by the operation of the image comparison module 482. For example, the correction target determination module 483 may identify the metadata of the test target belonging to the upper group (hereinafter referred to as the "upper metadata set") and the metadata of the test target belonging to the lower group (hereinafter referred to as the "lower metadata set"). The correction target determination module 483 may compare the upper metadata set and the lower metadata set to identify the metadata included in the upper metadata set and the lower metadata set (hereinafter referred to as the "duplicate metadata set"). The correction target determination module 483 may determine an image corresponding to one or more metadata of the upper metadata set and not corresponding to two or more metadata of the duplicate metadata set as the correction target. For example, if the upper metadata set includes a first ISO sensitivity value, a second ISO sensitivity value, a first brightness value, a second brightness value, a first sharpness value, and a second sharpness value, and the first ISO sensitivity value, the first brightness value, and the first sharpness value are determined as duplicate metadata, then an image having the first ISO sensitivity value, the second brightness value, and the second sharpness value may be determined as the correction target. An image having a first ISO sensitivity value, a first brightness value, and a second sharpness value may be excluded from the correction target.

[0091] According to an embodiment, metadata may be received from an external device (e.g., electronic device 400) together with an image, and may be stored in a database (e.g., raw image memory 479 or image memory 480) in association with the image. For example, metadata may be information acquired by an image sensor (e.g., image sensor 411), and may include focal length, automatic focus area, information related to left-right rotation during shooting (orientation), automatic white balance (AWB), color space, exposure time, aperture-related information (e.g., F-number, F-stop), exposure program (e.g., automatic, aperture priority, shutter priority, manual, etc.), ISO speed rating (ISO), image capture date (raw data time), etc. Metadata may also include information sensed by a sensor or clock other than the image sensor, such as illumination at an image capture position or a timestamp.

[0092] According to an embodiment, the component information may include information (e.g., image area, layer, vector, scene category, etc.) acquired by the image recognition module 473 by recognizing an image (e.g., a small original image) received from an external device (e.g., the electronic device 400).

[0093] According to an embodiment, the correction target determination module 483 may determine the correction target based on the tuning strategy (e.g., effect). For example, when an image is corrected by a correction method (e.g., second ISP 476b) to which a modification (e.g., replacement or upgrade) has been applied, if the expected effect is an increase in brightness, the correction target determination module 483 may apply a weight to the difference in brightness (e.g., a predetermined value may be added to the difference) so as to preferentially classify the test target having a brightness increased by correction into the upper group.

[0094] The second electronic device 400 (e.g., electronic device 101) according to an embodiment may include a camera module 410, a display 420, a first processor 430, a first memory 440, an input device 450, and a sound output device 460. The camera module 410 according to an embodiment may include an image sensor 411, a second processor 412, and a second memory 418. The second processor 412 according to an embodiment may include a raw image processing module 413, an ISP 416, and an encoder 417. At least some operations performed by the second processor 412 may be performed by the first processor 430. The raw image processing module 413, the ISP 416, and the encoder 417 may be logic modules, and thus, the operations performed thereby may be performed by the second processor 412 (e.g., processor 120). In another embodiment, at least one of the raw image processing module 413, the ISP 416, and the encoder 417 may be implemented as an independent hardware module within the second processor 412.

[0095] Although not shown, the second electronic device 400 may include a communication module (eg, Figure 1 The communication module 190 in the first electronic device 470 may also include a communication module for performing data communication with the second electronic device 400.

[0096] In an embodiment, the image sensor 411 (e.g., Figure 2 The image sensor 230 or Figure 3 The image sensor 321 in the image sensor 411 can acquire various raw images of the object. The image sensor 411 can acquire various types of raw images according to the color filter array (CFA) pattern. The image sensor 411 can acquire a raw image with a dual pixel (DP) structure (or a dual photodiode (2PD) structure), in which the pixels include different parallax (or phase difference) information. The image sensor 411 may include multiple image sensors with the same or different characteristics (for example, a dual sensor (for example, RGB+RGB, RGB+Mono or Wide+Tele, etc.) or an array sensor (for example, two or more sensors), etc.). Multiple image sensors can be used to acquire one or more raw images. The acquired raw images can be stored in a second memory 418 (for example, DRAM) as is or after additional processing.

[0097] According to an embodiment, the acquired raw image may be configured in various formats (e.g., Bayer format, etc.). The raw image may be represented by one of red (R), green (G), and blue (B) according to each pixel, and may be represented by a bit depth of 8 to 16 bits. Various color filter array (CFA) patterns may be applied to the raw image. The raw image may have a layer structure including information about various colors of each pixel (e.g., multiple colors of R, G, and B). According to various configurations of the image sensor, the raw image may include not only color information but also parallax (phase difference) information. Information related to the captured image (e.g., time, position, illumination, etc.) may be generated as metadata and stored in association with the raw image. For example, the second processor 412 may use the image sensor 411 to obtain metadata about the raw image. The metadata that may be obtained by the image sensor 411 includes, for example, focal length, automatic focus area, information related to left and right rotation (orientation) during image capture, color space, and exposure time. In addition, the metadata may include location information of the image, which may be obtained by a sensor other than the image sensor (e.g., a GPS sensor).

[0098] In an embodiment, the raw image processing module 413 may perform various processes on the raw image acquired from the image sensor 411. For example, the raw image processing module 413 may perform lens distortion correction or noise removal on the raw image.

[0099] The original image processing module 413 according to an embodiment may include a small original image generation module 414 and a compression module 415. The small original image generation module 414 may generate a small original image from the original image by using various reduction methods (e.g., reducing the size or reducing the resolution) or downsampling methods (e.g., taking only one or some of a series of sampling samples). The compression module 415 may compress the original image or the small original image by using various compression algorithms, and may store the compressed original image or the compressed small original image in the second memory 418. The small original image may be temporarily or permanently stored in the second memory 418. The communication module (not shown) (e.g., the communication module 190) of the second electronic device 400 may send the small original image stored in the second memory 418 to the first electronic device 470.

[0100] In an embodiment, ISP 416 (e.g., Figure 2 The image signal processor 260 or Figure 3 The ISP 323 in the second memory 418 may perform image processing on the original image stored in the second memory 418. For example, the ISP 416 may process (e.g., correct) the original image by using component information (e.g., image segment, layer, vector, scene category, etc.) acquired from the first electronic device 470 through the communication module in various ways. According to an embodiment, the ISP 416 may send the original image or a small original image to the first electronic device 470 through the communication module so that the first electronic device 470 generates component information for image processing. The ISP 416 may process the original image by using the component information acquired from the first electronic device 470 via the communication module. The ISP 416 may compress the processed original image into, for example, JPEG, and may store it in the first memory 440.

[0101] In an embodiment, the ISP 416 performing image processing in the second electronic device 400 may be changed from the first ISP 416a to the second ISP 416b. For example, the second ISP 416b may have a correction method in which at least a part of the correction method of the first ISP 416a has been upgraded.

[0102] In an embodiment, the encoder 417 may encode the original image processed by the ISP 416 to generate an image file (eg, JPEG, MPEG, 360-degree panoramic, etc.), and may store the image file in the first memory 440 .

[0103] In an embodiment, the first processor 430 (eg, Figure 1 The processor 120 in the embodiment of the present invention can be electrically connected to the camera module 410, the display 420, the first memory 440, the input device 450, the sound output device 460 and the communication module (not shown) to control at least one of them. In addition, the first processor 430 can perform various data processing and operations.

[0104] According to an embodiment, the first processor 430 may update the image stored in the first memory 440 in response to the fact that the ISP 416 has been changed from the first ISP 416a to the second ISP 416b. For example, the first processor 430 may perform the same functions as at least some of the functions of the processor 471 of the first electronic device 470 (e.g., at least one of the test target acquisition module 481, the image comparison module 482, and the correction target determination module 483).

[0105] According to an embodiment, an electronic device (e.g., Figure 1 The electronic device 101 or Figure 4 The second electronic device 400 in the embodiment may include: a camera (eg, Figure 1 ), which includes an image sensor (e.g., Figure 1 Camera module 180, Figure 2 The image sensor 230, Figure 3 Image sensor 321 or Figure 4 image sensor 411 in; communication module (eg, Figure 1 Communication module 190 in; memory (e.g., Figure 1 The memory 130, Figure 2 250 or Figure 3 325); and a processor (e.g., Figure 1 The processor 120, Figure 2 The image signal processor 260, Figure 3 ISP 323 or Figure 4The first processor 430 or the second processor 412 in the embodiment of the present invention is operably connected to the camera, the communication module and the memory. The memory may store instructions that, when executed, cause the processor to: acquire a first image including an external object by using the camera; acquire a second image including less information than the first image by using at least a portion of the first image; send the second image to an external electronic device (e.g., Figure 1 The electronic device 104 or the server 108, Figure 3 The external electronic device 300, or Figure 4 receiving security information corresponding to the second image from an external electronic device by using the communication module; and protecting the first image based at least in part on the security information.

[0106] According to an embodiment, the security information corresponding to the second image may relate to at least one biometric information region of the external object.

[0107] According to an embodiment, the instructions may also cause the processor (e.g., Figure 1 The processor 120, Figure 2 The image signal processor 260, Figure 3 ISP 323 in or Figure 4 The first processor 430 or the second processor 412 in the embodiment divides the first image into a first security level area and a second security level area based at least in part on the security information.

[0108] According to an embodiment, the instructions may also cause the processor (e.g., Figure 1 The processor 120, Figure 2 The image signal processor 260, Figure 3 ISP 323 in or Figure 4 The first processor 430 or the second processor 412 in the embodiment of the present invention generates a security key and stores the security key in a memory (e.g., Figure 1 The memory 130, Figure 2 250 in memory, or Figure 3 325 in the memory); obtaining a third image that has encoded the second security level area of ​​the first image based on the security key; and sending the third image to an external electronic device (eg, Figure 1 The electronic device 104 or the server 108, Figure 3 The external electronic device 300, or Figure 4 The first electronic device 470 in the embodiment of the present invention.

[0109] According to an embodiment, the instructions may also cause the processor (e.g., Figure 1 The processor 120, Figure 2The image signal processor 260, Figure 3 ISP 323 in or Figure 4 430 or the second processor 412 in the first processor 430 or the second processor 412 in the second processor 430): from an external electronic device (e.g., Figure 1 The electronic device 104 or the server 108, Figure 3 The external electronic device 300, or Figure 4 The first electronic device 470 in the embodiment receives a fourth image in which the first security level area of ​​the third image has been at least partially corrected; and uses the image stored in the memory (eg, Figure 1 The memory 130, Figure 2 250 in memory, or Figure 3 The fifth image in which the fourth image is decoded is obtained using a security key in the memory 325) in the memory.

[0110] According to an embodiment, the instructions may also cause the processor (e.g., Figure 1 The processor 120, Figure 2 The image signal processor 260, Figure 3 ISP 323 in or Figure 4 The first processor 430 or the second processor 412 in the image obtains a third image by mosaicking, masking or scrambling the second security level area of ​​the first image.

[0111] According to an embodiment, the instructions may also cause the processor (e.g., Figure 1 The processor 120, Figure 2 The image signal processor 260, Figure 3 ISP 323 in or Figure 4 The first processor 430 or the second processor 412 in the image): extracting a first security level area from the first image and generating a third image including the first security level area; and extracting a second security level area from the first image and generating a security area image based on the second security level area.

[0112] According to an embodiment, the instructions may also cause the processor (e.g., Figure 1 The processor 120, Figure 2 The image signal processor 260, Figure 3 ISP 323 in or Figure 4 The first processor 430 or the second processor 412 in the embodiment of the present invention): sending the third image to the external electronic device; and storing the safe area image in the memory (eg, Figure 1 The memory 130, Figure 2 250 in memory, or Figure 3 In the memory 325).

[0113] According to an embodiment, the instructions may also cause the processor (e.g., Figure 1 The processor 120, Figure 2 The image signal processor 260, Figure 3 ISP 323 in or Figure 4 430 or the second processor 412 in the first processor 430 or the second processor 412 in the second processor 430): from an external electronic device (e.g., Figure 1 The electronic device 104 or the server 108, Figure 3 The external electronic device 300, or Figure 4 The first electronic device 470 in the embodiment of the present invention receives a fourth image in which the third image has been at least partially corrected; and acquires a fifth image by synthesizing the fourth image and the safety area image.

[0114] According to an embodiment, the instructions may also cause the processor (e.g., Figure 1 The processor 120, Figure 2 The image signal processor 260, Figure 3 ISP 323 in or Figure 4 The first processor 430 or the second processor 412 in the embodiment of the present invention: based on the received security information, providing an image indicating a first security level area and a second security level area to a user; and based on a user input selecting a part or all of the first security level area or the second security level area, separating the first security level area and the second security level area from each other.

[0115] According to an embodiment, the instructions may also cause the processor (e.g., Figure 1 The processor 120, Figure 2 The image signal processor 260, Figure 3 ISP 323 in or Figure 4 The first processor 430 or the second processor 412 in the embodiment of the present invention is configured to: receive another user input specifying an additional area in the provided image other than the first security level area and the second security level area; and determine the specified additional area as the second security level area.

[0116] Figure 5 is a flowchart illustrating a method for protecting personal information included in an image in an electronic device according to an embodiment.

[0117] Reference Figure 5 In operation 510, according to an embodiment, the processor 120 (eg, Figure 4 The second electronic device 400 in the embodiment may be configured to include an image sensor (eg, Figure 1 Camera module 180, Figure 2 The image sensor 230, Figure 3The image sensor 321 in the Figure 4 The first image may be an original image (eg, Figure 3 The original image 322 in FIG. 1 ) and / or an image obtained by processing or correcting the original image (eg, a JPEG image).

[0118] According to an embodiment, the first image may be in various formats (e.g., a Bayer format, etc.). The first image may be represented by one of red (R), green (G), and blue (B) for each pixel, and may be represented by a bit depth of 8 to 16 bits. Various color filter array (CFA) patterns may be applied to the first image. The first image may have a layer structure including information about various colors of each pixel (e.g., multiple colors of R, G, and B). According to various configurations of the image sensor, the first image may include not only color information (e.g., RGB) but also phase difference information. Information related to the captured image (e.g., time, position, illumination, etc.) may be generated as metadata and stored in association with the first image.

[0119] In operation 520, according to an embodiment, the processor 120 of the electronic device 101 may acquire a second image including less information than the first image by using at least a portion of the first image. The second image may be a small image related to the original image ( Figure 3 A small original image 326 in (eg, a small original image 326) and / or a small image of a processed image (eg, a JPEG image).

[0120] According to an embodiment, the processor 120 can generate the second image by reducing the data in the first image. For example, the processor 120 can generate the second image from the first image by using various reduction methods or downsampling methods. Since the processor 120 adjusts the resolution of the first image, filters the first image by one or more frequency bands, and / or selects one or more bit plane levels of the FIR image, the amount of data of the second image can be less than the amount of data of the first image. The processor 120 can generate the second image by extracting the low frequency band from the first image. The processor 120 can generate the second image by selecting some from the multiple bit plane levels of the first image.

[0121] In operation 530, according to an embodiment, the processor 120 of the electronic device 101 may communicate with the user through a communication module (eg, Figure 1 The communication module 190 in the second image is sent to the external electronic device 300 (eg, Figure 1 The electronic device 104 or the server 108 or Figure 4 The first electronic device 470 in the embodiment of the present invention.

[0122] According to an embodiment, the processor 120 may transmit a second image having a smaller capacity than the first image to the external electronic device 300 so that the external electronic device 300 generates security component information for protecting the first or second image. According to an embodiment, the processor 120 may request security component information from the external electronic device 300 in order to divide the second image into a first security level area and a second security level area requiring higher security than the first security level area.

[0123] According to an embodiment, the security component information can be used to divide the second image into multiple security level areas (e.g., three security level areas). In this case, multiple servers can process the second image for multiple security levels. For example, a first server can be configured to process a first security level area included in the second image, and a second server can be configured to process a second security level area. In another example, the third security level area can be processed by the electronic device 101.

[0124] In operation 540, according to an embodiment, the processor 120 of the electronic device 101 may communicate with the user by using a communication module (eg, Figure 1 The communication module 190 in the embodiment receives security information included in the second image from the external electronic device 300.

[0125] According to an embodiment, the external electronic device 300 may use the second image to identify at least one image region. For example, the external electronic device 300 may determine at least one image region (eg, a security region) including personal information.

[0126] According to an embodiment, instead of determining personal information (e.g., fingerprint information, iris information, or facial information) or in addition thereto, the external electronic device 300 may apply an object recognition algorithm or a text recognition algorithm to the second image in order to recognize an object (e.g., a person) or recognize a specific part of an object (e.g., "a person's face," "a person's eyes," or "a person's hands"). For example, the external electronic device 300 may recognize an object (e.g., a driver's license) in the second image, but may not determine personal information (e.g., a license number) included in the object.

[0127] According to an embodiment, the external electronic device 300 may include at least one image region of the second image believed to include personal information (e.g., “person's face,” “person's eyes,” or “person's hands”) in the security information, and may send it to the electronic device 101. According to an embodiment, the security information may include location information (e.g., pixel coordinates) of at least one image region believed to include personal information and / or identification information of each image region (e.g., “person's face,” “person's eyes,” or “person's hands”).

[0128] According to an embodiment, the external electronic device 300 may transmit the correction area information (eg, correction area information 332 ) to the electronic device 101 together with the security information of the second image.

[0129] In operation 550 , according to an embodiment, the processor 120 of the electronic device 101 may protect the first image based at least in part on the security information.

[0130] According to an embodiment, the processor 120 may divide the first image into a first security level area and a second security level area based at least in part on the security information.

[0131] According to an embodiment, the processor 120 may generate a security key as part of a method for protecting a first image and store the security key in a memory, may acquire a third image in which a second security level region of the first image has been encoded based on the security key, and may send the third image to the external electronic device 300. For example, the processor 120 may acquire the third image by mosaicking, masking, or scrambling the second security level region of the first image. According to an embodiment, the processor 120 may receive a fourth image in which a first security level region of the third image has been at least partially corrected from the external electronic device 300, and may acquire a fifth image in which the fourth image has been decoded using the security key stored in the memory 130.

[0132] According to an embodiment, as part of the method for protecting the first image, the processor 120 may extract the first security level area from the first image and generate a third image including the first security level area, and may extract the second security level area from the first image and generate a security area image including the second security level area. According to an embodiment, the processor 120 may send the third image to the external electronic device 300, and may store the security area image in the memory 130.

[0133] According to an embodiment, the processor 120 may receive a fourth image in which the third image has been at least partially corrected from the external electronic device 300 , and acquire a fifth image by using the fourth image and the safety area image.

[0134] Figure 6 is a flowchart illustrating a method for protecting a first image in an electronic device according to an embodiment. Figure 6 The operations in may be included as part of or in whole in operation 550. Figure 5 middle.

[0135] Reference Figure 6In operation 610, according to an embodiment, the processor 120 of the electronic device 101 may acquire a third image in which the second security level area of ​​the first image has been encoded based on the security key.

[0136] According to an embodiment, the method includes: receiving a signal from an external electronic device 300 (eg, Figure 1 The electronic device 104 or the server 108 or Figure 4 The security information in the second image received by the first electronic device 470 in the processor 120 may include security component information for dividing the second image into a first security level area and / or a second security level area requiring higher security than the first security level area. The processor 120 may divide the first image into a first security level area and a second security level area based at least in part on the security information or the security component information. According to an embodiment, the first image may be divided into a plurality of security areas (e.g., three security level areas) according to the security information or the security component information.

[0137] According to an embodiment, the processor 120 may provide an image indicating at least one security area to the user based on security information received from the external electronic device 300, and may determine the first security level area and the second security level area based on a user input selecting a portion or the entirety of at least one security area. For example, the processor may display at least one security area (e.g., "human face", "human eyes", or "human hands") in the provided image, and if the user selects a specific area (e.g., "human eyes"), the selected specific area (e.g., "human eyes") may be designated as the second security level area.

[0138] According to an embodiment, the processor 120 may receive a user input designating an additional area (eg, an area of ​​the image displaying a document) other than at least one security area within the provided image, and may determine the designated additional area as a second security level area.

[0139] According to an embodiment, the processor 120 may generate a security key and then may store the security key in a memory.The processor 120 may generate an encoded third image from the first image by using the security key and a hash function.

[0140] In operation 620, according to an embodiment, the processor 120 of the electronic device 101 may communicate with the user through a communication module (eg, Figure 1 The communication module 190 in the embodiment sends the third image to the external electronic device 300 (eg, Figure 1 The electronic device 104 or the server 108 or Figure 4 For example, the third image may have a second security level area that has been mosaicked, masked, and scrambled.

[0141] In operation 630, according to an embodiment, the processor 120 of the electronic device 101 may communicate with the user through a communication module (eg, Figure 1 The communication module 190 in the embodiment receives, from the external electronic device 300, a fourth image in which the first security level area of ​​the third image is corrected.

[0142] According to an embodiment, the external electronic device 300 may correct at least a portion of the first security level area. For example, the external electronic device 300 may correct the first security level area by using the existing generated correction area information (e.g., correction area information 332), or may correct the first security level area by using the extended correction area information. The external electronic device 300 does not have a security key for the third image, and therefore may not determine the second security level area. For example, the external electronic device 300 may not correct the second security level area.

[0143] In operation 640 , according to an embodiment, the processor 120 of the electronic device 101 may decode the fourth image based on the security key stored in the memory 130 and then acquire the fifth image.

[0144] According to an embodiment, the processor 120 may acquire the fifth image only by decoding the fourth image. For example, the processor 120 may demodulate, demask, or descramble the second security level area in the fourth image. According to an embodiment, the processor 120 may decode the fourth image and then use the correction area information (e.g., correction area information 332) to correct the second security level area, thereby acquiring the fifth image. The fifth image may be stored in the memory 130.

[0145] Figure 7 is a flowchart illustrating a method for protecting a first image in an electronic device according to an embodiment. Figure 7 The operations in may be included as part of or in whole in operation 550. Figure 5 middle.

[0146] Reference Figure 7 In operation 710, according to an embodiment, the processor 120 of the electronic device 101 may extract a first security level area from the first image and generate a third security level area including the first security level area.

[0147] According to an embodiment, the method includes: receiving a signal from an external electronic device 300 (eg, Figure 1 The electronic device 104 or the server 108 or Figure 4The security information in the second image received by the first electronic device 470 in the processor 120 may include security component information for dividing the second image into a first security level area and / or a second security level area requiring higher security than the first security level area. The processor 120 may divide the first image into a first security level area and a second security level area based at least in part on the security information or the security component information. According to an embodiment, the security information may be used to divide the first image into a plurality of security areas (e.g., three security level areas).

[0148] According to an embodiment, the processor 120 may provide an image indicating at least one security area to the user based on security information received from the external electronic device 300, and may separate the first security level area and the second security level area from each other based on a user input selecting a portion or the entirety of at least one security area. For example, the processor may display at least one security area (e.g., "human face", "human eyes", or "human hands") in the provided image, and if the user selects a specific area (e.g., "human eyes"), the selected specific area (e.g., "human eyes") may be designated as the second security level area.

[0149] According to an embodiment, the processor 120 may receive a user input designating an additional area (eg, a portion of the image displaying a document) other than the at least one security area within the provided image, and may determine the designated additional area as a second security level area.

[0150] According to an embodiment, the processor 120 may extract the first safety area based at least in part on the safety information.For example, the third image may include only the first safety area within the first image.

[0151] In operation 720 , according to an embodiment, the processor 120 of the electronic device 101 may extract the second security level area from the first image and generate a security area image including the second security level area.

[0152] According to an embodiment, the processor 120 may extract the second security level area based at least in part on the security information. For example, the security area image may include only the second security level area within the first image. In this example, the security area image may be removed from the third image.

[0153] In operation 730, according to an embodiment, the processor 120 of the electronic device 101 may communicate with the user through a communication module (eg, Figure 1 The communication module 190 in the apparatus sends the third image to the external electronic device 300.

[0154] In operation 740, according to an embodiment, the processor 120 of the electronic device 101 may communicate with the user through a communication module (eg, Figure 1 The communication module 190 in the apparatus 100 receives a fourth image in which at least a portion of the third image is corrected from the external electronic device 300.

[0155] According to an embodiment, the external electronic device 300 may correct at least a portion of the third image including the first safety area. For example, the external electronic device 300 may correct the third image by using the existing generated correction area information (e.g., correction area information 332), or may correct the third image by using the extended correction area information.

[0156] In operation 750 , according to an embodiment, the processor 120 of the electronic device 101 may acquire a fifth image by using the fourth image and the safety area image.

[0157] According to an embodiment, the processor 120 may obtain a fifth image by synthesizing the fourth image and the safety area image. According to an embodiment, the processor 120 may correct the safety area image including the second security level area by using the correction area information (e.g., the correction area information 332), and may obtain a fifth image by synthesizing the fourth image and the corrected safety area image. The fifth image may be stored in the memory 130.

[0158] According to an embodiment, a method for protecting personal information included in an image may include: acquiring a first image including an external object by using a camera including an image sensor (operation 510); acquiring a second image including less information than the first image by using at least a portion of the first image (operation 520); sending the second image to an external electronic device through a communication module (operation 530); receiving security information corresponding to the second image from the external electronic device through the communication module (operation 540); and protecting the first image at least partially based on the security information (operation 550).

[0159] According to an embodiment, the security information corresponding to the second image may relate to at least one biometric information region of the external object.

[0160] According to an embodiment, the protecting (operation 550 ) may include dividing the first image into a first security level area and a second security level area based at least in part on the security information.

[0161] According to an embodiment, a method for protecting personal information included in an image may include: generating a security key and storing the security key in a memory; obtaining a third image in which a second security level area of ​​the first image has been encoded based on the security key (operation 610); and sending the third image to the external electronic device (operation 620).

[0162] According to an embodiment, a method for protecting personal information included in an image may include: receiving a fourth image in which a first security level area of ​​a third image has been at least partially corrected from an external electronic device (operation 630); and obtaining a fifth image in which the fourth image is decoded using a security key stored in a memory (operation 640).

[0163] According to an embodiment, a method for protecting personal information included in an image may include: extracting a first security level area from a first image and generating a third image including the first security level area (operation 710); and extracting the second security level area from the first image and generating a security area image based on the second security level area (operation 720).

[0164] According to an embodiment, a method for protecting personal information included in an image may include: transmitting a third image to an external electronic device; and storing the security area image in the memory.

[0165] According to an embodiment, a method for protecting personal information included in an image may include: receiving a fourth image in which a third image has been at least partially corrected from an external electronic device (operation 740); and acquiring a fifth image by synthesizing the fourth image and a safety area image (operation 750).

[0166] Figure 8 is a flowchart illustrating a method for protecting personal information included in an image in a system according to an embodiment.

[0167] Reference Figure 8 In operation 811, according to an embodiment, the electronic device 101 may use an image sensor (eg, Figure 1 Camera module 180, Figure 2 The image sensor 230, Figure 3 The image sensor 321 in the Figure 4 The image sensor 411 in the image sensor is used to acquire a first image including an external object.

[0168] In operation 813 , according to an embodiment, the electronic device 101 may acquire a second image including less information than the first image by using at least a portion of the first image.

[0169] In operation 815, according to an embodiment, the electronic device 101 may transmit the second image to the external electronic device 300 (eg, Figure 1 The electronic device 104 or the server 108 or Figure 4 The first electronic device 470 in the embodiment of the present invention.

[0170] In operation 817, according to an embodiment, the external electronic device 300 may extract at least one image region (e.g., a security region) that is considered to include personal information based on the second image. For example, the external electronic device 300 may apply an object recognition algorithm or a text recognition algorithm to the second image in order to recognize an object (e.g., a person) or recognize a specific region of an object (e.g., "a person's face," "a person's eyes," or "a person's hands").

[0171] In operation 819, according to an embodiment, the external electronic device 300 may transmit security information included in the second image to the electronic device 101. According to an embodiment, the security information may include location information (e.g., pixel coordinates) of at least one image region considered to include personal information or identification information of each image region (e.g., “face of a person”, “eyes of a person”, or “hands of a person”).

[0172] In operation 821, according to an embodiment, the electronic device 101 may acquire a third image based at least in part on the first image and the security information. According to an embodiment, the electronic device 101 may acquire a third image in which the second security level area of ​​the first image has been encoded based on the security key. According to an embodiment, the electronic device 101 may extract the first security level area from the first image, and may generate a third image including the first security level area.

[0173] In operation 823, according to an embodiment, the electronic device 101 can communicate with the user through a communication module (eg, Figure 1 The communication module 190 in the apparatus sends the third image to the external electronic device 300.

[0174] In operation 825 , according to an embodiment, the external electronic device 300 may correct at least a portion of the third image including the first security level area to acquire a fourth image.

[0175] In operation 827 , according to an embodiment, the external electronic device 300 may transmit the fourth image to the electronic device 101 .

[0176] In operation 829, according to an embodiment, the electronic device 101 may acquire a fifth image by using the fourth image. According to an embodiment, the processor 120 may acquire the fifth image by only decoding the fourth image. According to another embodiment, the processor 120 may acquire the fifth image by synthesizing the fourth image and the safety area image. The fifth image may be stored in the memory 130 of the electronic device 101.

[0177] According to an embodiment, an electronic device (e.g., Figure 1 The electronic device 104 or the server 108, Figure 3 The external electronic device 300 or Figure 4 The first electronic device 470 in the embodiment may include: a communication module; a memory (eg, Figure 3 335 in the memory); and a processor operably connected to the communication module and the memory (e.g., Figure 3 ISP 333 or Figure 4 The memory stores instructions that, when executed, cause the processor to: receive a signal from an external electronic device (e.g., Figure 1 The electronic device 101 or Figure 4 A second electronic device 400) receives an image; identifies at least one image area in the image; determines, based at least in part on a result of the identification, security information included in the image and associated with the at least one image area; and sends the security information to an external electronic device through a communication module.

[0178] 9A illustrates an example of a method for protecting personal information according to an embodiment, FIG. 9B illustrates an example of a method for protecting personal information according to an embodiment, and FIG. 9C illustrates an example of a method for protecting personal information according to an embodiment.

[0179] FIG. 9A shows a second image 901 (eg, Figure 3 By using at least a portion of the first image including the external object 911, the processor 120 can acquire a second image 901 including less information than the first image.

[0180] 9B shows an image 921 for indicating at least one safety area. Based on the safety information received from the external electronic device 300, the processor 120 may provide the user with an image 921 indicating at least one safety area (e.g., "human eyes 931," "human ears 933," or "human hands 935"). According to an embodiment, the processor 120 may divide the first image into a first security level area and a second security level area without providing the user with the image 921. For example, the processor may designate an area corresponding to "human eyes 931," "human ears 933," or "human hands 935" as a second security level area.

[0181] 9C shows a third image 941 in which the second security level area of ​​the first image has been encoded based on a security key. According to an embodiment, the processor 120 may determine a user input that selects a portion or all of at least one security area provided by the provided image, and may separate the first security level area and the second security level area from each other. For example, if a user selects a specific area (e.g., a person's eye 931) from at least one security area provided in the image (e.g., "a person's eye 931", "a person's ear 933", or "a person's hand 935"), the processor 120 may determine the area corresponding to the "a person's eye 931" as the second security level area. In addition, according to an embodiment, the processor 120 may acquire the third image 941 after dividing the first image into the first security level area and the second security level area. According to an embodiment, the third image 941 may have a second security level area that has been mosaicked, masked (951), or scrambled.

[0182] The electronic device according to various embodiments may be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a household appliance. According to an embodiment of the present disclosure, the electronic device is not limited to those electronic devices described above.

[0183] It should be understood that the various embodiments of the present disclosure and the terms used therein are not intended to limit the technical features set forth herein to specific embodiments, but include various changes, equivalent forms or alternative forms for corresponding embodiments. For the description of the accompanying drawings, similar reference numerals may be used to refer to similar or related elements. It will be understood that the nouns in the singular form corresponding to the term may include one or more things, unless the relevant context clearly indicates otherwise. As used herein, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C" and "at least one of A, B or C" may include any one or all possible combinations of the items listed together with the corresponding one of the multiple phrases. As used herein, terms such as "1st" and "2nd" or "first" and "second" may be used to simply distinguish the corresponding component from another component, and do not limit the component in other aspects (e.g., importance or order). It will be understood that if an element (e.g., a first element) is referred to as being “combined with another element (e.g., the second element)”, “combined to another element (e.g., the second element)”, “connected with another element (e.g., the second element)”, or “connected to another element (e.g., the second element)” when the terms “operably” or “communicatively” are used or when the terms “operably” or “communicatively” are not used, it means that the one element may be directly (e.g., wired) connected to the other element, wirelessly connected to the other element, or connected to the other element via a third element.

[0184] As used herein, the term "module" may include units implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "portion," or "circuit"). A module may be a single integrated component adapted to perform one or more functions or a minimum unit or portion of the single integrated component. For example, according to an embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0185] The various embodiments described herein may be implemented as software (e.g., program 140) including one or more instructions stored in a storage medium (e.g., internal memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101). For example, under the control of a processor, a processor (e.g., processor 120) of the machine (e.g., electronic device 101) may call at least one of the one or more instructions stored in the storage medium and execute the at least one instruction with or without the use of one or more other components. This enables the machine to operate to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be run by an interpreter. A machine-readable storage medium may be provided in the form of a non-transitory storage medium. Among them, the term "non-transitory" only means that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but the term does not distinguish between data being semi-permanently stored in the storage medium and data being temporarily stored in the storage medium.

[0186] According to an embodiment, the method according to various embodiments of the present disclosure may be included and provided in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be released in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or may be downloaded via an application store (e.g., PlayStore). TM ) The computer program product may be published (e.g., downloaded or uploaded) online, or the computer program product may be distributed (e.g., downloaded or uploaded) directly between two user devices (e.g., smart phones). If published online, at least part of the computer program product may be temporarily generated, or at least part of the computer program product may be at least temporarily stored in a machine-readable storage medium (such as a memory of a manufacturer's server, a server of an application store, or a forwarding server).

[0187] According to various embodiments, each component (e.g., module or program) in the above-mentioned components may include a single entity or multiple entities. According to various embodiments, one or more components in the above-mentioned components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform the one or more functions of each component in the multiple components in the same or similar manner as a corresponding component in the multiple components before integration. According to various embodiments, the operations performed by a module, program or another component may be performed sequentially, in parallel, repeatedly or in a heuristic manner, or one or more operations in the operations may be run or omitted in different orders, or one or more other operations may be added.

[0188] Some of the above-mentioned embodiments of the present disclosure may be implemented in hardware, firmware, or via software or computer code that may be stored in a medium such as a CD ROM, a digital versatile disk (DVD), a tape, a RAM, a floppy disk, a hard disk, or a magneto-optical disk, or the execution of a computer code that is initially stored on a remote recording medium or a non-transitory machine-readable medium, downloaded via a network, and stored on a local recording medium, so that the methods described herein may be presented via such software stored on a recording medium using a general-purpose computer or a dedicated processor or programmable or dedicated hardware (e.g., an ASIC or FPGA). As will be understood in the art, a computer, a processor, a microprocessor controller, or programmable hardware includes a memory component that can store or receive software or computer code, such as RAM, ROM, flash memory, etc., which implements the processing methods described herein when accessed and executed by a computer, processor, or hardware.

[0189] While the present disclosure has been shown and described with reference to various embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the present disclosure as defined by the appended claims and their equivalents.

Claims

1. An electronic device, include: Cameras, including image sensors; Communication module; Memory; as well as a processor operably connected to the camera, the communication module and the memory, The memory stores instructions which, when executed, cause the processor to: acquiring a first image including an external object by using the camera; generating a second image including less information than information included in the first image by using at least a portion of the first image; transmitting the second image to an external electronic device by using the communication module; receiving security information corresponding to the second image from the external electronic device by using the communication module; Based at least in part on the security information, dividing the first image into a first security level area and a second security level area, the second security level area having a higher security level than the first security level area; generating, based at least in part on the security information, a third image in which a second security level region of the first image is protected; sending the third image to the external electronic device; and A fourth image is received from the external electronic device in which the first security level area of ​​the third image is at least partially corrected by the external electronic device.

2. The electronic device according to claim 1, in, The security information corresponding to the second image relates to at least one biometric information area of ​​the external object.

3. The electronic device according to claim 1, in, The instructions further cause the processor to: generating a security key and storing the security key in the memory; acquiring the third image in which the second security level area of ​​the first image is encoded based on the security key; and The third image is transmitted to the external electronic device.

4. The electronic device according to claim 3, in, The instructions further cause the processor to: A fifth image in which the fourth image is decoded is acquired using the security key stored in the memory.

5. The electronic device according to claim 3, in, The instructions further cause the processor to acquire the third image by mosaicking, masking, or scrambling the second security level region of the first image.

6. The electronic device according to claim 3, in, The instructions further cause the processor to: extracting the first security level area from the first image, and generating the third image including the first security level area; and extracting the second security level area from the first image, and generating a security area image based on the second security level area; sending the third image to the external electronic device; storing the security area image in the memory; receiving, from the external electronic device, a fourth image in which the third image is at least partially corrected; as well as A fifth image is acquired by synthesizing the fourth image and the safety area image.

7. The electronic device according to claim 1, in, The instructions further cause the processor to: Based on the received security information, providing a user with an image indicating the first security level area and the second security level area; as well as Based on a user input selecting a part or all of the first security level area or the second security level area, the first security level area and the second security level area are separated from each other.

8. The electronic device according to claim 7, in, The instructions further cause the processor to: receiving another user input specifying an additional area in the provided image other than the first security level area and the second security level area; and The designated additional area is determined as the second security level area.

9. A method for protecting personal information included in an image, the method include: acquiring a first image including an external object by using a camera including an image sensor; generating a second image including less information than information included in the first image by using at least a portion of the first image; transmitting the second image to an external electronic device by using a communication module; receiving security information corresponding to the second image from the external electronic device by using the communication module; Based at least in part on the security information, dividing the first image into a first security level area and a second security level area, the second security level area having a higher security level than the first security level area; generating, based at least in part on the security information, a third image in which a second security level region of the first image is protected; sending the third image to the external electronic device; as well as A fourth image is received from the external electronic device in which the first security level area of ​​the third image is at least partially corrected by the external electronic device.

10. The method according to claim 9, in, The security information corresponding to the second image relates to at least one biometric information area of ​​the external object.

11. The method according to claim 9, further comprising: include: Generate a security key and store the security key in a memory; acquiring the third image in which the second security level area of ​​the first image is encoded based on the security key; and The third image is transmitted to the external electronic device.

12. The method according to claim 11, further comprising: include: A fifth image in which the fourth image is decoded is acquired using the security key stored in the memory.

13. The method according to claim 11, further comprising: include: extracting the first security level area from the first image, and generating the third image including the first security level area; extracting the second security level area from the first image, and generating a security area image based on the second security level area; sending the third image to the external electronic device; storing the security area image in the memory; receiving, from the external electronic device, a fourth image in which the third image is at least partially corrected; as well as A fifth image is acquired by synthesizing the fourth image and the safety area image.

Images