Electronic device for removing harmful information included in images and operation method of same

Abstract

Disclosed are an electronic device for removing harmful information included in images and an operation method of same. According to an embodiment, the electronic device comprises: a memory storing instructions; and at least one processor that executes the instructions, wherein the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to: for removal of at least one of first harmful information that is identified as text and second harmful information that is not identified as text, receive an input of an input image; determine whether the first harmful information is included in first text included in the input image; determine an intermediate image from the input image, on the basis of whether the first harmful information is included in the first text; generate second text based on the intermediate image; determine whether the second harmful information is included in the second text; and determine an output image from the intermediate image, on the basis of whether the second harmful information is included in the second text.

Description

Electronic device for removing harmful information within an image and method of operation thereof

[0001] An electronic device for removing harmful information within an image and a method of operating the same are disclosed.

[0002] Generative AI models can analyze images and extract information. The information analyzed and extracted by generative AI models can be utilized in various ways. The information analyzed and extracted by generative AI models can be processed and used for target operations such as image-based search, object detection, and image correction.

[0003] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0004] According to one embodiment, the electronic device may include a memory for storing instructions. The electronic device may include at least one processor for executing the instructions. When the instructions are executed individually or collectively by the at least one processor, the electronic device may receive an input image for the removal of at least one of first harmful information identified by text and second harmful information not identified by text. When the instructions are executed individually or collectively by the at least one processor, the electronic device may determine whether the first harmful information is included in the first text included in the input image. When the instructions are executed individually or collectively by the at least one processor, the electronic device may determine an intermediate image from the input image based on whether the first harmful information is included in the first text. When the instructions are executed individually or collectively by the at least one processor, the electronic device may generate a second text based on the intermediate image. When the above instructions are executed individually or collectively by the at least one processor, the electronic device may determine whether the second text contains the second harmful information. When the above instructions are executed individually or collectively by the at least one processor, the electronic device may determine an output image from the intermediate image based on whether the second text contains the second harmful information.

[0005] According to one embodiment, an electronic device may include a memory for storing instructions. The electronic device may include at least one processor for executing the instructions. When the instructions are executed individually or collectively by the at least one processor, the electronic device may receive an input image for removing harmful information that is not identified as text. When the instructions are executed individually or collectively by the at least one processor, the electronic device may generate text based on the input image. When the instructions are executed individually or collectively by the at least one processor, the electronic device may determine whether the text contains the harmful information. When the instructions are executed individually or collectively by the at least one processor, the electronic device may modify at least a portion of the input image if the text contains the harmful information. When the instructions are executed individually or collectively by the at least one processor, the electronic device may restore the input image with at least a portion modified to generate an output image from which the harmful information has been removed.

[0006] According to one embodiment, a method of operation of an electronic device may include an operation of receiving an input image for removing at least one of first harmful information identified by text and second harmful information not identified by said text. A method of operation of an electronic device may include an operation of determining whether the first harmful information is included in the first text included in said input image. A method of operation of an electronic device may include an operation of determining an intermediate image from said input image based on whether the first harmful information is included in said first text. A method of operation of an electronic device may include an operation of generating a second text based on said intermediate image. A method of operation of an electronic device may include an operation of determining whether the second harmful information is included in said second text. A method of operation of an electronic device may include an operation of determining an output image from said intermediate image based on whether the second harmful information is included in said second text.

[0007] According to one embodiment, a method of operation of an electronic device may include an operation of receiving an input image to remove harmful information not identified by text. A method of operation of an electronic device may include an operation of generating text based on the input image. A method of operation of an electronic device may include an operation of determining whether the text contains the harmful information. If the text contains the harmful information, a method of operation of an electronic device may include an operation of changing at least a part of the input image. A method of operation of an electronic device may include an operation of recovering the input image with at least a part changed to generate an output image from which the harmful information has been removed.

[0008] According to one embodiment, a non-transient computer-readable recording medium may store one or more computer programs including instructions that execute an operation of receiving an input image for the removal of at least one of first harmful information identified by text and second harmful information not identified by said text. The non-transient computer-readable recording medium may store one or more computer programs including instructions that execute an operation of determining whether the first harmful information is included in the first text included in said input image. The non-transient computer-readable recording medium may store one or more computer programs including instructions that execute an operation of determining an intermediate image from said input image based on whether the first harmful information is included in said first text. The non-transient computer-readable recording medium may store one or more computer programs including instructions that execute an operation of generating a second text based on said intermediate image. The non-transient computer-readable recording medium may store one or more computer programs including instructions that execute an operation of determining whether the second harmful information is included in said second text. A non-transient computer-readable recording medium may store one or more computer programs comprising instructions that execute an operation to determine an output image from an intermediate image based on whether the second text contains the second harmful information.

[0009] According to one embodiment, a non-transient computer-readable recording medium may store one or more computer programs including instructions for executing an operation of receiving an input image to remove harmful information not identified as text. The non-transient computer-readable recording medium may store one or more computer programs including instructions for executing an operation of generating text based on the input image. The non-transient computer-readable recording medium may store one or more computer programs including instructions for executing an operation of determining whether the text contains the harmful information. The non-transient computer-readable recording medium may store one or more computer programs including instructions for executing an operation of changing at least a part of the input image when the text contains the harmful information. The non-transient computer-readable recording medium may store one or more computer programs including instructions for executing an operation of recovering the input image with at least a part changed to generate an output image from which the harmful information has been removed.

[0010] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0011] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments.

[0012] FIGS. 2 and FIGS. 3 are drawings for explaining images containing harmful information according to one embodiment.

[0013] FIG. 4 is a flowchart for explaining the operation of an electronic device according to one embodiment.

[0014] FIG. 5 is a flowchart for explaining the operation of an electronic device for removing first harmful information according to one embodiment.

[0015] FIG. 6 is a diagram illustrating the determination of harmful information according to one embodiment.

[0016] FIG. 7 is a drawing for explaining the removal of first harmful information according to one embodiment.

[0017] FIG. 8 is a flowchart for explaining the operation of an electronic device for removing second harmful information according to one embodiment.

[0018] FIGS. 9 to 11 are drawings for explaining the modification and restoration of at least a portion of an image according to one embodiment.

[0019] FIG. 12 is a drawing for explaining the removal of second harmful information according to one embodiment.

[0020] FIGS. 13 and FIGS. 14 are drawings for explaining the processing of original images containing harmful information.

[0021] FIG. 15 is a diagram illustrating a method for removing harmful information from an image according to one embodiment.

[0022] FIG. 16 is a block diagram of an electronic device for explaining the removal of harmful information according to one embodiment.

[0023] FIG. 17 is a flowchart for explaining the removal of harmful information included in an image according to one embodiment.

[0024] FIG. 18 is a flowchart for explaining the removal of harmful information not identified in an image according to one embodiment.

[0025] FIG. 19 is a flowchart for explaining a method for first performing the removal of harmful information that is not identified in an image according to one embodiment.

[0026] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0027] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0028] The processor (120) can control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., a program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store the resulting data in a non-volatile memory (134).

[0029] The processor (120) may be implemented as a circuitry (e.g., a processing circuit) such as a system on chip (SoC) or an integrated circuit (IC). The processor (120) may include one or more processors. For example, the processor (120) may include a combination of one or more processors such as a CPU, GPU, MPU, AP, and CP. Additionally, the processor (120) may include various processing circuits and / or multiple processors. For example, as used in this specification and claims, the term "processor" may include various processing circuits including at least one processor, and one or more of the at least one processor may be configured to perform the various functions described in this specification, either alone or together in a distributed manner. Where "processor," "at least one processor," or "one or more processors" are described in this specification as being configured to perform various functions, these terms may include, for example without limitation, cases where one processor performs some of the described functions and one or more other processors perform the remaining functions, as well as cases where a single processor performs all of the described functions. Additionally, at least one processor may be a combination of multiple processors that perform the described or disclosed various functions in a distributed manner, etc. At least one processor may execute program instructions to achieve or perform the various functions.

[0030] According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0031] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0032] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0033] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0034] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0035] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0036] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0037] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0038] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0039] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0040] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0041] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0042] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0043] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0044] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0045] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0046] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0047] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0048] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0049] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0050] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0051] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0052] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0053] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0054] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0055] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0056] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0057]

[0058] FIGS. 2 and FIGS. 3 are drawings for explaining images containing harmful information according to one embodiment.

[0059] According to one embodiment, due to the development of computer vision technology, it may be possible to generate various information using images, such as image analysis, description, and summarization. For example, if an image and a command "provide a description of the image" are input to a generative AI model, the generative AI model can generate and provide a description of the image as text. The generative AI model can generate a secondary result based on the description of the image. The generative AI model may form at least part of an artificial intelligence neural network and may include a model that generates images or a model that generates language. The image generation model may include, for example, a generative adversarial network (GAN), a variational auto encoder (VAE), or a Diffusion-based model using a VAE and a Transformer. The language generation model may include, for example, a large language model (LLM), a large multimodal model (LMM), a large vision model (LVM), or a large action model (LAM). The LAM can automatically generate actions for an environment (e.g., a robot, a car, an electronic device (101), or a program (140)).

[0060] According to one embodiment, if harmful information is included in an image and the harmful information is input directly into an artificial intelligence model, a problem may arise in which a result different from the user's intention is generated.

[0061] For example, referring to FIG. 2, an input image (200) containing harmful information (210) is shown. A generative AI model may receive input images (200) and commands to provide descriptions of the input images (200). As harmful information (210) is included in the input images (200), the generative AI model may output a result (220) that includes the harmful information (210) as is. For example, the result (220) representing a description of the input images (200) may include the harmful information (210) identified in the input images (200) as is.

[0062] According to one embodiment, when an electronic device (e.g., the electronic device (101) of FIG. 1) executes a separate target operation (e.g., writing a report, writing an advertisement, etc.) using the result (220), a secondary result is generated in which harmful information (210) is included as is, thereby generating a result different from the user's intention.

[0063] According to one embodiment, if an electronic device identifies that harmful information (210) is included in an input image (200) and can remove the harmful information (210), a result (230) intended by the user can be output.

[0064] The harmful information described in this disclosure may include not only text identified as unethical, such as profanity, sexual language and / or violent language, but also text that threatens the security of a system or causes information leakage, such as prompt injection, prompt extraction and / or jail breaking. The harmful information of this disclosure is described later in FIG. 6.

[0065] For example, the input image (300) of FIG. 3 (e.g., the input image (200) of FIG. 2) may contain harmful information (310) (e.g., harmful information (210) of FIG. 2) that is not profanity or violent language but ignores the user's command and leaks the prompt (320) of the generative AI model. The input image (300) containing the harmful information (310) may be input into the generative AI model as is. The harmful information (310) may contain content that ignores the user's command and causes the content of the prompt (320) to leak. As the input image (300) containing the harmful information (310) is input into the generative AI model, the last content of the prompt (320) input into the generative AI model may leak and ignore the user's command included in the prompt (320).

[0066] The present disclosure describes a method for an electronic device to determine whether an input image contains harmful information and to remove the harmful information. The present disclosure describes a method for removing not only harmful information identified as text in an input image but also harmful information not identified as text.

[0067]

[0068] FIG. 4 is a flowchart for explaining the operation of an electronic device according to one embodiment.

[0069] The operations described below may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. Additionally, some operations may be omitted depending on some embodiments. Operations (410) through (460) may be performed by at least one component (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1).

[0070] According to one embodiment, instructions stored in memory (e.g., memory (130) of FIG. 1) by at least one processor may be executed individually and / or collectively, and the instructions may cause an electronic device to perform the following operations (410) to operations (460).

[0071] In operation (410), the electronic device may receive an input image (e.g., the input image (200) of FIG. 2 or the input image (300) of FIG. 3) for removing at least one of the first harmful information identified by text and the second harmful information not identified by text.

[0072] According to one embodiment, an input image may include at least one of first harmful information identified through text recognition such as optical character recognition (OCR) and second harmful information not identified through text recognition.

[0073] In operation (420), the electronic device can determine whether the first harmful information is included in the first text included in the input image.

[0074] According to one embodiment, an electronic device can determine whether an input image contains first harmful information identified as text. A method for determining whether the first text contains first harmful information is described later in FIG. 6.

[0075] In operation (430), the electronic device can determine an intermediate image from an input image based on whether the first text contains first harmful information.

[0076] According to one embodiment, when the first text contains first harmful information, the electronic device may determine an image containing a regenerated area containing the first harmful information as an intermediate image. The regenerated area may be an area in which the first harmful information is removed and which is regenerated to blend naturally with the remaining areas. The electronic device may determine an input image containing the regenerated area as an intermediate image.

[0077] According to one embodiment, the electronic device may determine an input image as an intermediate image if the first text does not contain first harmful information. A method for determining an intermediate image is described later in FIG. 5.

[0078] In operation (440), the electronic device can generate a second text based on an intermediate image.

[0079] According to one embodiment, the electronic device may perform an operation (440) to remove second harmful information that is not identified as text. The electronic device may input an intermediate image to an artificial intelligence model that provides a description of the image (e.g., a generative artificial intelligence model). The artificial intelligence model may output a second text, which is a description of the intermediate image, as a response to the input of the intermediate image.

[0080] In operation (450), the electronic device can determine whether the second text contains second harmful information.

[0081] According to one embodiment, the method for determining whether the second text contains second harmful information may be the same as the method for determining whether the first text contains first harmful information.

[0082] In operation (460), the electronic device can determine an output image from an intermediate image based on whether the second text contains second harmful information.

[0083] According to one embodiment, if the second text contains second harmful information, the electronic device may alter (e.g., modify) at least a portion of the intermediate image. Alteration of at least a portion of the intermediate image may include damage to the intermediate image. The electronic device may recover the intermediate image in which at least a portion has been altered. During the alteration and recovery of at least a portion of the intermediate image, the second text may be removed. The electronic device may determine the recovered image in which the second text has been removed as the output image.

[0084] According to one embodiment, if the second text does not contain second harmful information, the electronic device may determine an intermediate image as an output image. A method for determining the output image is described later in FIG. 8.

[0085] According to one embodiment, the removal of second harmful information that is not identified by text may be performed before the removal of first harmful information that is identified by text. For example, an electronic device may determine whether second harmful information that is not identified by text exists in a description of an input image, and if second harmful information exists, it may generate a first image through processing of the input image. The method of generating the first image may be the same as operation (460). The electronic device may determine whether first harmful information that is identified by text exists in the first image, and if first harmful information exists, it may determine a second image, which is an output image, through processing of the first image. The method of generating the second image, which is an output image, based on the first image may be the same as operation (430). A method of performing the removal of second harmful information that is not identified by text first is described later in FIG. 19.

[0086] According to one embodiment, when an electronic device receives a command to perform a target operation based on an input image, the electronic device may automatically perform operations (410) to (460) to determine whether the input image contains harmful information and to remove the harmful information if it exists.

[0087] According to one embodiment, the target operation may include various operations based on the input image. The target operation may include operations using a generative artificial intelligence model. For example, when attempting to add a specific object to the input image using a generative artificial intelligence model, the electronic device may perform operations (410) to (460). For example, when requesting the generation of an appreciation for the input image using a generative artificial intelligence model, the electronic device may perform operations (410) to (460).

[0088] According to one embodiment, target actions may be determined by policy. Actions such as downloading an image from a web page, downloading an image via a messenger, and / or capturing a screen may be determined by policy as target actions. When an electronic device downloads an image from a web page, it may perform actions (410) to (460) to determine whether the image to be downloaded contains harmful information and remove the harmful information.

[0089] Below, a method for removing the first text from an input image is described.

[0090]

[0091] FIG. 5 is a flowchart for explaining the operation of an electronic device for removing first harmful information according to one embodiment.

[0092] The operations described below may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. Additionally, some operations may be omitted depending on some embodiments. Operations (510) through (560) may be performed by at least one component (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1).

[0093] According to one embodiment, instructions stored in memory (e.g., memory (130) of FIG. 1) by at least one processor may be executed individually and / or collectively, and the instructions may cause an electronic device to perform the following operations (510) to operations (560).

[0094] In operation (510), the electronic device can extract a first text from an input image (e.g., the input image (200) of FIG. 2 or the input image (300) of FIG. 3).

[0095] According to one embodiment, an electronic device can perform text recognition on an input image. For example, the electronic device can perform text recognition on an input image by performing OCR or intelligent word recognition (IWR) on the input image. The electronic device can extract a first text by performing text recognition on the input image. However, the text recognition method described above is merely an example to aid understanding and should not be interpreted as limiting or restricting the embodiments of the present disclosure.

[0096] According to one embodiment, the first text may include only those identified as text as text is performed on the input image. For example, text written in a very small size (e.g., in pixels) on the input image may not be identified as text. For example, text may not be identified as text if it is written scattered across different areas within the input image in units of one character. For example, at least one of the brightness, saturation, and color of the pixels constituting the written text may not be recognized as text because it is similar to at least one of the brightness, saturation, and color of adjacent pixels.

[0097] According to one embodiment, text represented using a specific object within an input image can be identified as text and included in a first text by performing text recognition. For example, text represented using a tree branch included in an input image can be included in the first text through text recognition.

[0098] In operation (520), the electronic device can determine whether the first text contains first harmful information.

[0099] According to one embodiment, the electronic device can determine whether the first text contains the first harmful information based on at least one of a rule predetermined to be determined as harmful information and an artificial intelligence model that has learned the harmful information. A method for determining whether the first text contains the first harmful information is described later in FIG. 6.

[0100] According to one embodiment, the electronic device may perform an operation (530) if the first text contains first harmful information. The electronic device may perform an operation (560) if the first text does not contain first harmful information.

[0101] In operation (530), the electronic device can remove the area containing the first harmful information.

[0102] According to one embodiment, the electronic device can determine an area containing first harmful information in an input image.

[0103] According to one embodiment, an electronic device may obtain region information containing a first harmful information in an input image from a text identification module (e.g., an OCR module and an IWR module) used to identify text. Based on the region information, the electronic device may determine a region containing the first harmful information.

[0104] According to one embodiment, the electronic device can determine an area containing first harmful information by using a separate module that determines the location and / or area of ​​first harmful information identifiable by text.

[0105] According to one embodiment, the electronic device can remove an area containing first harmful information from an input image.

[0106] In operation (540), the electronic device can regenerate the removed area based on the input image.

[0107] According to one embodiment, an electronic device can regenerate a removed area using an artificial intelligence model that generates images. The electronic device can input an input image and an image from which an area containing first harmful information has been removed to the artificial intelligence model that generates images. The artificial intelligence model can regenerate the removed area based on the input image. The artificial intelligence model may include a large vision model (LVM) or a generative artificial intelligence model.

[0108] In operation (550), the electronic device can determine an input image containing a regenerated area as an intermediate image.

[0109] The intermediate image generated through operations (510) to (550) may not contain first harmful information identified as text compared to the input image. Through operations (510) to (550), an intermediate image in which the first harmful information identified through text recognition has been removed may be obtained.

[0110] In operation (560), the electronic device can determine the input image as an intermediate image.

[0111] According to one embodiment, the first text identified through text recognition may not contain the first harmful information. If the first text does not contain the first harmful information, the electronic device may determine the input image as an intermediate image.

[0112] According to one embodiment, when an intermediate image is determined through operation (560) or operation (550), the electronic device may perform operation (440) of FIG. 4. The electronic device may generate a second text based on the intermediate image and determine whether the second text contains second harmful information.

[0113] The following describes how to determine whether harmful information is included.

[0114]

[0115] FIG. 6 is a diagram illustrating the determination of harmful information according to one embodiment.

[0116] Referring to FIG. 6, an example of harmful information (600) (e.g., harmful information (210) of FIG. 2 or harmful information (310) of FIG. 3) is illustrated. Harmful information (600) may include multiple categories. For example, harmful information (600) may include categories such as illegal text, sexual text, child abuse text, violent text, derogatory text, toxic text, insult text, profanity, text related to self-harm and suicide, malicious code, prompt injection, prompt extraction, and / or jailbreak. The criteria for determining harmful information (600) may be determined differently for each category. However, the categories described above are merely examples for convenience of explanation and should not be interpreted as limiting or restricting the scope of other embodiments.

[0117] According to one embodiment, harmful information (600) may be classified into systemic harmful information that threatens the security of the electronic device system or causes a malfunction, and unethical harmful information identified as unethical.

[0118] According to one embodiment, systemic harmful information that threatens the security of an electronic device system or causes a malfunction may include text related to malicious code, prompt injection, prompt extraction, and / or a jailbreak. For example, systemic harmful information may include a URL that causes malicious code to be downloaded. However, the above-described examples are merely for convenience of explanation and should not be interpreted as limiting or restricting the scope of other embodiments.

[0119] According to one embodiment, unethical harmful information identified as unethical may include illegal text, sexual text, child abuse text, violent text, derogatory text, toxic text, insult text, profanity, text related to self-harm and suicide. However, the examples described above are merely for convenience of explanation and should not be interpreted as limiting or restricting the scope of other embodiments.

[0120] According to one embodiment, an electronic device may determine whether a text contains harmful information based on at least one of a rule determined to determine specific information as harmful information (600) and an artificial intelligence model that identifies harmful information (600). For example, the electronic device may determine whether a first text contains first harmful information based on at least one of a rule determined to determine specific text as harmful information (600) and an artificial intelligence model that identifies harmful information (600). For example, the electronic device may determine whether a second text contains second harmful information based on at least one of a rule determined to determine specific text as harmful information (600) and an artificial intelligence model that identifies harmful information (600).

[0121] According to one embodiment, the electronic device can determine whether the text contains harmful information (600) based on a rule determined in advance to determine specific information as harmful information (600). The electronic device can determine the category of harmful information contained in the text based on a rule determined in advance to determine specific information as harmful information (600). For example, assume that the text "XXX" is included in a rule determined in advance to be in the profanity category. If the electronic device determines in advance that the text contains "XXX" based on the rule, the harmful information (600) is included in the text and the category can be determined as profanity.

[0122] According to one embodiment, an electronic device can determine whether harmful information (600) is included in text based on an artificial intelligence model that identifies harmful information (600). The artificial intelligence model may include a DNN model or a generative artificial intelligence model trained to identify harmful information (600). The trained artificial intelligence model can determine whether harmful information (600) is included in text, and if the trained artificial intelligence model is trained to determine a category, it can also determine the category of harmful information (600) included in text.

[0123] According to one embodiment, after performing a judgment of harmful information (600) based on rules, a judgment of harmful information (600) based on an artificial intelligence model may be performed. However, this is merely an example for convenience of explanation and should not be interpreted as limiting the embodiments. For example, a judgment of harmful information (600) based on an artificial intelligence model may be performed first, and then a judgment of harmful information (600) based on rules may be performed. Since the rules are predetermined to determine what constitutes harmful information (600), harmful information (600) that is not included in the rules may not be judged as harmful information. For example, neologisms or new types of harmful information may not be determined as harmful information (600) through the rules. Since the artificial intelligence model determines harmful information (600) through learning, it may determine it as harmful information (600) even if it is not included in the rules.

[0124] Below, a method for determining an intermediate image using an input image (e.g., the input image (200) of FIG. 2 or the input image (300) of FIG. 3) is described.

[0125]

[0126] FIG. 7 is a drawing for explaining the removal of first harmful information according to one embodiment.

[0127] Referring to FIG. 7, an input image (700) (e.g., the input image (200) of FIG. 2 or the input image (300) of FIG. 3) is shown.

[0128] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) can perform text recognition on an input image (700) to extract a first text. For example, the first text may be extracted as "XXX like, all XXXX!, department store return costs completely free, Universe Club customer, shop without worry."

[0129] According to one embodiment, an electronic device can determine whether the first text contains first harmful information (710) (e.g., harmful information (210) of FIG. 2, harmful information (310) of FIG. 3, or harmful information (600) of FIG. 6). The electronic device can determine whether the first text contains first harmful information (710) based on at least one of an artificial intelligence model that identifies the first harmful information (710) and has predetermined to determine specific information as the first harmful information (710).

[0130] According to one embodiment, if the first harmful information exists, the electronic device may remove an area containing the first harmful information (710) from the input image (700). The electronic device may regenerate the removed area based on the input image (700). The electronic device may input the input image (700) or the image from which the area containing the first harmful information (710) has been removed into an artificial intelligence model that generates images. The artificial intelligence model may regenerate the area containing the first harmful information (710) to match the adjacent area and / or the input image (700). The electronic device may determine the input image (700) containing the regenerated area as an intermediate image (720).

[0131] Below, a method for removing second harmful information that is not identified as text based on an intermediate image (720) is described.

[0132]

[0133] FIG. 8 is a flowchart for explaining the operation of an electronic device for removing second harmful information according to one embodiment.

[0134] The operations described below may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. Additionally, some operations may be omitted depending on some embodiments. Operations (810) through (870) may be performed by at least one component (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1).

[0135] According to one embodiment, instructions stored in memory (e.g., memory (130) of FIG. 1) by at least one processor may be executed individually and / or collectively, and the instructions may cause an electronic device to perform the following operations (810) to operations (870).

[0136] In operation (810), the electronic device can determine whether the second text contains second harmful information (e.g., harmful information (600) of FIG. 6).

[0137] According to one embodiment, the electronic device may generate a second text prior to the operation (810). The electronic device may input an intermediate image (e.g., the intermediate image (720) of FIG. 7) to an artificial intelligence model that provides a description of the image (e.g., a generative artificial intelligence model). The artificial intelligence model may output a second text, which is a description of the intermediate image, in response to the input of the intermediate image.

[0138] According to one embodiment, the first text is identified through text recognition, but the second text is generated based on an artificial intelligence model that provides a description of the image, so text hidden within the image can be extracted. For example, text written in a very small size (e.g., in pixels) in the image, or text scattered in different areas within the input image (e.g., the input image (200) of FIG. 2, the input image (300) of FIG. 3, or the input image (700) of FIG. 7) in single characters can be extracted as the second text.

[0139] According to one embodiment, if the first text extracted from the input image does not contain the first harmful information (e.g., harmful information (210) of FIG. 2, harmful information (310) of FIG. 3, harmful information (600) of FIG. 6, or the first harmful information (710) of FIG. 7), the intermediate image may be the same as the input image. If the first text extracted from the input image contains the first harmful information, the image from which the first harmful information has been removed may be determined as the intermediate image.

[0140] According to one embodiment, an electronic device may determine whether a second text contains second harmful information based on at least one of a rule predetermined to determine specific information as second harmful information and an artificial intelligence model that identifies second harmful information. The term "second harmful information" is intended to indicate a difference from first harmful information that can be identified as text and removed. Accordingly, the meaning and determination method of the second harmful information and the first harmful information may be the same.

[0141] According to one embodiment, the electronic device may perform an operation (820) if the second text contains second harmful information. The electronic device may perform an operation (870) if the second text does not contain second harmful information.

[0142] In operation (820), the electronic device can change at least a portion of the intermediate image.

[0143] According to one embodiment, a change (e.g., modification) to at least a portion of an intermediate image may include damage to at least a portion of the intermediate image.

[0144] In operation (830), the electronic device can generate a recovery image by recovering an intermediate image in which at least a portion has been changed.

[0145] According to one embodiment, the electronic device can modify and restore an intermediate image in various ways. Methods for modifying and restoring an intermediate image are described later in FIGS. 9 to 11.

[0146] In operation (840), the electronic device can determine the similarity between the recovered image and the intermediate image.

[0147] According to one embodiment, the electronic device may determine similarity through pixel-by-pixel comparison between the reconstructed image and the intermediate image. The electronic device may determine similarity through histogram-based comparison between the reconstructed image and the intermediate image. However, this is merely an example and should not be interpreted as limiting other embodiments. For example, it is obvious to those skilled in the art that other methods for determining similarity may be used in addition to the pixel-by-pixel comparison and histogram-based comparison described above.

[0148] In operation (850), the electronic device can determine whether the similarity exceeds a threshold value.

[0149] According to one embodiment, the electronic device may perform an operation (860) when the similarity exceeds a first threshold. The electronic device may perform an operation (820) and / or an operation (830) when the similarity does not exceed the first threshold.

[0150] According to one embodiment, in an operation (820) based on the similarity not exceeding a first threshold, the electronic device may change at least a portion of the intermediate image to a current change degree that is reduced compared to the previous change degree of the intermediate image (e.g., the change degree in the previous operation (820)). In an operation (820) based on the similarity not exceeding a first threshold, the electronic device may reduce the current change degree of the intermediate image to a previous change degree performed on the recovered image where the similarity exceeded the first threshold. For example, the electronic device may reduce the current blurring (e.g., smoothing) degree of the intermediate image to a previous blurring degree of the intermediate image.

[0151] According to one embodiment, in an operation (820) based on the similarity not exceeding a first threshold, the electronic device may modify at least a portion of the intermediate image using a modification method different from the previous modification method of the intermediate image (e.g., the modification method in the previous operation (820)). For example, if the previous modification method of the intermediate image is blurring, the electronic device may use vectorization as the current modification method.

[0152] According to one embodiment, in operation (830), the electronic device can recover an intermediate image in which at least a portion has been altered. In operation (830), the electronic device can recover an intermediate image in which at least a portion has been altered generated in operation (820) as the similarity does not exceed a first threshold. In operation (830), the electronic device can recover an intermediate image in which at least a portion has been altered generated in the previous operation (820) as the similarity does not exceed a first threshold.

[0153] According to one embodiment, the electronic device may repeat the modification and recovery of at least a portion of an intermediate image or the recovery of at least a portion of the intermediate image until the similarity exceeds a first threshold. According to one embodiment, the number of repetitions for the modification and recovery of at least a portion and the number of repetitions for the recovery of at least a portion of the intermediate image may be set to a maximum number of repetitions. When the maximum number of repetitions is reached, the electronic device may determine the recovered image with the highest similarity as the output image.

[0154] According to one embodiment, an artificial intelligence model (e.g., LVM) may be used to recover an intermediate image in which at least a portion has been altered. The artificial intelligence model may be an artificial intelligence model that recovers an image in a manner corresponding to how at least a portion of the intermediate image has been altered. The artificial intelligence model that recovers the image may be a suitably fine-tuned model included inside an electronic device or a model included in an external electronic device (e.g., a server (108) of FIG. 1) that communicates with the electronic device.

[0155] According to one embodiment, the electronic device may use difference instead of similarity. For example, the electronic device may perform operation (820) or operation (830) if the difference between the recovered image and the intermediate image exceeds a second threshold. For example, the electronic device may perform operation (860) if the difference between the recovered image and the intermediate image does not exceed the second threshold.

[0156] In operation (860), the electronic device can determine the recovery image as the output image.

[0157] In operation (870), the electronic device can determine the intermediate image as the output image.

[0158] According to one embodiment, if the output image differs from the input image, the electronic device may notify the user that the image has been modified. The electronic device may notify the user that the image has been modified by displaying a pop-up window containing the reason for the image modification. For example, the input image may be modified because the input image contains first harmful information identified by text and / or second harmful information not identified by text, and the electronic device may notify the user that the image has been modified through a pop-up window.

[0159] According to one embodiment, the electronic device may store the reason why the image was modified in the detailed information (e.g., metadata) of the image.

[0160] According to one embodiment, the output image may not include first harmful information identified as text and second harmful information not identified as text.

[0161] According to one embodiment, if the input image does not contain both first harmful information identified as text and second harmful information not identified as text, the input image may be determined as an output image.

[0162] According to one embodiment, if the input image contains only first harmful information identified as text, an intermediate image from which the first harmful information has been removed may be determined as an output image.

[0163] According to one embodiment, if the input image contains only second harmful information that is not identified as text, the input image may be determined as an intermediate image. At least a part of the intermediate image may be modified, and the intermediate image with at least a part modified may be restored, and the restored image may be determined as an output image.

[0164] According to one embodiment, when an input image contains both first harmful information identified as text and second harmful information not identified as text, the first harmful information may be removed from the input image to determine an intermediate image, and the second harmful information may be removed from the intermediate image to determine an output image.

[0165] The following describes the process of modifying and restoring at least a portion of an intermediate image to remove second harmful information.

[0166]

[0167] FIGS. 9 to 11 are drawings for explaining the modification and restoration of at least a portion of an image according to one embodiment.

[0168] Referring to FIG. 9, an intermediate image (920) (e.g., the intermediate image (720) of FIG. 7) and second harmful information (950) (e.g., the harmful information (600) of FIG. 6) are shown. It should be understood that the second harmful information (950) is visually indicated for convenience of explanation as being included in the second text that is not identified as text within the intermediate image (920).

[0169] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) can determine whether the intermediate image (920) contains second harmful information (950). The determination of whether the second harmful information (950) is included in the second text describing the intermediate image (920) is described in detail in FIG. 6, so a detailed explanation is omitted.

[0170] According to one embodiment, when the second text contains second harmful information (950), the electronic device may alter at least a portion of the intermediate image (920). For example, the electronic device may perform blurring (e.g., smoothing) on ​​the intermediate image. As at least a portion of the intermediate image (920) is altered, an intermediate image (930) with at least a portion altered and altered second harmful information (960) may be generated. The altered second harmful information (960) may be damaged and difficult to identify.

[0171] According to one embodiment, an electronic device can recover an intermediate image (930) in which at least a portion has been altered. The intermediate image (930) in which at least a portion has been altered can be recovered as a recovered image (940). The intermediate image (930) in which at least a portion has been altered can be input into an artificial intelligence model (e.g., LVM) that performs recovery in a manner corresponding to blurring. The artificial intelligence model can recover the intermediate image (930) in which at least a portion has been altered in a manner corresponding to blurring and output a recovered image (940). During the recovery process of the intermediate image (930) in which at least a portion has been altered, the second harmful information (960) that was altered may not be recovered. The recovered image (940) may not contain the second harmful information (950).

[0172] According to one embodiment, the electronic device can determine the similarity between an intermediate image (920) and a restored image (940). If the similarity does not exceed a threshold (e.g., a first threshold), the electronic device may repeat the alteration and restoration of at least a portion of the intermediate image (920) until the similarity exceeds the threshold. For example, the electronic device may repeat the alteration and restoration of at least a portion of the intermediate image (920) by reducing the degree of blurring. If the similarity does not exceed the threshold, the electronic device may repeat the restoration of at least a portion of the altered intermediate image (930) until the similarity exceeds the threshold.

[0173] According to one embodiment, the electronic device may determine the recovery image (940) as the output image if the similarity between the intermediate image (920) and the recovery image (940) exceeds a threshold value.

[0174] Vectorization is explained below.

[0175] Referring to FIG. 10, an intermediate image (1020) (e.g., the intermediate image (720) of FIG. 7 or the intermediate image (920) of FIG. 9) and second harmful information (1050) (e.g., the harmful information (600) of FIG. 6 or the second harmful information (950) of FIG. 9) are shown. It should be understood that the second harmful information (1050) is visually indicated for convenience of explanation as being included in the second text that is not identified as text within the intermediate image (1020).

[0176] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) can determine whether the intermediate image (1020) contains second harmful information (1050). The determination of whether the second harmful information (1050) is included in the second text describing the intermediate image (1020) is described in detail in FIG. 6, so a detailed description is omitted.

[0177] According to one embodiment, when the second text contains second harmful information (1050), the electronic device may alter at least a portion of the intermediate image (1020). For example, the electronic device may perform vectorization on the intermediate image. By altering at least a portion of the intermediate image (1020), an intermediate image (1030) with at least a portion altered and altered second harmful information (1060) (e.g., altered second harmful information (960) of FIG. 9) may be generated. The intermediate image (1030) with at least a portion altered may be a vector image. The altered second harmful information (1060) may be damaged and difficult to identify.

[0178] According to one embodiment, an intermediate image (1020) may be vectorized to extract clear image lines within the intermediate image. An intermediate image (1030) with at least some parts modified may include clear image lines extracted from the intermediate image (1020). As clear image lines within the image are extracted through vectorization, it may become more difficult to identify the modified second harmful information (1060) contained in the intermediate image (1030) with at least some parts modified. The second harmful information (1060) that is more difficult to identify may be removed during recovery. For the vectorization of the intermediate image (1020), a generative artificial intelligence model, a separate artificial intelligence model for vectorization, and / or an engine for vectorization may be used.

[0179] According to one embodiment, an electronic device can recover an intermediate image (1030) in which at least a portion has been altered. The intermediate image (1030) in which at least a portion has been altered can be recovered into a recovered image (1040) (e.g., the recovered image (940) of FIG. 9). The intermediate image (1030) in which at least a portion has been altered can be input into an artificial intelligence model (e.g., LVM) that performs recovery in a manner corresponding to vectorization. The artificial intelligence model can recover the intermediate image (1030) in which at least a portion has been altered in a manner corresponding to vectorization and output a recovered image (1040). During the recovery process of the intermediate image (1030) in which at least a portion has been altered, the second harmful information (1060) that has been altered may not be recovered. The recovered image (1040) may not contain the second harmful information (1050).

[0180] According to one embodiment, the electronic device can determine the similarity between an intermediate image (1020) and a restored image (1040). If the similarity does not exceed a threshold, the electronic device may repeat the alteration and restoration of at least a portion of the intermediate image (1020) until the similarity exceeds the threshold. If the similarity does not exceed the threshold, the electronic device may repeat the restoration of at least a portion of the intermediate image (1030) until the similarity exceeds the threshold.

[0181] According to one embodiment, the electronic device may determine the recovery image (1040) as the output image if the similarity between the intermediate image (1020) and the recovery image (1040) exceeds a threshold value.

[0182] The following describes changes made using image scaling (e.g., SCALE transformation).

[0183] Referring to FIG. 11, an intermediate image (1120) (e.g., intermediate image (720) of FIG. 7, intermediate image (920) of FIG. 9, or intermediate image (1020) of FIG. 10) and second harmful information (1150) (e.g., harmful information (600) of FIG. 6, second harmful information (950) of FIG. 9, or second harmful information (1050) of FIG. 10) are shown. It should be understood that the second harmful information (1150) is included in the second text that is not identified as text within the intermediate image (1120) and is visually indicated for convenience of explanation.

[0184] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) can determine whether the intermediate image (1120) contains second harmful information (1150). The determination of whether the second text describing the intermediate image (1120) contains second harmful information (1150) is described in detail in FIG. 6, so a detailed description is omitted.

[0185] According to one embodiment, when the second text contains second harmful information (1150), the electronic device may alter at least a portion of the intermediate image (1120). For example, the electronic device may alter at least a portion of the intermediate image (1120) by scaling down the intermediate image (1120) and then scaling it back up. As at least a portion of the intermediate image (1120) is altered, an intermediate image (1130) with at least a portion altered and altered second harmful information (1160) (e.g., altered second harmful information (960) of FIG. 9 or altered second harmful information (1060) of FIG. 10) may be generated. The intermediate image (1130) with at least a portion altered may have reduced resolution compared to the intermediate image (1120). Due to the reduction in resolution, the altered second harmful information (1160) may be damaged and difficult to identify.

[0186] According to one embodiment, an electronic device can recover an intermediate image (1130) in which at least a portion has been altered. The intermediate image (1130) in which at least a portion has been altered can be recovered into a recovered image (1140) (e.g., the recovered image (940) of FIG. 9 or the recovered image (1040) of FIG. 10). The intermediate image (1130) in which at least a portion has been altered can be input into an artificial intelligence model (e.g., LVM) that performs recovery in a manner corresponding to scaling. The artificial intelligence model can recover the intermediate image (1130) in which at least a portion has been altered in a manner corresponding to scaling and output a recovered image (1140). During the recovery process of the intermediate image (1130) in which at least a portion has been altered, the second harmful information (1160) that has been altered may not be recovered. The recovered image (1140) may not contain the second harmful information (1150).

[0187] According to one embodiment, the electronic device can determine the similarity between an intermediate image (1120) and a restored image (1140). If the similarity does not exceed a threshold, the electronic device may repeat the alteration and restoration of at least a portion of the intermediate image (1120) until the similarity exceeds the threshold. For example, the electronic device may reduce the degree of scaling. If the similarity does not exceed the threshold, the electronic device may repeat the restoration of at least a portion of the intermediate image (1130) until the similarity exceeds the threshold.

[0188] According to one embodiment, the electronic device may determine the recovery image (1140) as the output image if the similarity between the intermediate image (1120) and the recovery image (1140) exceeds a threshold value.

[0189] The removal of the second harmful information is explained further below.

[0190]

[0191] FIG. 12 is a drawing for explaining the removal of second harmful information according to one embodiment.

[0192] Referring to FIG. 12, an intermediate image (1220) (e.g., intermediate image (720) of FIG. 7, intermediate image (920) of FIG. 9, intermediate image (1020) of FIG. 10, or intermediate image (1120) of FIG. 11) is shown. The area (1230) of the intermediate image (1220) may contain second harmful information (1250) that is not identified by text recognition (e.g., harmful information (600) of FIG. 6, second harmful information (950) of FIG. 9, second harmful information (1050) of FIG. 10, or second harmful information (1150) of FIG. 11). For example, second harmful information (1250) that is described in pixel units and is very small and not identified by text recognition may be included.

[0193] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) can obtain a second text based on an intermediate image (1220). The electronic device can obtain a second text, which is a description of the intermediate image (1220), by inputting the intermediate image (1220) into an artificial intelligence model that describes the image. The artificial intelligence model can analyze the input intermediate image (1220) and generate description information that describes the intermediate image (1220).

[0194] According to one embodiment, the electronic device can determine whether the second text contains second harmful information (1250). If the second text contains second harmful information (1250), the electronic device can restore at least a portion of the intermediate image (1220) after changing it.

[0195] According to one embodiment, the second harmful information may be removed during the process of changing and restoring at least a portion of the intermediate image (1220). For example, the second harmful information may be damaged. As the second harmful information is damaged, the second harmful information may no longer have meaning. For example, during the process of changing and restoring at least a portion of the intermediate image (1220), the changed second harmful information (1260) (e.g., the changed second harmful information (960) of FIG. 9, the changed second harmful information (1060) of FIG. 10, or the changed second harmful information (1160) of FIG. 11) may be completely removed or lose its meaning as most of the information is removed.

[0196] The following describes the processing of original images containing harmful information.

[0197]

[0198] FIGS. 13 and FIGS. 14 are drawings for explaining the processing of original images containing harmful information.

[0199] Referring to FIG. 13, a user interface (1300) is shown that provides processing options for an input image containing harmful information (e.g., the input image (200) of FIG. 2, the input image (300) of FIG. 3, or the input image (700) of FIG. 7) (e.g., the original image).

[0200] According to one embodiment, if only an output image from which harmful information has been removed is stored in an electronic device (e.g., the electronic device (101) of FIG. 1), it may cause confusion for the user. For example, an image different from the image downloaded by the user (e.g., the input image) and an image (e.g., the output image) may be stored, causing confusion for the user. To prevent confusion for the user, the electronic device may provide processing options for the input image.

[0201] According to one embodiment, the "automatic deletion of harmful images" option of the user interface (1300) may be turned ON depending on user input. When the "automatic deletion of harmful images" option is turned ON, the electronic device may delete an input image containing first harmful information identified by text (e.g., harmful information (210) of FIG. 2, harmful information (310) of FIG. 3, harmful information (600) of FIG. 6, or first harmful information (710) of FIG. 7) and / or second harmful information not identified by text (e.g., harmful information (600) of FIG. 6, second harmful information (950) of FIG. 9, second harmful information (1050) of FIG. 10, second harmful information (1150) of FIG. 11, or second harmful information (1250) of FIG. 12), and save only the output image.

[0202] According to one embodiment, the "automatic deletion of harmful images" option of the user interface (1300) may be turned OFF and the "display of harmful image information" option may be turned ON depending on user input. At the bottom of the "display of harmful image information" option, a preview may be provided for the location where a masking window (1310) indicating that it is a harmful image is displayed. The masking window (1310) may be a UI object (user interface object) indicating that it is a harmful image. The location where the masking window (1310) is displayed may be predetermined. For example, the location of the masking window (1310) may be predetermined as the bottom, middle, top, and / or diagonal direction of the input image. As the "automatic deletion of harmful images" option is turned OFF and the "display of harmful image information" option is turned ON, the electronic device may also store an input image containing first harmful information identified by text and / or second harmful information not identified by text.

[0203] According to one embodiment, the electronic device may create a masking window (1310) on an input image containing first harmful information identified by text and / or second harmful information not identified by text. The electronic device may display the reason why the input image is masked (e.g., containing harmful information) through the masking window (1310).

[0204] According to one embodiment, the location of the masking window (1310) may be determined based on the location of the first harmful information and / or the second harmful information. The electronic device may identify the location and / or size of the first harmful information and / or the second harmful information while determining whether the input image contains the first harmful information and / or the second harmful information. The electronic device may generate a masking window (1310) having a size corresponding to the size of the first harmful information and / or the second harmful information, and display the masking window (1310) at a location corresponding to the location of the first harmful information and / or the second harmful information.

[0205] According to one embodiment, the input image may include a plurality of first harmful information and / or second harmful information. The electronic device may generate a plurality of shielding windows (1310) capable of shielding the plurality of first harmful information and / or second harmful information.

[0206] According to one embodiment, the input image may include a plurality of first harmful information and / or second harmful information. The electronic device may generate a covering window (1310) capable of covering all of the plurality of first harmful information and / or second harmful information based on the location and / or size of the plurality of first harmful information and / or second harmful information.

[0207] According to one embodiment, when storing an input image containing first harmful information identified by text and / or second harmful information not identified by text, said input image may be stored separately from the output image. For example, the electronic device may create a separate folder and store only the output image in said folder.

[0208] According to one embodiment, the electronic device may regenerate an input image to include a warning message and / or a warning reason. The regenerated input image may include a warning message and / or a warning reason. The warning message and / or the warning reason may indicate that the input image is a harmful image and the reason why it was determined to be a harmful image. Unlike the masking window (1310), the warning message and / or the warning reason may be part of the regenerated input image. In the regenerated input image, the area containing the first harmful information and / or the second harmful information may be regenerated as the warning message and / or the warning reason. A generative artificial intelligence model may be used for the regeneration of the input image. The electronic device may display the regenerated input image instead of the input image.

[0209] According to one embodiment, an input image regenerated to include a warning message and / or a reason for the warning may be stored separately from the input image. For example, the electronic device may create a separate folder and store only the regenerated input image in that folder.

[0210] According to one embodiment, the "automatic deletion of harmful images" option may be turned OFF and the "display of harmful image information" option may also be turned OFF in the user interface (1300). When the "automatic deletion of harmful images" option is turned OFF and the "display of harmful image information" option is also turned OFF, the electronic device may display an input image without displaying a separate warning message and / or a reason for the warning.

[0211] According to one embodiment, in the user interface (1300), the "automatic deletion of harmful images" option may be turned OFF and the "display of harmful image information" option may be turned ON. When the "automatic deletion of harmful images" option is turned OFF and the "display of harmful image information" option is turned ON, the electronic device may display a regenerated input image. While displaying the regenerated input image, the electronic device may receive a command to display the input image. When the electronic device receives a command to display the input image, it may display a warning message indicating that the input image contains harmful information. For example, the warning message may include phrases such as "The input image contains harmful information. Do you still wish to view the input image?". When the electronic device receives input via the confirmation button of the warning message, it may display the input image containing harmful information.

[0212] According to one embodiment, the electronic device displays an output image and may further provide a display of an input image corresponding to the output image based on user input. For example, the electronic device may further display an object along with the output image. The object may be a link for displaying the input image corresponding to the output image. When the electronic device receives input regarding the object, it may display a warning message indicating that the input image corresponding to the output image contains harmful information. For example, the warning message may include a phrase such as, "The input image contains harmful information. Do you still wish to view the input image?" When the electronic device receives input via a confirmation button of the warning message, it may display the input image containing harmful information.

[0213] Below, an example of displaying a cover window (1310) is described.

[0214] Referring to FIG. 14, a plurality of screens (1410, 1420, 1430) (e.g., the screen (1310) of FIG. 13) are shown.

[0215] According to one embodiment, a masking window (1410) may be displayed at the bottom of an input image (1415) (e.g., the input image (200) of FIG. 2, the input image (300) of FIG. 3, or the input image (700) of FIG. 7) (e.g., the original image). The masking window (1410) may include the reason why the masking window (1410) is displayed on the input image (1415).

[0216] According to one embodiment, the occlusion window (1420) may be displayed diagonally on the input image (1425) (e.g., the input image (200) of FIG. 2, the input image (300) of FIG. 3, or the input image (700) of FIG. 7). The occlusion window (1420) may include the reason why the occlusion window (1420) is displayed on the input image (1425).

[0217] According to one embodiment, the masking window (1430) may be displayed in the center of the input image (1435) (e.g., the input image (200) of FIG. 2, the input image (300) of FIG. 3, or the input image (700) of FIG. 7). The masking window (1410) may include the reason why the masking window (1430) is displayed on the input image (1435).

[0218] According to one embodiment, the electronic device may display a masking window over an area in an input image where harmful information (e.g., first harmful information and / or second harmful information) exists. For example, if the harmful information is located at the bottom of the input image (1415), the electronic device may identify the location and / or size of the harmful information and generate a masking window corresponding to the location and / or size of the harmful information. The electronic device may mask at least a portion of the harmful information by displaying a masking window corresponding to the location and / or size of the harmful information over the area in which the harmful information exists.

[0219] However, the above-described method of displaying the covering window is merely an example to aid understanding and should not be interpreted as limiting or restricting the embodiments of the present disclosure.

[0220]

[0221] FIG. 15 is a diagram illustrating a method for removing harmful information from an image according to one embodiment.

[0222] According to one embodiment, an electronic device can determine whether harmful information exists not only in still images but also in videos (e.g., movies, streaming videos, etc.) and / or live streaming videos (e.g., real-time VR chat, etc.) and remove the harmful information. If harmful information is included in any frame or a part of any frame of a video and / or live streaming video, the user may not be aware of it. However, an electronic device performing analysis on the video and / or live streaming video may recognize it, which may cause a problem. The electronic device can determine whether harmful information exists for each frame of the video and / or live streaming video and remove the harmful information. For example, the electronic device can determine whether harmful information exists using each frame as an input image and remove the harmful information.

[0223] For example, the t-th frame (1500) may correspond to an input image (e.g., input image (200) of FIG. 2, input image (300) of FIG. 3, input image (700) of FIG. 7, or input image (1415, 1425, 1435) of FIG. 14. It may be determined whether the first harmful information (e.g., harmful information (210) of FIG. 2, harmful information (310) of FIG. 3, harmful information (600) of FIG. 6, or first harmful information (710) of FIG. 7) is included in the first text identified by the text included in the t-th frame (1500). If the first harmful information is included, the electronic device may perform removal of the first harmful information. The electronic device may obtain a second text that is a description of an intermediate frame image from which the first harmful information has been removed (e.g., intermediate image (720) of FIG. 7, intermediate image (920) of FIG. 9, intermediate image (1020) of FIG. 10, intermediate image (1120) of FIG. 11, or intermediate image (1220) of FIG. 12). The electronic device may determine whether the second harmful information not identified by the text (e.g., harmful information (600) of FIG. 6, second harmful information (950) of FIG. 9, second harmful information (1050) of FIG. 10, second harmful information (1150) of FIG. 11, or second harmful information (1250) of FIG. 12) is included in the second text. If the second harmful information is included, the electronic device may perform removal of the second harmful information to generate an output frame image. The electronic device may display a modified t-th frame (1540) which is an output frame image from which the harmful image has been removed.

[0224] According to one embodiment, in the case of a video, when downloading to an electronic device, it is possible to determine whether harmful information is included frame by frame and perform removal of harmful information frame by frame. When determining whether harmful information is included frame by frame during download, the electronic device stores the video from which harmful information has been removed and can subsequently play the video from which harmful information has been removed.

[0225] According to one embodiment, in the case of a video, the electronic device can determine whether harmful information is included frame by frame when playing the video and perform removal of harmful information. When performing removal of harmful information while playing the video, the electronic device can perform removal of harmful information in the background for frames to be displayed after the current frame while displaying the current frame.

[0226] According to one embodiment, harmful information may be included in audio contained in video (e.g., movies, streaming videos, etc.) and live streaming video (e.g., real-time VR chat, etc.). An electronic device can convert the audio into text. For example, the electronic device can convert the audio into a script using a generative artificial intelligence model and / or a speech-to-text (STT) model. The electronic device can determine whether the script contains harmful information. As the determination of harmful information is described in detail in FIG. 6, a detailed explanation is omitted. The electronic device can regenerate text corresponding to the harmful information. The electronic device can regenerate text corresponding to the harmful information to fit the context. The electronic device can modify the script by replacing the harmful information with the regenerated text. The electronic device can generate audio based on the script. For example, the electronic device can generate audio using a generative artificial intelligence model and / or a text-to-speech (TTS) model.

[0227] According to one embodiment, an electronic device comprising high-performance hardware capable of parallel processing may be required for removing harmful information from the above-described video (e.g., movies, streaming video, etc.) and live streaming video (e.g., real-time VR chat, etc.) and / or audio included therein.

[0228]

[0229] FIG. 16 is a block diagram of an electronic device for explaining the removal of harmful information according to one embodiment.

[0230] Referring to FIG. 16, an image analyzer (1600) is illustrated. The image analyzer (1600) may be provided as software, such as an application. The image analyzer (1600) may include a plurality of modules (1601, 1602, 1603, 1604, 1605, 1606, 1607). Only components related to the embodiments of the present disclosure are illustrated in the image analyzer (1600). Accordingly, it is obvious to those skilled in the art that general-purpose modules other than the plurality of modules (1601, 1602, 1603, 1604, 1605, 1606, 1607) illustrated in FIG. 16 may be included.

[0231] In addition, the functions of the multiple modules (1601, 1602, 1603, 1604, 1605, 1606, 1607) described below do not necessarily have to be performed in each module. For example, there may be a module among the multiple modules (1601, 1602, 1603, 1604, 1605, 1606, 1607) that performs two or more functions. Therefore, it should be understood that the description of each module described below does not necessarily correspond one-to-one with each module.

[0232] According to one embodiment, an image analyzer (1600) determines whether harmful information is included in an input image (e.g., input image (200) of FIG. 2, input image (300) of FIG. 3, input image (700) of FIG. 7, or input image (1415, 1425, 1435) of FIG. 14), and if harmful information is included, the harmful information can be removed.

[0233] According to one embodiment, an image analyzer (1600) can perform analysis on an input image in response to a request received from a plurality of applications (1620, 1630). The plurality of applications (1620, 1630) may be applications that perform target operations based on the input image.

[0234] According to one embodiment, the reliable image setting module (1601) can manage settings for the image analyzer (1600). For example, the reliable image setting module (1601) can provide ON / OFF functionality for the image analyzer (1600). For example, the reliable image setting module (1601) can provide whether to delete the original image or whether to display a masking window (e.g., the masking window (1310) of FIG. 13 or a plurality of masking windows (1410, 1420, 1430) of FIG. 14).

[0235] According to one embodiment, an optical character recognition module (1602) can perform the extraction of text included in an input image. For example, a first text identified as text can be extracted from the input image. The OCR module can obtain location information of the extracted text.

[0236] According to one embodiment, the text safety filter module (1603) can determine whether first harmful information (e.g., harmful information (210) of FIG. 2, harmful information (310) of FIG. 3, harmful information (600) of FIG. 6, or first harmful information (710) of FIG. 7) exists within the first text. The text safety filter module (1603) can determine whether second harmful information (e.g., harmful information (600) of FIG. 6, second harmful information (950) of FIG. 9, second harmful information (1050) of FIG. 10, second harmful information (1150) of FIG. 11, or second harmful information (1250) of FIG. 12) exists within the second text obtained from the intermediate image. The determination of whether harmful information exists is described in detail in FIG. 6, so a detailed explanation is omitted.

[0237] According to one embodiment, the partial image designer module (1604) may determine an area containing harmful information based on location information of harmful information received from the OCR module (1602) when it is determined that first harmful information exists in the first text. The partial image designer module (1604) may delete the area containing harmful information. The partial image designer module (1604) may perform blurring, vectorization, or scaling on the area containing harmful information. The area containing harmful information on which blurring, vectorization, or scaling has been performed may be recycled.

[0238] According to one embodiment, the LVM module (large vision model module) (1606) can regenerate a deleted area containing harmful information. The LVM module (1606) can regenerate harmful information that has been blurred, vectorized, or scaled. The LVM module (1606) can perform regeneration based on an input image.

[0239] According to one embodiment, the LVM module (1606) may use an external LVM module (1610) for regeneration. For example, if available resources are insufficient, the LVM module (1606) may use an external LVM module (1610) to perform regeneration.

[0240] According to one embodiment, the full image designer module (1605) can generate an intermediate image (e.g., the intermediate image (720) of FIG. 7, the intermediate image (920) of FIG. 9, the intermediate image (1020) of FIG. 10, the intermediate image (1120) of FIG. 11, or the intermediate image (1220) of FIG. 12) based on the region and input image regenerated from the LVM module (1606).

[0241] According to one embodiment, if the text safety filter module (1603) determines that the second text for the intermediate image contains harmful information, the image cleaner module (1607) can modify and restore at least a portion of the intermediate image. The image cleaner module (1607) may include a smoother, a vectorizer, and a reformator. The smoother can perform smoothing (e.g., blurring) on ​​the image. The vectorizer can perform vectorization on the image. The reformator can perform scaling on the image.

[0242] According to one embodiment, the image analyzer (1600) may further include a generative artificial intelligence model. The generative artificial intelligence model may be used in various ways. For example, the generative artificial intelligence model may be used to output a description of an image input to the generative artificial intelligence model. For example, the generative artificial intelligence model may be used to regenerate an area containing first harmful information. For example, the generative artificial intelligence model may perform modifications to at least a portion of the image input to the generative artificial intelligence model.

[0243] According to one embodiment, the image analyzer (1600) may not include a generative artificial intelligence model, and an external generative artificial intelligence model may be used for the operation described above. The external generative artificial intelligence model may be included in an external server (e.g., the server (108) of FIG. 1).

[0244]

[0245] FIG. 17 is a flowchart for explaining the removal of harmful information included in an image according to one embodiment.

[0246] The operations described below may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. Additionally, some operations may be omitted depending on some embodiments. Operations (1710) to (1720) may be performed by at least one component (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1).

[0247] According to one embodiment, instructions stored in memory (e.g., memory (130) of FIG. 1) by at least one processor may be executed individually and / or collectively, and the instructions may cause an electronic device to perform the following operations (1710) to operations (1730).

[0248] In operation (1710), the electronic device may receive an input image (e.g., input image of FIG. 2 (200), input image of FIG. 3 (300), input image of FIG. 7 (700), or input image of FIG. 14 (1415, 1425, 1435)) for removing harmful information identified by text (e.g., harmful information of FIG. 2 (210), harmful information of FIG. 3 (310), harmful information of FIG. 6 (600), or first harmful information of FIG. 7 (710)).

[0249] According to one embodiment, the electronic device can input an input image to an image analyzer (e.g., the image analyzer (1600) of FIG. 16).

[0250] In operation (1720), the electronic device can determine whether the text included in the input image contains harmful information.

[0251] According to one embodiment, the electronic device can perform the extraction of text (e.g., first text) included in an input image. For example, when an input image is input to an image analyzer, an OCR module (e.g., OCR module (1602) of FIG. 16) can perform the extraction of text included in the input image.

[0252] According to one embodiment, an electronic device can determine whether the extracted text contains harmful information. For example, text may be input into a text safety filter module (e.g., text safety filter module (1603)). The text safety filter module can determine whether harmful information is present in the text. The text safety filter module can determine whether harmful information is included, such as harmful information (e.g., profanity, sexual language and / or violent language) or aggressive intent (e.g., prompt injection, prompt extraction and / or jail break).

[0253] In operation (1730), the electronic device can determine an output image from an input image based on whether the text contains harmful information.

[0254] According to one embodiment, the electronic device can determine an input image as an output image if it does not contain harmful information.

[0255] According to one embodiment, the electronic device may determine an input image from which harmful information has been removed as an output image by performing removal of harmful information when harmful information is included. For example, a partial image designer module (e.g., the partial image designer module (1604) of FIG. 16) may determine an area containing harmful information based on location information of harmful information received from an OCR module when it is determined that harmful information exists in the text. The partial image designer module may delete the area containing harmful information.

[0256] According to one embodiment, the electronic device can regenerate an area from which harmful information has been removed. For example, an LVM module (e.g., LVM module (1606) of FIG. 16) can regenerate an area from which harmful information has been removed.

[0257] According to one embodiment, a full image designer module (e.g., the full image designer module (1605) of FIG. 16) can generate an output image based on the regenerated area and the input image.

[0258] According to one embodiment, the electronic device can determine an image in which an area containing harmful information is regenerated as an output image.

[0259] A detailed description of operations (1710) to (1730) is omitted as it is described in detail in FIGS. 1 to 16.

[0260]

[0261] FIG. 18 is a flowchart for explaining the removal of harmful information not identified in an image according to one embodiment.

[0262] The operations described below may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. Additionally, some operations may be omitted depending on some embodiments. Operations (1810) to (1850) may be performed by at least one component (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1).

[0263] According to one embodiment, instructions stored in memory (e.g., memory (130) of FIG. 1) by at least one processor may be executed individually and / or collectively, and the instructions may cause an electronic device to perform the following operations (1810) to operations (1850).

[0264] In operation (1810), the electronic device may receive an input image (e.g., input image (200) of FIG. 2, input image (300) of FIG. 3, input image (700) of FIG. 7, or input image (1415, 1425, 1435) of FIG. 14) for removing harmful information that is not identified by text (e.g., harmful information (600) of FIG. 6, second harmful information (950) of FIG. 9, second harmful information (1050) of FIG. 10, second harmful information (1150) of FIG. 11, or second harmful information (1250) of FIG. 12).

[0265] According to one embodiment, the electronic device can input an input image to an image analyzer (e.g., the image analyzer (1600) of FIG. 16).

[0266] In operation (1820), the electronic device can generate text based on the input image.

[0267] According to one embodiment, an electronic device may input an input image to an artificial intelligence model (e.g., a generative artificial intelligence model) that provides a description of the image in order to remove harmful information not identified by text. The artificial intelligence model may generate text (e.g., a second text) that is a description of the input image as a response to the input of the input image.

[0268] In operation (1830), the electronic device can determine whether the text contains harmful information.

[0269] According to one embodiment, an electronic device can determine whether the generated text contains harmful information. For example, text may be input into a text safety filter module (e.g., text safety filter module (1603)). The text safety filter module can determine whether harmful information is present in the text. The text safety filter module can determine whether harmful information is included, such as harmful information (e.g., profanity, sexual language and / or violent language) or aggressive intent (e.g., prompt injection, prompt extraction and / or jail break).

[0270] In operation (1840), the electronic device can change at least part of the input image if the text contains harmful information.

[0271] According to one embodiment, an image cleaner module (e.g., the image cleaner module (1607) of FIG. 16) can modify at least a portion of an input image. The image cleaner module may include a smoother, a vectorizer, and a reformatter. The image cleaner module can modify at least a portion of an input image using at least one of the smoother, the vectorizer, and the reformatter.

[0272] In operation (1850), the electronic device can recover an input image in which at least a portion has been altered to generate an output image from which harmful information has been removed.

[0273] According to one embodiment, an LVM module (e.g., the LVM module (1606) of FIG. 16) can recover an input image in which at least a part has been changed.

[0274] According to one embodiment, a generative artificial intelligence model may be used to recover an input image in which at least a part has been changed.

[0275] A detailed description of operations (1810) to operations (1850) is omitted as it is described in detail in FIGS. 1 to 17.

[0276]

[0277] FIG. 19 is a flowchart for explaining a method for first performing the removal of harmful information that is not identified in an image according to one embodiment.

[0278] The operations described below may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. Additionally, some operations may be omitted depending on some embodiments. Operations (110) through (1960) may be performed by at least one component (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1).

[0279] According to one embodiment, instructions stored in memory (e.g., memory (130) of FIG. 1) by at least one processor may be executed individually and / or collectively, and the instructions may cause an electronic device to perform the following operations (1910) to operations (1960).

[0280] In operation (1910), the electronic device may receive an input image (e.g., input image of FIG. 2 (200), input image of FIG. 3 (310), harmful information of FIG. 6 (600) or first harmful information of FIG. 7 (710)) identified by text and second harmful information not identified by text (e.g., harmful information of FIG. 6 (600), second harmful information of FIG. 9 (950), second harmful information of FIG. 10 (1050), second harmful information of FIG. 11 (1150) or second harmful information of FIG. 12 (1250)) for the removal of at least one of the first harmful information identified by text (e.g., input image of FIG. 2 (200), input image of FIG. 3 (300), input image of FIG. 7 (700) or input image of FIG. 14 (1415, 1425, 1435)).

[0281] According to one embodiment, the electronic device can input an input image to an image analyzer (e.g., the image analyzer (1600) of FIG. 16).

[0282] In operation (1920), the electronic device can generate text based on the input image (e.g., second text).

[0283] According to one embodiment, the electronic device may perform an operation (1920) to remove harmful information that is not identified as text. The electronic device may input an input image to an artificial intelligence model that provides a description of the image (e.g., a generative artificial intelligence model). The artificial intelligence model may output text (e.g., a second text) which is a description of the input image as a response to the input of the input image.

[0284] In operation (1930), the electronic device can determine whether the text contains second harmful information.

[0285] According to one embodiment, the electronic device can determine whether the generated text contains harmful information. For example, a text safety filter module (e.g., text safety filter module (1603)) may be used to determine whether harmful information is included.

[0286] In operation (1940), the electronic device can determine an intermediate image from the input image based on whether the text contains second harmful information.

[0287] According to one embodiment, if the text contains second harmful information, the electronic device may change (e.g., modify) at least a part of the input image. The change of at least a part of the input image may include damage to the input image. The electronic device may recover the input image in which at least a part has been changed. Text may be removed during the process of changing and recovering at least a part of the input image. The electronic device may determine the recovered image in which the text has been removed as an intermediate image. An image cleaner module (e.g., the image cleaner module (1607) of FIG. 16) may be used for changing and recovering at least a part of the input image.

[0288] According to one embodiment, if the text does not contain second harmful information, the input image may be determined as an intermediate image.

[0289] In operation (1950), the electronic device can determine whether the text included in the intermediate image contains the first harmful information.

[0290] According to one embodiment, the first harmful information identified as text may not be removed through the modification and recovery of at least a portion of the input image. The intermediate image may still contain the first harmful information identified as text.

[0291] According to one embodiment, an electronic device can extract text contained in an intermediate image through text recognition such as OCR. An OCR module (e.g., the OCR module (1602) of FIG. 16) may be used to extract text contained in an intermediate image.

[0292] According to one embodiment, the electronic device can determine whether the extracted text contains harmful information. For example, a text safety filter module (e.g., text safety filter module (1603)) may be used to determine whether harmful information is included.

[0293] In operation (1960), the electronic device can determine an output image from an intermediate image based on whether the text contains first harmful information.

[0294] According to one embodiment, when the text contains first harmful information, the electronic device may determine an image containing a regenerated area containing the area containing the harmful information as an output image. The regenerated area may be an area in which the first harmful information is removed and the area is regenerated to blend naturally with the remaining areas. For the removal of the first harmful information and the generation of the output image, a partial image designer module (e.g., the partial image designer module (1604) of FIG. 16), an LVM module (e.g., the LVM module (1606) of FIG. 16), a generative artificial intelligence model and / or a full image designer module (e.g., the full image designer module (1605) of FIG. 16)) may be used.

[0295] According to one embodiment, the electronic device can determine an intermediate image as an output image if the text does not contain first harmful information.

[0296]

[0297] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) may include a memory (e.g., the memory (130) of FIG. 1) for storing instructions. The electronic device may include at least one processor (e.g., the processor (120) of FIG. 1) for executing instructions. When the instructions are executed individually or collectively by at least one processor, the electronic device may receive an input image (e.g., input image of FIG. 2 (200), input image of FIG. 3 (310), harmful information of FIG. 6 (600) or first harmful information of FIG. 7 (710)) for the removal of at least one of first harmful information identified by text (e.g., harmful information of FIG. 2 (210), harmful information of FIG. 3 (300), harmful information of FIG. 6 (600)), harmful information of FIG. 7 (310), harmful information of FIG. 6 (600)), second harmful information of FIG. 9 (950), second harmful information of FIG. 10 (1050), second harmful information of FIG. 11 (1150) or second harmful information of FIG. 12 (1250)) and second harmful information not identified by text (e.g., input image of FIG. 2 (200), input image of FIG. 3 (300), input image of FIG. 7 (700)), or input image of FIG. 14 (1415, 1425, 1435)). When the instructions are executed individually or collectively by at least one processor, the electronic device may determine whether the first text included in the input image contains first harmful information. When the instructions are executed individually or collectively by at least one processor, the electronic device may determine an intermediate image (e.g., intermediate image (720) of FIG. 7, intermediate image (920) of FIG. 9, intermediate image (1020) of FIG. 10, intermediate image (1120) of FIG. 11, or intermediate image (1220) of FIG. 12) from the input image based on whether the first text contains first harmful information. When the instructions are executed individually or collectively by at least one processor, the electronic device may generate a second text based on the intermediate image.When the instructions are executed individually or collectively by at least one processor, the electronic device may determine whether the second text contains second harmful information. When the instructions are executed individually or collectively by at least one processor, the electronic device may determine an output image from an intermediate image based on whether the second text contains second harmful information.

[0298] According to one embodiment, the first harmful information and the second harmful information may include systemic harmful information that threatens the security of the electronic device system or causes a malfunction, and unethical harmful information identified as unethical.

[0299] According to one embodiment, when the instructions are executed individually or collectively by at least one processor, the electronic device may extract a first text from an input image. When the instructions are executed individually or collectively by at least one processor, the electronic device may determine whether the first text contains first harmful information. When the instructions are executed individually or collectively by at least one processor, the electronic device may remove an area containing the first harmful information from the input image if the first text contains the first harmful information. When the instructions are executed individually or collectively by at least one processor, the electronic device may regenerate the removed area based on the input image. When the instructions are executed individually or collectively by at least one processor, the electronic device may determine the input image containing the regenerated area as an intermediate image.

[0300] According to one embodiment, when instructions are executed individually or collectively by at least one processor, the electronic device may determine whether the first harmful information is included in the first text based on at least one of a rule determined to determine specific information as the first harmful information and a first artificial intelligence model that identifies the first harmful information.

[0301] According to one embodiment, when the instructions are executed individually or collectively by at least one processor, the electronic device may be able to regenerate the removed area using a second artificial intelligence model that generates an image (e.g., the LVM module (1606) of FIG. 16 or the external LVM module (1610) of FIG. 16).

[0302] According to one embodiment, when the instructions are executed individually or collectively by at least one processor, the electronic device may be input into a third artificial intelligence model describing an intermediate image. When the instructions are executed individually or collectively by at least one processor, the electronic device may be configured to obtain a second text describing an intermediate image from the third artificial intelligence model. When the instructions are executed individually or collectively by at least one processor, the electronic device may be configured to determine whether the second text contains second harmful information.

[0303] According to one embodiment, when the instructions are executed individually or collectively by at least one processor, the electronic device may be able to modify at least a portion of an intermediate image when the second text contains second harmful information. When the instructions are executed individually or collectively by at least one processor, the electronic device may be able to restore the intermediate image in which at least a portion has been modified to generate a restored image (e.g., the restored image (940) of FIG. 9, the restored image (1040) of FIG. 10, or the restored image (1140) of FIG. 11), and determine the restored image as an output image.

[0304] According to one embodiment, instructions, when executed individually or collectively by at least one processor, can cause an electronic device to determine the similarity between a recovered image and an intermediate image. When instructions, when executed individually or collectively by at least one processor, can cause an electronic device to determine the intermediate image as the output image if the similarity exceeds a threshold value.

[0305] According to one embodiment, when instructions are executed individually or collectively by at least one processor, the electronic device may be able to regenerate a recovery image if the similarity does not exceed a threshold.

[0306] According to one embodiment, when instructions are executed individually or collectively by at least one processor, the electronic device may determine an input image as an intermediate image when it is determined that the first text does not contain first harmful information.

[0307] According to one embodiment, when instructions are executed individually or collectively by at least one processor, the electronic device may be configured to determine an intermediate image as an output image when it is determined that the second text does not contain second harmful information.

[0308] According to one embodiment, the electronic device may include a memory for storing instructions. The electronic device may include at least one processor for executing instructions. When the instructions are executed individually or collectively by at least one processor, the electronic device may receive an input image for removing harmful information that is not identified as text. When the instructions are executed individually or collectively by at least one processor, the electronic device may generate text based on the input image. When the instructions are executed individually or collectively by at least one processor, the electronic device may determine whether the text contains harmful information. When the instructions are executed individually or collectively by at least one processor, the electronic device may modify at least a portion of the input image if the text contains harmful information. When the instructions are executed individually or collectively by at least one processor, the electronic device may restore the input image with at least a portion modified to generate an output image from which harmful information has been removed.

[0309] According to one embodiment, harmful information may include systemic harmful information that threatens the security of the electronic device system or causes a malfunction, and unethical harmful information identified as unethical.

[0310] According to one embodiment, when instructions are executed individually or collectively by at least one processor, the electronic device may determine the similarity between a recovered image and an input image. When instructions are executed individually or collectively by at least one processor, the electronic device may determine the recovered image as an output image if the similarity exceeds a threshold value.

[0311] According to one embodiment, a method of operation of an electronic device may include an operation of receiving an input image for removing at least one of first harmful information identified by text and second harmful information not identified by text. A method of operation of an electronic device may include an operation of determining whether the first harmful information is included in the first text included in the input image. A method of operation of an electronic device may include an operation of determining an intermediate image from the input image based on whether the first harmful information is included in the first text. A method of operation of an electronic device may include an operation of generating a second text based on the intermediate image. A method of operation of an electronic device may include an operation of determining whether the second harmful information is included in the second text. A method of operation of an electronic device may include an operation of determining an output image from the intermediate image based on whether the second harmful information is included in the second text.

[0312] According to one embodiment, the first harmful information and the second harmful information may include systemic harmful information that threatens the security of the electronic device system or causes a malfunction, and unethical harmful information identified as unethical.

[0313] According to one embodiment, the operation of determining whether the first harmful information is included may include the operation of extracting the first text from the input image. The operation of determining whether the first harmful information is included may include the operation of determining whether the first harmful information is included in the first text. The operation of determining an intermediate image from the input image may include the operation of removing the region containing the first harmful information from the input image when the first text contains the first harmful information. The operation of determining an intermediate image from the input image may include the operation of regenerating the removed region based on the input image. The operation of determining an intermediate image from the input image may include the operation of determining the input image containing the regenerated region as the intermediate image.

[0314] According to one embodiment, the operation of generating a second text based on an intermediate image may include the operation of inputting the intermediate image into a third artificial intelligence model that describes the intermediate image. The operation of generating a second text based on an intermediate image may include the operation of obtaining a second text describing the intermediate image from the third artificial intelligence model.

[0315] According to one embodiment, the operation of determining an output image from an intermediate image may include an operation of changing at least a portion of the intermediate image when the second text contains second harmful information. The operation of determining an output image from an intermediate image may include an operation of generating a recovered image by recovering the intermediate image in which at least a portion has been changed. The operation of determining an output image from an intermediate image may include an operation of determining the recovered image as the output image.

[0316] According to one embodiment, a non-transient computer-readable recording medium may store one or more computer programs including instructions that execute an operation of receiving an input image for the removal of at least one of first harmful information identified as text and second harmful information not identified as text. A non-transient computer-readable recording medium may store one or more computer programs including instructions that execute an operation of determining whether the first harmful information is included in the first text included in the input image. A non-transient computer-readable recording medium may store one or more computer programs including instructions that execute an operation of determining an intermediate image from the input image based on whether the first harmful information is included in the first text. A non-transient computer-readable recording medium may store one or more computer programs including instructions that execute an operation of generating a second text based on the intermediate image. A non-transient computer-readable recording medium may store one or more computer programs including instructions that execute an operation of determining whether the second harmful information is included in the second text. A non-transient computer-readable recording medium may store one or more computer programs including instructions that execute an operation to determine an output image from an intermediate image based on whether the second text contains second harmful information.

[0317]

[0318] Furthermore, the embodiments of the present invention disclosed in this specification and drawings are merely specific examples provided to facilitate the explanation of the technical content according to the embodiments of the present invention and to aid in understanding the embodiments of the present invention, and are not intended to limit the scope of the embodiments of the present invention. Accordingly, the scope of the various embodiments of the present invention should be interpreted to include all modifications or variations derived based on the technical concept of the various embodiments of the present invention, in addition to the embodiments disclosed herein.

Claims

1. In an electronic device (101), Memory (130) for storing instructions; and At least one processor (120) that executes the above instructions Includes, When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) is enabled, Input images (200, 300, 700, 1415, 1425, 1435) are received for the removal of at least one of first harmful information (210, 310, 600, 710) identified by text and second harmful information (600, 950, 1050, 1150, 1250) not identified by text, and Determining whether the first harmful information (210, 310, 600, 710) is included in the first text included in the above input images (200, 300, 700, 1415, 1425, 1435), and Determining an intermediate image (720, 920, 720, 1120, 1220) from the input image (200, 300, 700, 1415, 1425, 1435) based on whether the first harmful information (210, 310, 600, 710) is included in the first text, and Generate a second text based on the above intermediate images (720, 920, 720, 1120, 1220), and Determining whether the second text contains the second harmful information (600, 950, 1050, 1150, 1250), and Determining an output image from the intermediate image (720, 920, 720, 1120, 1220) based on whether the second harmful information (600, 950, 1050, 1150, 1250) is included in the second text, Electronic device (101).

2. In Paragraph 1, The above first harmful information (210, 310, 600, 710) and the above second harmful information (600, 950, 1050, 1150, 1250) are, including systemic harmful information that threatens security or causes malfunction of the system of the electronic device (101) and unethical harmful information identified as unethical, Electronic device (101).

3. In either Paragraph 1 or Paragraph 2, When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) is enabled, Extract the first text from the above input images (200, 300, 700, 1415, 1425, 1435), and Determining whether the first harmful information (210, 310, 600, 710) is included in the first text, and If the first text contains the first harmful information (210, 310, 600, 710), the area containing the first harmful information (210, 310, 600, 710) is removed from the input image (200, 300, 700, 1415, 1425, 1435), and Regenerate the removed region based on the above input images (200, 300, 700, 1415, 1425, 1435), and Determining the input image (200, 300, 700, 1415, 1425, 1435) including the regenerated area as the intermediate image (720, 920, 720, 1120, 1220), Electronic device (101).

4. In any one of paragraphs 1 through 3, When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) is enabled, A method for determining whether the first text contains the first harmful information (210, 310, 600, 710) based on at least one of a rule predetermined to determine specific information as the first harmful information (210, 310, 600, 710) and a first artificial intelligence model that identifies the first harmful information (210, 310, 600, 710). Electronic device (101).

5. In any one of paragraphs 1 through 4, When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) is enabled, Regenerating the removed area using a second artificial intelligence model (1606, 1610) that generates an image, Electronic device (101).

6. In any one of paragraphs 1 through 5, When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) is enabled, The above intermediate images (720, 920, 720, 1120, 1220) are input into a third artificial intelligence model that describes images, and Obtaining the second text describing the intermediate images (720, 920, 720, 1120, 1220) from the third artificial intelligence model, and Determining whether the second harmful information (600, 950, 1050, 1150, 1250) is included in the second text, Electronic device (101).

7. In any one of paragraphs 1 through 6, When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) is enabled, If the second text contains the second harmful information (600, 950, 1050, 1150, 1250), at least a portion of the intermediate image (720, 920, 720, 1120, 1220) is modified, the intermediate image (720, 920, 720, 1120, 1220) with at least a portion modified is restored to generate a restored image (940, 1040, 1140), and the restored image (940, 1040, 1140) is determined as the output image. Electronic device (101).

8. In any one of paragraphs 1 through 7, When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) is enabled, Determining the similarity between the above-mentioned recovery image (940, 1040, 1140) and the above-mentioned intermediate image (720, 920, 720, 1120, 1220), and if the similarity exceeds a threshold value, determining the above-mentioned intermediate image (720, 920, 720, 1120, 1220) as the output image, Electronic device (101).

9. In any one of paragraphs 1 through 8, When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) is enabled, If the above similarity does not exceed a threshold value, the recovery image (940, 1040, 1140) is regenerated, Electronic device (101).

10. In any one of paragraphs 1 through 9, When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) is enabled, If it is determined that the first harmful information (210, 310, 600, 710) is not included in the first text, the input image (200, 300, 700, 1415, 1425, 1435) is determined to be the intermediate image (720, 920, 720, 1120, 1220). Electronic device (101).

11. In any one of paragraphs 1 through 10, When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) is enabled, If it is determined that the second harmful information (600, 950, 1050, 1150, 1250) is not included in the second text, the intermediate image (720, 920, 720, 1120, 1220) is determined as the output image. Electronic device (101).

12. In the method of operating the electronic device (101), An operation of receiving an input image (200, 300, 700, 1415, 1425, 1435) for removing at least one of first harmful information (210, 310, 600, 710) identified by text and second harmful information (600, 950, 1050, 1150, 1250) not identified by text; An operation to determine whether the first harmful information (210, 310, 600, 710) is included in the first text included in the input images (200, 300, 700, 1415, 1425, 1435); An operation of determining an intermediate image (720, 920, 720, 1120, 1220) from the input image (200, 300, 700, 1415, 1425, 1435) based on whether the first harmful information (210, 310, 600, 710) is included in the first text; An operation to generate a second text based on the above intermediate images (720, 920, 720, 1120, 1220); An operation to determine whether the second harmful information (600, 950, 1050, 1150, 1250) is included in the second text; and Operation of determining an output image from the intermediate image (720, 920, 720, 1120, 1220) based on whether the second harmful information (600, 950, 1050, 1150, 1250) is included in the second text. including, Method of operation.

13. In Paragraph 12, The above first harmful information (210, 310, 600, 710) and the above second harmful information (600, 950, 1050, 1150, 1250) are, including systemic harmful information that threatens security or causes malfunction of the system of the electronic device (101) and unethical harmful information identified as unethical, Method of operation.

14. In either Paragraph 12 or Paragraph 13, The operation of determining whether the above-mentioned first harmful information (210, 310, 600, 710) is included is, The operation of extracting the first text from the above input images (200, 300, 700, 1415, 1425, 1435); and An operation to determine whether the first harmful information (210, 310, 600, 710) is included in the first text. Includes, The operation of determining an intermediate image (720, 920, 720, 1120, 1220) from the above input images (200, 300, 700, 1415, 1425, 1435) is. If the first text contains the first harmful information (210, 310, 600, 710), an operation to remove an area containing the first harmful information (210, 310, 600, 710) from the input image (200, 300, 700, 1415, 1425, 1435); An operation to regenerate the removed region based on the above input images (200, 300, 700, 1415, 1425, 1435); and Operation of determining the input image (200, 300, 700, 1415, 1425, 1435) including the regenerated area as the intermediate image (720, 920, 720, 1120, 1220). including, Method of operation.

15. In any one of paragraphs 12 through 14, The operation of generating a second text based on the above intermediate images (720, 920, 720, 1120, 1220) is, The operation of inputting the above intermediate images (720, 920, 720, 1120, 1220) into a third artificial intelligence model that describes images; and The operation of obtaining the second text describing the intermediate images (720, 920, 720, 1120, 1220) from the third artificial intelligence model. including, Method of operation.

Images