Electronic device protecting personal information when sharing contents to external electronic devices and operation method thereof

Abstract

Disclosed are an electronic device which protects personal information when sharing content to external electronic devices, and an operation method thereof. The electronic device may comprise: at least one processor including a processing circuit; and a memory for storing instructions. When the at least one processor individually or collectively executes the instructions, the instructions may instruct the electronic device to: acquire a connection request with an external electronic device for sharing content; identify a first image of the content for sharing to the external electronic device; determine a first area and a second area in the first image on the basis of a security level for sharing the content; generate a second image on the basis of the second area; and transmit, to the external electronic device, a third image including the second area of the first image and at least a part of the second image.

Description

Electronic devices and their operating methods for protecting personal information when sharing content with external electronic devices.

[0001] An electronic device and a method of operating the same are disclosed for protecting personal information when sharing content to an external electronic device.

[0002] Advances in wired and wireless communication networks have enabled interconnected electronic devices capable of displaying content for users. For example, electronic devices can share content with external devices through mirroring or casting technologies. With the advancement of technology, not only traditional personal information (e.g., resident registration number, address, etc.) but also search history, favorites, and recommended posts are now recognized as personal information. Therefore, when sharing content, protection of search history, favorites, and other information, in addition to traditional personal information, may be necessary.

[0003] The background technology described above is something that was possessed or acquired during the process of deriving the present disclosure, and cannot necessarily be said to be a publicly known technology disclosed to the general public prior to the filing of the present disclosure.

[0004] According to one embodiment, an electronic device may include at least one processor including a processing circuit. The electronic device may include a memory storing instructions. When the at least one processor individually or collectively executes the instructions, the instructions may cause the electronic device to obtain a connection request with an external electronic device for sharing content. When the at least one processor individually or collectively executes the instructions, the instructions may cause the electronic device to identify a first image of the content for sharing with the external electronic device. When the at least one processor individually or collectively executes the instructions, the instructions may cause the electronic device to determine a first region and a second region in the first image based on a security level for sharing the content. When the at least one processor individually or collectively executes the commands, the commands may cause the electronic device to generate a second image to replace the first area based on the second area. When the at least one processor individually or collectively executes the commands, the commands may cause the electronic device to transmit a third image including the second area of ​​the first image and the second image to the external electronic device.

[0005] According to one embodiment, an external electronic device may include at least one processor including a processing circuit. The external electronic device may include a memory for storing instructions. When the at least one processor individually or collectively executes the instructions, the instructions may cause the external electronic device to receive a first image and a security level for the content from an electronic device that provides content sharing. When the at least one processor individually or collectively executes the instructions, the instructions may cause the external electronic device to determine a first region and a second region in the first image based on the security level. When the at least one processor individually or collectively executes the instructions, the instructions may cause the external electronic device to generate a second image to replace the first region based on the second region. When the at least one processor individually or collectively executes the instructions, the instructions may cause the external electronic device to display a third image including the second region of the first image and the second image on a display module.

[0006] According to one embodiment, a method of operating an electronic device may include an operation of obtaining a connection request with an external electronic device for sharing content. The method may include an operation of identifying a first image of the content for sharing with the external electronic device. The method may include an operation of determining a first region and a second region in the first image based on a security level for sharing the content. The method may include an operation of generating a second image to replace the first region based on the second region. The method may include an operation of transmitting a third image including the second region of the first image and the second image to the external electronic device.

[0007] According to one embodiment, a non-transitory computer-readable recording medium can store computer programs including commands for executing the operating method of the electronic device described above.

[0008] According to one embodiment, a method of operating an external electronic device may include receiving a first image and a security level for content from an electronic device that provides sharing of the content. The method may include determining a first region and a second region in the first image based on the security level. The method may include generating a second image to replace the first region based on the second region. The method may include displaying a third image including the second region of the first image and the second image on a display module.

[0009] According to one embodiment, a non-transitory computer-readable recording medium can store computer programs including commands for executing the method of operating the external electronic device described above.

[0010] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0011] FIG. 2 is a diagram for explaining communication between an electronic device and an external electronic device according to one embodiment of the present disclosure.

[0012] FIG. 3 is a flowchart for explaining an operating method of an electronic device according to one embodiment of the present disclosure.

[0013] FIG. 4 is a flowchart illustrating a method for determining whether personal information is personal information and protecting it according to one embodiment of the present disclosure.

[0014] FIG. 5 and FIG. 6 are diagrams for explaining a method for obtaining a sharing purpose among attribute information of an external electronic device according to one embodiment of the present disclosure.

[0015] FIG. 7 is a flowchart illustrating a method for protecting personal information based on a security level when protection of information is required according to one embodiment of the present disclosure.

[0016] FIG. 8 is a diagram for explaining display restrictions on personal information according to a security level according to one embodiment of the present disclosure.

[0017] FIG. 9 is a flowchart for explaining an operation method of an electronic device according to one embodiment of the present disclosure.

[0018] FIGS. 10 to 15 are drawings for explaining content display of an external electronic device according to one embodiment of the present disclosure.

[0019] FIG. 16 and FIG. 17 are diagrams for explaining dynamic control of a security level according to one embodiment of the present disclosure.

[0020] FIG. 18 is a diagram illustrating protection for a plurality of pieces of information having different sensitivities according to one embodiment of the present disclosure.

[0021] FIG. 19 is a drawing for explaining a system according to one embodiment of the present disclosure.

[0022] FIG. 20 is a diagram illustrating content sharing according to one embodiment of the present disclosure.

[0023] FIG. 21 is a diagram for explaining content transmission according to one embodiment of the present disclosure.

[0024] FIG. 22 is a flowchart for explaining an operation method of an external electronic device according to one embodiment of the present disclosure.

[0025] FIG. 23 is a block diagram of a generative artificial intelligence system according to various embodiments.

[0026] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are assigned the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted.

[0027] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0028] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134).

[0029] According to one embodiment, the processor (120) may be implemented as a circuit (e.g., a processing circuit) such as a system on chip (SoC) or an integrated circuit (IC). The processor (120) may include one or more processors. For example, the processor (120) may include a combination of one or more processors such as a CPU, a GPU, an MPU (micro processing unit), an AP, and a CP.

[0030] According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0031] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, in the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include a plurality of artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0032] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto.

[0033] According to one embodiment, the memory (130) may include one or more memories. The instructions stored in the memory (130) may be stored in one memory. The instructions stored in the memory (130) may be divided and stored in multiple memories. The instructions stored in the memory (130) may be individually or collectively executed by at least one processor (120) to cause the electronic device (101) to perform and / or control the operations described with reference to FIGS. 2 to 23. The instructions stored in the memory (130) may be individually or collectively executed by at least one processor to cause the electronic device (101) to perform and / or control the operations described with reference to FIGS. 2 to 23. According to one embodiment, the memory (130) may include a volatile memory (132) or a nonvolatile memory (134).

[0034] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0035] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0036] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0037] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0038] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0039] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0040] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0041] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0042] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0043] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0044] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0045] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0046] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0047] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0048] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0049] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0050] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0051] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0052] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0053] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0054] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0055] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0056] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smart phones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0057] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0058] FIG. 2 is a diagram for explaining communication between an electronic device and an external electronic device according to one embodiment of the present disclosure.

[0059] Referring to FIG. 2, an electronic device (201) (e.g., electronic device (101) of FIG. 1), an external electronic device (202) (e.g., electronic device (102) of FIG. 2), and a server (208) (e.g., server (108) of FIG. 1) are illustrated.

[0060] An electronic device (201) can obtain a connection request from a user to an external electronic device (202). The electronic device (201) can obtain a connection request to display content on the external electronic device (202).

[0061] Sharing content as described herein may refer to sharing an image of the content. The external electronic device (202) may function as an external display that displays the shared content. Accordingly, the external electronic device (202) may receive an image of the content, rather than the content itself. For example, when an electronic device (201) shares a video for display on the external electronic device (202), the external electronic device (202) may receive a frame image included in the video file, rather than the video file itself. For example, when an electronic device (201) shares a document with the external electronic device (202), the external electronic device (202) may receive an image of the document, rather than the document file itself.

[0062]

[0063] According to one embodiment, the electronic device (201) may be connected to an external electronic device (202) via a wire based on a connection request. For example, the electronic device (201) may communicate with the external electronic device (202) via a connection terminal (e.g., connection terminal (178) of FIG. 1). The electronic device (201) may be connected wirelessly with the external electronic device (202) based on the connection request. For example, the electronic device (201) may communicate with the external electronic device (202) via a short-range wireless communication network (e.g., first network (198) of FIG. 1).

[0064] In one embodiment, the electronic device (201) may be connected to an external electronic device (202) and a server (208) based on a connection request. For example, the electronic device (201) may communicate with the external electronic device (202) via a long-range wireless communication network (e.g., the second network (199) of FIG. 1).

[0065] According to one embodiment, an electronic device (201) that has established a communication connection with an external electronic device (202) can share an image of content with the external electronic device (202). For example, the electronic device (210) can share an image of content with the external electronic device (202) using a mirroring technology or a casting technology. However, this is merely an example, and the present disclosure is not limited thereto, and it will be apparent to those skilled in the art that the method of the present disclosure can be applied to various situations in which content is shared. For example, it will be apparent to those skilled in the art that the method of the present disclosure can be applied even when transmitting an image (e.g., a photograph or a captured screen, etc.) stored in the electronic device (201) to the external electronic device (202).

[0066] According to one embodiment, the content may include all forms of data and information that the electronic device (201) can share with an external electronic device (202). For example, the content may include files such as images, videos, and documents stored in the electronic device (201). For example, the content may include applications stored in the electronic device (201). For example, the content may include data and information that is not stored in the electronic device (201) but can be received from an external source (e.g., a server (208)). For example, the content may include a video received from a server via a video streaming application. For example, the content may include a screen being displayed on a display module of the electronic device (201) (e.g., the display module (160) of FIG. 1).

[0067] According to one embodiment, content shared with an external electronic device (202) may include information that the user does not wish to share. Information that the user does not wish to share may be referred to as personal information. Personal information may include both traditional personal information (e.g., resident registration number, address, account number, facial image, etc.) and a broader range of personal information (e.g., messenger notifications, video recommendation lists, favorites lists, search history, SNS follow lists, etc.). Traditional personal information may be information that identifies the user.

[0068] In one embodiment, there may be situations where it is acceptable to share personal information when sharing content, and situations where it should not be shared. For example, if the external electronic device (202) is located in a conference room, personal information may be displayed on the external electronic device (202) during screen sharing for a meeting. In this case, the personal information may not be shared to protect privacy. On the other hand, if the external electronic device (202) is located in the user's personal room and is being used solely by the user, it may not matter if personal information is displayed on the external electronic device (202).

[0069] Additionally, individual differences may exist in how they perceive certain personal information as sensitive. For example, User A may perceive a video recommendation algorithm as sensitive personal information, while User B may not.

[0070] Therefore, it may be necessary to assess personal information and make decisions about whether to share it flexibly based on the context of the content being shared.

[0071] Below, we will explain how to protect personal information when an electronic device (201) shares content with an external electronic device (202).

[0072] FIG. 3 is a flowchart for explaining an operating method of an electronic device according to one embodiment of the present disclosure.

[0073] In the following embodiments, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. In addition, some operations may be omitted in some cases. Operations (310) to (370) may be performed by at least one component (e.g., the processor (120) of FIG. 1) of an electronic device (e.g., the electronic device (101) of FIG. 1 and the electronic device (210) of FIG. 2). For example, instructions stored in a memory (e.g., the memory (130) of FIG. 1) may be executed by at least one processor including processing circuitry, and the instructions may cause the electronic device to perform the following operations.

[0074] In operation (310), the electronic device may display content. In one embodiment, the electronic device may display content after being connected to an external electronic device (e.g., the electronic device (102) of FIG. 1 and the external electronic device (202) of FIG. 2) upon obtaining a connection request with the external electronic device.

[0075] According to one embodiment, the content may include various types of information and / or data that can be shared with an external electronic device, such as applications, programs, images, videos, documents, and screens displayed on the electronic device, stored in the electronic device. According to one embodiment, the electronic device may display images of the content on a display module (e.g., the display module (160) of FIG. 1 ). For example, the electronic device may display images of images, videos, and documents stored in the electronic device on the display module.

[0076] In one embodiment, the electronic device may display content received from an external electronic device or server (e.g., server (108) of FIG. 1 and server (208) of FIG. 2). Depending on the embodiment, operation (310) may be omitted. For example, when sharing content to an external electronic device via a casting technique, operation (310) may be omitted.

[0077] In operation (320), the electronic device can obtain a connection request with an external electronic device.

[0078] According to one embodiment, an electronic device may obtain a connection request from a user to an external electronic device for content sharing. For example, the connection request may include a user input for screen mirroring or screen casting via a wireless connection with the external electronic device. For example, the connection request may include an input for an electronic device, such as a smartphone or tablet PC, to provide the user with an experience using another electronic device (e.g., a desktop, laptop, etc.). For example, the connection request may include a user input for screen duplication and / or screen extension via a cable connection with the external electronic device. However, this is merely an example and the present disclosure is not limited thereto. According to one embodiment, the content shared by the electronic device to the external electronic device may be different from the content being displayed in operation (310).

[0079] According to one embodiment, the electronic device may obtain a connection request from a user through an input module (e.g., the input module (150) of FIG. 1). For example, the electronic device may obtain a connection request through a user's touch input, voice input, or the like.

[0080] According to one embodiment, the electronic device may identify an external electronic device based on a connection request and attempt to connect to the identified external electronic device. According to one embodiment, if the electronic device is capable of connecting to multiple external electronic devices, the electronic device may identify one of the multiple external electronic devices based on the connection request and attempt to connect to the identified external electronic device.

[0081] In one embodiment, when a user selects content to share, the electronic device can receive information about the content and the external electronic device with which the user is attempting to share. The content may include the electronic device's screen in the case of screen sharing. The content may also include applications, programs, or specific content, such as videos or images, selected by the user.

[0082] According to one embodiment, when a user receives information about an external electronic device that the user is attempting to share, the electronic device may check the connection history for the external electronic device. If a connection history exists, the electronic device may retrieve existing connection information for the external electronic device. The existing connection information may include attribute information of the external electronic device. The attribute information may include at least one of the purpose of connection between the electronic device and the external electronic device and information related to the external electronic device. The information related to the external electronic device may include at least one of the ID of the external electronic device, type information of the external electronic device, location information of the external electronic device, network information of the external electronic device, account information of the external electronic device, and previous security level information of the external electronic device. If a connection history for the external electronic device does not exist, the electronic device may register connection information (e.g., attribute information) for the external electronic device.

[0083] According to one embodiment, the electronic device may automatically attempt to connect to an external electronic device with the strongest connection strength among a plurality of external electronic devices. For example, the electronic device may automatically attempt to connect to one of the plurality of external electronic devices based on at least one of signal strength information, distance information, and attribute information.

[0084] In one embodiment, the electronic device may identify a connection request for a cable connection with an external electronic device as a connection request for content sharing. For example, if content sharing occurs automatically through a cable connection, the electronic device may identify the cable connection as a connection request.

[0085] In operation (330), the electronic device can establish a communication connection with an external electronic device.

[0086] According to one embodiment, an electronic device can establish a communication connection with an external electronic device in response to a connection request. The communication connection with the external electronic device can be achieved via at least one of a wired and a wireless method.

[0087] In operation (340), the electronic device can obtain attribute information of an external electronic device.

[0088] The method for determining the attribute information of an external electronic device will be described later in FIGS. 5 and 6.

[0089] In operation (350), the electronic device may generate a prompt based on property information of the external electronic device and / or a protection pattern of personal information.

[0090] In one embodiment, an electronic device can identify a personal information protection pattern. The personal information protection pattern may indicate whether the user perceives certain information as sensitive. The method for determining the personal information protection pattern is described below in Figure 4.

[0091] According to one embodiment, the electronic device can determine the security level based on the attribute information and / or the protection pattern of personal information of an external electronic device. The method for determining the security level is described below in FIG. 7.

[0092] In one embodiment, an electronic device may generate a prompt based on a security level. The prompt may include a command that causes a generative AI model to perform a specific action. The prompt may include a command that causes the generative AI model to remove personal information from an image of shared content based on the security level. For example, if the shared security level is high, the electronic device may determine that information (i.e., personal information) above a certain sensitivity level corresponding to the security level should be removed. The electronic device may generate a prompt including a command that causes the information above the certain sensitivity level to be removed. For example, if the shared security level is low, the electronic device may determine that information above a certain sensitivity level corresponding to the security level should be removed. The electronic device may generate a prompt including a command that causes the information above the certain sensitivity level to be removed.

[0093] In one embodiment, the prompt may include instructions to cause the generative AI model to regenerate an image of a region from which personal information has been removed. The prompt may cause the image to be regenerated based on at least a portion of an area adjacent to the region from which personal information has been removed, where personal information has not been removed. The area from which personal information has not been removed may be an area that does not require protection. The prompt may include instructions to generate an image of the region from which personal information has been removed based on information about the surrounding area of ​​the region from which personal information has been removed and the size of the region from which personal information has been removed. For example, the prompt may include instructions to regenerate the image so that at least a portion of the image of the region from which personal information has been removed blends seamlessly with the area from which personal information has not been removed, even if the image is placed within the region from which personal information has been removed.

[0094] For example, the command may include instructions to create an image and use the created image to replace the removed personal information. For example, the command to remove personal information may include instructions to replace the personal information with another image within the content. However, this is merely an example and the present disclosure is not limited thereto.

[0095] In action (360), the electronic device can obtain an image generated based on the prompt.

[0096] According to one embodiment, an electronic device may input at least one of a first image, personal information included in the first image, and a prompt into a generative AI model. The personal information included in the first image may be personal information detected by a personal information detection module described later in FIG. 19. The generative AI model may generate an image based on the input values ​​received. The image generated by the generative AI model may be an image from which personal information has been removed based on a security level, or an image generated based on a portion of the image that is not related to personal information.

[0097] In one embodiment, an electronic device may transmit a prompt to a server (e.g., server (108) of FIG. 1 and server (208) of FIG. 2). The server may input the received prompt into a generative artificial intelligence model. The generative artificial intelligence model may generate an image based on the prompt. The image generated by the generative artificial intelligence model may be an image from which personal information has been removed based on a security level. The server may transmit the image generated by the generative artificial intelligence model to the electronic device.

[0098] In one embodiment, further operations may be performed to determine areas containing personal information in an image of the content based on a prompt. For example, the electronic device may determine areas containing personal information using an AI model trained to determine areas containing personal information in an image of the content based on a prompt. For example, the electronic device may determine areas containing personal information in an image of the content based on a user's selection command. For example, the electronic device may determine areas containing personal information through a separate module.

[0099] In operation (370), the electronic device can transmit an image generated based on the prompt to an external electronic device.

[0100] In one embodiment, the external electronic device can obtain an image generated based on the prompt. The external electronic device can display the image generated based on the prompt on a display module of the external electronic device.

[0101] For example, an external electronic device may act as an external display for the electronic device.

[0102] Below, we will explain how electronic devices determine privacy protection patterns based on the sharing history of personal information.

[0103] FIG. 4 is a flowchart illustrating a method for determining whether personal information is personal information and protecting it according to one embodiment of the present disclosure.

[0104] According to one embodiment, the electronic device can determine whether or not the information is personal information based on the flowchart of FIG. 4. The operations described below may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. Furthermore, some operations may be omitted depending on the embodiment. Operations (410) to (470) may be performed by at least one component of the electronic device (e.g., the processor (120) of FIG. 1 ). For example, instructions stored in a memory (e.g., the memory (130) of FIG. 1 ) may be executed by at least one processor including a processing circuit, and the instructions may cause the electronic device to perform the following operations.

[0105] In operation (410), an electronic device (e.g., electronic device (101) of FIG. 1 and electronic device (201) of FIG. 2) can determine whether information included in the shared content has been previously shared.

[0106] According to one embodiment, an electronic device can check the sharing history of information contained in content. Based on the sharing history, the electronic device can determine whether the user has protected the information contained in the content when sharing the content. If there is a history of protection, the electronic device can perform operation (420). If there is no history of protection, the electronic device can perform operation (430).

[0107] In one embodiment, an electronic device can determine whether information contained in content (e.g., message notifications, algorithmically-based SNS recommended posts, resident registration numbers, account numbers, addresses, names, algorithmically-based video recommendation lists, user facial images, mailbox lists, etc.) has been previously shared based on sharing history. For example, the electronic device can determine whether information contained in the content has been shared beyond a threshold. For example, the electronic device can determine whether information contained in the content has been shared sufficiently to generate training data for training an artificial intelligence model that determines whether information is personal information.

[0108] In one embodiment, the electronic device may perform operation (420) if the information included in the content has not been previously shared. The electronic device may perform operation (430) if the information included in the content has been previously shared. For example, if the information included in the content is a video recommendation list and the video recommendation algorithm has not been shared based on the sharing history, the electronic device may perform operation (420). For example, if the information included in the content is a mailbox list and has been shared based on the sharing history, the electronic device may perform operation (430).

[0109] In one embodiment, the electronic device may perform operation (430) or operation (420) based on whether information contained in the content has been shared a threshold number of times. The threshold number may be the minimum number of times required to generate training data necessary to train an initial AI model that has learned a first condition into a personalized AI model that has learned a second condition.

[0110] In operation (420), the electronic device can determine whether information contained in the content constitutes personal information according to a first condition. If the information contained in the content does not constitute personal information according to the first condition, the electronic device can perform operation (440). If the information contained in the content constitutes personal information according to the first condition, the electronic device can perform operation (450).

[0111] According to one embodiment, the first condition may include attribute information of an external electronic device (electronic device 102 of FIG. 1 and / or external electronic device 202 of FIG. 2), user settings, and judgment criteria such as whether the personal information is traditionally personal information. However, this is merely an example and the present disclosure is not limited thereto. For example, if the external electronic device is a display device in a conference room, the electronic device may determine at least one piece of information included in the content as personal information based on attribute information of the external electronic device. For example, the electronic device may determine at least one piece of information included in the content as personal information based on the user's settings, who has determined that the user's name is personal information that must always be protected. For example, the electronic device may determine a resident registration number, which is traditionally personal information, as personal information.

[0112] In one embodiment, an electronic device may use a non-personalized initial AI model to determine whether information contained in content constitutes personal information. The initial AI model may be an AI model that has not been fine-tuned based on sharing history. The initial AI model may be a model that has learned the first condition. The initial AI model may be fine-tuned based on sharing history and sharing release commands and / or sharing enable commands acquired in operation (460) to learn the second condition.

[0113] In operation (430), the electronic device may determine whether information contained in the content constitutes personal information based on a second condition. The second condition may include a condition for determining whether specific information constitutes personal information under certain conditions (e.g., sharing situation, sharing target, etc.). The second condition may include a method for determining the user's protection pattern and security level for the information.

[0114] According to one embodiment, an electronic device can use an AI model that has learned the second condition to determine whether information contained in content constitutes personal information. The AI ​​model that has learned the second condition may be a personalized AI model. The AI ​​model may be a model trained to determine a protection pattern and security level for information, and to determine whether information contained in the content constitutes personal information based on sensitivity and security levels. Upon receiving content as input, the AI ​​model can determine whether the information contained in the content constitutes personal information. Below, the method by which the AI ​​model learns the protection pattern will be described. The method for determining the security level and determining the information contained in the content as personal information will be described later in FIG. 7.

[0115] In one embodiment, the AI ​​model can learn a user's protection pattern based on various information, such as attribute information of an external electronic device, the user's sharing setting commands (e.g., a protection release command) and / or sharing release commands (e.g., a protection release command), the frequency of protection, and whether the information is traditionally personal information. The protection pattern can represent a user's pattern of sharing (or protecting) specific information. For example, if a user has no history of protecting an address, which is traditionally personal information, the AI ​​model can learn a protection pattern indicating that the user rarely protects the address.

[0116] According to one embodiment, an AI model that has learned a protection pattern can determine the sensitivity of information contained in content based on the user's protection pattern. The sensitivity of information may be information about the degree to which the user perceives the need for protection for personal information. Sensitivity may be categorized as "high," "medium," "low," and "none." However, this is merely an example and the present disclosure is not limited thereto. For example, the sensitivity of personal information may be categorized into fewer or more than four groups (e.g., levels 1 to 5). However, for convenience of explanation, it is assumed below that the sensitivity is categorized as "high," "medium," "low," and "none."

[0117] For example, if the frequency of protecting a video recommendation list is high regardless of the connected external electronic device, the AI ​​model can be trained to determine the sensitivity of the video recommendation list as "high." For example, if the frequency of hiding a video recommendation list is 60% or less, the AI ​​model can determine the sensitivity of the video recommendation list as "medium." For example, if a user repeatedly removes protection for a traditional personal information, such as an account number, the AI ​​model can be trained to determine the sensitivity of the account number as "low." However, this is merely an example, and the electronic device can train the AI ​​model based on various information, such as the attribute information of the external electronic device, the user's sharing setting commands (e.g., a protection setting command) and / or sharing removal commands (e.g., a protection removal command), the frequency of protection, and whether the information is traditional personal information.

[0118] In one embodiment, the AI ​​model can generate and store a sensitivity determined based on a protection pattern learned for specific information contained in the content, as shown in Table 1. If the content includes pre-stored information, the AI ​​model can load and utilize the stored information without having to use AI to determine the sensitivity.

[0119] Sensitivity information: Messenger content, facial photos, etc., video recommendation algorithm, search history, address, account number, etc., mailbox list, etc.

[0120] According to one embodiment, the electronic device may perform operation (440) if the information included in the content does not correspond to personal information according to the second condition. The electronic device may perform operation (450) if the information included in the content corresponds to personal information according to the second condition. In operation (440), the electronic device may share the information included in the content with an external electronic device. The electronic device may share the information included in the content with the external electronic device without protecting it.

[0121] In operation (450), the electronic device may not share information included in the content with an external electronic device. The electronic device may protect information included in the content and not share the information with the external electronic device.

[0122] In operation (460), the electronic device may receive a command to unshare and / or a command to set up sharing for information being shared and / or personal information not being shared. If the command to unshare and / or the command to set up sharing is not received, the electronic device may terminate the operation. If the command to unshare and / or the command to set up sharing is received, the electronic device may perform operation (470).

[0123] In action (470), the electronic device can update the second condition.

[0124] According to one embodiment, an electronic device may obtain a command to unshare and / or a command to set up sharing from a user while sharing content. The unshare command may be a command to determine that information being shared with an external electronic device is personal information and to stop sharing with the external electronic device. The command to set up sharing may be a command to disable protection for personal information that is determined to be personal information and is not shared with the external electronic device, thereby allowing sharing with the external electronic device.

[0125] For example, an electronic device may obtain a sharing setting command for message notifications, which are information contained in the content, while sharing the content with an external electronic device. For example, an electronic device may obtain a sharing undo command for a video recommendation list, which is information contained in the content, while sharing the content with an external electronic device.

[0126] In one embodiment, the unshare command may include a command to prevent personal information from being displayed on an external electronic device, such as a block command, a blur command, or the like.

[0127] In one embodiment, an electronic device can manage sharing history, including unsharing commands and / or sharing settings obtained from a user when sharing content. The electronic device can update an AI model based on the sharing history. The AI ​​model can determine whether information contained in the content is personal information based on the updated second condition.

[0128] In one embodiment, the trained AI model learns a user-tailored protection pattern, so that the sensitivity of information can be determined differently for each user. For example, if User A recognizes a facial image as sensitive information and protects the facial image whenever sharing the image, the AI ​​model can be trained to classify the facial image as "high" in sensitivity for User A. For example, if User B does not recognize a facial image as sensitive personal information and does not protect the facial image whenever sharing the image, the AI ​​model can be trained to classify the facial image as "none" in sensitivity for User B. The following describes the attribute information of an external electronic device.

[0129] FIG. 5 and FIG. 6 are diagrams for explaining a method for obtaining a sharing purpose among attribute information of an external electronic device according to one embodiment of the present disclosure.

[0130] FIG. 5 is a diagram for explaining a method for obtaining a sharing purpose among attribute information of an external electronic device when there is no connection history (e.g., sharing history) with the external electronic device.

[0131] According to one embodiment, an electronic device (501) (e.g., electronic device (101) of FIG. 1 and electronic device (201) of FIG. 2) can obtain attribute information of an external electronic device (e.g., electronic device (102) of FIG. 1 and external electronic device (202) of FIG. 2). The attribute information can be used to determine a security level, as described below in FIG. 7.

[0132] According to one embodiment, the attribute information of the external electronic device may include the purpose of the connection between the electronic device (510) and the external electronic device. For example, the purpose of the connection may include information about personal use, use with people in a meeting, or use with people belonging to a limited group.

[0133] According to one embodiment, the attribute information of the external electronic device may include at least one of an ID of the external electronic device, type information of the external electronic device, location information of the external electronic device, network information of the external electronic device, connection purpose, account information of the external electronic device, and previous security level information.

[0134] According to one embodiment, the ID of the external electronic device may include information that can identify the external electronic device, such as the serial number of the external electronic device and the nickname of the external electronic device. The ID of the external electronic device may be received from the external electronic device or input by the user when registering the attribute information of the external electronic device.

[0135] According to one embodiment, the type information of the external electronic device may include information regarding the type of the external electronic device. For example, the type information of the external electronic device may include information regarding whether the external electronic device is a computing device such as a smart phone, tablet, e-book device, laptop, personal computer, desktop, workstation, or server; a wearable device such as smart glasses or a head-mounted display (HMD); or a display device such as a smart TV or monitor. The type information of the external electronic device may be received from the external electronic device or input by the user when registering attribute information of the external electronic device.

[0136] According to one embodiment, the location information of an external electronic device may include GPS information of the external electronic device. If the external electronic device is connected to the Internet, the location information of the external electronic device may include an IP address. The location information of the external electronic device may be received from the external electronic device or input by the user when registering the attribute information of the external electronic device.

[0137] According to one embodiment, the network information of an external electronic device may include information about the network to which the external electronic device is connected. The network information may include information such as whether the external electronic device is connected via wired and / or wireless connections, its IP address, and the strength of the network connection. The network information of the external electronic device may be received from the external electronic device or input by the user when registering the attribute information of the external electronic device.

[0138] According to one embodiment, the account information of the external electronic device may include information about an account logged into the external electronic device. The account information of the external electronic device may be received from the external electronic device or input by the user when registering attribute information of the external electronic device.

[0139] According to one embodiment, the previous security level information may include information about the security level when the electronic device (501) was previously connected to the external electronic device, if there is a connection history to the external electronic device.

[0140] According to one embodiment, the electronic device (501) may acquire and store attribute information of an external electronic device. When a connection request with an external electronic device is received, the electronic device (501) may check previous settings. When a connection request with an external electronic device is received from a user, the electronic device (501) may determine whether attribute information of the external electronic device exists.

[0141] According to one embodiment, the case where attribute information of an external electronic device does not exist may include the case where attribute information is deleted or the case where the external electronic device is connected for the first time.

[0142] In one embodiment, when a connection request for an external electronic device is received, the electronic device (501) may determine whether attribute information of the external electronic device exists. If attribute information of the external electronic device does not exist, the electronic device (501) may ask the user about the purpose of sharing. The electronic device (501) may determine attribute information of the external electronic device based on the user input in response to the question.

[0143] For example, the electronic device (501) may display a question, "Are you connecting to an external electronic device for work purposes?" The electronic device (501) may obtain a user input for the question, "YES" or "NO." If "YES" is entered, the electronic device (501) may determine the sharing purpose of the external electronic device based on the user input. For example, if the user selects to connect for work purposes (e.g., selecting "YES"), the electronic device (501) may determine at least some of the attribute information (e.g., the sharing purpose) of the external electronic device to be for work purposes. The electronic device (501) may receive the remaining attribute information from the external electronic device or input it from the user and store it together with the sharing purpose. The stored attribute information may be used to determine the security level. If "NO" is entered, the electronic device (501) may display a question, "For what purpose are you connecting to the external electronic device?" The electronic device (501) may obtain a user input for the question. Upon obtaining user input, the electronic device (501) may display the question, "Do you use an external electronic device for personal purposes?" The electronic device (501) may receive user input for the question as "YES" or "NO." Based on the received user input, the electronic device (501) may determine the sharing purpose as personal. The electronic device (501) may receive the remaining attribute information from the external electronic device or input it from the user and store it along with the sharing purpose. However, the above-described questions are merely examples, and the present disclosure is not limited thereto.

[0144] In one embodiment, the electronic device (501) may use an artificial intelligence model to answer a question asking about the purpose of a connection.

[0145] In one embodiment, the electronic device (501) may determine the connection purpose based on default settings. For example, if the attribute information of the external electronic device does not exist, the electronic device (501) may determine the connection purpose based on default settings without asking a question regarding the connection purpose. For example, if the attribute information of the external electronic device does not exist, the electronic device (501) may determine the connection as a meeting purpose based on the attribute information of the external electronic device according to the default settings. The electronic device (501) may receive a modification command from the user and update the connection purpose determined based on the default settings.

[0146] In one embodiment, the electronic device (501) may provide a question sufficient to determine the sharing purpose of the external electronic device, and may determine the sharing purpose of the external electronic device based on user input regarding the question. The electronic device may store the sharing purpose along with other attribute information. When a connection request is received from an external electronic device for content sharing, the electronic device may determine a security level based on the attribute information for the external electronic device.

[0147] According to one embodiment, the electronic device (501) can update attribute information of an external electronic device based on contextual information. The contextual information is information related to the connection status of the external electronic device and may include information that can be changed, such as GPS information, IP address, and nickname of the external electronic device.

[0148] FIG. 6 is a diagram for explaining a method for updating attribute information of an external electronic device according to context information when attribute information of an external electronic device exists.

[0149] According to one embodiment, an electronic device (601) (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2, and electronic device (501) of FIG. 5) may determine whether an update is required for attribute information of an external electronic device (e.g., electronic device (102) of FIG. 1 and external electronic device (202) of FIG. 2) based on contextual information. The electronic device (601) may check the contextual information. The electronic device (601) may check the contextual information according to a previous setting. The electronic device (601) may determine whether an update is required for attribute information of the external electronic device based on the contextual information.

[0150] For example, when a connection request is received, the electronic device (601) can check contextual information. For example, if the location of the external electronic device changes from work to home, the contextual information may indicate that the GPS information of the external electronic device has changed. For example, if the nickname of the external electronic device has changed, the contextual information may indicate that the nickname of the external electronic device has changed. According to one embodiment, if it is determined that at least one of the attribute information of the external electronic device has changed based on the contextual information, the electronic device can check whether the attribute information has changed. For example, the electronic device (601) may ask the user again about the sharing purpose. The electronic device (601) may update the sharing purpose of the external electronic device based on the user input in response to the question. For example, the electronic device (601) may display the question, "Has the connection purpose for the external electronic device changed to personal?" The electronic device (601) may obtain the user input in response to the question as "YES" or "NO." The electronic device (601) may update the sharing purpose based on the obtained user input.

[0151] The display of the questions described in FIGS. 5 and 6 is merely exemplary, and the present disclosure is not limited thereto. For example, it will be apparent to those skilled in the art that the questions may be provided in audio or video format.

[0152] Below, we will explain how to determine the security level based on the above-described protection pattern and attribute information of the external electronic device.

[0153] FIG. 7 is a flowchart illustrating a method for protecting personal information based on a security level when protection of information is required according to one embodiment of the present disclosure.

[0154] The operations described below may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. In addition, some operations may be omitted depending on the embodiment. Operations (700) to (750) may be performed by at least one component (e.g., the processor (120) of FIG. 1) of an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIG. 2, the electronic device (501) of FIG. 5, and the electronic device (601) of FIG. 6). For example, instructions stored in a memory (e.g., the memory (130) of FIG. 1) may be executed by at least one processor including a processing circuit, and the instructions may cause the electronic device to perform the following operations.

[0155] In operation (700), the electronic device can identify an external electronic device attempting to share content.

[0156] According to one embodiment, when the electronic device obtains a connection request from a user with an external electronic device (e.g., the electronic device (102) of FIG. 1 and the external electronic device (202) of FIG. 2), the electronic device can identify the external electronic device for the connection request.

[0157] In operation (710), the electronic device can determine whether attribute information of an external electronic device exists. The electronic device can determine whether attribute information for the external electronic device identified in operation (700) exists.

[0158] In operation (720), the electronic device can obtain attribute information of an external electronic device.

[0159] According to one embodiment, if attribute information of an external electronic device does not exist, the electronic device can obtain attribute information according to the method described above in FIGS. 5 and 6. According to one embodiment, if attribute information of an external electronic device exists, the electronic device can further determine whether the attribute information of the external electronic device has changed.

[0160] In operation (730), the electronic device can determine whether information requires protection. The electronic device can determine whether information requires protection based on the information contained in the content. For example, if the content includes a resident registration number and a list of mailboxes, the electronic device may determine that the resident registration number requires protection, but the list of mailboxes does not.

[0161] In one embodiment, the electronic device may determine whether information requires protection based on attribute information of the external electronic device. For example, the electronic device may determine that information requires protection if the external electronic device, identified based on attribute information, is not logged into the user's account.

[0162] According to one embodiment, an electronic device can determine whether information requires protection based on an AI model. The electronic device can determine whether information requires protection based on protection patterns learned by the AI ​​model. If the electronic device determines, based on the AI ​​model, that at least one piece of information contained in content exceeds a threshold sensitivity level, the electronic device can determine that protection is required. For example, if the electronic device determines, based on the AI ​​model, that at least one piece of information contained in content exceeds a sensitivity level of "low," the electronic device can determine that protection is required.

[0163] However, the method for determining whether the above-described information needs to be protected is only an example and the present disclosure is not limited thereto.

[0164] In operation (740), the electronic device can share content without protecting the information. The electronic device can share content with an external electronic device without protecting the information.

[0165] In operation (750), the electronic device can share content with protected personal information determined based on the second condition. The electronic device determines whether the information contained in the content is personal information and can share the content with protected personal information. Methods for protecting personal information will be described later in FIG. 7.

[0166] According to one embodiment, an electronic device may determine whether information contained in content is personal information using an artificial intelligence model that has learned a second condition. The second condition may include a method for determining a security level and the user's protection pattern described above in FIG. 4 . The artificial intelligence model may be trained to determine the user's protection pattern and security level. The protection pattern, as described above in FIG. 4 , will not be further described. The artificial intelligence model may be trained to determine the security level based on at least one of attribute information of an external electronic device and the user's protection pattern. The security level may refer to the degree to which information contained in the content requires protection when shared.

[0167] According to one embodiment, the AI ​​model can determine a security level based on at least one of the user's protection pattern and attribute information of an external electronic device. For example, the AI ​​model can be trained to determine the security level as "High," "Medium," "Low," and "None." However, this is merely an example and the present disclosure is not limited thereto. For example, the security level can be divided into fewer or more groups than four (e.g., Levels 1 to 5). However, for convenience of explanation, it is assumed below that the sensitivity is divided into "High," "Medium," "Low," and "None."

[0168] For example, an AI model can be trained to determine a security level of "High" based on the attribute information of the external electronic device if the purpose of the connection to the external electronic device is for a meeting. For example, the AI ​​model can be trained to determine a security level of "High" if the image of the content being shared contains personal information with a sensitivity level of "High" in excess of a threshold. For example, the AI ​​model can determine a security level of "High" based on at least one of the attribute information of the external electronic device and the sensitivity of the personal information included in the protection pattern.

[0169] For example, an AI model may be trained to determine a security level of "medium" based on attribute information of the external electronic device if the purpose of connecting to the external electronic device is to share content in the user's living room. For example, the electronic device may be trained to determine a security level of "medium" if personal information with a sensitivity of "medium" exists in an image of the content being shared in excess of a threshold percentage. For example, the AI ​​model may determine a security level of "medium" based on at least one of attribute information of the external electronic device and the sensitivity of personal information included in the protection pattern.

[0170] For example, an AI model may be trained to determine a security level of "low" based on attribute information of the external electronic device if the purpose of connecting to the external electronic device is to view content with multiple people in the user's personal room via the external electronic device. For example, the AI ​​model may be trained to determine a security level of "low" if personal information with a sensitivity of "low" exists in an image of the content to be shared in excess of a threshold percentage. For example, an electronic device may determine a security level of "low" based on at least one of attribute information of the external electronic device and the sensitivity of personal information included in the protection pattern.

[0171] For example, an AI model may be trained to determine a security level of "None" based on attribute information of the external electronic device if the purpose of connecting to the external electronic device is to share content for exclusive use on the external electronic device in the user's personal room. For example, the AI ​​model may be trained to determine a security level of "None" if personal information is not present in the image of the content being shared. For example, an electronic device may be trained to determine a security level of "None" based on at least one of attribute information of the external electronic device and the sensitivity of personal information included in the protection pattern.

[0172] However, the training of the aforementioned AI model is merely an example, and the present disclosure is not limited thereto. Therefore, it is readily apparent to those skilled in the art that AI models can be trained to determine security levels based on various criteria, methods, and other factors.

[0173] In one embodiment, the security level may be determined by the user's selection when requesting a network connection for content sharing with an external electronic device. For example, if the user selects "High" as the security level when requesting a network connection for content sharing with an external electronic device, the security level may be determined to be "High" based on the user's selection.

[0174] In one embodiment, the security level may be determined based on an application displayed on a display module of the electronic device (e.g., display module (160) of FIG. 1 ). For example, if the application being displayed is an application that allows the user to write a personal diary, the security level may be determined to be "high."

[0175] In one embodiment, the security level may be determined based on previous security level information included in the attribute information of the external electronic device. When the electronic device determines that the attribute information for the external electronic device has not changed upon connection with the external electronic device, the electronic device may determine the security level based on the previous security level information. The absence of a change in attribute information may indicate that the sharing situation has not changed. For example, if the attribute information of the external electronic device has not changed and the previous security level information is a security level of "low," the electronic device may determine the security level to be "low."

[0176] According to one embodiment, an electronic device can determine a security level based on peripheral devices identified during a screen sharing operation. The electronic device can obtain information related to access rights from the peripheral devices. The electronic device can determine the security level based on the information related to access rights obtained from the peripheral devices. For example, the electronic device can obtain information related to access rights from the electronic devices of participants in a meeting. Based on this information, the electronic device can determine whether participants in the meeting have access rights to content shared during the meeting and determine the security level.

[0177] In one embodiment, the AI ​​model can determine at least some of the information contained in the content as personal information based on the determined sensitivity and security levels for the information contained in the content. For example, the AI ​​model can be trained to determine information with a sensitivity level higher than the corresponding security level as personal information. For example, a security level of "high" may correspond to a sensitivity level of "low" because it requires a high level of privacy protection and thus requires personal information to be protected regardless of the sensitivity level. A security level of "medium" may correspond to a sensitivity level of "medium" because it requires a medium level of privacy protection. A security level of "low" may correspond to a sensitivity level of "high" because it requires a low level of privacy protection and thus requires personal information with a high need for protection to be protected.

[0178] In one embodiment, an electronic device may generate a prompt based on determined personal information (i.e., information about the judgment regarding the personal information). The prompt may include instructions for a generative AI model to remove personal information from an image of shared content and generate an image in the removed area. The prompt may also include instructions for regenerating areas containing information (e.g., personal information) with a sensitivity level higher than the security level. However, this is merely an example and the present disclosure is not limited thereto.

[0179] In one embodiment, the prompt may include instructions to cause the generative AI model to regenerate an image for a region from which personal information has been removed. The prompt may cause the image to be regenerated based on at least a portion of an area from which personal information has not been removed that is adjacent to the area from which personal information has been removed. The area from which personal information has not been removed may be an area that does not require protection. The prompt may include instructions to regenerate the image for the area from which personal information has been removed so that at least a portion of the regenerated image blends seamlessly with the area from which personal information has not been removed, even if the image is placed within the area from which personal information has been removed.

[0180] Referring to Table 2 below, an example of a prompt generated by determining the security level and the information with the corresponding sensitivity as personal information is shown.

[0181] In the image for content with a security level prompt (e.g., conference room, lecture room, etc.), remove all information with a sensitivity level of "low" or higher by determining it as personal information, and regenerate the image for the area from which the personal information was removed. In the image for content with a security level prompt (e.g., living room TV, etc.), remove all information with a sensitivity level of "medium" or higher by determining it as personal information, and regenerate the image for the area from which the personal information was removed. In the image for content with a sensitivity level of "low" or higher by determining it as personal information, and regenerate the image for the area from which the personal information was removed.

[0182] Below, we will describe the sharing results for external electronic devices. Figure 8 is a diagram illustrating the display restrictions on personal information based on security levels according to one embodiment of the present disclosure.

[0183] Referring to FIG. 8, an electronic device (801) for sharing content (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIG. 2, the electronic device (501) of FIG. 5, and the electronic device (601) of FIG. 6) is illustrated. For example, the electronic device (801) may use a video streaming application to mirror a video to an external electronic device (802) (e.g., the electronic device (102) of FIG. 1 and the external electronic device (202) of FIG. 2) or an external electronic device (803) (e.g., the electronic device (102) of FIG. 1 and the external electronic device (202) of FIG. 2).

[0184] In one embodiment, the electronic device (801) may be displaying content. For example, the electronic device may be displaying an image of the content. For example, the image of the content may include a frame image of a video and a video recommendation list (810). The sensitivity for the video recommendation list (810) will be assumed to be "low" based on the user's protection pattern learned by the artificial intelligence model.

[0185] In one embodiment, the external electronic device (802) may be a display device (e.g., a TV) placed in a conference room. When the electronic device (801) shares content with the external electronic device (802), the external electronic device (802) assumes that the security level is determined to be "high" based on the security level determination method learned by the artificial intelligence model. Accordingly, the electronic device (801) may provide the external electronic device (802) with an image from which the video recommendation list (810) has been removed. The external electronic device (802) may display the image from which the video recommendation list (810) has been removed.

[0186] In one embodiment, the external electronic device (803) may be a display device placed in the user's living room. When the electronic device (801) shares content with the external electronic device (803), the external electronic device (803) assumes that the security level is determined to be "medium" based on the security level determination method learned by the artificial intelligence model. Accordingly, the electronic device may share an image from which the video recommendation list (810) is not removed with the external electronic device (803). The external electronic device (803) may display an image including the video recommendation list (810).

[0187] Below, we will explain how to create a new image for an area that contains personal information.

[0188] FIG. 9 is a flowchart for explaining an operation method of an electronic device according to one embodiment of the present disclosure.

[0189] The operations described below may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. In addition, some operations may be omitted according to some embodiments. Operations (910) to (950) may be performed by at least one component (e.g., the processor (120) of FIG. 1) of an electronic device (e.g., the electronic device (101) of FIG. 1 , the electronic device (201) of FIG. 2 , the electronic device (501) of FIG. 5 , the electronic device (601) of FIG. 6 , and the electronic device (801) of FIG. 8 ). For example, instructions stored in a memory (e.g., the memory (130) of FIG. 1 ) may be executed by at least one processor including a processing circuit, and the instructions may cause the electronic device to perform the following operations.

[0190] In operation (910), the electronic device may obtain a connection request from a user to an external electronic device (e.g., the electronic device (102) of FIG. 1, the external electronic device (202) of FIG. 2, the external electronic device (802) of FIG. 8, and the external electronic device (803) of FIG. 8) for sharing content.

[0191] The acquisition of a connection request is omitted as described above in Fig. 3.

[0192] In operation (920), the electronic device may identify a first image for the content for sharing the content with an external electronic device.

[0193] In one embodiment, an electronic device may share an image of the content without sharing the content itself with an external electronic device. For example, the external electronic device may function as an external display for the electronic device. For example, when an electronic device shares a video stored on the electronic device with an external electronic device, only the video playback screen, not the video file itself, may be shared. For example, when an electronic device shares a screen displayed on a display module with an external electronic device, the screen currently being displayed on the display module may be shared with the external electronic device.

[0194] According to one embodiment, the first image for the content may include an execution screen of the content displayed on the display module according to the execution of the content. The content may include various types of information and / or data that can be shared with an external electronic device, such as applications, programs, images, videos, documents stored in the electronic device, and screens displayed on the electronic device. For example, when screen sharing is performed during an online conference, the first image may include an execution screen of the content displayed on the display module (e.g., the display module (160) of FIG. 1) of the electronic device of a conference participant performing screen sharing. For example, when a user is watching a movie on an electronic device with a small display module and mirrors the movie screen to an electronic device with a larger display, the first image may include an execution screen of the movie on the electronic device with the small display module.

[0195] According to one embodiment, an electronic device may identify at least a portion of an execution screen of at least one application running in the background on the electronic device as a first image. According to one embodiment, the first image for the content may include an execution screen of the content running in the background but not subsequently displayed on the display module. For example, when a user casts an application of the electronic device to another electronic device, the application may run in the background on the electronic device and the execution screen of the application may be displayed on the other electronic device. While the electronic device casts the application to the other electronic device, the user may operate the electronic device for another purpose (e.g., sending a message, browsing the Internet, etc.). For example, while the electronic device casts the application to the other electronic device, the electronic device may display a screen for another purpose on the display module. In this case, the first image may include an execution screen of the content that would have been displayed on the display module of the electronic device if the content (e.g., an application) running in the background on the electronic device had been run in the foreground.

[0196] According to one embodiment, the electronic device can identify at least a portion of a screen being displayed on a display module of the electronic device as the first image.

[0197] According to one embodiment, the first image for the content may include an image for content transmitted from the electronic device to an external electronic device (e.g., a photograph or captured image).

[0198] In one embodiment, the display module can display multiple frames per second based on a refresh rate. The first image may include frames displayed on the display module based on the refresh rate when the display module displays content. Alternatively, the first image may include frames displayed on the display module based on the refresh rate when content running in the background is displayed on the display module.

[0199] In operation (930), the electronic device can determine the first area and the second area in the first image based on the security level for sharing the content. The method for determining the security level is described above in FIG. 7, so a detailed description will be omitted.

[0200] According to one embodiment, the first region may be determined based on a condition (e.g., the second condition). The electronic device may determine the sensitivity of information included in the content and the security level for sharing the content based on an artificial intelligence model. The electronic device may determine personal information based on the sensitivity of information included in the content and the security level for sharing the content based on the artificial intelligence model. The electronic device may determine an area corresponding to personal information in the first image as the first region. The electronic device may determine information with a sensitivity level higher than or equal to the security level as personal information, and may determine an area corresponding to the personal information in the first image as the first region. For example, the first region may be an area containing personal information with a sensitivity level of "low" or higher when the security level is "high." For example, the first region may be an area containing personal information with a sensitivity level of "medium" or higher when the security level is "medium." For example, the first region may be an area containing personal information with a sensitivity level of "high" or higher when the security level is "low."

[0201] In one embodiment, the second region may be determined as an area excluding the first region in the first image.

[0202] In one embodiment, if the content being shared is an application, the first area may be determined based on at least one of the user's settings history or application property information. The application property information may include at least one of the application's category information and usage history information.

[0203] In one embodiment, if the content being shared is an application, the first area may be determined based on an artificial intelligence model trained based on at least one of the user's settings history or application property information. The artificial intelligence model may be trained based on a reinforcement learning algorithm.

[0204] In one embodiment, the first region may be determined based on a user's selection. The electronic device may determine one or more first regions in response to the user's selection of the first image when requesting connection with an external electronic device.

[0205] In operation (940), the electronic device can generate a second image based on the second area.

[0206] According to one embodiment, an electronic device may generate a second image based on a prompt. The electronic device may input the prompt into a generative artificial intelligence model to generate the second image. At least a portion of the second image may be displayed in the first area. The size of at least a portion of the second image may correspond to the size of the first area. At least a portion of the second image may be naturally connected to at least a portion of the second area adjacent to the first area. A description of the prompt will be omitted, as it has been described above with reference to FIGS. 3 and 7.

[0207] In one embodiment, the prompt may include instructions for the generative AI model to regenerate images for the first region and images for the second region. For example, images for regions that do not require protection (e.g., the second region) may also be regenerated to fit the overall composition.

[0208] In one embodiment, the second image may include a second region proximate the first region and an image regenerated based on a previously generated image. The previously generated image may include a previously generated second image to protect personal information corresponding to the first region. By utilizing the previously generated image, resources for the generative AI model that generates the second image can be conserved.

[0209] In one embodiment, the generative AI model can generate a second image based on various information (e.g., personalized information, recipient information, shared device information, information displayed on the screen, contextual information, etc.). For example, if a user shares a photo based on personalized information and the photo is then shared with multiple users by modifying the user's appearance in the photo into a caricature, the generative AI model can generate a caricature that corresponds to the user's appearance when the photo is shared. For example, if a captured image of a recommended SNS post based on recipient information is shared to a person in the contact list, the generative AI model can regenerate only the area determined to be personal information. For example, if a captured image of a recommended SNS post based on recipient information is shared to a person not in the contact list, the generative AI model can regenerate the entire image based on the guest account.

[0210] In one embodiment, the second image may include a second region proximate the first region and an image regenerated based on information stored on the electronic device. The information stored on the electronic device may include at least some of the user's account information, schedule information, location information, network information, images, videos, and text. By utilizing the information stored on the electronic device, resources for the generative AI model that generates the second image can be conserved.

[0211] In one embodiment, the second image may include a second area adjacent to the first area and an image regenerated based on information about the application. The information about the application may include at least some of the images, videos, and text stored on the electronic device in connection with the application. The information about the application may be received from a server providing application services.

[0212] In operation (950), the electronic device can transmit a third image including a second area of ​​the first image and at least a portion of the second image to an external electronic device.

[0213] According to one embodiment, the electronic device may transmit a third image, in which a first region of a first image is replaced with at least a portion of a second image, to an external electronic device. The third image may be an image in which a first region of the first image is replaced with at least a portion of a second image. For example, the third image may be an image in which personal information having a sensitivity corresponding to a security level in the first image is replaced with at least a portion of a second image regenerated based on a second region proximate the first region. For example, the third image may be an image that does not include an image corresponding to the first region of the first image. For example, the third image may be an image in which personal information having a sensitivity corresponding to a security level in the first image is removed.

[0214] According to one embodiment, the third image may be an image in which at least a portion of the second region of the first image is positioned at the center.

[0215] In one embodiment, the electronic device may transmit the first image and the second image generated for the first area to an external electronic device. The external electronic device may then generate a third image by combining the received first image and the second image.

[0216] In one embodiment, the external electronic device may display a third image on the display module of the external electronic device. The third image may be an image with personal information obscured, depending on the security level.

[0217] FIGS. 10 to 15 are drawings for explaining content display of an external electronic device according to one embodiment of the present disclosure.

[0218] Referring to FIG. 10, an electronic device (1001) (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIG. 2, the electronic device (501) of FIG. 5, the electronic device (601) of FIG. 6, and the electronic device (801) of FIG. 8) is illustrated. The electronic device (1001) can share a screen displayed on the electronic device (1001) with an external electronic device (1002) (e.g., the electronic device (102) of FIG. 1, the external electronic device (202) of FIG. 2, the external electronic device (802) of FIG. 8, and the external electronic device (803) of FIG. 8). It will be assumed that the external electronic device (1002) is a display device placed in a conference room, and that the electronic device (1001) has obtained a connection request with the external electronic device (1002) for conference purposes.

[0219] According to one embodiment, the electronic device (1001) shares the screen being displayed with an external electronic device, so that the screen being displayed on the electronic device (1001) can be identified as a first image for the content.

[0220] Referring to the first image, a message pop-up may be included. The message pop-up is assumed to be personal information with a sensitivity level of 5 out of 5 based on the artificial intelligence model. Since the electronic device (1001) and the external electronic device (1002) are connected for conference purposes, the security level is assumed to be 5 out of 5 based on the artificial intelligence model. The method for determining the sensitivity of information (e.g., the protection pattern of personal information) and the security level based on the artificial intelligence model is omitted for brevity.

[0221] In one embodiment, security level 5 may correspond to sensitivity level 1. At security level 5, all information may be determined as personal information and must be protected. The area where the message pop-up appears may be determined as the first area (1010), which contains personal information requiring protection. The area of ​​the first image, excluding the first area (1010), may be determined as the second area (1020). For example, the second area (1020) may be an area that does not require protection.

[0222] According to one embodiment, the electronic device (1001) can generate a second image. The electronic device (1001) can generate the second image based on at least a portion of the second area (1020) for natural display with the second area (1020). The second image can be an image generated based on at least a portion of the second area (1020) and can be an image that can naturally blend in with the first image. At least a portion (1040) of the second image can be included in the third image (1030).

[0223] According to one embodiment, the electronic device (1001) may generate a prompt to generate a second image based on information based on a first region (1010) of a first image. The information based on the first region (1010) may include information about an image of a second region (1020) proximate to the first region (1010), information such as the size of the first region (1010), etc. The electronic device (1001) may input the prompt into a generative artificial intelligence model to generate the second image.

[0224] In one embodiment, the electronic device (1001) may generate a prompt that causes the generative artificial intelligence model to generate a third image (1030) based on the first image. By inputting the prompt to the generative artificial intelligence model, the electronic device (1001) may generate the third image (1030) directly based on the first image without generating a second image.

[0225] According to one embodiment, the electronic device (1001) can transmit a third image (1030) including a second area (1020) of the first image and the second image (or at least a portion (1040) of the second image) to an external electronic device (1002). The external electronic device (1002) can display the third image (1030).

[0226] According to one embodiment, the electronic device (1001) may display the third image (1030) together with the first image on at least a portion of the display module (e.g., the display module (160) of FIG. 1). For example, the electronic device (1001) may display the third image (1030) in a reduced size on a lower left area (1050) of the display module. The electronic device (1001) may provide monitoring by displaying the third image (1030) together with the first image on the display module, thereby allowing a user to check how the third image (1030) is being displayed on an external electronic device (1002). By monitoring the third image (1030) being displayed, the user may provide the electronic device (1001) with a protection release command or a protection command for protected or unprotected personal information.

[0227] In one embodiment, the electronic device (1001) may transmit the first image instead of the third image (1030) in response to a user input while transmitting the third image (1030) to the external electronic device (1002). For example, the user may disable personal information protection if he or she determines that personal information protection is no longer necessary. Upon disabling personal information protection, the first image may be transmitted to the external electronic device (1002) instead of the third image (1030).

[0228] Referring to FIG. 11, an electronic device (1101) (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIG. 2, the electronic device (501) of FIG. 5, the electronic device (601) of FIG. 6, the electronic device (801) of FIG. 8, and the electronic device (1001) of FIG. 10) is illustrated. The electronic device (1101) can share a screen displayed on the electronic device (1101) with an external electronic device (1102) (e.g., the electronic device (102) of FIG. 1, the external electronic device (202) of FIG. 2, the external electronic device (802) of FIG. 8, the external electronic device (803) of FIG. 8, and the external electronic device (1002) of FIG. 10). It will be assumed that the external electronic device (1102) is a display device placed in a conference room, and that the electronic device (1101) has obtained a connection request with the external electronic device (1102) for conference purposes. For example, FIG. 11 is a diagram illustrating a configuration for regenerating an image of an area requiring protection based on an image of an area that does not require protection according to one embodiment of the present disclosure.

[0229] According to one embodiment, the electronic device (1101) shares the screen being displayed with an external electronic device (1102), so that the screen being displayed on the electronic device (1101) can be identified as a first image for the content.

[0230] In one embodiment, the first image may include recommended SNS posts. The recommended SNS posts may be generated by a recommendation algorithm based on the user's search history. For example, Region A and Region B may include recommended posts recommended by a recommendation algorithm based on the user's search history. For example, Region A and Region B may include recommended posts related to posts frequently searched by the user. Assume that an AI model analyzes the first image and determines that the recommended SNS posts are personal information that should not be shared based on the second condition. A detailed description of the AI ​​model that learned the second condition is omitted, as it has been described above.

[0231] In one embodiment, since the SNS recommended post is determined to be personal information, the area where the SNS recommended post is displayed may be determined to be a first area (1110) containing personal information requiring protection (e.g., the first area (1010) of FIG. 10 ). An area in the first image from which the first area (1110) is excluded may be a second area (1120) (e.g., the second area (1020) of FIG. 10 ). For example, the second area (1120) may be an area that does not require protection. To determine the first area (1110) and the second area (1120), personal information may be detected through analysis of the first image. The detection of personal information will be described later in FIG. 19 .

[0232] According to one embodiment, the electronic device (1101) may generate a second image. The second image may be an image generated based on at least a portion of the second area (1120) and may be an image that can naturally blend in with the first image. At least a portion (1140) of the second image (e.g., at least a portion (1040) of the second image of FIG. 10) may be included in the third image (1030).

[0233] According to one embodiment, the electronic device (1101) can generate a second image using a generative artificial intelligence model. The electronic device (1101) can generate a prompt to be input to the generative artificial intelligence model. The electronic device (1101) can generate a prompt to cause the generative artificial intelligence model to generate a second image to replace the first region (1110), and generate the second image based on at least a portion of the second region (1120). The electronic device can generate a prompt to generate the second image based on information based on the first region (1110) of the first image. The information based on the first region (1110) can include information about an image of a second region (1120) proximate to the first region (1110), and information such as the size of the first region (1110). The electronic device (1101) can input the prompt to the generative artificial intelligence model to generate the second image. The second image may be generated based on at least a portion of the second region (1120) and may blend in naturally with the second region (1120). For example, at least a portion (1140) of the second image included in regions C and D may include a puppy.

[0234] According to one embodiment, the external electronic device (1102) can display a third image (1130) that includes an image included in a second area (1120) and at least a portion (1140) of the second image.

[0235] In one embodiment, the electronic device (1101) may generate a prompt that causes the generative artificial intelligence model to generate a third image (1130) based on the first image. By inputting the prompt to the generative artificial intelligence model, the electronic device (1101) may generate the third image (1130) directly based on the first image without generating the second image.

[0236] According to one embodiment, the electronic device (1101) can display a first region (1110) and a second region (1120) separately. For example, the electronic device (1101) can highlight the border of the first region (1110) or display the first region (1110) darker than the second region (1120) to display the first region (1110) and the second region (1120) separately. By displaying the first region (1110) and the second region (1120) separately, the electronic device (1101) can enable a user to intuitively recognize the protected region and the unprotected region. For example, the electronic device (1101) can display a third image (1130) on a display module of the electronic device (1101) and distinguish between a second area (1120) and a first area (1110) replaced with at least a portion (1140) of the second image.

[0237] According to one embodiment, the electronic device (1101) may further display a button (1150). The button (1150) may be a button that provides ON / OFF protection for personal information. For example, the electronic device (1101) may obtain a protection setting command or a protection release command through the button (1150). For example, the electronic device (1101) may release protection for personal information included in the first area (1110) based on the protection release command. For example, the electronic device (1101) may provide protection for unprotected information based on the protection setting command. The protection command or protection release command obtained through the button (1150) may be stored as a shared history and used to update the protection pattern.

[0238] Referring to FIG. 12, an electronic device (1201) (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIG. 2, the electronic device (501) of FIG. 5, the electronic device (601) of FIG. 6, the electronic device (801) of FIG. 8, the electronic device (1001) of FIG. 10, and the electronic device (1101) of FIG. 11) is illustrated. The electronic device (1201) can share the screen displayed on the electronic device (1201) with an external electronic device (1202) (e.g., the electronic device (102) of FIG. 1, the external electronic device (202) of FIG. 2, the external electronic device (802) of FIG. 8, the external electronic device (803) of FIG. 8, the external electronic device (1002) of FIG. 10, and the external electronic device (1102) of FIG. 11). The external electronic device (1202) is a display device placed in a conference room, and it is assumed that the electronic device (1201) has obtained a connection request with the external electronic device (1202) for conference purposes. For example, FIG. 12 is a diagram for explaining a configuration for regenerating an image of an area requiring protection based on an image of an area that does not require protection according to one embodiment of the present disclosure.

[0239] According to one embodiment, the electronic device (1201) shares the screen being displayed with an external electronic device (1202), so that the screen being displayed on the electronic device (1201) can be identified as a first image for the content.

[0240] Referring to the first image, a list of mailboxes may be included. In one embodiment, sensitivity may be determined for the list of mailboxes. Alternatively, in one embodiment, sensitivity may be determined differently for each email included in the list of mailboxes. For example, sensitivity may be determined differently for each email based on the content, sender, and / or subject of the email. For convenience of explanation, it is assumed below that the list of mailboxes, rather than each individual email, is determined to be personal information requiring protection. However, it will be apparent to those skilled in the art that the following description can be equally applied even if each email is determined to be personal information.

[0241] According to one embodiment, referring to the electronic device (1201), the screen may include a first area (1210) including a mailbox list (e.g., the first area (1010) of FIG. 10 and the first area (1110) of FIG. 11) and a second area (1220) that does not require security (e.g., the second area (1020) of FIG. 10 and the second area (1120) of FIG. 11) (e.g., an area where advertisements are displayed). For example, the second area (1220) may be an area that does not require protection. To determine the first area (1210) and the second area (1220), personal information detection may be performed through analysis of the first image. The detection of personal information will be described later with reference to FIG. 19.

[0242] According to one embodiment, the electronic device (1201) can generate one or more second images. The second images can be generated based on the second area (1220). The electronic device can generate one or more second images based on at least a portion of the second area (1220). The one or more second images can be images generated based on at least a portion of the second area (1220) and can be images that can naturally blend in with the first image. At least a portion (1241, 1243) of the one or more second images (e.g., at least a portion (1040) of the second image of FIG. 10 and at least a portion (1140) of the second image of FIG. 11) can be included in the third image (1230).

[0243] In one embodiment, the electronic device (1201) may generate a prompt to generate one or more second images based on information based on a first region (1210) of a first image. The information based on the first region (1210) may include information about an image of a second region (1220) proximate to the first region (1010), information such as the size of the first region (1010). The generative artificial intelligence model may generate the second images based on the prompt.

[0244] In one embodiment, the electronic device (1201) may generate a prompt that causes the generative artificial intelligence model to generate a third image (1230) based on the first image. By inputting the prompt into the generative artificial intelligence model, the electronic device (1201) may generate the third image (1230) directly based on the first image.

[0245] According to one embodiment, the electronic device may transmit an image included in a second area (1220) and a third image (1230) including one or more second images (or at least a portion (1241, 1243) of one or more second images) (e.g., the third image (1130) of FIG. 1) to an external electronic device (1202). The arrangement of the image included in the second area (1220) and at least a portion (1241, 1243) of one or more second images in the third image (1230) may be different from the arrangement of the first area (1210) and the second area (1220) in the first image. For example, the second area (1220) may be arranged on the right side on the screen of the electronic device (1201), whereas the image included in the second area (1220) in the third image (1230) may be arranged in the center.

[0246] In one embodiment, the electronic device (1201) can protect personal information by changing the layout. For example, if the first area (1210) and the second area (1220) are displayed in separate windows, the electronic device (1201) can change the layout so that the window including the second area (1220) is displayed in full screen.

[0247] In one embodiment, the electronic device (1201) can protect personal information through blur processing. For example, the first area (1210) can be blurred to make it difficult to identify.

[0248] In one embodiment, the electronic device (1201) can protect personal information through blocking. For example, the first area (1210) can be blocked to obscure personal information.

[0249] According to one embodiment, the electronic device (1201) can display the first area (1210) by deleting it.

[0250] However, the protection of personal information through the aforementioned blurring and blocking may not naturally align with the second area (1220). Therefore, blurring and blocking may be utilized when it is difficult to create the aforementioned second image.

[0251] Referring to FIG. 13, a screen being displayed by an electronic device (e.g., an electronic device (101) of FIG. 1, an electronic device (201) of FIG. 2, an electronic device (501) of FIG. 5, an electronic device (601) of FIG. 6, an electronic device (801) of FIG. 8, an electronic device (1001) of FIG. 10, an electronic device (1101) of FIG. 11, and an electronic device (1201) of FIG. 12) is illustrated. The electronic device can share the screen being displayed on the electronic device with an external electronic device (e.g., an electronic device (102) of FIG. 1, an external electronic device (202) of FIG. 2, an external electronic device (802) of FIG. 8, an external electronic device (803) of FIG. 8, an external electronic device (1002) of FIG. 10, and an external electronic device (1102) of FIG. 11). It is assumed that the external electronic device is a display device placed in a conference room, and that the electronic device has obtained a connection request with the external electronic device for conference purposes. For example, FIG. 13 is a diagram illustrating a configuration for replacing an image of an area requiring protection based on an image that has already been loaded but is not being displayed on the screen according to one embodiment of the present disclosure.

[0252] In one embodiment, the electronic device shares the screen it is displaying with an external electronic device, so that the screen it is displaying on the electronic device can be identified as a first image for the content.

[0253] In one embodiment, the first image may include recommended SNS posts. The recommended SNS posts may be generated based on a recommendation algorithm based on the user's search history. For example, regions A and B may include recommended posts recommended by a recommendation algorithm based on the user's search history. For example, regions A and B may include recommended posts related to posts frequently searched by the user. It is assumed that the SNS recommended posts based on the user's search history are personal information with a sensitivity level of 4 out of 5 based on an artificial intelligence model. Since the electronic device and the external electronic device were connected for the purpose of a meeting, it is assumed that the security level is determined to be 5 out of 1 to 5 based on the artificial intelligence model. The method for determining the sensitivity of information (e.g., the protection pattern of personal information) and the security level based on the artificial intelligence model is omitted as described above.

[0254] In one embodiment, security level 3 corresponds to sensitivity level 3. At security level 3, information with a sensitivity level of 3 or higher can be determined as personal information. For example, recommended posts on social media can be determined as personal information that must be protected.

[0255] In one embodiment, the electronic device can replace the first region (1310) (e.g., the first region (1010) of FIG. 10 , the first region (1110) of FIG. 11 , and the first region (1210) of FIG. 12 ) based on the non-display region (1370). The electronic device may have a non-display region (1370) that is loaded but not displayed due to limitations in the size of the display module (e.g., the display module (160) of FIG. 1 ). For example, the display module of the electronic device may display only the display region (1360), and the non-display region (1370) may need to be scrolled to be displayed on the display module.

[0256] According to one embodiment, the electronic device may determine an image included in an area (1350) that does not include personal information in a non-display area (1370) as the second image (1340). For example, the electronic device may determine an image included in areas C and D that do not include a recommendation algorithm as the second image (1340) to replace the first area (1310). To determine the first area (1310) and the second area (1320) (e.g., the second area (1020) of FIG. 10 , the second area (1120) of FIG. 11 , and the second area (1220) of FIG. 12 ), personal information detection may be performed on the first image. The detection of personal information will be described later with reference to FIG. 19 .

[0257] According to one embodiment, the electronic device can transmit a third image (1330) (e.g., the third image (1130) of FIG. 11 and the third image (1230) of FIG. 12) including an image included in a second area (1320) and a second image (1340) (e.g., an image included in area C and area D) to an external electronic device.

[0258] In one embodiment, the electronic device can display a third image (1330). By utilizing pre-loaded information, computational resources can be reduced by eliminating the need to create a separate image, and a more natural third image (1330) can be obtained. The method of utilizing pre-loaded information described above can be performed through a preprocessing method when casting content to an external electronic device.

[0259] According to one embodiment, the above-described method may also be used when transmitting an image captured through screen capture to an external electronic device. For example, the display area (1360) may be an area captured during screen capture. The non-display area (1370) may be an area that is loaded but not displayed on the display module and thus not captured during capture. The electronic device may modify the display area (1360) captured based on the non-display area (1370) during capture. For example, the electronic device may modify the captured image by replacing the first area (1310) with an image included in an area that does not correspond to personal information in the non-display area (1370). The modified captured image may be the third image (1330).

[0260] Referring to FIG. 14, a screen being displayed by an electronic device (e.g., an electronic device (101) of FIG. 1, an electronic device (201) of FIG. 2, an electronic device (501) of FIG. 5, an electronic device (601) of FIG. 6, an electronic device (801) of FIG. 8, an electronic device (1001) of FIG. 10, an electronic device (1101) of FIG. 11, and an electronic device (1201) of FIG. 12) is illustrated. The electronic device can share the screen being displayed on the electronic device with an external electronic device (e.g., an electronic device (102) of FIG. 1, an external electronic device (202) of FIG. 2, an external electronic device (802) of FIG. 8, an external electronic device (803) of FIG. 8, an external electronic device (1002) of FIG. 10, and an external electronic device (1102) of FIG. 11). It is assumed that the external electronic device is a display device placed in a conference room, and that the electronic device has obtained a connection request with the external electronic device for conference purposes. For example, FIG. 14 is a diagram for explaining a configuration for sharing by replacing the first image (1400) based on an image obtainable by a guest account according to one embodiment of the present disclosure.

[0261] In one embodiment, the electronic device shares the screen it is displaying with an external electronic device, so that the screen it is displaying on the electronic device can be identified as a first image (1400) for the content.

[0262] In one embodiment, the first image (1400) may include SNS recommended posts. The SNS recommended posts may include posts based on a recommendation algorithm based on the user's search history. For example, areas A and B may include posts based on the recommendation algorithm. It is assumed that the SNS recommended posts based on the user's search history are determined by an AI model to be personal information that needs to be protected. For example, the AI ​​model may determine the SNS recommended posts as personal information based on the protection pattern and security level of the SNS recommended posts included in the first image.

[0263] In one embodiment, an electronic device can access a social networking service (SNS) application using a guest account. The electronic device can obtain at least one image from the SNS application accessed using the guest account. Based on the at least one image, the electronic device can generate a third image (1430) (e.g., the third image (1130) of FIG. 11 , the third image (1230) of FIG. 12 , and the third image (1330) of FIG. 13 ). The electronic device can replace the first image (1400) with the third image (1430) without distinction between the first area (1410) (e.g., the first area (1010) of FIG. 10, the first area (1110) of FIG. 11, the first area (1210) of FIG. 12, and the first area (1310) of FIG. 13) and the second area (1420) (e.g., the second area (1020) of FIG. 10, the second area (1120) of FIG. 11, the second area (1220) of FIG. 12, and the second area (1320) of FIG. 13). The electronic device can transmit the third image (1430) to an external electronic device. The external electronic device can display the third image (1430).

[0264] Referring to FIG. 15, an electronic device (1501) (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIG. 2, the electronic device (501) of FIG. 5, the electronic device (601) of FIG. 6, the electronic device (801) of FIG. 8, the electronic device (1001) of FIG. 10, the electronic device (1101) of FIG. 11, and the electronic device (1201) of FIG. 12) sharing content with an external electronic device (1502) (e.g., the electronic device (102) of FIG. 1, the external electronic device (202) of FIG. 2, the external electronic device (802) of FIG. 8, the external electronic device (803) of FIG. 8, the external electronic device (1002) of FIG. 10, and the external electronic device (1102) of FIG. 11)) is illustrated. FIG. 15 is a diagram illustrating a configuration for generating a caricature of a user according to one embodiment of the present disclosure to replace an area requiring protection.

[0265] According to one embodiment, an electronic device (1501) may share an image including a user's face (e.g., a first image (e.g., the first image (1400) of FIG. 14)) with an external electronic device (1502). If the user's face is included in a sensitivity level corresponding to a security level or higher, the user's face may need to be protected from being shared. The electronic device (1501) may use a generative artificial intelligence model to deform the user's face to prevent the user's face from being shared. The electronic device (1501) may generate a prompt including a command that causes the generative artificial intelligence model to deform the user's face. For example, the prompt may include a command that causes the generative artificial intelligence model to generate a caricature of the user. For example, the prompt may include a command that causes the generative artificial intelligence model to deform landmarks that can identify the user, such as the shape of the eyes, nose, mouth, ears, and face. However, this is merely an example and the present disclosure is not limited thereto.

[0266] According to one embodiment, an area in which a user is included in an image may be a first area (e.g., a first area (1010) in FIG. 10, a first area (1110) in FIG. 11, a first area (1210) in FIG. 12, a first area (1310) in FIG. 13, and a first area (1410) in FIG. 14), and an area in which the user is not included may be a second area (e.g., a second area (1020) in FIG. 10, a second area (1120) in FIG. 11, a second area (1220) in FIG. 12, a second area (1320) in FIG. 13, and a second area (1420) in FIG. 14). A caricature of the user may correspond to the second image. An image in which at least a portion of a caricature of the user and the first image are synthesized may correspond to a third image (1530) (e.g., the third image (1130) of FIG. 11, the third image (1230) of FIG. 12, the third image (1330) of FIG. 13, and the third image (1430) of FIG. 14).

[0267] In one embodiment, the electronic device (1501) can transmit a third image (1530) including a caricature of the user and a second area to an external electronic device (1502). The external electronic device (1502) can display the third image (1530).

[0268] FIG. 16 and FIG. 17 are diagrams for explaining dynamic control of a security level according to one embodiment of the present disclosure.

[0269] In one embodiment, the security level can be dynamically changed. The security level can be changed after it has been determined during sharing. For example, the security level can adaptively change depending on the sharing context of the content. The security level can change as the sharing context changes, including the content being shared, the recipients of the content (e.g., conference participants), and the external electronic device being shared. When the sharing context changes, the security level may need to be changed accordingly. For example, if content being shared on a display device is changed during a conference and the changed content should not be visible to the conference participants, the sharing context has changed and a change in the security level may be required. For example, if an uninvited external party joins the conference during the conference, the sharing context has changed and a change in the security level may be required.

[0270] In one embodiment, the security level can be determined not only to protect the personal information of an individual user, but also to protect specific information of a group. For example, members working on a specific project together can be viewed as a single group, and the security level can be set to low or no level when sharing content with members of the same group. The descriptions of Figures 1 through 15 above can be applied equally when sharing content related to a specific project with outsiders who are not members of the group.

[0271] Referring to FIG. 16, an external electronic device (1602) displaying shared content (e.g., the electronic device (102) of FIG. 1, the external electronic device (202) of FIG. 2, the external electronic device (802) of FIG. 8, the external electronic device (803) of FIG. 8, the external electronic device (1002) of FIG. 10, the external electronic device (1102) of FIG. 11, and the external electronic device (1502) of FIG. 15) is illustrated.

[0272] In one embodiment, for example, as shown in FIG. 16, it is assumed that content is being shared through an external electronic device (1602) in a meeting attended by members belonging to a group performing a specific project. The external electronic device (1602) may be displaying a third image (1630) (e.g., the third image (1130) of FIG. 11, the third image (1230) of FIG. 12, the third image (1330) of FIG. 13, the third image (1430) of FIG. 14, and the third image (1530) of FIG. 15) in which information corresponding to a sensitivity level of 3 or higher is protected, based on an artificial intelligence model, with a protection level determined to be level 3 out of levels 1 to 5.

[0273] According to one embodiment, an electronic device (e.g., an electronic device (101) of FIG. 1, an electronic device (201) of FIG. 2, an electronic device (501) of FIG. 5, an electronic device (601) of FIG. 6, an electronic device (801) of FIG. 8, an electronic device (1001) of FIG. 10, an electronic device (1101) of FIG. 11, an electronic device (1201) of FIG. 12, and an electronic device (1501) of FIG. 15) and / or an external electronic device (1602) can recognize the participation of an outsider who does not belong to a group in a meeting. For example, the electronic device and / or the external electronic device (1602) can recognize that an outsider is attempting to join a meeting when the outsider tags an ID card to a reader that manages access to a conference room. For example, in a video conferencing environment, the electronic device and / or the external electronic device (1602) can recognize that an outsider is attempting to join a meeting when the outsider attempts to join a video conference.

[0274] In one embodiment, the electronic device can change its security level when an outsider is identified to prevent information from being leaked to outsiders. For example, the electronic device can dynamically change its security level.

[0275] For example, an electronic device can change the security level to level 5 and generate a third image (1640) by regenerating the third image (1630) based on the security level. The electronic device can generate a third image (1640) in which information corresponding to a sensitivity level of level 1 or higher corresponding to the security level of level 5 is protected. The third image (1640) may be an image from which specific information has been removed compared to the third image (1630).

[0276] In one embodiment, the electronic device can transmit the third image (1640) to an external electronic device (1602). The external electronic device (1602) can display the regenerated third image (1640). Accordingly, the electronic device can prevent the leakage of specific information by dynamically changing the security level.

[0277] Referring to FIG. 17, an HMD (1702) (head mounted display) displaying shared content (e.g., electronic device (102) of FIG. 1, external electronic device (202) of FIG. 2, external electronic device (802) of FIG. 8, external electronic device (803) of FIG. 8, external electronic device (1002) of FIG. 10, external electronic device (1102) of FIG. 11, external electronic device (1502) of FIG. 15, and external electronic device (1602) of FIG. 16) is illustrated.

[0278] According to one embodiment, a user (1710) of the HMD (1702) may be viewing content shared from an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2, electronic device (501) of FIG. 5, electronic device (601) of FIG. 6, electronic device (801) of FIG. 8, electronic device (1001) of FIG. 10, electronic device (1101) of FIG. 11, electronic device (1201) of FIG. 12, and electronic device (1501) of FIG. 15). For example, the user (1710) of the HMD (1702) may be viewing a video shared from the electronic device. The HMD (1702) may be displaying a third image (1730) generated based on the security level “low” (e.g., the third image (1130) of FIG. 11, the third image (1230) of FIG. 12, the third image (1330) of FIG. 13, the third image (1430) of FIG. 14, the third image (1530) of FIG. 15, and the third image (1630) of FIG. 16). For example, the third image (1430) may include a video recommendation algorithm.

[0279] In one embodiment, the user of the HMD (1702) may change from user (1710) to user (1720) while content is being shared. For example, an unauthorized person (e.g., user (1720)) may change the user of the HMD (1702). The HMD (1702) may determine whether the user has changed through multiple sensor modules. If the user has changed, the HMD (1702) may notify the electronic device that the user has changed.

[0280] In one embodiment, the electronic device can change its security level based on user changes. For example, the electronic device can dynamically change its security level based on user changes.

[0281] For example, an electronic device may change the security level of the HMD (1702) to "High" if the user is changed to an unauthorized person. Based on the changed security level, the electronic device may regenerate the third image (1730) into a third image (1740). The third image (1740) may be an image from which the video recommendation list has been removed from the third image (1730).

[0282]

[0283] FIG. 18 is a diagram illustrating protection of multiple personal information having different sensitivities according to one embodiment of the present disclosure.

[0284] FIG. 18 illustrates a first image (1800) including a plurality of personal information items (1801, 1803, 1805, 1807, 1809). For convenience of explanation, electronic devices (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2, electronic device (501) of FIG. 5, electronic device (601) of FIG. 6, electronic device (801) of FIG. 8, electronic device (1001) of FIG. 10, electronic device (1101) of FIG. 11, electronic device (1201) of FIG. 12, and electronic device (1501) of FIG. 15)) are omitted, and only the first image (1800) displayed on the screen of the electronic device is illustrated.

[0285] In FIG. 18, an area where multiple personal information (1801, 1803, 1805, 1807, 1809) is displayed may be a first area (e.g., a first area (1010) of FIG. 10, a first area (1110) of FIG. 11, a first area (1210) of FIG. 12, a first area (1310) of FIG. 13, and a first area (1410) of FIG. 14). An area in the first image (1800) excluding the first area may be a second area (e.g., a second area (1020) of FIG. 10, a second area (1120) of FIG. 11, a second area (1220) of FIG. 12, a second area (1320) of FIG. 13, and a second area (1420) of FIG. 14). For example, in the first image (1800), the area that does not require protection may be the second area.

[0286] Personal information (1801) and personal information (1802) may be determined to have a sensitivity level of "high" based on the personal information protection pattern. Personal information (1809) may be determined to have a sensitivity level of "medium" based on the personal information protection pattern. Personal information (1805) and personal information (1807) may be determined to have a sensitivity level of "low" based on the personal information protection pattern.

[0287] According to one embodiment, when content is shared in a shared environment with a security level of "high," the electronic device can perform protection for personal information with a sensitivity level of "low" or higher. For example, a third image (1810) (e.g., a third image (1130) of FIG. 11, a third image (1230) of FIG. 12, a third image (1330) of FIG. 13, a third image (1430) of FIG. 14, a third image (1530) of FIG. 15, a third image (1630) of FIG. 16, and a third image (1730) of FIG. 17) displayed on an external electronic device (e.g., an electronic device (102) of FIG. 1, an external electronic device (202) of FIG. 2, an external electronic device (802) of FIG. 8, an external electronic device (1002) of FIG. 10, an external electronic device (1102) of FIG. 11, an external electronic device (1502) of FIG. 15, and an external electronic device (1602) of FIG. 16)) may not include personal information.

[0288] In one embodiment, when content is shared in a shared environment with a security level of "medium," the electronic device can protect personal information with a sensitivity level of "medium" or higher. For example, the third image (1820) displayed on the external electronic device may include personal information (1805) and personal information (1807) with a sensitivity level of "low."

[0289] In one embodiment, when content is shared in a shared environment with a security level of "low," the electronic device can protect personal information with a sensitivity level of "high" or higher. For example, the third image (1830) displayed on the external electronic device may include personal information (1809) with a sensitivity level of "medium," personal information (1805), and personal information (1807) with a sensitivity level of "low."

[0290] For example, based on the sensitivity and security level of the personal information, the electronic device may perform protection for multiple pieces of personal information (1801, 1803, 1805, 1807, 1809) in whole or in part.

[0291] In one embodiment, an electronic device may obtain a command to deprotect protected personal information from a user. The electronic device may then deprotect the personal information for which the command has been obtained. For example, the electronic device may obtain a command to deprotect personal information (1801) while the third image (1830) is being displayed on an external electronic device, and the electronic device may deprotect the personal information (1801). Accordingly, the personal information (1801) may be displayed on the third image (1830).

[0292] The deactivation command obtained from the user can be used by the AI ​​model to update the protection pattern of personal information.

[0293] In one embodiment, an electronic device may obtain a protection setting command for information that is not currently protected from a user. The electronic device may then perform protection on the information for which the protection setting command has been obtained. For example, the electronic device may obtain a protection setting command for personal information (1805) while the third image (1820) is displayed on an external electronic device, and perform protection on the personal information (1805). Accordingly, the personal information (1805) may disappear from the third image (1820). The protection setting command obtained from the user may be used by an artificial intelligence model to update the protection pattern of the personal information.

[0294] According to one embodiment, the electronic device can perform protection or release protection of personal information upon obtaining a protection command or release command from a user, regardless of the security level.

[0295] Below, the system of the present disclosure will be described.

[0296]

[0297] FIG. 19 is a drawing for explaining a system according to one embodiment of the present disclosure.

[0298] Referring to FIG. 19, a first DB (1910) and a second DB (1920) are illustrated. The first DB (1910) can store attribute information on an external electronic device (e.g., an electronic device (102) of FIG. 1, an external electronic device (202) of FIG. 2, an external electronic device (802) of FIG. 8, an external electronic device (803) of FIG. 8, an external electronic device (1002) of FIG. 10, an external electronic device (1102) of FIG. 11, an external electronic device (1502) of FIG. 15, and an external electronic device (1602) of FIG. 16). The first DB can store attribute information of an external electronic device that has been connected at least once. Descriptions regarding attribute information of an external electronic device and a method for obtaining the same are omitted, as described above with reference to FIGS. 5 and 6.

[0299] According to one embodiment, the second DB (1920) can store a protection pattern. According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIG. 2, the electronic device (501) of FIG. 5, the electronic device (601) of FIG. 6, the electronic device (801) of FIG. 8, the electronic device (1001) of FIG. 10, the electronic device (1101) of FIG. 11, the electronic device (1201) of FIG. 12, and the electronic device (1501) of FIG. 15) can include a personal information protection module (1940).

[0300] According to one embodiment, the privacy protection module (1940) may include an artificial intelligence model that determines whether information contained in content is personal information. For example, the privacy protection module (1940) may include an artificial intelligence model that determines whether information contained in a first image of the content (e.g., the first image (1400) of FIG. 14 and the first image (1800) of FIG. 18) is personal information.

[0301] According to one embodiment, the artificial intelligence model included in the personal information protection module (1940) may be a model that has learned the second condition and is trained to determine whether information contained in the content is personal information based on the second condition. The artificial intelligence model may determine a protection pattern and security level for the information contained in the content based on the second condition and, based on this, determine whether the information contained in the content is personal information. The protection pattern has been described in detail in FIG. 4, and the security level has been described in detail in FIG. 7, so a detailed description thereof will be omitted.

[0302] In one embodiment, information for which a security pattern has been determined by an artificial intelligence model may be stored in a second database (1920). If the content includes information stored in the second database (1920), the artificial intelligence model may load and use a security pattern (e.g., sensitivity) for the corresponding information from the second database (1920), thereby reducing computational resources.

[0303] According to one embodiment, an electronic device may receive a connection request from a user to an external electronic device. Upon receiving the connection request, the electronic device may identify the external electronic device based on the connection request. The electronic device may then receive a command to share content with the external electronic device.

[0304] According to one embodiment, the electronic device can determine whether attribute information of an external electronic device identified based on a connection request exists based on a first database (DB) (1910). If attribute information of the external electronic device does not exist, the electronic device can obtain attribute information of the external electronic device using the method described above in FIG. 5.

[0305] According to one embodiment, the electronic device can check the protection pattern (e.g., sensitivity) for information included in the first image from the second DB (1920). If the second DB (1920) does not include the protection pattern for information included in the content, the protection pattern is determined based on the artificial intelligence model of the personal information protection module (1940).

[0306] You can decide the turn and save it in the second DB (1920).

[0307] According to one embodiment, the electronic device may transmit attribute information and / or protection patterns of external electronic devices identified in the first database (1910) and the second database (1920) to the personal information protection module (1940). The personal information protection module (1940) may determine a security level based on at least one of the attribute information and / or protection patterns of the external electronic device using an artificial intelligence model. A description of the method for determining the security level is omitted, as described above with reference to FIG. 7.

[0308] In one embodiment, the artificial intelligence model can determine which personal information needs to be protected among the information included in the first image based on the protection pattern and security level.

[0309] According to one embodiment, the electronic device may transmit judgment information about personal information to a prompt generation module (1950). The prompt generation module (1950) may generate a prompt to generate a second image to replace an image included in a first area (e.g., the first area (1010) of FIG. 10, the first area (1110) of FIG. 11, the first area (1210) of FIG. 12, the first area (1310) of FIG. 13, and the first area (1410) of FIG. 14) based on the judgment information about personal information and information based on the first image. Information based on the first image may include at least one of information about the first image, the size of the first region, information about a portion of the second region (e.g., the second region (1020) of FIG. 10, the second region (1120) of FIG. 11, the second region (1220) of FIG. 12, the second region (1320) of FIG. 13, and the second region (1420) of FIG. 14) close to the first region, and the location of the first region.

[0310] For example, the prompt may include instructions to generate a second image based on at least a portion of a second area proximate the first area.

[0311] According to one embodiment, the personal information detection module (1960) can detect personal information from a first image. The first image may be an image of content to be shared. The description of the first image, as described above in FIG. 9, is omitted.

[0312] In one embodiment, the personal information detection module (1960) can detect personal information in a first image and determine a first area and a second area based on a security level included in the prompt. The personal information detection module (1960) can determine an area containing personal information with a sensitivity level higher than or equal to the security level as the first area. The personal information detection module (1960) can determine an area in the first image excluding the first area as the second area.

[0313] According to one embodiment, the personal information detection module (1960) may include an artificial intelligence model trained to detect personal information in a first image. The artificial intelligence model may include an artificial intelligence model trained to detect personal information in the first image based on a shared history. According to one embodiment, the personal information detection module (1960) may include an artificial intelligence model trained to detect personal information in the first image and determine a first region and a second region based on a security level. The artificial intelligence model may include an artificial intelligence model trained to detect personal information in the first image based on a shared history and determine a first region and a second region based on a security level. However, this is merely an example and the present disclosure is not limited thereto. For example, an electronic device may receive a selection of a first region and a second region from a user.

[0314] In one embodiment, the regeneration module (1970) may generate an image (e.g., a second image) to replace the first region based on a first image and a prompt obtained from the personal information detection module (1960) in which the first region and the second region are determined. The regeneration module (1970) may include a generative artificial intelligence model. The regeneration model may input a prompt into the generative artificial intelligence model to obtain a second image to replace the first region.

[0315] According to one embodiment, the electronic device may be displaying a second area of ​​the first image and a third image including the second image (e.g., the third image (1130) of FIG. 11, the third image (1230) of FIG. 12, the third image (1330) of FIG. 13, the third image (1430) of FIG. 14, the third image (1530) of FIG. 15, the third image (1630) of FIG. 16, and the third image (1730) of FIG. 17). For example, the third image (1430) may be transmitted to an external electronic device. According to one embodiment, the electronic device may transmit the second area of ​​the first image and the second image to the external electronic device, and the electronic device may obtain the third image by combining the second area and the second image. The external electronic device may display the third image.

[0316] FIG. 20 is a diagram illustrating content sharing according to one embodiment of the present disclosure.

[0317] Referring to FIG. 20, an electronic device (2001) displaying an execution screen (2010) of an SNS application (e.g., an electronic device (101) of FIG. 1, an electronic device (201) of FIG. 2, an electronic device (501) of FIG. 5, an electronic device (601) of FIG. 6, an electronic device (801) of FIG. 8, an electronic device (1001) of FIG. 10, an electronic device (1101) of FIG. 11, an electronic device (1201) of FIG. 12, and an electronic device (1501) of FIG. 15) is illustrated.

[0318] According to one embodiment, the electronic device (2001) may activate a sharing function while displaying an execution screen (2010) of an SNS application. For example, the electronic device (2001) may receive a connection request with an external electronic device (2002) (e.g., the electronic device (102) of FIG. 1 , the external electronic device (202) of FIG. 2 , the external electronic device (802) of FIG. 8 , the external electronic device (803) of FIG. 8 , the external electronic device (1002) of FIG. 10 , the external electronic device (1102) of FIG. 11 , the external electronic device (1502) of FIG. 15 , and the external electronic device (1602) of FIG. 16 ). For example, the connection request may include a user's input for screen mirroring or screen casting through a wireless connection with the external electronic device (2002). For example, the connection request may include a user's input for screen duplication and / or screen expansion through a cable connection with the external electronic device (2002).

[0319] In one embodiment, the electronic device (2001) may activate a sharing function. When the sharing function is activated, the electronic device (2001) may display at least one area (e.g., area (2023), area (2025)) containing personal information on the execution screen (2010).

[0320] According to one embodiment, the electronic device (2001) may display buttons (2027, 2029) (e.g., button (1150) of FIG. 11) indicating whether to share at least one area containing personal information with an external electronic device (2002). If a button (2027, 2029) is selected, the area corresponding to the selected button (2027, 2029) is not shared, and if the button (2027, 2029) is not selected (e.g., not selected), the area corresponding to the unselected button (2027, 2029) may be shared. For example, in the execution screen (2010), the area (2023) corresponding to the button (2023) may not be shared with the external electronic device (2002). For example, in the execution screen (2010), the area (2025) corresponding to the button (2025) can be shared with an external electronic device (2002). The electronic device (2001) can obtain a selection command or a deselection command for the area (2023, 2025) from the user through the buttons (2023, 2025). However, it is not necessarily required to have buttons (2027, 2029) indicating whether to share with the external electronic device (2002), and it is obvious to those skilled in the art that sharing can be initiated through a cable connection with an external display device.

[0321] According to one embodiment, when the sharing function is activated, the electronic device (2001) may recommend whether to share the area corresponding to the button (2027, 2029) with the external electronic device (2002) by selecting or not selecting the button (2027, 2029). If there is no sharing history for personal information included in the area (2023, 2025), the electronic device (2001) may determine whether to share or not share the area (2023, 2025) according to the first condition, and may recommend sharing or not sharing to the user by displaying selection or not selecting the button (2027, 2029) based on the determination. If there is a sharing history for personal information included in the areas (2023, 2025), the electronic device (2001) can determine whether to share or not share the areas (2023, 2025) based on a second condition determined based on the sharing history, and can recommend sharing or not sharing to the user by displaying selection or non-selection of a button (2027, 2029) based on the determination.

[0322] In one embodiment, the electronic device (2001) may receive a command to execute sharing via a pop-up window (2020) that receives a sharing command. Upon receiving the sharing command, the electronic device may share content (e.g., the execution screen (2010)) to the external electronic device (2002) based on the selection and / or non-selection of areas (2023, 2025). For example, assume that area (2023) is selected and area (2025) is non-selected. For example, it is assumed that the region (2023) is at least intentionally determined as a first region (e.g., the first region (1010) of FIG. 10, the first region (1110) of FIG. 11, the first region (1210) of FIG. 12, the first region (1310) of FIG. 13, and the first region (1410) of FIG. 14), and the region (2025) is at least intentionally determined as a second region (e.g., the second region (1020) of FIG. 10, the second region (1120) of FIG. 11, the second region (1220) of FIG. 12, the second region (1320) of FIG. 13, and the second region (1420) of FIG. 14). An external electronic device (2002) can display a screen (2003) including a regenerated image (e.g., at least a portion (1140) of the second image of FIG. 11, at least a portion (1241) of the second image of FIG. 12, at least a portion (1243) of the second image of FIG. 12, and the second image (1340) of FIG. 13) for an area (2023).

[0323] According to one embodiment, when the execution screen (2010) is shared, the electronic device (2001) may further display an area (2030) (e.g., the lower left area (1050) of FIG. 10) for monitoring the screen (2003) being displayed on the external electronic device (2002). The area (2030) may display a reduced image of the screen (2003). For example, an area corresponding to area (2023) in the screen (2003) displayed on the external electronic device (2002) may include a regenerated image. Area (2023) may include a reduced regenerated image because the screen (2003) displays a reduced image.

[0324] According to one embodiment, the electronic device (2001) may obtain a selection command and / or a deselection command for areas (2023, 2025) while sharing the execution screen (2010). Hereinafter, it is assumed that the electronic device (2001) obtains a selection command for area (2025). The electronic device (2001) may change the display of the button (2029) based on the selection command for area (2025). Based on the acquisition of the selection command for area (2025), the area (2025) may be determined as a first area. Based on the acquisition of the selection command for area (2025), the electronic device (2001) may regenerate an image for area (2025). The electronic device (2001) may share the execution screen including the regenerated image for area (2025). The external electronic device (2001) can display a screen (2005) containing an additionally regenerated image for the area (2025).

[0325] Below, we will explain how to protect personal information when transmitting content.

[0326] FIG. 21 is a diagram for explaining content transmission according to one embodiment of the present disclosure.

[0327] Referring to FIG. 21, an electronic device (2101) displaying a photo (2110) including a user's face (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2, electronic device (501) of FIG. 5, electronic device (601) of FIG. 6, electronic device (801) of FIG. 8, electronic device (1001) of FIG. 10, electronic device (1101) of FIG. 11, electronic device (1201) of FIG. 12, electronic device (1501) of FIG. 15, and electronic device (2001) of FIG. 20) is illustrated. The electronic device (2101) can provide a file sharing service (e.g., airdrop, quick share, etc.) with an external electronic device (2102) (e.g., the electronic device (102) of FIG. 1, the external electronic device (202) of FIG. 2, the external electronic device (802) of FIG. 8, the external electronic device (803) of FIG. 8, the external electronic device (1002) of FIG. 10, the external electronic device (1102) of FIG. 11, the external electronic device (1502) of FIG. 15, the external electronic device (1602) of FIG. 16, and the external electronic device (2002) of FIG. 20).

[0328] In one embodiment, the electronic device (2101) may receive a command to transmit a photo (2110) to an external electronic device (2102). For example, the electronic device (2101) may receive a command to transmit the photo (2110) to the external electronic device (2102) through a “Transmit” menu (2120) displayed through a right-click of a mouse.

[0329] According to one embodiment, upon receiving a transmission command, the electronic device (2101) may display at least one area (e.g., area (2130)) containing personal information in the photo (2110). For example, the electronic device (2101) may determine a user included in the photo (2110) as area (2130).

[0330] According to one embodiment, the electronic device (2101) may include a button (2140) (e.g., button (1150) of FIG. 11 and buttons (2027, 2029) of FIG. 20) that indicates whether to share the area (2130) with an external electronic device (2102). The electronic device (2101) may obtain a selection command or a deselection command for the area (2130) from a user through the button (2140).

[0331] According to one embodiment, the electronic device (2101) can recommend whether to share the area corresponding to the button (2140) with the external electronic device (2102) through selection or non-selection of the button (2140). If there is no sharing history for personal information included in the area (2130), the electronic device (2101) can determine whether to share or not share the area (2130) according to a first condition, and can recommend sharing or non-sharing to the user by displaying selection or non-selection of the button (2140) based on the determination. If there is a sharing history for personal information included in the area (2130), the electronic device (2101) can determine whether to share or not share the area (2130) according to a second condition determined based on the sharing history, and can recommend sharing or non-sharing to the user by displaying selection or non-selection of the button (2140) based on the determination.

[0332] According to one embodiment, the electronic device (2101) may receive a command to execute sharing through a pop-up window (2150) for receiving a sharing command (e.g., a pop-up window (2020) of FIG. 20). When receiving the sharing command, the electronic device (2101) may share content (e.g., a photo (2110)) to an external electronic device (2102) based on selection and / or non-selection of an area (2130). For example, it is assumed that the area (2130) is selected. For example, it is assumed that the area (2130) is determined to be a first area (e.g., the first area (1010) of FIG. 10, the first area (1110) of FIG. 11, the first area (1210) of FIG. 12, the first area (1310) of FIG. 13, and the first area (1410) of FIG. 14). An external electronic device (2102) can receive a photograph (2103) comprising a regenerated image (e.g., at least a portion (1140) of the second image of FIG. 11, at least a portion (1241) of the second image of FIG. 12, at least a portion (1243) of the second image of FIG. 12, and the second image (1340) of FIG. 13) for an area (2130).

[0333] FIG. 22 is a flowchart for explaining an operation method of an external electronic device according to one embodiment of the present disclosure.

[0334] According to one embodiment, an external electronic device (e.g., an electronic device (101) of FIG. 1, an electronic device (201) of FIG. 2, an electronic device (501) of FIG. 5, an electronic device (601) of FIG. 6, an electronic device (801) of FIG. 8, an electronic device (1001) of FIG. 10, an electronic device (1101) of FIG. 11, an electronic device (1201) of FIG. 12, an electronic device (1501) of FIG. 15, an electronic device (2001) of FIG. 20, and an electronic device (2101) of FIG. 21)) other than an electronic device (e.g., an electronic device (102) of FIG. 1, an external electronic device (202) of FIG. 2, an external electronic device (802) of FIG. 8, an external electronic device (803) of FIG. 8, an external electronic device (1002) of FIG. 10, an external electronic device (1102) of FIG. 11, an external electronic device (1502) of FIG. 15, The external electronic device (1602) of FIG. 16, the external electronic device (2002) of FIG. 20, and the external electronic device (2102) of FIG. 21) can perform protection of personal information.

[0335] The operations described below may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. Furthermore, some operations may be omitted according to some embodiments. Operations (2210) to (2240) may be performed by an external electronic device. For example, instructions stored in a memory may be executed by at least one processor including a processing circuit, and the instructions may cause the external electronic device to perform the following operations.

[0336] In operation (2210), the external electronic device may receive a first image (e.g., the first image (1400) of FIG. 14 and the first image (1800) of FIG. 18) and a security level for the content from the electronic device providing sharing for the content.

[0337] The security level can be determined in the electronic device. The method for determining the security level is described above in Fig. 7, and is thus omitted.

[0338] In operation (2220), the external electronic device can determine a first area and a second area in the first image based on a security level.

[0339] The method for determining the first and second regions will be omitted as it has been described above in FIGS. 10, 12 and 19.

[0340] In action (2230), the external electronic device generates a second image based on the second area.

[0341] An external electronic device can generate a prompt based on the security level. The generation of the prompt is described above in FIGS. 3, 7, 9, 11, and 19, and will be omitted.

[0342] In operation (2240), the external electronic device may display a third image (e.g., the third image (1130) of FIG. 11, the third image (1230) of FIG. 12, the third image (1330) of FIG. 13, the third image (1430) of FIG. 14, the third image (1530) of FIG. 15, the third image (1630) of FIG. 16, and the third image (1730) of FIG. 17) including a second area of ​​the first image and at least a portion of the second image) on the display module.

[0343] Since the matters described above through FIGS. 1 to 21 are applied to each operation illustrated in FIG. 22, a more detailed description is omitted.

[0344] FIG. 23 is a block diagram of a generative artificial intelligence system according to various embodiments.

[0345] The User Query / Response Interface (2310) can receive user input. The user input may be in the form of natural language, images, and / or videos. Context information may also be transmitted when the user input is transmitted. Context information may include various additional information at the time of user input. For example, it may include information on the application currently being used by the user or location information of the user. Furthermore, the user input may be in a mixed form of the aforementioned natural language, images, sounds, and context information. Furthermore, the user input may also be in a non-natural language form, such as selecting a menu. The User Query / Response Interface (2310) can output the results of the generative artificial intelligence system (2300) to the user. The output may be in the form of natural language or specific content, and may also be provided in the form of an action requested by the user. The User Query Interface / Response Interface (2310) can output the results of the generative artificial intelligence system (2300) to the user. The output can be in natural language form, in the form of specific content, or in the form of actions requested by the user.

[0346] The AI ​​framework (2320) can receive user input and coordinate and control each component necessary to perform the user's intention based on the user's query.

[0347] User input received from the User Query / Response Interface (2310) can be transmitted to the Prompt design component (2330). The Prompt design component (2330) (e.g., the prompt generation module (1950) of FIG. 19) can be used to generate prompts suitable for inputting user input into large language models (LLMs) or large multimodal models (LMMs). The Prompt design component (2330) can be an AI component that uses a machine learning algorithm or a neural network to develop better prompts over time. The Prompt design component (2330) can access a knowledge component (e.g., knowledge repositories (2370)) containing user preference data, a prompt library, and prompt examples based on the user input to generate a prompt, and transmit the generated prompt to the LLM or LMM.

[0348] The API / Plugins management component (2340) can communicate with external information when there is a request for additional information when passing user input as input to a generative model. The API / Plugins management component (2340) establishes a channel for communication with the outside of the AI ​​Interface through an API, and can enable access to various data sources through the established channel. In addition, the API / Plugins management component (2340) can request an action through an API (application programming interface; API) when an application or service needs to perform an action that performs the user input as a final result rather than an intermediate result. Information obtained from an external source can be used to generate a prompt in the Prompt design component (2330) along with user input, or can be passed as input to the generative model.

[0349] The Refiner component (e.g., the output modification component (2350)) can fine-tune the output from a generative model. For example, the Refiner component can verify that the content generated by the LLM and / or LMM is not irrelevant, biased, or harmful. Furthermore, the Refiner component can determine the degree to which the output matches the user's desired result and, if necessary, initiate additional processing. The Refiner component can also configure and provide hints to the user to avoid undesirable output.

[0350] A Generative AI Model (2360) (e.g., the regeneration module (1970) of FIG. 19) may generally refer to an artificial intelligence neural network that generates new types of data based on user input information. The Generative AI Model (2360) may include a model that generates images and / or a model that generates languages. Representative models that generate images include a generative adversarial network (GAN) and a variational auto encoder (VAE), and examples include a diffusion-based generative model that uses a VAE and a transformer structure. A model that generates languages ​​is a model that is trained to statistically output the most appropriate output based on input values, and representative examples include models such as CHAT-GPT 3 and CHAT-GPT 4. In addition, there is also an LMM that can recognize various types of data input, such as text, images, and voice, and generate new data corresponding to them.

[0351]

[0352] According to one embodiment, an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2, electronic device (501) of FIG. 5, electronic device (601) of FIG. 6, electronic device (801) of FIG. 8, electronic device (1001) of FIG. 10, electronic device (1101) of FIG. 11, electronic device (1201) of FIG. 12, electronic device (1501) of FIG. 15, electronic device (2001) of FIG. 20, and electronic device (2101) of FIG. 21) may include at least one processor (e.g., processor (120) of FIG. 1) including processing circuitry. The electronic device may include a memory (e.g., memory (130) of FIG. 1) that stores instructions. When at least one processor individually or collectively executes instructions, the instructions may cause the electronic device to obtain a connection request from a user with an external electronic device (e.g., the electronic device (102) of FIG. 1, the external electronic device (202) of FIG. 2, the external electronic device (802) of FIG. 8, the external electronic device (803) of FIG. 8, the external electronic device (1002) of FIG. 10, the external electronic device (1102) of FIG. 11, the external electronic device (1502) of FIG. 15, the external electronic device (1602) of FIG. 16, the external electronic device (2002) of FIG. 20, and the external electronic device (2102) of FIG. 21) for sharing content. When at least one processor individually or collectively executes instructions, the instructions may cause the electronic device to identify a first image of the content (e.g., the first image (1400) of FIG. 14 and the first image (1800) of FIG. 18) for sharing the content with an external electronic device.When at least one processor individually or collectively executes the instructions, the instructions may cause the electronic device to determine a first region (e.g., a first region (1010) of FIG. 10, a first region (1110) of FIG. 11, a first region (1210) of FIG. 12, a first region (1310) of FIG. 13, and a first region (1410) of FIG. 14)) and a second region (e.g., a second region (1020) of FIG. 10, a second region (1120) of FIG. 11, a second region (1220) of FIG. 12, a second region (1320) of FIG. 13, and a second region (1420) of FIG. 14)) in a first image based on a security level for sharing of the content. When at least one processor individually or collectively executes the instructions, the instructions may cause the electronic device to generate a second image to replace the first region based on the second region. When at least one processor individually or collectively executes instructions, the instructions may cause the electronic device to transmit a third image to an external electronic device, the third image including a second area of ​​the first image and at least a portion of the second image (e.g., at least a portion (1140) of the second image of FIG. 11, at least a portion (1241) of the second image of FIG. 12, at least a portion (1243) of the second image of FIG. 12, and the second image (1340) of FIG. 13).

[0353] In one embodiment, when at least one processor individually or collectively executes commands, the commands can cause the electronic device to determine a region containing information with a sensitivity level higher than or equal to a security level as a first region. When at least one processor individually or collectively executes commands, the commands can cause the electronic device to determine a region in a first image, excluding the first region, as a second region.

[0354] According to one embodiment, when at least one processor individually or collectively executes instructions, the instructions may cause the electronic device to generate a second image based on at least a portion of a second area proximate the first area.

[0355] According to one embodiment, the third image may include a first region of the first image that includes at least a portion of the second image.

[0356] According to one embodiment, when at least one processor individually or collectively executes instructions, the instructions may cause the electronic device to identify at least a portion of a screen being displayed on a display module of the electronic device as a first image.

[0357] According to one embodiment, when at least one processor individually or collectively executes instructions, the instructions may cause the electronic device to identify at least a portion of an execution screen of at least one application running in the background by the electronic device as a first image.

[0358] According to one embodiment, when at least one processor individually or collectively executes instructions, the instructions may cause the electronic device to determine a security level based on at least one of a protection pattern for information included in a first image and attribute information of an external electronic device.

[0359] In one embodiment, the protection pattern may include a sensitivity determined based on the user's sharing history with respect to the information.

[0360] According to one embodiment, the attribute information may include at least one of a connection purpose between the electronic device and the external electronic device and information related to the external electronic device.

[0361] According to one embodiment, when at least one processor individually or collectively executes instructions, the instructions may cause the electronic device to display a third image on the display module and to distinguish between the second area and the first area replaced by the second image.

[0362] In one embodiment, the external electronic device may include at least one processor comprising a processing circuit. The external electronic device may include a memory storing instructions. When the at least one processor individually or collectively executes the instructions, the instructions may cause the external electronic device to receive a first image of the content and a security level from an electronic device providing content sharing. When the at least one processor individually or collectively executes the instructions, the instructions may cause the external electronic device to determine a first region and a second region in the first image based on the security level. When the at least one processor individually or collectively executes the instructions, the instructions may cause the external electronic device to generate a second image based on the second region. When the at least one processor individually or collectively executes the instructions, the instructions may cause the external electronic device to display a third image including a second region of the first image and at least a portion of the second image on a display module.

[0363] According to one embodiment, a method of operating an electronic device may include an operation of obtaining a connection request from a user with an external electronic device for sharing content. The method of operating the electronic device may include an operation of identifying a first image of content for sharing the content with the external electronic device. The method of operating the electronic device may include an operation of determining a first region and a second region in the first image based on a security level for sharing the content. The method of operating the electronic device may include an operation of generating a second image based on the second region. The method of operating the electronic device may include an operation of transmitting a third image including a second region of the first image and at least a portion of the second image to the external electronic device.

[0364] According to one embodiment, the operation of determining the first region and the second region in the first image may determine an region including information of a sensitivity level higher than or equal to a security level as the first region, and determine an region in the first image excluding the first region as the second region.

[0365] In one embodiment, the operation of generating the second image may generate the second image based on at least a portion of a second area proximate the first area.

[0366] According to one embodiment, the third image may include a first region of the first image that includes at least a portion of the second image.

[0367] According to one embodiment, the act of identifying a first image for content may identify at least a portion of a screen being displayed on a display module of the electronic device as the first image.

[0368] According to one embodiment, the act of identifying a first image for content may include identifying at least a portion of an execution screen of at least one application running in the background by the electronic device as the first image.

[0369] According to one embodiment, the method of operating the electronic device may further include an operation of determining a security level based on at least one of a protection pattern for information included in a first image and attribute information of an external electronic device.

[0370] According to one embodiment, the method of operating the electronic device may further include displaying a third image on a display module and distinguishing between a second area and a first area replaced with the second image.

[0371] According to one embodiment, a non-transitory computer-readable recording medium can store computer programs including instructions for executing any one of the methods described above.

[0372] The embodiments of the present invention disclosed in this specification and drawings are merely specific examples presented to easily explain the technical contents according to the embodiments of the present invention and to help understand the embodiments of the present invention, and are not intended to limit the scope of the embodiments of the present invention. Therefore, the scope of the various embodiments of the present invention should be interpreted as including all changes or modified forms derived based on the technical ideas of the various embodiments of the present invention in addition to the embodiments disclosed herein.

Claims

In electronic devices (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101), At least one processor (120) comprising processing circuitry; and Memory (130) that stores instructions Including, When the at least one processor (120) individually or collectively executes the instructions, the instructions cause the electronic device (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101) to: Obtain a connection request from a user to an external electronic device (102; 202; 802; 803; 1002; 1102; 1502; 1602; 2002; 2102) for sharing content, identify a first image (1400; 1800) for the content for sharing the content with the external electronic device (102; 202; 802; 803; 1002; 1102; 1502; 1602; 2002; 2102), and, based on a security level for sharing the content, generate a first area (1010; 1110; 1210; 1310; 1410) and a second area (1020; 1110; 1210; 1310; 1410) in the first image (1400; 1800). 1120; 1220; 1320; 1420) and generate a second image (1140; 1241; 1243; 1340) based on the second area (1020; 1120; 1220; 1320; 1420), and generate a third image (1130; 1230; 1330; 1430; 1530; 1630; 1730) including the second area (1020; 1120; 1220; 1320; 1420) of the first image (1400; 1800) and at least a part (1140; 1241; 1243; 1340) of the second image, using the external electronic device (102; 202; 802; To transmit to 803; 1002; 1102; 1502; 1602; 2002; 2102), Electronic devices (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101). In the first paragraph, When the at least one processor (120) individually or collectively executes the instructions, the instructions cause the electronic device (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101) to: An area including information having a sensitivity higher than that corresponding to the security level is determined as the first area (1010; 1110; 1210; 1310; 1410), and an area excluding the first area (1010; 1110; 1210; 1310; 1410) in the first image (1400; 1800) is determined as the second area (1020; 1120; 1220; 1320; 1420). Electronic devices (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101). In any one of paragraphs 1 and 2, When the at least one processor (120) individually or collectively executes the instructions, the instructions cause the electronic device (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101) to: Generating the second image (1140; 1241; 1243; 1340) based on at least a portion of the second area (1020; 1120; 1220; 1320; 1420) adjacent to the first area (1010; 1110; 1210; 1310; 1410). Electronic devices (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101). In any one of the first to third paragraphs, The third image above (1130; 1230; 1330; 1430; 1530; 1630; 1730) is The first region (1010; 1110; 1210; 1310; 1410) of the first image (1400; 1800) includes at least a part of the second image (1140; 1241; 1243; 1340). Electronic devices (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101). In any one of the first to fourth paragraphs, When the at least one processor (120) individually or collectively executes the instructions, the instructions cause the electronic device (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101) to: Identifying at least a part of the screen being displayed on the display module of the electronic device (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101) as the first image (1400; 1800), Electronic devices (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101). In any one of the first to fifth paragraphs, When the at least one processor (120) individually or collectively executes the instructions, the instructions cause the electronic device (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101) to: Identifying at least a part of the execution screen of at least one application running in the background by the electronic device (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101) as the first image (1400; 1800). Electronic devices (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101). In any one of claims 1 to 6, When the at least one processor (120) individually or collectively executes the instructions, the instructions cause the electronic device (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101) to: Determine the security level based on at least one of the protection pattern for information included in the first image (1400; 1800) and attribute information of the external electronic device (102; 202; 802; 803; 1002; 1102; 1502; 1602; 2002; 2102). Electronic devices (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101). In any one of the first to seventh paragraphs, The above protection pattern is, Including a sensitivity determined based on the user's sharing history for the above information; Electronic devices (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101). In any one of claims 1 to 8, The above property information is, At least one of the purpose of connection between the electronic device (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101) and the external electronic device (102; 202; 802; 803; 1002; 1102; 1502; 1602; 2002; 2102) and information related to the external electronic device (102; 202; 802; 803; 1002; 1102; 1502; 1602; 2002; 2102), Electronic devices (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101). In any one of claims 1 to 9, When the at least one processor (120) individually or collectively executes the instructions, the instructions cause the electronic device (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101) to: Displaying the third image (1130; 1230; 1330; 1430; 1530; 1630; 1730) on the display module, and distinguishingly displaying the first area (1010; 1110; 1210; 1310; 1410) replaced by the second area (1020; 1120; 1220; 1320; 1420) and the second image (1140; 1241; 1243; 1340). Electronic devices (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101). In a method of operating an electronic device (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101), An action of obtaining a connection request from a user to an external electronic device (102; 202; 802; 803; 1002; 1102; 1502; 1602; 2002; 2102) for sharing content; An operation of identifying a first image (1400; 1800) for the content for sharing the content with the external electronic device (102; 202; 802; 803; 1002; 1102; 1502; 1602; 2002; 2102); An operation of determining a first area (1010; 1110; 1210; 1310; 1410) and a second area (1020; 1120; 1220; 1320; 1420) in the first image (1400; 1800) based on a security level for sharing the content; An operation of generating a second image (1140; 1241; 1243; 1340) based on the second area (1020; 1120; 1220; 1320; 1420); and An operation of transmitting a third image (1130; 1230; 1330; 1430; 1530; 1630; 1730) including at least a portion of the second area (1020; 1120; 1220; 1320; 1420) of the first image (1400; 1800) and the second image (1140; 1241; 1243; 1340) to the external electronic device (102; 202; 802; 803; 1002; 1102; 1502; 1602; 2002; 2102) including, How it works. In Article 11, The operation of determining the first region (1010; 1110; 1210; 1310; 1410) and the second region (1020; 1120; 1220; 1320; 1420) in the first image (1400; 1800) is as follows: An area including information higher than the sensitivity corresponding to the security level is determined as the first area (1010; 1110; 1210; 1310; 1410), and an area excluding the first area (1010; 1110; 1210; 1310; 1410) in the first image (1400; 1800) is determined as the second area (1020; 1120; 1220; 1320; 1420). How it works. In any one of the 11th and 12th clauses, The operation of generating the above second image (1140; 1241; 1243; 1340) is as follows: Generating the second image (1140; 1241; 1243; 1340) based on at least a portion of the second area (1020; 1120; 1220; 1320; 1420) adjacent to the first area (1010; 1110; 1210; 1310; 1410). How it works. In any one of Articles 11 to 13, The third image above (1130; 1230; 1330; 1430; 1530; 1630; 1730) is The first region (1010; 1110; 1210; 1310; 1410) of the first image (1400; 1800) includes at least a part of the second image (1140; 1241; 1243; 1340). How it works. In any one of Articles 11 to 14, The operation of identifying the first image (1400; 1800) for the above content is as follows: Identifying at least a part of a screen being displayed on a display module of the electronic device (101; 201; 501; 601; 801; 1001; 1101; 1201; 1501; 2001; 2101) as the first image (1400; 1800), How it works.

Images