Image filtering method, electronic device and computer program

Abstract

To provide methods, systems, and non-transitory computer-readable recording media for filtering images using keywords extracted from the images.SOLUTION: An image filtering method implemented on an electronic device comprises: receiving a chat message including information on an image sent from another electronic device through a chat room of a messenger; and filtering an image corresponding to the chat message using a keyword extracted from the image.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The following description relates to techniques for filtering images. [Background technology]

[0002] Instant messenger, a common communication tool, is software that allows users to send and receive messages and data in real time. When a user registers a conversation partner on the messenger, he or she can exchange messages with the people registered in the conversation partner list in real time.

[0003] Such messenger functions are now widely used not only in personal computers (PCs) but also in the mobile environment of mobile communication terminals.

[0004] For example, Patent Document 1 (publication date: September 30, 2002) discloses a mobile messenger service system and method for a mobile terminal that uses a wireless communication network to enable a messenger service to be provided between mobile messengers installed on the mobile terminal.

[0005] Generally, messengers support a function for users to share various information such as photos, videos, files, contacts, locations, schedules, etc. in a chat room, and a function for searching conversation messages and shared information. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Korean Patent Publication No. 10-2002-0074304 Summary of the Invention [Problem to be solved by the invention]

[0007] You can filter images using keywords extracted from the images you want to share in the chat room. [Means for solving the problem]

[0008] Provided is an image filtering method executed by an electronic device, the electronic device including at least one processor configured to execute computer-readable instructions contained in a memory, the image filtering method including a step of receiving, by the at least one processor, a conversation message including information regarding an image sent from another electronic device in a messenger chat room, and a step of filtering, by the at least one processor, an image corresponding to the conversation message using keywords extracted from the image.

[0009] According to one aspect, the receiving step may include receiving a conversation message including the keyword related to the image from a messenger server.

[0010] According to another aspect, the keywords may be extracted by object detection targeting the image on the messenger server or on the other electronic device.

[0011] According to another aspect, the receiving step may include receiving a conversation message for the image from a messenger server, the conversation message including a thumbnail of the image, and the image filtering method may include extracting the keyword by object detection targeting the thumbnail by the at least one processor.

[0012] According to another aspect, the filtering may include restricting display of the image if the keyword matches a keyword set as a condition for image filtering.

[0013] According to another aspect, the filtering step may include, if the keyword matches a keyword set as a condition for image filtering, displaying the keyword without displaying a thumbnail of the image when displaying the conversation message in the chat room.

[0014] According to another aspect, the filtering step may include applying a filtering effect to a thumbnail of the image when displaying the conversation message in the chat room if the keyword matches a keyword set as a condition for image filtering.

[0015] According to another aspect, the filtering step may include a step of requesting user authentication for the image when displaying the conversation message in the chat room if the keyword matches a keyword set as a condition for image filtering.

[0016] According to another aspect, the image filtering method may include a step of, when a user request for the image is input to the chat room, downloading the image from a messenger server and recording it together with the keyword by the at least one processor.

[0017] According to another aspect, the image filtering method may include, when a keyword for entering a message or searching in the chat room is entered by the at least one processor, recommending an image from among the recorded images that corresponds to the entered keyword.

[0018] According to another aspect, the image filtering method may include immediately downloading and recording the image by the at least one processor if the keyword matches a predefined specific keyword.

[0019] According to yet another aspect, the filtering may include automatically hiding or deleting the image if the image corresponds to a particular type of content, regardless of the condition.

[0020] A computer program for causing a computer to execute the image filtering method, and a non-transitory computer-readable recording medium having the computer program recorded thereon are provided.

[0021] An electronic device is provided, comprising: a memory; and at least one processor communicatively connected to the memory and configured to execute computer-readable instructions contained in the memory, the at least one processor comprising: a message receiving unit that receives conversation messages including information regarding images sent from other electronic devices in a messenger chat room; and an image filtering unit that filters images corresponding to the conversation messages using keywords extracted from the images. [Brief description of the drawings]

[0022] [Figure 1] FIG. 1 illustrates an example of a network environment in accordance with an embodiment of the present invention. [Diagram 2] 2 is a block diagram illustrating the internal configuration of an electronic device and a server according to an embodiment of the present invention. FIG. [Diagram 3] FIG. 2 illustrates an example of a basic technical scenario for sending images in a messenger in accordance with an embodiment of the present invention. [Figure 4]FIG. 2 is a block diagram illustrating example components that may be included in a server processor in one embodiment of the present invention. [Diagram 5] 4 is a flow chart illustrating an example method that may be performed by a server in one embodiment of the present invention. [Figure 6] FIG. 2 is a block diagram illustrating example components that may be included in a processor of an electronic device in accordance with an embodiment of the present invention. [Figure 7] 4 is a flowchart illustrating an example of a method that can be performed by an electronic device in accordance with an embodiment of the present invention. [Figure 8] 4 is a flowchart illustrating an image filtering process according to an embodiment of the present invention. [Figure 9] 4 is a flowchart illustrating an image filtering process according to an embodiment of the present invention. [Figure 10] 4 is a flowchart illustrating an image filtering process according to an embodiment of the present invention. [Figure 11] FIG. 13 is a diagram showing an example of a chat room screen that restricts the display of images in one embodiment of the present invention. [Figure 12] FIG. 13 is a diagram showing an example of a chat room screen where image display is restricted in one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0024] SUMMARY OF THE DISCLOSURE The present invention relates to techniques for filtering images.

[0025] Embodiments including those specifically disclosed in this specification can automatically filter images using keywords extracted from images, thereby achieving significant advantages in aspects such as image regulation, image management, convenience, efficiency, and cost savings.

[0026] Fig. 1 is a diagram showing an example of a network environment in one embodiment of the present invention. The network environment in Fig. 1 shows an example including a plurality of electronic devices 110, 120, 130, 140, a plurality of servers 150, 160, and a network 170. Fig. 1 is merely an example for explaining the invention, and the number of electronic devices and the number of servers are not limited to those shown in Fig. 1.

[0027] The electronic devices 110, 120, 130, and 140 may be fixed terminals or mobile terminals realized by computer systems. Examples of the electronic devices 110, 120, 130, and 140 include smartphones, mobile phones, navigation systems, personal computers (PCs), notebook PCs, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), tablets, game consoles, wearable devices, internet of things (IoT) devices, virtual reality (VR) devices, and augmented reality (AR) devices. As an example, FIG. 1 shows a smartphone as an example of the electronic device 110, but in the embodiment of the present invention, the electronic device 110 may mean one of various physical computer systems that can communicate with other electronic devices 120, 130, and 140 and / or servers 150 and 160 via a network 170 using a substantially wireless or wired communication method.

[0028] The communication method is not limited, and may include not only a communication method using a communication network (for example, a mobile communication network, a wired Internet, a wireless Internet, a broadcast network, a satellite network, etc.) that the network 170 can include, but also short-range wireless communication between devices. For example, the network 170 may include any one or more of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, etc. Furthermore, the network 170 may include any one or more of a network topology including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, etc.

[0029] Each of the servers 150 and 160 may be realized by one or more computer devices that communicate with the multiple electronic devices 110, 120, 130, and 140 via the network 170 to provide instructions, codes, files, contents, services, and the like. For example, the server 150 may be a system that provides a first service to the multiple electronic devices 110, 120, 130, and 140 connected via the network 170, and the server 160 may be a system that provides a second service to the multiple electronic devices 110, 120, 130, and 140 connected via the network 170. As a more specific example, the server 150 may provide a service (e.g., a messenger service, for example) targeted by an application, which is a computer program installed and executed in the multiple electronic devices 110, 120, 130, and 140, as a first service to the multiple electronic devices 110, 120, 130, and 140. As another example, the server 160 may provide, as a second service, a service of distributing files for installing and executing the above-mentioned application to the multiple electronic devices 110, 120, 130, and 140.

[0030] Fig. 2 is a block diagram for explaining the internal configuration of an electronic device and a server in an embodiment of the present invention. In Fig. 2, the internal configuration of electronic device 110 and the internal configuration of server 150 are explained as examples of electronic devices. In addition, other electronic devices 120, 130, 140 and server 160 may have the same or similar internal configuration as electronic device 110 or server 150 described above.

[0031] The electronic device 110 and the server 150 may include memories 211 and 221, processors 212 and 222, communication modules 213 and 223, and input / output interfaces 214 and 224. The memories 211 and 221 may be non-transitory computer-readable recording media and may include persistent mass storage devices such as random access memory (RAM), read only memory (ROM), disk drives, solid state drives (SSD), flash memories, and the like. Here, persistent mass storage devices such as ROM, SSD, flash memories, and disk drives may be included in the electronic device 110 and the server 150 as persistent storage devices separate from the memories 211 and 221. In addition, the memories 211 and 221 may store an operating system and at least one program code (for example, a code for a browser installed and executed in the electronic device 110, an application installed in the electronic device 110 to provide a specific service, and the like). Such software components may be loaded from a computer-readable recording medium separate from the memories 211 and 221. Such another computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. In another embodiment, the software components may be loaded into the memory 211, 221 through a communication module 213, 223 that is not a computer-readable recording medium. For example, at least one program may be loaded into the memory 211, 221 based on a computer program (for example, the above-mentioned application) that is installed by a file provided via the network 170 by a developer or a file distribution system that distributes an installation file of the application (for example, the above-mentioned server 160).

[0032] The processors 212, 222 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to the processors 212, 222 by the memory 211, 221 or the communication modules 213, 223. For example, the processors 212, 222 may be configured to execute instructions received according to program code stored in a storage device such as the memory 211, 221.

[0033] The communication modules 213 and 223 may provide a function for the electronic device 110 and the server 150 to communicate with each other via the network 170, or may provide a function for the electronic device 110 and / or the server 150 to communicate with other electronic devices (for example, the electronic device 120) or other servers (for example, the server 160). For example, a request generated by the processor 212 of the electronic device 110 according to a program code recorded in a recording device such as the memory 211 may be transmitted to the server 150 via the network 170 under the control of the communication module 213. Conversely, a control signal, an instruction, a content, a file, etc. provided under the control of the processor 222 of the server 150 may be received by the electronic device 110 through the communication module 213 of the electronic device 110 via the communication module 223 and the network 170. For example, control signals, instructions, content, files, etc. from the server 150 received through the communication module 213 may be transmitted to the processor 212 or memory 211, and the content, files, etc. may be recorded on a recording medium (the above-mentioned permanent recording device) that the electronic device 110 may further include.

[0034] The input / output interface 214 may be a means for interfacing with the input / output device 215. For example, the input device may include a device such as a keyboard, a mouse, a microphone, or a camera, and the output device may include a device such as a display, a speaker, or a haptic feedback device. As another example, the input / output interface 214 may be a means for interfacing with a device in which functions for input and output are integrated into one, such as a touch screen. The input / output device 215 may be configured as one device together with the electronic device 110. Also, the input / output interface 224 of the server 150 may be a means for interfacing with a device for input or output (not shown) that may be connected to the server 150 or included in the server 150. As a more specific example, when the processor 212 of the electronic device 110 processes the instructions of the computer program loaded in the memory 211, a service screen or content configured using data provided by the server 150 or the electronic device 120 may be displayed on a display through the input / output interface 214.

[0035] Also, in other embodiments, the electronic device 110 and the server 150 may include more components than those in FIG. 2. However, it is not necessary to clearly show most of the conventional technical components in the figure. For example, the electronic device 110 may be realized to include at least some of the above-mentioned input / output devices 215, and may further include other components such as a transceiver, a GPS (Global Positioning System) module, a camera, various sensors, a database, etc. As a more specific example, if the electronic device 110 is a smartphone, various components such as an acceleration sensor, a gyro sensor, a camera module, various physical buttons, a button using a touch panel, an input / output port, a vibrator for vibration, etc., which are generally included in a smartphone, may be realized to be further included in the electronic device 110.

[0036] FIG. 3 illustrates an example of a basic technical scenario for sending images in a messenger in one embodiment of the present invention.

[0037] Figure 3 shows an example of the process by which users exchange images using a messenger.

[0038] 1. A user (User A) of electronic device 110 runs Messenger installed on electronic device 110 and then selects an image that he or she would like to share with a user (User B) of electronic device 120 in a chat room with User B. The chat room is a Messenger conversation screen that corresponds to a communication session established between User A's account and User B's account.

[0039] 2. In response to the file transmission request from the messenger, the electronic device 110 transmits the image selected by the user A to the server 150.

[0040] 3. The server 150 receives the image to be sent to the user B from the electronic device 110 and records it in a file DB (for example, the memory 221).

[0041] 4. In the chat room with user A on the messenger, the electronic device 120 downloads the image sent by user A from the server 150 in response to a request from user B.

[0042] 5. Server 150 temporarily stores the image sent from user A to user B, but once user B has downloaded it, it deletes the image from the file DB after a certain period of time.

[0043] The server 150 does not immediately send the image file sent by user A to user B, but first transmits an image type message including a thumbnail of the image as a conversation message for the image, and downloads the image file to the electronic device 120 when requested by user B.

[0044] Specific embodiments of methods and systems for filtering images are described below.

[0045] In this specification, the term "image" may refer to all image-based content, including still images such as photographs as well as video.

[0046] FIG. 4 is a block diagram illustrating example components that may be included in a server processor in one embodiment of the present invention, and FIG. 5 is a flowchart illustrating an example method that may be performed by a server in one embodiment of the present invention.

[0047] The server 150 according to the present embodiment serves as a platform that provides a messenger service to a plurality of electronic devices 110, 120, 130, and 140 that are clients. The server 150 may provide the messenger service in conjunction with an application installed on the electronic devices 110, 120, 130, and 140.

[0048] The server 150 may execute the image filtering method shown in Fig. 5. To this end, the processor 222 of the server 150 may include, as components, an image receiving unit 410, a keyword extracting unit 420, and a message transmitting unit 430, as shown in Fig. 4. Depending on the embodiment, the components of the processor 222 may be selectively included or excluded from the processor 222. Also, depending on the embodiment, the components of the processor 222 may be separated or combined for expressing the functions of the processor 222.

[0049] Such processor 222 and components of processor 222 may control server 150 to execute steps 510 to 540 included in the image filtering method of Fig. 5. For example, processor 222 and components of processor 222 may be realized to execute instructions according to operating system code and at least one program code included in memory 221.

[0050] Here, the components of the processor 222 may represent different functions of the processor 222 that are executed by the processor 222 according to instructions provided by program code recorded on the server 150. For example, the image receiving unit 410 may be used as a functional representation of the processor 222 that controls the server 150 according to the instructions described above so that the server 150 receives images that the server 150 wishes to share in a chat room.

[0051] In step 510, the processor 222 may read necessary instructions from the memory 221 in which instructions related to the control of the server 150 are loaded. In this case, the read instructions may include instructions for controlling the processor 222 to execute steps 520 to 540 described below.

[0052] In operation 520, the image receiving unit 410 may receive an image from the electronic device 110 that the user of the electronic device 110 wishes to share in a chat room. The user of the electronic device 110 uploads an image that the user wishes to share with other users to the server 150 through the chat room.

[0053] In operation 530, the keyword extraction unit 420 may extract keywords associated with the image from the image received from the electronic device 110. For example, the keyword extraction unit 420 may extract keywords associated with the image by recognizing an object included in the image through object detection and acquiring the name of the object. The keyword extraction unit 420 may include a machine learning engine (e.g., TensorFlow, etc.) and apply an object detection algorithm based on a deep learning model previously trained by a learning image set. When an image received from the electronic device 110 is input, the keyword extraction unit 420 may find an object included in the corresponding image based on an object detection algorithm based on a deep learning model and extract features of the corresponding object. In this case, the keyword extraction unit 420 may extract features of the object detected from the image by applying a learning model modeled based on various attributes (e.g., color, texture, category, etc.) that can classify objects. In addition to the above-mentioned method, keywords may be extracted from an image by extracting features included in the image using at least one of well-known image recognition algorithms, for example, scale invariant feature transform (SIFT), speeded up robust features (SURF), histogram of oriented gradient (HOG), modified census transform (MCT), etc., and calculating the similarity between the extracted features and features of an object preregistered in a database on the server 150 (for example, the memory 221) or another database that can be linked to the server 150. In addition to extracting keywords, it is also possible to classify a particular type of content, such as content that violates public order and morals, based on the image analysis results.

[0054] In step 540, the message transmitting unit 430 may transmit a conversation message including the keyword extracted in step 530 for the image received from the electronic device 110 to the electronic device 120 of another user through the chat room. When the user of the electronic device 110 uploads an image to the chat room, the server 150 may record the image as data related to the chat room, and may also record the keyword extracted from the image. In this case, the message transmitting unit 430 may transmit a conversation message including information about the image uploaded by the user of the electronic device 110 to the other user. In other words, the message transmitting unit 430 does not immediately transmit the image file, but first transmits an image type message including information about the image. In particular, when transmitting the image type message, the message transmitting unit 430 may also transmit the keyword extracted from the image. In this case, in the case of an image type message, the keyword extracted from the image may be included, or a thumbnail of the image may be included together with the keyword extracted from the image.

[0055] Therefore, the server 150 can extract keywords from an image to be shared in a chat room and transmit a conversation message including the extracted keywords to the chat room.

[0056] FIG. 6 is a block diagram illustrating an example of components that may be included in a processor of an electronic device in one embodiment of the present invention, and FIG. 7 is a flowchart illustrating an example of a method that may be performed by an electronic device in one embodiment of the present invention.

[0057] An image filtering system implemented by a computer may be configured in the electronic device 110 according to the present embodiment. For example, the image filtering system may be implemented in the form of a program that operates independently, or may be implemented in the form of an in-app of a specific application so as to be operable on the specific application, and may perform image filtering in conjunction with the server 150.

[0058] Based on instructions provided by an application installed on the electronic device 110, the image filtering system implemented in the electronic device 110 may execute an image filtering method. To execute the image filtering method shown in FIG. 7, the processor 212 of the electronic device 110 may include, as components, a filtering setting unit 610, a message receiving unit 620, and an image filtering unit 630, as shown in FIG. 6. In some embodiments, the components of the processor 212 may be selectively included or excluded in the processor 212. In some embodiments, the components of the processor 212 may be separated or combined to represent the functions of the processor 212.

[0059] Such processor 212 and components of processor 212 may control electronic device 110 to execute steps 710 to 740 included in the image filtering method of Fig. 7. For example, processor 212 and components of processor 212 may be implemented to execute instructions according to operating system code and at least one program code included in memory 211.

[0060] Here, the components of the processor 212 may represent different functions of the processor 212 that are executed by the processor 212 according to instructions provided by a program code recorded in the electronic device 110 (for example, instructions provided by an application executed by the electronic device 110). For example, the filtering setting unit 610 may be used as a functional representation of the processor 212 that controls the electronic device 110 according to the above-mentioned instructions so that the electronic device 110 sets conditions for image filtering.

[0061] In step 710, the processor 212 may read necessary instructions from the memory 211 loaded with instructions related to the control of the electronic device 110. In this case, the read instructions may include instructions for controlling the processor 212 to execute steps 720 to 740 described below.

[0062] In operation 720, the filtering setting unit 610 may set a keyword list including at least one keyword as a condition for image filtering. The keyword list for image filtering may be determined by a user setting of the electronic device 110, and may further include a keyword determined as a default value based on information (e.g., age, etc.) related to the user of the electronic device 110. In addition, the keyword list may be set collectively for all chat rooms of the messenger installed in the electronic device 110, or may be set as an individual condition for each chat room.

[0063] In operation 730, the message receiving unit 620 may receive a conversation message sent by another user in a chat room in which the user of the electronic device 110 is included, and if the received message is an image type message including information about an image sent by another user, the message may also receive keywords extracted from the image. In other words, the message receiving unit 620 receives a conversation message including keywords extracted from an image sent to the user of the electronic device 110 in a chat room from the server 150. In this case, the conversation message may further include a thumbnail of the image sent to the user of the electronic device 110.

[0064] In step 740, in the process of displaying conversation messages exchanged by the user of the electronic device 110 in the chat room, if the message is an image type message, the image filtering unit 630 may restrict the display of the image in the chat room if a keyword received together with the message is included in a keyword list. The image filtering unit 630 may record a keyword extracted from the image for each image type message in a local storage space (e.g., the memory 211), and then perform a matching process between the recorded keyword and a keyword set as a condition for image filtering, and perform image filtering if they match. As an example, the image filtering unit 630 may indicate that the image received in the chat room is an image belonging to a filtering target by displaying only the keyword included in the image type message, i.e., the keyword extracted from the image, in the chat room. As another example, when the image filtering unit 630 displays a thumbnail of the image when displaying the image type message in the chat room, the image filtering unit 630 may indicate that the image received in the chat room is an image belonging to a filtering target by applying a filtering effect (e.g., a blur process, etc.) that restricts the image display to the thumbnail. As another example, the image filtering unit 630 may indicate that an image received in a chat room is an image that belongs to a filtering target by requesting a predetermined authentication (e.g., adult authentication or password authentication) when displaying an image-type message in a chat room. When a user request is input or authentication is completed for an image to be filtered, the image filtering unit 630 may request the server 150 to download the image and record it locally. At this time, the image filtering unit 630 may record keywords extracted from the image for each downloaded image.

[0065] Furthermore, if an image transmitted to the user of the electronic device 110 corresponds to content that violates public order and morals, the image filtering unit 630 may automatically hide or delete the image regardless of the filtering conditions. Meanwhile, in contrast to restricting image display, if an image transmitted to the user of the electronic device 110 matches a specific keyword previously set by the user, the image filtering unit 630 may immediately download and automatically record the image locally without a user request. In addition, for accessibility to visually impaired people, if an image type message is received in a chat room, the image filtering unit 630 may output a keyword extracted from the image by voice using TTS (text to speech). Furthermore, if a keyword is input in a message input window or a search window of the chat room, the image filtering unit 630 may automatically recommend an image corresponding to the input keyword among images downloaded and recorded in the chat room.

[0066] Therefore, on the client side where Messenger is installed, image filtering can be performed on images exchanged in Messenger chat rooms by using keywords extracted from the images.

[0067] The entity that extracts keywords associated with an image may be the server 150 as described above, but in some cases, the originating client or the receiving client may be the entity that extracts keywords.

[0068] 8 to 10 are flow charts showing an image filtering process according to an embodiment of the present invention. The description will be given on the premise that a user A of the electronic device 110 sends an image to a user B of the electronic device 120.

[0069] FIG. 8 shows an image filtering process in which the server 150 is the subject of keyword extraction.

[0070] When user A selects an image that he or she wishes to share in a chat room with user B, electronic device 110 may upload the selected image to server 150 (S81).

[0071] When the image is received from the electronic device 110, the server 150 may extract keywords associated with the image from the received image (S82).

[0072] The server 150 may record the image received from the electronic device 110 together with the keywords extracted from the image (S83).

[0073] The server 150 may transmit an image type message as a conversation message for the image received from the electronic device 110, i.e., the image sent by user A, to the electronic device 120 of user B through the corresponding chat room (S84).

[0074] The image-type message transmitted to the electronic device 120 may include keywords extracted from the image, or may include a thumbnail of the image along with the keywords extracted from the image.

[0075] In the process of displaying conversation messages exchanged with user A in a chat room with user A, if the message is an image type message, the electronic device 120 may perform image filtering in the chat room using keywords extracted from the image (S85).

[0076] FIG. 9 shows an image filtering process in which the electronic device 110, which is a sending client, is the subject of keyword extraction.

[0077] When user A selects an image that he or she wishes to share in a chat room with user B, electronic device 110 may extract keywords associated with the image from the selected image (S91).

[0078] The electronic device 110 may provide the keywords extracted from the image to the user A through an edit screen before uploading the image to the server 150. At this time, the user A may check the keywords extracted from the image on the edit screen, and may directly edit the keywords, such as modifying, adding, or deleting, if necessary.

[0079] The electronic device 110 may locally record the image selected by user A and the associated keywords (the extracted keywords or the edited keywords), and then upload them to the server 150 (S92).

[0080] The server 150 may record both the image and the keyword received from the electronic device 110 (S93).

[0081] The server 150 may transmit an image type message as a conversation message for the image received from the electronic device 110, i.e., the image sent by user A, to the electronic device 120 of user B through the corresponding chat room (S94).

[0082] The image-type message transmitted to the electronic device 120 may include keywords extracted from the image, or may include a thumbnail of the image along with the keywords extracted from the image.

[0083] In the process of displaying conversation messages exchanged with user A in a chat room with user A, if the message is an image type message, the electronic device 120 may perform image filtering in the chat room using keywords extracted from the image (S95).

[0084] FIG. 10 shows an image filtering process in which the electronic device 120, which is a receiving client, is the subject of keyword extraction.

[0085] When user A selects an image that he or she wishes to share in a chat room with user B, electronic device 110 may upload the selected image to server 150 (S101).

[0086] When the image is received from the electronic device 110, the server 150 may record the received image (S102).

[0087] The server 150 may transmit an image type message as a conversation message for the image received from the electronic device 110, i.e., the image sent by user A, to the electronic device 120 of user B through the corresponding chat room (S103).

[0088] An image type message communicated to the electronic device 120 may include a thumbnail of the image.

[0089] In the process of displaying conversation messages exchanged with user A in a chat room with user A, if the message is an image type message, the electronic device 120 may extract keywords associated with the image from the thumbnail image (S104).

[0090] The electronic device 120 may locally record the thumbnail image and keywords of the image sent by user A, and then perform image filtering in the corresponding chat room using the keywords extracted from the thumbnail image (S105).

[0091] The subject that extracts keywords related to an image is not limited to the server 150. If a sending client or a receiving client includes a learning model for keyword extraction, the sending client or the receiving client can be used as the subject for keyword extraction.

[0092] In the receiving client, if a keyword extracted from an image is included in a keyword list set as a filtering condition, the display of the image in the chat room may be restricted.

[0093] Referring to FIG. 11, the electronic device 120 displays conversation messages exchanged with user A in a chat room 1100 with user A. At this time, if user A sends an image, the image display may be restricted by displaying an image type message 1110 with only keywords extracted from the image, without displaying a thumbnail.

[0094] 12, when the electronic device 120 displays an image type message 1220 using a thumbnail image in a chat room 1200 with user A, the electronic device 120 may limit the image display by blurring the thumbnail. At this time, the electronic device 120 may display what kind of object the image is, as well as keywords extracted from the image.

[0095] Even if the image display is restricted and the image is a filtered image, when user B inputs a download menu 1111, 1221 associated with the image type message 1110, 1220, the electronic device 120 can make a request to the server 150 to download the image corresponding to the image type message 1110, 1220 and record it locally.

[0096] In this manner, according to an embodiment of the present invention, images that are desired to be shared in a chat room can be automatically filtered using keywords extracted from the images.

[0097] The above-mentioned devices may be realized by hardware components, software components, and / or a combination of hardware and software components. For example, the devices and components described in the embodiments may be realized using one or more general-purpose or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or various devices capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications that run on the OS. The processing device may also respond to the execution of the software to access, record, manipulate, process, and generate data. For convenience of understanding, a single processing device may be described as being used, but those skilled in the art will understand that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, the processing device may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.

[0098] The software may include computer programs, codes, instructions, or a combination of one or more of these, and may configure a processing device to operate as desired or instruct the processing device, either independently or collectively. The software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device to be interpreted by the processing device or to provide instructions or data to the processing device. The software may be distributed and stored or executed in a distributed manner on computer systems connected by a network. The software and data may be stored on one or more computer-readable storage media.

[0099] The method according to the embodiment may be realized in the form of program instructions executable by various computer means and recorded on a computer-readable medium. Here, the medium may be one that continuously records a computer-executable program or one that temporarily records the program for execution or download. The medium may be one of various recording means or storage means in the form of a single or multiple hardware devices combined, and may not be limited to a medium directly connected to a certain computer system, but may be one that is distributed over a network. Examples of the medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and ROMs, RAMs, flash memories, etc., configured to record program instructions. Other examples of the medium include recording media or storage media managed by application stores that distribute applications, or sites, servers, etc. that supply or distribute various other software.

[0100] Although the embodiments have been described based on the limited embodiments and drawings as above, those skilled in the art will be able to make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from that described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a manner different from that described, or may be counterbalanced or replaced by other components or equivalents, and still achieve suitable results.

[0101] Therefore, different embodiments are within the scope of the appended claims provided they are equivalent to the claims. [Explanation of symbols]

[0102] 212: Processor 610: Filtering setting section 620: Message receiving unit 630: Image filtering section

Claims

1. 1. An image filtering method implemented by an electronic device, comprising: receiving, from the server, a conversation message including a keyword extracted from the image and a thumbnail of the image, in place of the image sent by another electronic device in a chat room of the messenger to the server of the messenger; displaying only the keyword extracted from the image on the chat room screen when the keyword extracted from the image matches a keyword set as a condition for image filtering; The image filtering method includes:

2. 1. An image filtering method implemented by an electronic device, comprising: receiving, from the server, a conversation message including a thumbnail of an image instead of the image sent by another electronic device in a chat room of a messenger to the server of the messenger; obtaining keywords extracted from the images by object detection on thumbnails of the images; and displaying only the keyword extracted from the image on the chat room screen when the keyword extracted from the image matches a keyword set as a condition for image filtering; The image filtering method includes:

3. 1. An image filtering method implemented by an electronic device, comprising: receiving, from the server, a conversation message including a keyword extracted from the image and a thumbnail of the image, in place of the image sent by another electronic device in a chat room of the messenger to the server of the messenger; displaying, on a screen of the chat room, a thumbnail of the image to which a predetermined filtering effect has been applied and the keyword extracted from the image when the keyword extracted from the image matches a keyword set as a condition for image filtering. The image filtering method includes:

4. 1. An image filtering method implemented by an electronic device, comprising: receiving, from the server, a conversation message including a thumbnail of an image instead of the image sent by another electronic device in a chat room of a messenger to the server of the messenger; obtaining keywords extracted from the images by object detection on thumbnails of the images; and displaying, on a screen of the chat room, a thumbnail of the image to which a predetermined filtering effect has been applied and the keyword extracted from the image when the keyword extracted from the image matches a keyword set as a condition for image filtering. The image filtering method includes:

5. The step of displaying further includes displaying a download menu on the screen of the chat room, when a user request for the image is input to the chat room by inputting the download menu, downloading the image from the messenger server and recording the image together with keywords extracted from the image. The image filtering method according to any one of claims 1 to 4, comprising:

6. 1. An image filtering method implemented by an electronic device, comprising: receiving, from the server, a conversation message including a keyword extracted from an image instead of the image sent by another electronic device in a chat room of a messenger to the server of the messenger; requesting user authentication for the image if the keyword extracted from the image matches a keyword set as a condition for image filtering. The image filtering method includes: a step of downloading the image from a server of the messenger and recording the image together with keywords extracted from the image when the user authentication is completed.

7. The image filtering method of claim 6, comprising:

8. recommending an image corresponding to a keyword input for message input or search on the chat room screen from among the recorded images when the keyword is input.

8. The image filtering method according to claim 5 or 7, comprising:

9. The keywords extracted from the image are extracted by object detection on the image on the messenger server or on the other electronic device.

8. An image filtering method according to claim 1, 3, 6 or 7.

10. if the keyword extracted from the image matches a predefined specific keyword, immediately downloading and recording the image from the messenger server; The image filtering method according to any one of claims 1 to 9, comprising:

11. A computer program that causes the electronic device to execute the image filtering method according to any one of claims 1 to 10.

12. An electronic device, means for receiving, after another electronic device transmits an image to the messenger server in a chat room of the messenger, a conversation message from the server, the conversation message including, in place of the image, a keyword extracted from the image and a thumbnail of the image; A means for displaying only the keyword extracted from the image on the chat room screen when the keyword extracted from the image matches a keyword set as a condition for image filtering. Including electronic devices.

13. An electronic device, means for receiving, after another electronic device in a messenger chat room has sent an image to the messenger server, a conversation message from the server including a thumbnail of the image in place of the image; means for obtaining keywords extracted from the image by object detection on thumbnails of the image; and A means for displaying only the keyword extracted from the image on the chat room screen when the keyword extracted from the image matches a keyword set as a condition for image filtering. Including electronic devices.

14. An electronic device, means for receiving, after another electronic device transmits an image to the messenger server in a chat room of the messenger, a conversation message from the server, the conversation message including, in place of the image, a keyword extracted from the image and a thumbnail of the image; A means for displaying, on a screen of the chat room, a keyword extracted from the image and a thumbnail of the image to which a predetermined filtering effect has been applied, when the keyword extracted from the image matches a keyword set as a condition for image filtering. Including electronic devices.

15. An electronic device, means for receiving, after another electronic device in a messenger chat room has sent an image to the messenger server, a conversation message from the server including a thumbnail of the image in place of the image; means for obtaining keywords extracted from the image by object detection on thumbnails of the image; and A means for displaying, on a screen of the chat room, a keyword extracted from the image and a thumbnail of the image to which a predetermined filtering effect has been applied, when the keyword extracted from the image matches a keyword set as a condition for image filtering. Including electronic devices.

16. An electronic device, A means for receiving, after another electronic device transmits an image to a server of the messenger in a chat room of the messenger, a conversation message including a keyword extracted from the image in place of the image from the server; a means for requesting user authentication for the image when a keyword extracted from the image matches a keyword set as a condition for image filtering; Including electronic devices.

Images