System

Abstract

A system is provided.SOLUTION: A system comprising: means for uploading an image; means for analyzing a background from the uploaded image to extract location-specific risk factors; means for converting the extracted risk factors into alternative natural background elements; means for generating a plurality of candidate images that have been converted; means for presenting to a user; and means for generating a final image upon user selection and confirmation.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] It is widely recognized that when posting images to social networking sites, there is a risk that the poster's location may be identified from the background of the image. If the poster's location is identified, there is a risk of stalking or other privacy violations. This risk is particularly important for users who use social networking sites on a daily basis. However, conventional methods such as painting over or pixelating the background detract from the beauty and naturalness of the image, making it difficult to post images that look unnatural. Therefore, there is a need for technology that can change the background in a natural way while reducing the risk of location identification. [Means for solving the problem]

[0005] The present invention first provides a means for users to upload images. Next, it provides a means for analyzing the background of the uploaded image and extracting risk elements that could lead to location identification. This provides a means for converting the extracted risk elements into other natural background elements. It also provides a means for generating multiple candidate images and presenting them to the user. The user selects and confirms the presented candidate images, and through the means for generating the final image, it becomes possible to post safe and natural images to SNS. In addition, in the case of videos, it provides a means for performing similar conversions while maintaining the continuity of the video frames, thereby realizing natural and safe video posting. Furthermore, it also includes a means for responding to reconversion instructions from the user, allowing for flexible changes.

[0006] "Means for uploading images" refers to a function that allows a user to select image data from their own device and send it to a server via a network.

[0007] "Means for analyzing the background and extracting risk elements that identify a location" refers to a function that analyzes uploaded images using an AI module, etc., and detects elements that can lead to the identification of a location, such as buildings and signs.

[0008] The "means for converting risk elements into other natural background elements" is a function that removes detected risk elements and synthesizes a different background image in their place to maintain a natural appearance.

[0009] The "means for generating a plurality of candidate images and presenting them to the user" is a function for creating a plurality of variations in which a risk element is replaced with another element and displaying the options to the user.

[0010] The "means for generating a final image upon user selection and confirmation" is a function in which the user selects one of the candidate images presented and generates the final image to be posted based on that selection.

[0011] "Means for converting into natural background elements while maintaining the continuity of video frames" refers to a function that changes the background for each frame in a video without losing continuity, thereby generating a natural image.

[0012] The "means for enabling re-conversion at the user's instruction" is a function that allows the user to instruct a different conversion again for a candidate image or video that has already been generated. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] This invention relates to a system that changes the background of images and videos posted to social networking sites in a natural way to reduce the risk of location identification. The program processing of this system is specifically explained below in natural language.

[0035] System Overview

[0036] The system is mainly composed of three entities: the server, the device, and the user. The server is responsible for the main processes, such as image and video analysis, background conversion, candidate generation, and final output generation. The device provides the user interface, uploads images and videos, and displays the results. The user uploads images and videos through the device and selects the best one from the converted candidates.

[0037] Program processing

[0038] 1. Upload an image

[0039] Users select an image from their device and click the upload button.

[0040] The terminal transmits the selected image data to the server.

[0041] 2. Image Analysis

[0042] The server passes the received image data to the AI ​​module.

[0043] The AI ​​module analyzes the images and detects elements that could lead to location identification, such as building shapes, address plates, and signs.

[0044] 3. Risk Assessment

[0045] The AI ​​module assesses the location risk based on the detected elements and assigns a risk level to each element.

[0046] 4. Background conversion

[0047] The server extracts risk elements and generates a background image that replaces them with other natural elements.

[0048] Example: In a photo that shows Tokyo Tower, replace the Tokyo Tower with natural scenery or an ordinary apartment building.

[0049] 5. Candidate Generation and Presentation

[0050] The server generates multiple candidate conversion images and transmits them to the terminal for presentation to the user.

[0051] The terminal displays the received candidate images to the user and provides a selection screen.

[0052] 6. User Choice

[0053] The user checks the presented candidate images and selects the appropriate one.

[0054] If necessary, the user can also instruct reconversion.

[0055] 7. Generating the final image

[0056] The server finalizes the selected candidate images and sends them to the user.

[0057] The device displays the final image to the user for final confirmation.

[0058] 8. Submission

[0059] Users can then post the final image to social media.

[0060] Application to video

[0061] The same process is applied to videos, but the same process is performed on each frame while maintaining continuity, resulting in a natural background transition for the entire video.

[0062] 1. Upload a video

[0063] The user selects a video from their device and clicks the upload button.

[0064] The device transmits the video file to the server.

[0065] 2. Frame analysis and transformation

[0066] The server receives the video data and inputs each frame into the AI ​​module.

[0067] The AI ​​module analyzes each frame, detects elements that pose a risk of location identification, and transforms them into natural backgrounds.

[0068] 3. Maintaining continuity

[0069] The server performs conversion processing that takes into account the continuity of frames in order to generate images that do not look out of place between consecutive frames.

[0070] 4. Final video generation

[0071] The server generates the final video and sends it to the device.

[0072] The device will then display the final video to the user and ask for confirmation.

[0073] 5. Post

[0074] Users can then post the final video to social media.

[0075] This makes it possible to provide a system that reduces the risk of location identification and allows users to safely post images and videos with natural-looking backgrounds to social media.

[0076] The processing flow will be explained below.

[0077] Step 1:

[0078] Users select the image they want to upload from their device and click the upload button.

[0079] Step 2:

[0080] The terminal transmits the selected image data to the server.

[0081] Step 3:

[0082] The server receives the uploaded image data.

[0083] Step 4:

[0084] The server passes the image to the AI ​​module and begins analysis.

[0085] Step 5:

[0086] The AI ​​module detects elements in the image that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[0087] Step 6:

[0088] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[0089] Step 7:

[0090] The server extracts risk elements and generates a background image that replaces them with other natural background elements.

[0091] Step 8:

[0092] The server generates a number of candidate transformed images.

[0093] Step 9:

[0094] The server sends the generated candidate images to the user's terminal.

[0095] Step 10:

[0096] The terminal displays the received candidate images to the user and provides a confirmation screen.

[0097] Step 11:

[0098] The user reviews the different candidate images presented and selects the best one.

[0099] Step 12:

[0100] If the user desires reconversion, he or she inputs the instruction into the terminal.

[0101] Step 13:

[0102] The server receives the user's instruction for reconversion, generates candidate images again, and sends them to the terminal.

[0103] Step 14:

[0104] The user makes the final selection and sends confirmation to the device.

[0105] Step 15:

[0106] The server finalizes the candidate images selected by the user to generate the final image.

[0107] Step 16:

[0108] The server sends the final image to the user's device.

[0109] Step 17:

[0110] The terminal displays the final image to the user and provides an interface for final confirmation.

[0111] Step 18:

[0112] The user then makes a final confirmation and enters instructions into the device to post to social media.

[0113] Step 19:

[0114] The device retrieves the final image from the server and posts it to a social media platform.

[0115] The above is the flow of image processing.

[0116] Application to video

[0117] Step 1:

[0118] Users select the video they want to upload from their device and click the upload button.

[0119] Step 2:

[0120] The terminal transmits the selected video data to the server.

[0121] Step 3:

[0122] The server receives the uploaded video data.

[0123] Step 4:

[0124] The server passes each frame of the video to the AI ​​module and begins analysis.

[0125] Step 5:

[0126] The AI ​​module detects elements in each frame that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[0127] Step 6:

[0128] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[0129] Step 7:

[0130] The server extracts risk elements for each frame and generates a background image that replaces them with other natural background elements.

[0131] Step 8:

[0132] The server performs the conversion process so that there is no sense of incongruity between consecutive frames, and generates the final video.

[0133] Step 9:

[0134] The server sends the final video to the user's device.

[0135] Step 10:

[0136] The device displays the final video to the user and provides a confirmation screen.

[0137] Step 11:

[0138] The user reviews the final video and enters instructions into the device to post it to social media.

[0139] Step 12:

[0140] The device retrieves the final video from the server and posts it to a social media platform.

[0141] Example 1

[0142] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0143] When posting images or videos to social networking sites, there is a risk that elements such as buildings, address plates, and signs that appear in the background may be identified by a third party. This risk can lead to a violation of privacy, making it difficult for posters to use social networking sites safely. Furthermore, in the case of videos, background conversion technology is also required to prevent the appearance of incongruity between consecutive frames. Therefore, a system is needed that can convert backgrounds to natural ones while reducing the risk of location identification.

[0144] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0145] In this invention, the server includes means for uploading images or videos, means for analyzing the background from each frame of the uploaded image or video and extracting risk elements for location identification, means for converting the extracted risk elements into other natural background elements, means for generating a natural background image using a generative AI model and replacing the risk elements for location identification, means for generating a plurality of candidate converted images or videos and presenting them to the user, and means for generating a final image or video after receiving a selection and confirmation from the user. This makes it possible to achieve conversion to a natural and safe background while reducing the risk of location identification.

[0146] "Means for uploading images or videos" refers to the function for sending image or video data from a user's device to a server.

[0147] "Means for analyzing the background from each frame of an image or video and extracting location-specific risk elements" refers to a function that uses an AI module to analyze each frame of an image or video and detect location-specific risk elements such as buildings, address plates, and signs.

[0148] "Means for transforming location risk elements into other natural background elements" refers to the function responsible for the transformation process to replace location risk elements with other natural backgrounds.

[0149] "Means for generating natural-looking background images using generative AI models to replace location-specific risk elements" refers to a function that uses AI technology to generate new background images and replace them with location-specific risk elements, thereby providing a natural-looking background.

[0150] "Means for generating multiple converted candidate images or videos and presenting them to the user" refers to a function that allows the server to present candidate images or videos generated using multiple conversion patterns to the user.

[0151] "Means for generating a final image or video upon user selection and confirmation" refers to the function that allows the user to finally confirm the image or video selected from the candidates and generate the approved final image or video.

[0152] This invention relates to a system that changes the background of images and videos to a more natural look when they are posted to a social networking site, reducing the risk of location identification. This system is composed of three entities: a server, a terminal, and a user.

[0153] System configuration

[0154] The system mainly uses the following major hardware and software:

[0155] 1. Server

[0156] Hardware: A computer system with a powerful processor, memory, and storage devices.

[0157] Software: Install AI modules (e.g., OpenCV, TENSORFLOW®), image processing libraries, and generative AI models (e.g., DALL-E, GAN).

[0158] 2. Terminal

[0159] Hardware: The devices that users regularly use, such as smartphones, tablets, and PCs.

[0160] Software: A dedicated application or web browser for uploading images and videos and communicating with the server.

[0161] 3. Users

[0162] An individual who uses the system to upload images and videos to be posted on social media and check the final generated content.

[0163] System Features

[0164] Image / Video Upload

[0165] Users select the image or video they want to post from their device and click the upload button. The device then sends the selected image or video data to the server.

[0166] Image / video analysis

[0167] The server passes the received image or video data to an AI module (e.g., OpenCV or TensorFlow) for background analysis and location-specific risk element detection.

[0168] Location-specific element detection and risk assessment

[0169] The AI ​​module detects location-specific risk factors such as buildings, address plates, and signs, and the server performs risk assessment based on these risk factors.

[0170] Converting to a natural background

[0171] The server uses a generative AI model (e.g., DALL-E, GAN) to replace risk elements with a different, more natural background image. Specifically, it uses a generative AI model (e.g., DALL-E, GAN) and inputs the following prompt:

[0172] Example prompt: "Change the background of a photo showing Tokyo Tower to a typical apartment building, and generate an image that eliminates the risk of location identification."

[0173] Generation and presentation of candidate images / videos

[0174] The server generates multiple candidates for the converted images and videos, and sends them to the device, which then presents the candidate images and videos to the user.

[0175] User selection and confirmation

[0176] The user can select and confirm the most suitable image or video from the presented candidates. If necessary, the user can also instruct reconversion.

[0177] Final image / video generation

[0178] The server generates the selected final image or video and sends it to the device, which displays the final output to the user and supports posting to social media.

[0179] This allows users to safely post images and videos with natural-looking backgrounds to social media, while reducing the risk of their location being identified.

[0180] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0181] Step 1:

[0182] Image / Video Upload

[0183] Users use their device to select the image or video they want to post and click the upload button.

[0184] Input: An image or video file selected by the user.

[0185] Output: Image or video data sent from the device to the server.

[0186] Specific operation: Triggered by user operation, the device sends an HTTP request to the server for the selected file, including image or video data.

[0187] Step 2:

[0188] Image / video analysis

[0189] The server passes the received image or video data to the AI ​​module for analysis.

[0190] Input: Image or video data received by the server.

[0191] Output: Analysis results from the AI ​​module (detection of location-specific risk elements such as buildings, house numbers, signs, etc.).

[0192] Specific operation: The server inputs the received image or video data into an AI module such as OpenCV or TensorFlow, and analyzes the location-specific risk elements for each frame, extracting the location information and characteristics of the risk elements as a result.

[0193] Step 3:

[0194] Location-specific element detection and risk assessment

[0195] The AI ​​module assesses the location risk based on the detected elements.

[0196] Input: Analyzed risk elements (buildings, house numbers, signs, etc.).

[0197] Output: Risk level assessment for each risk factor.

[0198] Specific operation: Based on the analysis results, the AI ​​module returns the location-specific risk level of each risk element to the server, which then receives this and performs risk assessment.

[0199] Step 4:

[0200] Converting to a natural background

[0201] The server uses a generative AI model to replace risk elements with alternative, natural-looking background images.

[0202] Input: Risk assessment results and original image or video data.

[0203] Output: The generated natural background image, or the altered image or video data with the risk elements replaced.

[0204] How it works: The server uses a generative AI model (e.g., DALL-E, GAN) to replace risk elements with a different background. Specifically, it generates a prompt sentence and inputs it into the AI ​​model to generate an appropriate background image.

[0205] Example prompt: "Change the background of a photo showing Tokyo Tower to a typical apartment building, and generate an image that eliminates the risk of location identification."

[0206] Step 5:

[0207] Generation and presentation of candidate images / videos

[0208] The server generates multiple candidate images or videos using different transformation patterns and sends them to the terminal.

[0209] Input: Generated background image or modified image / video data.

[0210] Output: Multiple candidate images or videos.

[0211] Specific operation: The server synthesizes the background image obtained by the generative AI model with the original image or video, generates multiple candidates with different conversion patterns, and sends them to the device as an HTTP response.

[0212] Step 6:

[0213] User selection and confirmation

[0214] The user selects and confirms the most suitable image or video from the presented candidates.

[0215] Input: Multiple candidate images or videos, selected by the user.

[0216] Output: User-selected final candidate image or video, with instructions for reconversion if necessary.

[0217] Specific operation: The device receives multiple candidate images or videos from the server, displays them on the user interface, and allows the user to select the most suitable one. Once the user has made their selection, the selection information is sent to the server.

[0218] Step 7:

[0219] Final image / video generation

[0220] The server performs final processing on the selected candidate images and videos and sends them to the terminal.

[0221] Input: Information about the candidate image or video selected by the user.

[0222] Output: The final image or video data.

[0223] Specific operation: The server checks the candidate images and videos selected by the user, performs any necessary final processing, and then sends the final data to the device for user confirmation.

[0224] Step 8:

[0225] Posting to social media

[0226] Users post the final images and videos to social media.

[0227] Input: The final image or video data.

[0228] Output: Images or videos posted to social media.

[0229] Specific operation: The device displays the final image or video to the user, and if the user is satisfied, provides the ability to post it to a social networking site. When the user clicks the post button, the data is uploaded to the social networking site.

[0230] (Application example 1)

[0231] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0232] In modern society, when posting images or videos to social media, there is an increasing risk that individuals and locations may be identified from the content of the posts. It is necessary to resolve these privacy issues and provide an environment where users can share content with peace of mind.

[0233] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0234] In this invention, the server includes means for uploading images or videos, means for analyzing the background from the uploaded images or videos and extracting risk elements for location identification, means for converting the extracted risk elements into other natural background elements, means for generating a plurality of converted candidate images or video frames and presenting them to the user, means for the user to select an appropriate one from the presented candidates, means for generating a final image or final video frame upon receiving the user's selection and confirmation, and means for posting the final image or final video frame to an SNS. This allows users to post images or videos to an SNS with peace of mind while reducing the risk of location identification.

[0235] "Means for uploading images or videos" refers to a function that allows a user to select an image or video from their device and send it to the server.

[0236] "Means for analyzing the background and extracting location-specific risk elements" is a function that uses an AI module to identify and extract location-specific risk elements contained in the background.

[0237] The "means for converting risk elements into other natural background elements" is an algorithm for replacing the extracted risk elements with other natural background elements.

[0238] The "means for generating multiple candidate transformed images or video frames and presenting them to the user" refers to a function for creating different versions of transformed images or video frames and offering them to the user as options.

[0239] The "means for the user to select the appropriate one" is an interface that allows the user to select the most appropriate one from multiple candidate images or video frames presented.

[0240] "Means for generating a final image or video frame upon receiving a user selection and confirmation" refers to functionality for creating a final image or video frame based on the user's selection.

[0241] The "means for posting the final image or final video frame to an SNS" is a function for uploading the generated final image or video frame to an SNS.

[0242] The "means for converting into natural background elements while maintaining the continuity of video frames" is an algorithm for performing background conversion that does not look out of place between a series of video frames.

[0243] A "means for enabling reconversion at the user's command" is an interface that allows the user to request conversion again.

[0244] This invention relates to a system that changes the background of images and videos posted on social networking sites in a natural way, reducing the risk of location identification. The system is mainly composed of three entities: a server, a device, and a user.

[0245] Hardware and software used

[0246] 1. Hardware:

[0247] Server: A server with high-performance computing power.

[0248] Devices: Smartphones, smart glasses, head-mounted displays.

[0249] 2. Software:

[0250] AI module: Uses TensorFlow or PyTorch for image analysis and background conversion.

[0251] Image processing library: OpenCV is used for various editing of generated images.

[0252] Communication and data management: Flask is used for communication between the server and the terminal.

[0253] System Operation

[0254] 1. Upload an image or video:

[0255] Users select images or videos from their smartphones or other devices and upload them to the server through the application.

[0256] 2. Background analysis and risk factor extraction:

[0257] The server receives the uploaded content and passes it to an AI module (e.g., TensorFlow, PyTorch), which analyzes the image or video and extracts location-specific risk factors (e.g., building shapes, house number plates, signs, etc.).

[0258] 3. Background transformation:

[0259] After extracting risk elements, the AI ​​module converts them into other natural background elements. For example, if a photo shows Tokyo Tower, it will replace the Tokyo Tower with natural scenery or a typical apartment building.

[0260] 4. Generate and present the transformed candidate image or video frame:

[0261] The server generates multiple conversion candidates and sends them to the terminal for presentation to the user, who then displays them and offers a choice.

[0262] 5. User Choice and Confirmation:

[0263] The user can select the most suitable one from the presented candidates, and can also instruct reconversion if necessary.

[0264] 6. Generate the final image or video frame:

[0265] The server generates the final image or video frame based on the user's selection and sends it to the user, who then displays it on the device and asks for final confirmation.

[0266] 7. Posting to social media:

[0267] Users can post the final image or video frame to social media.

[0268] Specific examples

[0269] For example, if a user wants to post a photo taken at a tourist spot to a social networking site, the system can detect famous landmarks or signs in the photo and replace them with other natural backgrounds, allowing users to enjoy posting while reducing the risk of being identified.

[0270] Prompt Sentence Examples

[0271] Analyze the input image and detect location risk elements. Next, replace those elements with a natural background and generate an image. Use a method to present multiple candidate images. In this case, the generated image must be consistent between consecutive video frames.

[0272] This will allow users to post images and videos on social media with peace of mind, strengthening privacy protection.

[0273] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0274] Step 1:

[0275] The user selects an image or video from the device and uploads it to the server through the application. The input is the image or video file selected on the device, and the output is the image or video data sent to the server. Specifically, the user clicks the "Upload" button and selects the specified file.

[0276] Step 2:

[0277] The server receives uploaded image or video data and passes it to an AI module (e.g., TensorFlow, PyTorch). The input is the uploaded image or video data, and the output is the analysis request passed to the AI ​​module. Specifically, the server saves the received data in a specified folder and passes the path to the AI ​​module.

[0278] Step 3:

[0279] The AI ​​module analyzes images or videos to extract location-specific risk elements. The input is the uploaded image or video data, and the output is a list of detected risk elements. Specifically, the AI ​​module uses a convolutional neural network (CNN) to identify risk elements in the image, such as building shapes, house number plates, and signs.

[0280] Step 4:

[0281] The server receives the list of risk elements from the AI ​​module and converts them into natural background elements. The input is the list of extracted risk elements, and the output is the converted background image or video frames composited onto the background. Specifically, the server uses the OpenCV library to replace the risk elements with common landscapes and buildings.

[0282] Step 5:

[0283] The server generates multiple candidate transformed images or video frames and sends them to the device for presentation to the user. The input is the transformed background image or video frame, and the output is multiple candidate images or video frames. Specifically, the server generates different background variations, saves each as a candidate, and sends it to the device.

[0284] Step 6:

[0285] The user selects the best one from multiple candidates presented on the device and sends the selection information to the server. The input is multiple candidate images or video frames, and the output is the user's selection information. Specifically, the user selects one of the presented candidate images or video frames and clicks the "Select" button.

[0286] Step 7:

[0287] The server receives the user's selection information and generates the final image or video frame. The input is the user's selection information and the corresponding candidate image or video frame, and the output is the final image or video frame. Specifically, the server performs final processing on the selected background to generate the final image or video.

[0288] Step 8:

[0289] The terminal displays the final image or video frame to the user and asks for the user's final confirmation. The input is the final image or video frame, and the output is the user's final confirmation information. Specifically, the terminal displays the final image or video and provides a "Confirm" button.

[0290] Step 9:

[0291] The user posts the final image or video after confirmation to the social networking site. The input is the image or video frame that has been finalized, and the output is the information to be posted to the social networking site. Specifically, the user clicks the "Post" button to upload the image or video to the social networking site.

[0292] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0293] This invention combines a system that changes the background of images and videos posted to social networking sites in a natural way to reduce the risk of location identification with an emotion engine that recognizes the user's emotions.The system adjusts background conversion candidates according to the user's emotions, allowing it to provide more appropriate images and videos.

[0294] System Overview

[0295] The system is mainly composed of four entities: the server, the device, the user, and the emotion engine. The server is responsible for the main processes, such as image and video analysis, background conversion, candidate generation, and final output generation. The device provides the user interface, uploads images and videos, and displays the results. The user uploads images and videos through the device and selects the best one from the converted candidates. The emotion engine recognizes the user's emotions and provides this information to the server.

[0296] Program processing

[0297] 1. Upload an image

[0298] Users select an image from their device and click the upload button.

[0299] The terminal transmits the selected image data to the server.

[0300] 2. Emotion recognition

[0301] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, input text, etc. to recognize emotions.

[0302] The emotion engine sends the recognized emotion information to the server.

[0303] 3. Image Analysis

[0304] The server passes the received image data to the AI ​​module and begins analysis.

[0305] The AI ​​module detects elements in the image that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[0306] 4. Risk Assessment

[0307] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[0308] 5. Background conversion

[0309] The server extracts risk factors and changes the background based on information from the emotion engine. For example, if the user is in a happy mood, the background will be bright, and if the user is in a calm mood, the background will be changed to a natural landscape.

[0310] For example, if a photo shows Tokyo Tower, and the user is feeling calm, the Tokyo Tower part can be replaced with mountains or a lake.

[0311] 6. Candidate Generation and Presentation

[0312] The server generates multiple candidate conversion images and transmits them to the terminal for presentation to the user.

[0313] The terminal displays the received candidate images to the user and provides a selection screen.

[0314] 7. User Choice

[0315] The user reviews the different candidate images presented and selects the best one.

[0316] If necessary, the user can also instruct reconversion.

[0317] 8. Generating the final image

[0318] The server finalizes the selected candidate images and sends them to the user.

[0319] The device displays the final image to the user for final confirmation.

[0320] 9. Submission

[0321] Users can then post the final image to social media.

[0322] Application to video

[0323] The same process is applied to videos, but the same process is performed on each frame while maintaining continuity, resulting in a natural background transition for the entire video.

[0324] 1. Upload a video

[0325] Users select a video from their device and click the upload button.

[0326] The device transmits the video file to the server.

[0327] 2. Emotion recognition

[0328] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, input text, etc. to recognize emotions.

[0329] The emotion engine sends the recognized emotion information to the server.

[0330] 3. Frame Analysis and Transformation

[0331] The server receives the video data and inputs each frame into the AI ​​module.

[0332] The AI ​​module analyzes each frame, detects elements that pose a risk of location identification, and transforms them into natural backgrounds.

[0333] 4. Maintaining continuity

[0334] The server performs the conversion process so that there is no sense of incongruity between consecutive frames, and generates the final video.

[0335] 5. Final video generation

[0336] The server generates the final video and offers variations based on the user's emotions.

[0337] The device displays the final video to the user and provides a confirmation screen.

[0338] 6. Submission

[0339] Users can then post the final video to social media.

[0340] This will reduce the risk of location identification and provide a system that allows users to safely post images and videos to social media with backgrounds that change naturally based on emotions.

[0341] The processing flow will be explained below.

[0342] Image processing flow

[0343] Step 1:

[0344] Users select the image they want to upload from their device and click the upload button.

[0345] Step 2:

[0346] The terminal transmits the selected image data to the server.

[0347] Step 3:

[0348] The server receives the uploaded image data.

[0349] Step 4:

[0350] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, and input text to recognize their emotions.

[0351] Step 5:

[0352] The emotion engine sends the user's recognized emotion information to the server. For example, if the user is smiling, the emotion of "joy" is recognized.

[0353] Step 6:

[0354] The server passes the received image data to the AI ​​module and begins analyzing the image.

[0355] Step 7:

[0356] The AI ​​module detects elements in the image that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[0357] Step 8:

[0358] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[0359] Step 9:

[0360] The server extracts risk factors and generates a background image that replaces them with other natural background factors. For example, if a risk factor is high, it is set to a tranquil landscape that completely covers it.

[0361] Step 10:

[0362] The server uses emotion information from the emotion engine to adjust the style of background transformation, for example, if the user's emotion is "joy", it selects a light-colored background.

[0363] Step 11:

[0364] The server generates a number of candidate transformed images.

[0365] Step 12:

[0366] The server sends the generated candidate images to the user's terminal.

[0367] Step 13:

[0368] The terminal displays the received candidate images to the user and provides a confirmation screen.

[0369] Step 14:

[0370] The user reviews the different candidate images presented and selects the best one.

[0371] Step 15:

[0372] If the user desires reconversion, he or she inputs the instruction into the terminal.

[0373] Step 16:

[0374] The server receives the user's instruction for reconversion, generates candidate images again, and sends them to the terminal.

[0375] Step 17:

[0376] The user makes the final selection and sends confirmation to the device.

[0377] Step 18:

[0378] The server finalizes the candidate images selected by the user to generate the final image.

[0379] Step 19:

[0380] The server sends the final image to the user's device.

[0381] Step 20:

[0382] The terminal displays the final image to the user and provides an interface for final confirmation.

[0383] Step 21:

[0384] The user then makes a final confirmation and enters instructions into the device to post to social media.

[0385] Step 22:

[0386] The device retrieves the final image from the server and posts it to a social media platform.

[0387] Video processing flow

[0388] Step 1:

[0389] Users select the video they want to upload from their device and click the upload button.

[0390] Step 2:

[0391] The terminal transmits the selected video data to the server.

[0392] Step 3:

[0393] The server receives the uploaded video data.

[0394] Step 4:

[0395] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, and input text to recognize their emotions.

[0396] Step 5:

[0397] The emotion engine sends the recognized user emotion information to the server.

[0398] Step 6:

[0399] The server passes each frame of video data to the AI ​​module and begins analysis.

[0400] Step 7:

[0401] The AI ​​module detects elements in each frame that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[0402] Step 8:

[0403] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[0404] Step 9:

[0405] The server extracts risk elements for each frame and converts them into a background image that replaces them with other natural background elements.

[0406] Step 10:

[0407] The server uses the emotion information from the emotion engine to adjust the style of the background transformation.

[0408] Step 11:

[0409] The server performs the conversion process so that there is no sense of incongruity between consecutive frames, and generates the final video.

[0410] Step 12:

[0411] The server sends the final video to the user's device.

[0412] Step 13:

[0413] The device displays the final video to the user and provides a confirmation screen.

[0414] Step 14:

[0415] The user reviews the final video and enters instructions into the device to post it to social media.

[0416] Step 15:

[0417] The device retrieves the final video from the server and posts it to a social media platform.

[0418] Example 2

[0419] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0420] With conventional SNS posts, the background of images and videos is exposed as is, increasing the risk of location identification. Furthermore, because the background is not changed to take into account the user's feelings, there is also the issue of it being difficult to change the background appropriately. There is a need to solve these problems and provide an environment where users can post images and videos on SNS with peace of mind.

[0421] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0422] In this invention, the server includes means for uploading images, means for recognizing a user's emotion from the uploaded image, means for adjusting background conversion candidates based on the recognized emotion, means for analyzing the uploaded image and extracting risk elements for location identification, means for converting the extracted risk elements into other natural background elements, means for generating a plurality of converted candidate images and presenting them to the user, and means for generating a final image after receiving a selection and confirmation from the user. This allows background conversion according to the user's emotion, reducing the risk of location identification and making it possible to provide images and videos that can be posted safely to SNS.

[0423] "Means for uploading images" is a function that allows a user to send image data selected from the terminal to the server.

[0424] "Means for recognizing user emotions" refers to a function that analyzes emotions from the user's facial expressions, tone of voice, input text, etc., and provides that information to the server.

[0425] The "means for adjusting background conversion candidates" is a function that selects and adjusts suggested background image variations based on the recognized user emotions.

[0426] "Means for extracting risk factors for location identification" is a function that analyzes elements that can lead to location identification from uploaded images and evaluates the risk.

[0427] The "means for converting into background elements" is a function that uses a generative AI model to convert the extracted risk elements into another natural background.

[0428] The "means for generating multiple candidate images and presenting them to the user" is a function for generating multiple converted background variations and displaying them on the user's device.

[0429] The "means for generating a final image" is a function for finalizing the candidate image selected by the user and providing it to the user.

[0430] This invention relates to a system that changes the background of images and videos posted to social networking sites to create a natural look. This system has a function to recognize the user's emotions, reducing the risk of location identification and allowing users to post content with peace of mind. Specific embodiments of this system are described below.

[0431] This system mainly consists of four components: a server, a terminal, a user, and an emotion engine. The server is responsible for analyzing images and videos, converting backgrounds, generating candidates, and generating the final output. The terminal provides the user interface, uploads images and videos, and displays the results. Users upload images and videos through their terminals and select the best one from the converted candidates. The emotion engine recognizes the user's emotions and provides that information to the server.

[0432] Hardware / Software Used

[0433] 1. Server: Analyzes images and videos, converts backgrounds, generates candidates, and generates final output. Specifically, it uses AI frameworks such as TensorFlow and PyTorch.

[0434] 2. Device: Provides the user interface, uploads images and videos, and displays the results. This can be a smartphone or PC.

[0435] 3. Emotion Engine: Recognizes user emotions using computer vision and voice recognition technologies, such as OpenCV and Google Cloud AI emotion analysis API.

[0436] 4. AI module: Analyzes and evaluates location risks in images. Built using TensorFlow and PyTorch.

[0437] System operation example

[0438] First, the user selects the image they want to post to the SNS from their device and clicks the upload button. The device then sends the selected image data to the server, where it is encoded in Base64 format.

[0439] Next, the device's built-in emotion engine analyzes the user's facial expressions, tone of voice, and input text. For example, if the user is smiling at the camera, the emotion engine recognizes "happiness." The recognition results are sent to the server in JSON format.

[0440] The server inputs the received image data into the AI ​​module, which then analyzes the image's pixel data and uses OCR technology to detect address plates and signs. It also extracts elements that can help identify the location, such as the shape of the building and the background scenery.

[0441] Next, the AI ​​module evaluates the location risk based on each detected element and assigns a score. For example, if an address plate is clearly visible, it is evaluated as a high-risk element. Based on this result, the server lists the risk elements and passes them on to the next conversion process.

[0442] The server applies a conversion algorithm based on this list and emotion data. If the user is in a "happy" mood, the background will be replaced with a "seaside scene." The generated background candidate images are seamlessly synthesized using techniques such as GAN (Generative Adversarial Network).

[0443] The server then generates multiple conversion candidate images and sends them to the terminal for presentation to the user. The terminal displays the received candidate images to the user in thumbnail format and provides a selection screen. The user can select the most appropriate one from the presented candidate images. If necessary, the user can also instruct conversion again.

[0444] Finally, the server processes the selected candidate images, applying edge correction and color balancing before providing them to the user. The device then displays the final image to the user for final confirmation. The user can then post the confirmed image to a social networking site.

[0445] Examples and prompts

[0446] For example, a user selects a family photo from their smartphone's photo library and taps the "Upload" button in the app. The device sends the image to the server and displays the message "Recognizing emotions...". The emotion engine recognizes the user's smile and detects "enjoyment". The AI ​​module marks the house number plate at a specific position in the image as a "high-risk element". Based on the user's emotions, the server generates multiple background images, such as seaside, mountains, and city night views, and displays them on the device. The user selects the best one from these candidates to generate the final image.

[0447] Example prompt sentence:

[0448] "I want to upload photos from a family vacation, but I want to change the background so my home address is not identified. The sentiment is fun."

[0449] "I'm feeling calm right now, so I'd like you to change the background of the videos I upload to natural scenery."

[0450] This allows users to safely post images and videos to social media with reduced risk of their location being identified.

[0451] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0452] Step 1:

[0453] Image upload

[0454] The user selects an image from the file selection dialog on the device and clicks the upload button.

[0455] Input: An image file selected by the user.

[0456] The device sends the selected image data to the server via an HTTP request, where the image data is encoded in Base64 format.

[0457] Output: Base64 encoded image data is sent to the server.

[0458] Step 2:

[0459] emotion recognition

[0460] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, and input text.

[0461] Input: User's real-time facial expression data, voice data, and input text data.

[0462] The device uses emotion recognition algorithms to analyze the user's emotions in real time. For example, if the user smiles at the camera, it will recognize "happiness."

[0463] The emotion engine sends the recognized emotion data in JSON format to the server.

[0464] Output: Emotion data in JSON format is sent to the server.

[0465] Step 3:

[0466] Image analysis

[0467] The server passes the received image data to an AI module (e.g., TensorFlow or PyTorch model) and begins analysis.

[0468] Input: Base64 encoded image data.

[0469] The AI ​​module analyzes the pixel data of the image and uses OCR technology to detect address plates and signs, as well as extracting elements relevant to location identification, such as building shapes and background scenery.

[0470] Output: The analysis results in a list of risk factors.

[0471] Step 4:

[0472] Risk Assessment

[0473] The AI ​​module assesses the location risk of each detected element and assigns a risk score.

[0474] Input: A list of risk factors.

[0475] The server lists elements with high risk scores and passes the data to the next background conversion process.

[0476] Output: A list of elements for which risk assessment has been completed is generated.

[0477] Step 5:

[0478] Background Transformation

[0479] The server applies a transformation algorithm based on the risk factor list and emotion data.

[0480] Input: List of risk-assessed elements, sentiment data.

[0481] The conversion algorithm generates the most appropriate background based on the user's emotions. For example, if the user is in a happy mood, the background will be replaced with a bright seaside scene.

[0482] The generative AI model uses a Generative Adversarial Network (GAN) to generate new backgrounds and seamlessly combine them with the original image.

[0483] Output: Multiple composite background candidate images.

[0484] Step 6:

[0485] Candidate generation and presentation

[0486] The server generates a plurality of candidate conversion images and transmits them to the terminal for presentation to the user.

[0487] Input: Multiple synthesized background candidate images.

[0488] The terminal displays the received candidate images to the user in thumbnail format and provides a selection screen.

[0489] Output: Candidate images in thumbnail format presented to the user.

[0490] Step 7:

[0491] User Selection

[0492] The user selects the best image from the different candidate images presented.

[0493] Input: Candidate images displayed in thumbnail format.

[0494] The terminal accepts the user's selection and transmits the selection information to the server. If the user wishes to reconvert, the terminal can also transmit an instruction to the server.

[0495] Output: Data of selected candidate images.

[0496] Step 8:

[0497] Generating the final image

[0498] The server performs final processing on the selected candidate images, applying edge correction and color balancing.

[0499] Input: The data of the selected candidate image.

[0500] The server sends the final image back to the terminal for display to the user.

[0501] The terminal displays the final image to the user for final confirmation.

[0502] Output: The final image.

[0503] Step 9:

[0504] Post

[0505] Users can then post the final image they have confirmed to social media.

[0506] Input: The final image as viewed by the user.

[0507] The device uses the SNS API to upload the image to the specified account.

[0508] Output: Images posted to social media.

[0509] (Application example 2)

[0510] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0511] Conventional image and video posting systems pose the risk of revealing the location where a user took a photo. In particular, analyzing features such as buildings and signs in the background of a photo can make the location easily identifiable. Furthermore, changing the background without considering the user's emotions can result in lower user satisfaction. A new system is needed to solve these problems and enable safer and more satisfying image and video posting.

[0512] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0513] In this invention, the server includes means for uploading images, means for analyzing the background from the uploaded image and extracting risk elements for location identification, means for converting the extracted risk elements into other natural background elements, means for generating candidates for background conversion based on emotions using an emotion engine that recognizes the user's emotions, means for generating a plurality of converted candidate images and presenting them to the user, and means for generating a final image after receiving a selection and confirmation from the user. This enables a natural background change according to the user's emotions while reducing the risk of location identification.

[0514] The "means for uploading images" provides a function for users to send image data they have taken to the system.

[0515] "Means for analyzing the background and extracting risk elements for identifying a location" provides the functionality to detect and analyze elements (e.g., buildings and signs) that identify a specific location from uploaded images.

[0516] The "means for converting extracted risk elements into other natural background elements" provides a function for replacing detected location-specific risk elements with other safe and natural backgrounds.

[0517] The "emotion engine that recognizes user emotions" is a mechanism for analyzing and recognizing a user's emotional state from facial expressions, tone of voice, input text, etc.

[0518] The "means for generating background conversion candidates based on emotions" provides a function for generating optimal background conversion candidates according to the recognized emotions of the user.

[0519] The "means for generating a plurality of converted candidate images and presenting them to the user" provides a function for displaying a plurality of background conversion candidate images generated based on emotion to the user and making them selectable.

[0520] The "means for generating a final image upon receiving a user's selection and confirmation" provides a function for applying the background conversion candidate selected by the user and generating a final image.

[0521] MODE FOR CARRYING OUT THE INVENTION

[0522] This invention is a system that changes the background of images and videos posted on social media in a natural way to reduce the risk of location identification, and combines it with an emotion engine that recognizes the user's emotions. The system works as follows.

[0523] System Configuration

[0524] This system mainly consists of four components: a server, a device (smartphone), a user, and an emotion engine. The server is responsible for the main processes, such as analyzing images and videos, converting backgrounds, generating candidates, and generating the final output. The device provides the user interface, uploads images and videos, and displays the results. Users upload images and videos through their devices and select the best converted background from the candidates. The emotion engine recognizes the user's emotions and provides that information to the server.

[0525] What the program does

[0526] 1. Image upload: The user selects an image from their smartphone and uploads it to the app. At this time, the image data is read using an image processing library such as OpenCV.

[0527] 2. Emotion recognition: The device's built-in emotion engine (e.g., Google Cloud's Face API) recognizes emotions from the user's facial expressions and input text. This information is sent to the server.

[0528] 3. Background analysis: The server passes the received image data to the AI ​​module, which then begins analysis. The AI ​​module detects elements in the image that can lead to location identification, such as building shapes, signs, and background scenery.

[0529] 4. Risk Assessment: The AI ​​module assesses the risk of location identification based on the detected elements and sets a risk level for each element.

[0530] 5. Background conversion: The server extracts risk factors and converts the background based on information from the emotion engine. For example, if the user is in a happy mood, the background will be changed to a bright one, and if the user is in a calm mood, the background will be changed to a natural landscape.

[0531] 6. Candidate generation and presentation: The server generates multiple background conversion candidate images and sends them to the terminal for presentation to the user.

[0532] 7. User Selection: The user reviews the different candidate images and selects the best one. If necessary, the user can also request reconversion.

[0533] 8. Final image generation: The server finalizes the selected candidate images and sends them to the user. The terminal displays the final image to the user for final confirmation.

[0534] 9. Post: Users can post the final image to social media.

[0535] Hardware and software used

[0536] Hardware: Smartphone (iOS or ANDROID compatible)

[0537] Software: OpenCV, emotion engine libraries (e.g. Google Cloud's Face API), and Python and TensorFlow on the server side for image analysis and background conversion.

[0538] Specific examples

[0539] For example, if a user uploads a smiling selfie, the emotion engine will determine the emotion of "joy." After analyzing risk factors, the server will generate a bright, natural background. The user can then select the best image from the presented candidate images and safely post it to social media.

[0540] Prompt Sentence Examples

[0541] Identify the emotion in an image uploaded by a user and automatically select the background that best matches that emotion to change the image background. For example, change the background to a bright natural scene for a "smiling face." This reduces the risk of location identification based on the original image.

[0542] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0543] Step 1:

[0544] The device provides an interface for the user to select an image from within the smartphone. The user selects an image and clicks the upload button within the app to input the image data into the device. The device then sends the selected image data to the server.

[0545] Step 2:

[0546] The server passes the received image data to the AI ​​module, which begins background analysis. Specifically, it uses OpenCV and an AI model to detect elements in the image that identify specific locations. The AI ​​model identifies location-specific risk elements, such as buildings and signs, and outputs their location information.

[0547] Step 3:

[0548] The server evaluates the risk of the location based on the output from the AI ​​module. The risk assessment is based on the type of element detected and its risk level. The output is a list of the risk level for each element.

[0549] Step 4:

[0550] The emotion engine installed on the device analyzes the user's emotions before or immediately after uploading. This emotion analysis is performed based on the user's facial expressions, voice, input text, etc. The emotion engine sends the analysis results to the server, which receives the user's emotional information as input.

[0551] Step 5:

[0552] The server performs background conversion based on the list of location-specific risk factors and the user's emotional information. Specifically, it removes or converts risk factors and generates an appropriate background based on the emotional information. For example, if the user is in a happy emotional state, the system generates a bright natural landscape.

[0553] Step 6:

[0554] The server prepares the generated multiple background conversion candidates for display and sends them to the device. The device presents the multiple conversion candidate images to the user and displays a selection screen. The user can review the conversion candidate images and select the most appropriate one.

[0555] Step 7:

[0556] The server receives the user-selected background candidate image as input and generates the final image. Specifically, it applies the selected background to the original image, adjusts unnecessary areas, and performs final processing.

[0557] Step 8:

[0558] The server sends the final image to the user's device, which displays it to the user, who can then post it to social media after reviewing it.

[0559] Step 9:

[0560] The user posts the final image to the SNS. Specifically, a dedicated button is provided to upload the generated final image data to the SNS, completing the posting process.

[0561] The above steps reduce the risk of location identification while changing the background according to the user's emotions, creating an environment in which images and videos can be posted to social media more safely.

[0562] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0563] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0564] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0565] [Second embodiment]

[0566] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0567] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0568] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0569] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0570] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0571] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0572] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0573] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0574] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0575] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0576] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0577] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0578] This invention relates to a system that changes the background of images and videos posted to social networking sites in a natural way to reduce the risk of location identification. The program processing of this system is specifically explained below in natural language.

[0579] System Overview

[0580] The system is mainly composed of three entities: the server, the device, and the user. The server is responsible for the main processes, such as image and video analysis, background conversion, candidate generation, and final output generation. The device provides the user interface, uploads images and videos, and displays the results. The user uploads images and videos through the device and selects the best one from the converted candidates.

[0581] Program processing

[0582] 1. Upload an image

[0583] Users select an image from their device and click the upload button.

[0584] The terminal transmits the selected image data to the server.

[0585] 2. Image Analysis

[0586] The server passes the received image data to the AI ​​module.

[0587] The AI ​​module analyzes the images and detects elements that could lead to location identification, such as building shapes, address plates, and signs.

[0588] 3. Risk Assessment

[0589] The AI ​​module assesses the location risk based on the detected elements and assigns a risk level to each element.

[0590] 4. Background conversion

[0591] The server extracts risk elements and generates a background image that replaces them with other natural elements.

[0592] Example: In a photo that shows Tokyo Tower, replace the Tokyo Tower with natural scenery or an ordinary apartment building.

[0593] 5. Candidate Generation and Presentation

[0594] The server generates multiple candidate conversion images and transmits them to the terminal for presentation to the user.

[0595] The terminal displays the received candidate images to the user and provides a selection screen.

[0596] 6. User Choice

[0597] The user checks the presented candidate images and selects the appropriate one.

[0598] If necessary, the user can also instruct reconversion.

[0599] 7. Generating the final image

[0600] The server finalizes the selected candidate images and sends them to the user.

[0601] The device displays the final image to the user for final confirmation.

[0602] 8. Submission

[0603] Users can then post the final image to social media.

[0604] Application to video

[0605] The same process is applied to videos, but the same process is performed on each frame while maintaining continuity, resulting in a natural background transition for the entire video.

[0606] 1. Upload a video

[0607] The user selects a video from their device and clicks the upload button.

[0608] The device transmits the video file to the server.

[0609] 2. Frame analysis and transformation

[0610] The server receives the video data and inputs each frame into the AI ​​module.

[0611] The AI ​​module analyzes each frame, detects elements that pose a risk of location identification, and transforms them into natural backgrounds.

[0612] 3. Maintaining continuity

[0613] The server performs conversion processing that takes into account the continuity of frames in order to generate images that do not look out of place between consecutive frames.

[0614] 4. Final video generation

[0615] The server generates the final video and sends it to the device.

[0616] The device will then display the final video to the user and ask for confirmation.

[0617] 5. Post

[0618] Users can then post the final video to social media.

[0619] This makes it possible to provide a system that reduces the risk of location identification and allows users to safely post images and videos with natural-looking backgrounds to social media.

[0620] The processing flow will be explained below.

[0621] Step 1:

[0622] Users select the image they want to upload from their device and click the upload button.

[0623] Step 2:

[0624] The terminal transmits the selected image data to the server.

[0625] Step 3:

[0626] The server receives the uploaded image data.

[0627] Step 4:

[0628] The server passes the image to the AI ​​module and begins analysis.

[0629] Step 5:

[0630] The AI ​​module detects elements in the image that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[0631] Step 6:

[0632] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[0633] Step 7:

[0634] The server extracts risk elements and generates a background image that replaces them with other natural background elements.

[0635] Step 8:

[0636] The server generates a number of candidate transformed images.

[0637] Step 9:

[0638] The server sends the generated candidate images to the user's terminal.

[0639] Step 10:

[0640] The terminal displays the received candidate images to the user and provides a confirmation screen.

[0641] Step 11:

[0642] The user reviews the different candidate images presented and selects the best one.

[0643] Step 12:

[0644] If the user desires reconversion, he or she inputs the instruction into the terminal.

[0645] Step 13:

[0646] The server receives the user's instruction for reconversion, generates candidate images again, and sends them to the terminal.

[0647] Step 14:

[0648] The user makes the final selection and sends confirmation to the device.

[0649] Step 15:

[0650] The server finalizes the candidate images selected by the user to generate the final image.

[0651] Step 16:

[0652] The server sends the final image to the user's device.

[0653] Step 17:

[0654] The terminal displays the final image to the user and provides an interface for final confirmation.

[0655] Step 18:

[0656] The user then makes a final confirmation and enters instructions into the device to post to social media.

[0657] Step 19:

[0658] The device retrieves the final image from the server and posts it to a social media platform.

[0659] The above is the flow of image processing.

[0660] Application to video

[0661] Step 1:

[0662] Users select the video they want to upload from their device and click the upload button.

[0663] Step 2:

[0664] The terminal transmits the selected video data to the server.

[0665] Step 3:

[0666] The server receives the uploaded video data.

[0667] Step 4:

[0668] The server passes each frame of the video to the AI ​​module and begins analysis.

[0669] Step 5:

[0670] The AI ​​module detects elements in each frame that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[0671] Step 6:

[0672] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[0673] Step 7:

[0674] The server extracts risk elements for each frame and generates a background image that replaces them with other natural background elements.

[0675] Step 8:

[0676] The server performs the conversion process so that there is no sense of incongruity between consecutive frames, and generates the final video.

[0677] Step 9:

[0678] The server sends the final video to the user's device.

[0679] Step 10:

[0680] The device displays the final video to the user and provides a confirmation screen.

[0681] Step 11:

[0682] The user reviews the final video and enters instructions into the device to post it to social media.

[0683] Step 12:

[0684] The device retrieves the final video from the server and posts it to a social media platform.

[0685] Example 1

[0686] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0687] When posting images or videos to social networking sites, there is a risk that elements such as buildings, address plates, and signs that appear in the background may be identified by a third party. This risk can lead to a violation of privacy, making it difficult for posters to use social networking sites safely. Furthermore, in the case of videos, background conversion technology is also required to prevent the appearance of incongruity between consecutive frames. Therefore, a system is needed that can convert backgrounds to natural ones while reducing the risk of location identification.

[0688] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0689] In this invention, the server includes means for uploading images or videos, means for analyzing the background from each frame of the uploaded image or video and extracting risk elements for location identification, means for converting the extracted risk elements into other natural background elements, means for generating a natural background image using a generative AI model and replacing the risk elements for location identification, means for generating a plurality of candidate converted images or videos and presenting them to the user, and means for generating a final image or video after receiving a selection and confirmation from the user. This makes it possible to achieve conversion to a natural and safe background while reducing the risk of location identification.

[0690] "Means for uploading images or videos" refers to the function for sending image or video data from a user's device to a server.

[0691] "Means for analyzing the background from each frame of an image or video and extracting location-specific risk elements" refers to a function that uses an AI module to analyze each frame of an image or video and detect location-specific risk elements such as buildings, address plates, and signs.

[0692] "Means for transforming location risk elements into other natural background elements" refers to the function responsible for the transformation process to replace location risk elements with other natural backgrounds.

[0693] "Means for generating natural-looking background images using generative AI models to replace location-specific risk elements" refers to a function that uses AI technology to generate new background images and replace them with location-specific risk elements, thereby providing a natural-looking background.

[0694] "Means for generating multiple converted candidate images or videos and presenting them to the user" refers to a function that allows the server to present candidate images or videos generated using multiple conversion patterns to the user.

[0695] "Means for generating a final image or video upon user selection and confirmation" refers to the function that allows the user to finally confirm the image or video selected from the candidates and generate the approved final image or video.

[0696] This invention relates to a system that changes the background of images and videos to a more natural look when they are posted to a social networking site, reducing the risk of location identification. This system is composed of three entities: a server, a terminal, and a user.

[0697] System configuration

[0698] The system mainly uses the following major hardware and software:

[0699] 1. Server

[0700] Hardware: A computer system with a powerful processor, memory, and storage devices.

[0701] Software: Install AI modules (e.g., OpenCV, TensorFlow), image processing libraries, and generative AI models (e.g., DALL-E, GAN).

[0702] 2. Terminal

[0703] Hardware: The devices that users regularly use, such as smartphones, tablets, and PCs.

[0704] Software: A dedicated application or web browser for uploading images and videos and communicating with the server.

[0705] 3. Users

[0706] An individual who uses the system to upload images and videos to be posted on social media and check the final generated content.

[0707] System Features

[0708] Image / Video Upload

[0709] Users select the image or video they want to post from their device and click the upload button. The device then sends the selected image or video data to the server.

[0710] Image / video analysis

[0711] The server passes the received image or video data to an AI module (e.g., OpenCV or TensorFlow) for background analysis and location-specific risk element detection.

[0712] Location-specific element detection and risk assessment

[0713] The AI ​​module detects location-specific risk factors such as buildings, address plates, and signs, and the server performs risk assessment based on these risk factors.

[0714] Converting to a natural background

[0715] The server uses a generative AI model (e.g., DALL-E, GAN) to replace risk elements with a different, more natural background image. Specifically, it uses a generative AI model (e.g., DALL-E, GAN) and inputs the following prompt:

[0716] Example prompt: "Change the background of a photo showing Tokyo Tower to a typical apartment building, and generate an image that eliminates the risk of location identification."

[0717] Generation and presentation of candidate images / videos

[0718] The server generates multiple candidates for the converted images and videos, and sends them to the device, which then presents the candidate images and videos to the user.

[0719] User selection and confirmation

[0720] The user can select and confirm the most suitable image or video from the presented candidates. If necessary, the user can also instruct reconversion.

[0721] Final image / video generation

[0722] The server generates the selected final image or video and sends it to the device, which displays the final output to the user and supports posting to social media.

[0723] This allows users to safely post images and videos with natural-looking backgrounds to social media, while reducing the risk of their location being identified.

[0724] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0725] Step 1:

[0726] Image / Video Upload

[0727] Users use their device to select the image or video they want to post and click the upload button.

[0728] Input: An image or video file selected by the user.

[0729] Output: Image or video data sent from the device to the server.

[0730] Specific operation: Triggered by user operation, the device sends an HTTP request to the server for the selected file, including image or video data.

[0731] Step 2:

[0732] Image / video analysis

[0733] The server passes the received image or video data to the AI ​​module for analysis.

[0734] Input: Image or video data received by the server.

[0735] Output: Analysis results from the AI ​​module (detection of location-specific risk elements such as buildings, house numbers, signs, etc.).

[0736] Specific operation: The server inputs the received image or video data into an AI module such as OpenCV or TensorFlow, and analyzes the location-specific risk elements for each frame, extracting the location information and characteristics of the risk elements as a result.

[0737] Step 3:

[0738] Location-specific element detection and risk assessment

[0739] The AI ​​module assesses the location risk based on the detected elements.

[0740] Input: Analyzed risk elements (buildings, house numbers, signs, etc.).

[0741] Output: Risk level assessment for each risk factor.

[0742] Specific operation: Based on the analysis results, the AI ​​module returns the location-specific risk level of each risk element to the server, which then receives this and performs risk assessment.

[0743] Step 4:

[0744] Converting to a natural background

[0745] The server uses a generative AI model to replace risk elements with alternative, natural-looking background images.

[0746] Input: Risk assessment results and original image or video data.

[0747] Output: The generated natural background image, or the altered image or video data with the risk elements replaced.

[0748] How it works: The server uses a generative AI model (e.g., DALL-E, GAN) to replace risk elements with a different background. Specifically, it generates a prompt sentence and inputs it into the AI ​​model to generate an appropriate background image.

[0749] Example prompt: "Change the background of a photo showing Tokyo Tower to a typical apartment building, and generate an image that eliminates the risk of location identification."

[0750] Step 5:

[0751] Generation and presentation of candidate images / videos

[0752] The server generates multiple candidate images or videos using different transformation patterns and sends them to the terminal.

[0753] Input: Generated background image or modified image / video data.

[0754] Output: Multiple candidate images or videos.

[0755] Specific operation: The server synthesizes the background image obtained by the generative AI model with the original image or video, generates multiple candidates with different conversion patterns, and sends them to the device as an HTTP response.

[0756] Step 6:

[0757] User selection and confirmation

[0758] The user selects and confirms the most suitable image or video from the presented candidates.

[0759] Input: Multiple candidate images or videos, selected by the user.

[0760] Output: User-selected final candidate image or video, with instructions for reconversion if necessary.

[0761] Specific operation: The device receives multiple candidate images or videos from the server, displays them on the user interface, and allows the user to select the most suitable one. Once the user has made their selection, the selection information is sent to the server.

[0762] Step 7:

[0763] Final image / video generation

[0764] The server performs final processing on the selected candidate images and videos and sends them to the terminal.

[0765] Input: Information about the candidate image or video selected by the user.

[0766] Output: The final image or video data.

[0767] Specific operation: The server checks the candidate images and videos selected by the user, performs any necessary final processing, and then sends the final data to the device for user confirmation.

[0768] Step 8:

[0769] Posting to social media

[0770] Users post the final images and videos to social media.

[0771] Input: The final image or video data.

[0772] Output: Images or videos posted to social media.

[0773] Specific operation: The device displays the final image or video to the user, and if the user is satisfied, provides the ability to post it to a social networking site. When the user clicks the post button, the data is uploaded to the social networking site.

[0774] (Application example 1)

[0775] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0776] In modern society, when posting images or videos to social media, there is an increasing risk that individuals and locations may be identified from the content of the posts. It is necessary to resolve these privacy issues and provide an environment where users can share content with peace of mind.

[0777] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0778] In this invention, the server includes means for uploading images or videos, means for analyzing the background from the uploaded images or videos and extracting risk elements for location identification, means for converting the extracted risk elements into other natural background elements, means for generating a plurality of converted candidate images or video frames and presenting them to the user, means for the user to select an appropriate one from the presented candidates, means for generating a final image or final video frame upon receiving the user's selection and confirmation, and means for posting the final image or final video frame to an SNS. This allows users to post images or videos to an SNS with peace of mind while reducing the risk of location identification.

[0779] "Means for uploading images or videos" refers to a function that allows a user to select an image or video from their device and send it to the server.

[0780] "Means for analyzing the background and extracting location-specific risk elements" is a function that uses an AI module to identify and extract location-specific risk elements contained in the background.

[0781] The "means for converting risk elements into other natural background elements" is an algorithm for replacing the extracted risk elements with other natural background elements.

[0782] The "means for generating multiple candidate transformed images or video frames and presenting them to the user" refers to a function for creating different versions of transformed images or video frames and offering them to the user as options.

[0783] The "means for the user to select the appropriate one" is an interface that allows the user to select the most appropriate one from multiple candidate images or video frames presented.

[0784] "Means for generating a final image or video frame upon receiving a user selection and confirmation" refers to functionality for creating a final image or video frame based on the user's selection.

[0785] The "means for posting the final image or final video frame to an SNS" is a function for uploading the generated final image or video frame to an SNS.

[0786] The "means for converting into natural background elements while maintaining the continuity of video frames" is an algorithm for performing background conversion that does not look out of place between a series of video frames.

[0787] A "means for enabling reconversion at the user's command" is an interface that allows the user to request conversion again.

[0788] This invention relates to a system that changes the background of images and videos posted on social networking sites in a natural way, reducing the risk of location identification. The system is mainly composed of three entities: a server, a device, and a user.

[0789] Hardware and software used

[0790] 1. Hardware:

[0791] Server: A server with high-performance computing power.

[0792] Devices: Smartphones, smart glasses, head-mounted displays.

[0793] 2. Software:

[0794] AI module: Uses TensorFlow or PyTorch for image analysis and background conversion.

[0795] Image processing library: OpenCV is used for various editing of generated images.

[0796] Communication and data management: Flask is used for communication between the server and the terminal.

[0797] System Operation

[0798] 1. Upload an image or video:

[0799] Users select images or videos from their smartphones or other devices and upload them to the server through the application.

[0800] 2. Background analysis and risk factor extraction:

[0801] The server receives the uploaded content and passes it to an AI module (e.g., TensorFlow, PyTorch), which analyzes the image or video and extracts location-specific risk factors (e.g., building shapes, house number plates, signs, etc.).

[0802] 3. Background transformation:

[0803] After extracting risk elements, the AI ​​module converts them into other natural background elements. For example, if a photo shows Tokyo Tower, it will replace the Tokyo Tower with natural scenery or a typical apartment building.

[0804] 4. Generate and present the transformed candidate image or video frame:

[0805] The server generates multiple conversion candidates and sends them to the terminal for presentation to the user, who then displays them and offers a choice.

[0806] 5. User Choice and Confirmation:

[0807] The user can select the most suitable one from the presented candidates, and can also instruct reconversion if necessary.

[0808] 6. Generate the final image or video frame:

[0809] The server generates the final image or video frame based on the user's selection and sends it to the user, who then displays it on the device and asks for final confirmation.

[0810] 7. Posting to social media:

[0811] Users can post the final image or video frame to social media.

[0812] Specific examples

[0813] For example, if a user wants to post a photo taken at a tourist spot to a social networking site, the system can detect famous landmarks or signs in the photo and replace them with other natural backgrounds, allowing users to enjoy posting while reducing the risk of being identified.

[0814] Prompt Sentence Examples

[0815] Analyze the input image and detect location risk elements. Next, replace those elements with a natural background and generate an image. Use a method to present multiple candidate images. In this case, the generated image must be consistent between consecutive video frames.

[0816] This will allow users to post images and videos on social media with peace of mind, strengthening privacy protection.

[0817] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0818] Step 1:

[0819] The user selects an image or video from the device and uploads it to the server through the application. The input is the image or video file selected on the device, and the output is the image or video data sent to the server. Specifically, the user clicks the "Upload" button and selects the specified file.

[0820] Step 2:

[0821] The server receives uploaded image or video data and passes it to an AI module (e.g., TensorFlow, PyTorch). The input is the uploaded image or video data, and the output is the analysis request passed to the AI ​​module. Specifically, the server saves the received data in a specified folder and passes the path to the AI ​​module.

[0822] Step 3:

[0823] The AI ​​module analyzes images or videos to extract location-specific risk elements. The input is the uploaded image or video data, and the output is a list of detected risk elements. Specifically, the AI ​​module uses a convolutional neural network (CNN) to identify risk elements in the image, such as building shapes, house number plates, and signs.

[0824] Step 4:

[0825] The server receives the list of risk elements from the AI ​​module and converts them into natural background elements. The input is the list of extracted risk elements, and the output is the converted background image or video frames composited onto the background. Specifically, the server uses the OpenCV library to replace the risk elements with common landscapes and buildings.

[0826] Step 5:

[0827] The server generates multiple candidate transformed images or video frames and sends them to the device for presentation to the user. The input is the transformed background image or video frame, and the output is multiple candidate images or video frames. Specifically, the server generates different background variations, saves each as a candidate, and sends it to the device.

[0828] Step 6:

[0829] The user selects the best one from multiple candidates presented on the device and sends the selection information to the server. The input is multiple candidate images or video frames, and the output is the user's selection information. Specifically, the user selects one of the presented candidate images or video frames and clicks the "Select" button.

[0830] Step 7:

[0831] The server receives the user's selection information and generates the final image or video frame. The input is the user's selection information and the corresponding candidate image or video frame, and the output is the final image or video frame. Specifically, the server performs final processing on the selected background to generate the final image or video.

[0832] Step 8:

[0833] The terminal displays the final image or video frame to the user and asks for the user's final confirmation. The input is the final image or video frame, and the output is the user's final confirmation information. Specifically, the terminal displays the final image or video and provides a "Confirm" button.

[0834] Step 9:

[0835] The user posts the final image or video after confirmation to the social networking site. The input is the image or video frame that has been finalized, and the output is the information to be posted to the social networking site. Specifically, the user clicks the "Post" button to upload the image or video to the social networking site.

[0836] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0837] This invention combines a system that changes the background of images and videos posted to social networking sites in a natural way to reduce the risk of location identification with an emotion engine that recognizes the user's emotions.The system adjusts background conversion candidates according to the user's emotions, allowing it to provide more appropriate images and videos.

[0838] System Overview

[0839] The system is mainly composed of four entities: the server, the device, the user, and the emotion engine. The server is responsible for the main processes, such as image and video analysis, background conversion, candidate generation, and final output generation. The device provides the user interface, uploads images and videos, and displays the results. The user uploads images and videos through the device and selects the best one from the converted candidates. The emotion engine recognizes the user's emotions and provides this information to the server.

[0840] Program processing

[0841] 1. Upload an image

[0842] Users select an image from their device and click the upload button.

[0843] The terminal transmits the selected image data to the server.

[0844] 2. Emotion recognition

[0845] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, input text, etc. to recognize emotions.

[0846] The emotion engine sends the recognized emotion information to the server.

[0847] 3. Image Analysis

[0848] The server passes the received image data to the AI ​​module and begins analysis.

[0849] The AI ​​module detects elements in the image that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[0850] 4. Risk Assessment

[0851] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[0852] 5. Background conversion

[0853] The server extracts risk factors and changes the background based on information from the emotion engine. For example, if the user is in a happy mood, the background will be bright, and if the user is in a calm mood, the background will be changed to a natural landscape.

[0854] For example, if a photo shows Tokyo Tower, and the user is feeling calm, the Tokyo Tower part can be replaced with mountains or a lake.

[0855] 6. Candidate Generation and Presentation

[0856] The server generates multiple candidate conversion images and transmits them to the terminal for presentation to the user.

[0857] The terminal displays the received candidate images to the user and provides a selection screen.

[0858] 7. User Choice

[0859] The user reviews the different candidate images presented and selects the best one.

[0860] If necessary, the user can also instruct reconversion.

[0861] 8. Generating the final image

[0862] The server finalizes the selected candidate images and sends them to the user.

[0863] The device displays the final image to the user for final confirmation.

[0864] 9. Submission

[0865] Users can then post the final image to social media.

[0866] Application to video

[0867] The same process is applied to videos, but the same process is performed on each frame while maintaining continuity, resulting in a natural background transition for the entire video.

[0868] 1. Upload a video

[0869] Users select a video from their device and click the upload button.

[0870] The device transmits the video file to the server.

[0871] 2. Emotion recognition

[0872] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, input text, etc. to recognize emotions.

[0873] The emotion engine sends the recognized emotion information to the server.

[0874] 3. Frame Analysis and Transformation

[0875] The server receives the video data and inputs each frame into the AI ​​module.

[0876] The AI ​​module analyzes each frame, detects elements that pose a risk of location identification, and transforms them into natural backgrounds.

[0877] 4. Maintaining continuity

[0878] The server performs the conversion process so that there is no sense of incongruity between consecutive frames, and generates the final video.

[0879] 5. Final video generation

[0880] The server generates the final video and offers variations based on the user's emotions.

[0881] The device displays the final video to the user and provides a confirmation screen.

[0882] 6. Submission

[0883] Users can then post the final video to social media.

[0884] This will reduce the risk of location identification and provide a system that allows users to safely post images and videos to social media with backgrounds that change naturally based on emotions.

[0885] The processing flow will be explained below.

[0886] Image processing flow

[0887] Step 1:

[0888] Users select the image they want to upload from their device and click the upload button.

[0889] Step 2:

[0890] The terminal transmits the selected image data to the server.

[0891] Step 3:

[0892] The server receives the uploaded image data.

[0893] Step 4:

[0894] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, and input text to recognize their emotions.

[0895] Step 5:

[0896] The emotion engine sends the user's recognized emotion information to the server. For example, if the user is smiling, the emotion of "joy" is recognized.

[0897] Step 6:

[0898] The server passes the received image data to the AI ​​module and begins analyzing the image.

[0899] Step 7:

[0900] The AI ​​module detects elements in the image that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[0901] Step 8:

[0902] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[0903] Step 9:

[0904] The server extracts risk factors and generates a background image that replaces them with other natural background factors. For example, if a risk factor is high, it is set to a tranquil landscape that completely covers it.

[0905] Step 10:

[0906] The server uses emotion information from the emotion engine to adjust the style of background transformation, for example, if the user's emotion is "joy", it selects a light-colored background.

[0907] Step 11:

[0908] The server generates a number of candidate transformed images.

[0909] Step 12:

[0910] The server sends the generated candidate images to the user's terminal.

[0911] Step 13:

[0912] The terminal displays the received candidate images to the user and provides a confirmation screen.

[0913] Step 14:

[0914] The user reviews the different candidate images presented and selects the best one.

[0915] Step 15:

[0916] If the user desires reconversion, he or she inputs the instruction into the terminal.

[0917] Step 16:

[0918] The server receives the user's instruction for reconversion, generates candidate images again, and sends them to the terminal.

[0919] Step 17:

[0920] The user makes the final selection and sends confirmation to the device.

[0921] Step 18:

[0922] The server finalizes the candidate images selected by the user to generate the final image.

[0923] Step 19:

[0924] The server sends the final image to the user's device.

[0925] Step 20:

[0926] The terminal displays the final image to the user and provides an interface for final confirmation.

[0927] Step 21:

[0928] The user then makes a final confirmation and enters instructions into the device to post to social media.

[0929] Step 22:

[0930] The device retrieves the final image from the server and posts it to a social media platform.

[0931] Video processing flow

[0932] Step 1:

[0933] Users select the video they want to upload from their device and click the upload button.

[0934] Step 2:

[0935] The terminal transmits the selected video data to the server.

[0936] Step 3:

[0937] The server receives the uploaded video data.

[0938] Step 4:

[0939] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, and input text to recognize their emotions.

[0940] Step 5:

[0941] The emotion engine sends the recognized user emotion information to the server.

[0942] Step 6:

[0943] The server passes each frame of video data to the AI ​​module and begins analysis.

[0944] Step 7:

[0945] The AI ​​module detects elements in each frame that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[0946] Step 8:

[0947] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[0948] Step 9:

[0949] The server extracts risk elements for each frame and converts them into a background image that replaces them with other natural background elements.

[0950] Step 10:

[0951] The server uses the emotion information from the emotion engine to adjust the style of the background transformation.

[0952] Step 11:

[0953] The server performs the conversion process so that there is no sense of incongruity between consecutive frames, and generates the final video.

[0954] Step 12:

[0955] The server sends the final video to the user's device.

[0956] Step 13:

[0957] The device displays the final video to the user and provides a confirmation screen.

[0958] Step 14:

[0959] The user reviews the final video and enters instructions into the device to post it to social media.

[0960] Step 15:

[0961] The device retrieves the final video from the server and posts it to a social media platform.

[0962] Example 2

[0963] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0964] With conventional SNS posts, the background of images and videos is exposed as is, increasing the risk of location identification. Furthermore, because the background is not changed to take into account the user's feelings, there is also the issue of it being difficult to change the background appropriately. There is a need to solve these problems and provide an environment where users can post images and videos on SNS with peace of mind.

[0965] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0966] In this invention, the server includes means for uploading images, means for recognizing a user's emotion from the uploaded image, means for adjusting background conversion candidates based on the recognized emotion, means for analyzing the uploaded image and extracting risk elements for location identification, means for converting the extracted risk elements into other natural background elements, means for generating a plurality of converted candidate images and presenting them to the user, and means for generating a final image after receiving a selection and confirmation from the user. This allows background conversion according to the user's emotion, reducing the risk of location identification and making it possible to provide images and videos that can be posted safely to SNS.

[0967] "Means for uploading images" is a function that allows a user to send image data selected from the terminal to the server.

[0968] "Means for recognizing user emotions" refers to a function that analyzes emotions from the user's facial expressions, tone of voice, input text, etc., and provides that information to the server.

[0969] The "means for adjusting background conversion candidates" is a function that selects and adjusts suggested background image variations based on the recognized user emotions.

[0970] "Means for extracting risk factors for location identification" is a function that analyzes elements that can lead to location identification from uploaded images and evaluates the risk.

[0971] The "means for converting into background elements" is a function that uses a generative AI model to convert the extracted risk elements into another natural background.

[0972] The "means for generating multiple candidate images and presenting them to the user" is a function for generating multiple converted background variations and displaying them on the user's device.

[0973] The "means for generating a final image" is a function for finalizing the candidate image selected by the user and providing it to the user.

[0974] This invention relates to a system that changes the background of images and videos posted to social networking sites to create a natural look. This system has a function to recognize the user's emotions, reducing the risk of location identification and allowing users to post content with peace of mind. Specific embodiments of this system are described below.

[0975] This system mainly consists of four components: a server, a terminal, a user, and an emotion engine. The server is responsible for analyzing images and videos, converting backgrounds, generating candidates, and generating the final output. The terminal provides the user interface, uploads images and videos, and displays the results. Users upload images and videos through their terminals and select the best one from the converted candidates. The emotion engine recognizes the user's emotions and provides that information to the server.

[0976] Hardware / Software Used

[0977] 1. Server: Analyzes images and videos, converts backgrounds, generates candidates, and generates final output. Specifically, it uses AI frameworks such as TensorFlow and PyTorch.

[0978] 2. Device: Provides the user interface, uploads images and videos, and displays the results. This can be a smartphone or PC.

[0979] 3. Emotion Engine: Recognizes user emotions using computer vision and speech recognition technologies, such as OpenCV and Google Cloud AI's emotion analysis API.

[0980] 4. AI module: Analyzes and evaluates location risks in images. Built using TensorFlow and PyTorch.

[0981] System operation example

[0982] First, the user selects the image they want to post to the SNS from their device and clicks the upload button. The device then sends the selected image data to the server, where it is encoded in Base64 format.

[0983] Next, the device's built-in emotion engine analyzes the user's facial expressions, tone of voice, and input text. For example, if the user is smiling at the camera, the emotion engine recognizes "happiness." The recognition results are sent to the server in JSON format.

[0984] The server inputs the received image data into the AI ​​module, which then analyzes the image's pixel data and uses OCR technology to detect address plates and signs. It also extracts elements that can help identify the location, such as the shape of the building and the background scenery.

[0985] Next, the AI ​​module evaluates the location risk based on each detected element and assigns a score. For example, if an address plate is clearly visible, it is evaluated as a high-risk element. Based on this result, the server lists the risk elements and passes them on to the next conversion process.

[0986] The server applies a conversion algorithm based on this list and emotion data. If the user is in a "happy" mood, the background will be replaced with a "seaside scene." The generated background candidate images are seamlessly synthesized using techniques such as GAN (Generative Adversarial Network).

[0987] The server then generates multiple conversion candidate images and sends them to the terminal for presentation to the user. The terminal displays the received candidate images to the user in thumbnail format and provides a selection screen. The user can select the most appropriate one from the presented candidate images. If necessary, the user can also instruct conversion again.

[0988] Finally, the server processes the selected candidate images, applying edge correction and color balancing before providing them to the user. The device then displays the final image to the user for final confirmation. The user can then post the confirmed image to a social networking site.

[0989] Examples and prompts

[0990] For example, a user selects a family photo from their smartphone's photo library and taps the "Upload" button in the app. The device sends the image to the server and displays the message "Recognizing emotions...". The emotion engine recognizes the user's smile and detects "enjoyment". The AI ​​module marks the house number plate at a specific position in the image as a "high-risk element". Based on the user's emotions, the server generates multiple background images, such as seaside, mountains, and city night views, and displays them on the device. The user selects the best one from these candidates to generate the final image.

[0991] Example prompt sentence:

[0992] "I want to upload photos from a family vacation, but I want to change the background so my home address is not identified. The sentiment is fun."

[0993] "I'm feeling calm right now, so I'd like you to change the background of the videos I upload to natural scenery."

[0994] This allows users to safely post images and videos to social media with reduced risk of their location being identified.

[0995] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0996] Step 1:

[0997] Image upload

[0998] The user selects an image from the file selection dialog on the device and clicks the upload button.

[0999] Input: An image file selected by the user.

[1000] The device sends the selected image data to the server via an HTTP request, where the image data is encoded in Base64 format.

[1001] Output: Base64 encoded image data is sent to the server.

[1002] Step 2:

[1003] emotion recognition

[1004] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, and input text.

[1005] Input: User's real-time facial expression data, voice data, and input text data.

[1006] The device uses emotion recognition algorithms to analyze the user's emotions in real time. For example, if the user smiles at the camera, it will recognize "happiness."

[1007] The emotion engine sends the recognized emotion data in JSON format to the server.

[1008] Output: Emotion data in JSON format is sent to the server.

[1009] Step 3:

[1010] Image analysis

[1011] The server passes the received image data to an AI module (e.g., TensorFlow or PyTorch model) and begins analysis.

[1012] Input: Base64 encoded image data.

[1013] The AI ​​module analyzes the pixel data of the image and uses OCR technology to detect address plates and signs, as well as extracting elements relevant to location identification, such as building shapes and background scenery.

[1014] Output: The analysis results in a list of risk factors.

[1015] Step 4:

[1016] Risk Assessment

[1017] The AI ​​module assesses the location risk of each detected element and assigns a risk score.

[1018] Input: A list of risk factors.

[1019] The server lists elements with high risk scores and passes the data to the next background conversion process.

[1020] Output: A list of elements for which risk assessment has been completed is generated.

[1021] Step 5:

[1022] Background Transformation

[1023] The server applies a transformation algorithm based on the risk factor list and emotion data.

[1024] Input: List of risk-assessed elements, sentiment data.

[1025] The conversion algorithm generates the most appropriate background based on the user's emotions. For example, if the user is in a happy mood, the background will be replaced with a bright seaside scene.

[1026] The generative AI model uses a Generative Adversarial Network (GAN) to generate new backgrounds and seamlessly combine them with the original image.

[1027] Output: Multiple composite background candidate images.

[1028] Step 6:

[1029] Candidate generation and presentation

[1030] The server generates a plurality of candidate conversion images and transmits them to the terminal for presentation to the user.

[1031] Input: Multiple synthesized background candidate images.

[1032] The terminal displays the received candidate images to the user in thumbnail format and provides a selection screen.

[1033] Output: Candidate images in thumbnail format presented to the user.

[1034] Step 7:

[1035] User Selection

[1036] The user selects the best image from the different candidate images presented.

[1037] Input: Candidate images displayed in thumbnail format.

[1038] The terminal accepts the user's selection and transmits the selection information to the server. If the user wishes to reconvert, the terminal can also transmit an instruction to the server.

[1039] Output: Data of selected candidate images.

[1040] Step 8:

[1041] Generating the final image

[1042] The server performs final processing on the selected candidate images, applying edge correction and color balancing.

[1043] Input: The data of the selected candidate image.

[1044] The server sends the final image back to the terminal for display to the user.

[1045] The terminal displays the final image to the user for final confirmation.

[1046] Output: The final image.

[1047] Step 9:

[1048] Post

[1049] Users can then post the final image they have confirmed to social media.

[1050] Input: The final image as viewed by the user.

[1051] The device uses the SNS API to upload the image to the specified account.

[1052] Output: Images posted to social media.

[1053] (Application example 2)

[1054] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1055] Conventional image and video posting systems pose the risk of revealing the location where a user took a photo. In particular, analyzing features such as buildings and signs in the background of a photo can make the location easily identifiable. Furthermore, changing the background without considering the user's emotions can result in lower user satisfaction. A new system is needed to solve these problems and enable safer and more satisfying image and video posting.

[1056] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1057] In this invention, the server includes means for uploading images, means for analyzing the background from the uploaded image and extracting risk elements for location identification, means for converting the extracted risk elements into other natural background elements, means for generating candidates for background conversion based on emotions using an emotion engine that recognizes the user's emotions, means for generating a plurality of converted candidate images and presenting them to the user, and means for generating a final image after receiving a selection and confirmation from the user. This enables a natural background change according to the user's emotions while reducing the risk of location identification.

[1058] The "means for uploading images" provides a function for users to send image data they have taken to the system.

[1059] "Means for analyzing the background and extracting risk elements for identifying a location" provides the functionality to detect and analyze elements (e.g., buildings and signs) that identify a specific location from uploaded images.

[1060] The "means for converting extracted risk elements into other natural background elements" provides a function for replacing detected location-specific risk elements with other safe and natural backgrounds.

[1061] The "emotion engine that recognizes user emotions" is a mechanism for analyzing and recognizing a user's emotional state from facial expressions, tone of voice, input text, etc.

[1062] The "means for generating background conversion candidates based on emotions" provides a function for generating optimal background conversion candidates according to the recognized emotions of the user.

[1063] The "means for generating a plurality of converted candidate images and presenting them to the user" provides a function for displaying a plurality of background conversion candidate images generated based on emotion to the user and making them selectable.

[1064] The "means for generating a final image upon receiving a user's selection and confirmation" provides a function for applying the background conversion candidate selected by the user and generating a final image.

[1065] MODE FOR CARRYING OUT THE INVENTION

[1066] This invention is a system that changes the background of images and videos posted on social media in a natural way to reduce the risk of location identification, and combines it with an emotion engine that recognizes the user's emotions. The system works as follows.

[1067] System Configuration

[1068] This system mainly consists of four components: a server, a device (smartphone), a user, and an emotion engine. The server is responsible for the main processes, such as analyzing images and videos, converting backgrounds, generating candidates, and generating the final output. The device provides the user interface, uploads images and videos, and displays the results. Users upload images and videos through their devices and select the best converted background from the candidates. The emotion engine recognizes the user's emotions and provides that information to the server.

[1069] What the program does

[1070] 1. Image upload: The user selects an image from their smartphone and uploads it to the app. At this time, the image data is read using an image processing library such as OpenCV.

[1071] 2. Emotion recognition: The device's built-in emotion engine (e.g., Google Cloud's Face API) recognizes emotions from the user's facial expressions and input text. This information is sent to the server.

[1072] 3. Background analysis: The server passes the received image data to the AI ​​module, which then begins analysis. The AI ​​module detects elements in the image that can lead to location identification, such as building shapes, signs, and background scenery.

[1073] 4. Risk Assessment: The AI ​​module assesses the risk of location identification based on the detected elements and sets a risk level for each element.

[1074] 5. Background conversion: The server extracts risk factors and converts the background based on information from the emotion engine. For example, if the user is in a happy mood, the background will be changed to a bright one, and if the user is in a calm mood, the background will be changed to a natural landscape.

[1075] 6. Candidate generation and presentation: The server generates multiple background conversion candidate images and sends them to the terminal for presentation to the user.

[1076] 7. User Selection: The user reviews the different candidate images and selects the best one. If necessary, the user can also request reconversion.

[1077] 8. Final image generation: The server finalizes the selected candidate images and sends them to the user. The terminal displays the final image to the user for final confirmation.

[1078] 9. Post: Users can post the final image to social media.

[1079] Hardware and software used

[1080] Hardware: Smartphone (iOS or Android compatible)

[1081] Software: OpenCV, emotion engine libraries (e.g. Google Cloud's Face API), and Python and TensorFlow on the server side for image analysis and background conversion.

[1082] Specific examples

[1083] For example, if a user uploads a smiling selfie, the emotion engine will determine the emotion of "joy." After analyzing risk factors, the server will generate a bright, natural background. The user can then select the best image from the presented candidate images and safely post it to social media.

[1084] Prompt Sentence Examples

[1085] Identify the emotion in an image uploaded by a user and automatically select the background that best matches that emotion to change the image background. For example, change the background to a bright natural scene for a "smiling face." This reduces the risk of location identification based on the original image.

[1086] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1087] Step 1:

[1088] The device provides an interface for the user to select an image from within the smartphone. The user selects an image and clicks the upload button within the app to input the image data into the device. The device then sends the selected image data to the server.

[1089] Step 2:

[1090] The server passes the received image data to the AI ​​module, which begins background analysis. Specifically, it uses OpenCV and an AI model to detect elements in the image that identify specific locations. The AI ​​model identifies location-specific risk elements, such as buildings and signs, and outputs their location information.

[1091] Step 3:

[1092] The server evaluates the risk of the location based on the output from the AI ​​module. The risk assessment is based on the type of element detected and its risk level. The output is a list of the risk level for each element.

[1093] Step 4:

[1094] The emotion engine installed on the device analyzes the user's emotions before or immediately after uploading. This emotion analysis is performed based on the user's facial expressions, voice, input text, etc. The emotion engine sends the analysis results to the server, which receives the user's emotional information as input.

[1095] Step 5:

[1096] The server performs background conversion based on the list of location-specific risk factors and the user's emotional information. Specifically, it removes or converts risk factors and generates an appropriate background based on the emotional information. For example, if the user is in a happy emotional state, the system generates a bright natural landscape.

[1097] Step 6:

[1098] The server prepares the generated multiple background conversion candidates for display and sends them to the device. The device presents the multiple conversion candidate images to the user and displays a selection screen. The user can review the conversion candidate images and select the most appropriate one.

[1099] Step 7:

[1100] The server receives the user-selected background candidate image as input and generates the final image. Specifically, it applies the selected background to the original image, adjusts unnecessary areas, and performs final processing.

[1101] Step 8:

[1102] The server sends the final image to the user's device, which displays it to the user, who can then post it to social media after reviewing it.

[1103] Step 9:

[1104] The user posts the final image to the SNS. Specifically, a dedicated button is provided to upload the generated final image data to the SNS, completing the posting process.

[1105] The above steps reduce the risk of location identification while changing the background according to the user's emotions, creating an environment in which images and videos can be posted to social media more safely.

[1106] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1107] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1108] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1109] [Third embodiment]

[1110] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1111] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1112] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1113] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1114] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1115] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1116] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1117] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1118] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1119] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1120] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1121] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1122] This invention relates to a system that changes the background of images and videos posted to social networking sites in a natural way to reduce the risk of location identification. The program processing of this system is specifically explained below in natural language.

[1123] System Overview

[1124] The system is mainly composed of three entities: the server, the device, and the user. The server is responsible for the main processes, such as image and video analysis, background conversion, candidate generation, and final output generation. The device provides the user interface, uploads images and videos, and displays the results. The user uploads images and videos through the device and selects the best one from the converted candidates.

[1125] Program processing

[1126] 1. Upload an image

[1127] Users select an image from their device and click the upload button.

[1128] The terminal transmits the selected image data to the server.

[1129] 2. Image Analysis

[1130] The server passes the received image data to the AI ​​module.

[1131] The AI ​​module analyzes the images and detects elements that could lead to location identification, such as building shapes, address plates, and signs.

[1132] 3. Risk Assessment

[1133] The AI ​​module assesses the location risk based on the detected elements and assigns a risk level to each element.

[1134] 4. Background conversion

[1135] The server extracts risk elements and generates a background image that replaces them with other natural elements.

[1136] Example: In a photo that shows Tokyo Tower, replace the Tokyo Tower with natural scenery or an ordinary apartment building.

[1137] 5. Candidate Generation and Presentation

[1138] The server generates multiple candidate conversion images and transmits them to the terminal for presentation to the user.

[1139] The terminal displays the received candidate images to the user and provides a selection screen.

[1140] 6. User Choice

[1141] The user checks the presented candidate images and selects the appropriate one.

[1142] If necessary, the user can also instruct reconversion.

[1143] 7. Generating the final image

[1144] The server finalizes the selected candidate images and sends them to the user.

[1145] The device displays the final image to the user for final confirmation.

[1146] 8. Submission

[1147] Users can then post the final image to social media.

[1148] Application to video

[1149] The same process is applied to videos, but the same process is performed on each frame while maintaining continuity, resulting in a natural background transition for the entire video.

[1150] 1. Upload a video

[1151] The user selects a video from their device and clicks the upload button.

[1152] The device transmits the video file to the server.

[1153] 2. Frame analysis and transformation

[1154] The server receives the video data and inputs each frame into the AI ​​module.

[1155] The AI ​​module analyzes each frame, detects elements that pose a risk of location identification, and transforms them into natural backgrounds.

[1156] 3. Maintaining continuity

[1157] The server performs conversion processing that takes into account the continuity of frames in order to generate images that do not look out of place between consecutive frames.

[1158] 4. Final video generation

[1159] The server generates the final video and sends it to the device.

[1160] The device will then display the final video to the user and ask for confirmation.

[1161] 5. Post

[1162] Users can then post the final video to social media.

[1163] This makes it possible to provide a system that reduces the risk of location identification and allows users to safely post images and videos with natural-looking backgrounds to social media.

[1164] The processing flow will be explained below.

[1165] Step 1:

[1166] Users select the image they want to upload from their device and click the upload button.

[1167] Step 2:

[1168] The terminal transmits the selected image data to the server.

[1169] Step 3:

[1170] The server receives the uploaded image data.

[1171] Step 4:

[1172] The server passes the image to the AI ​​module and begins analysis.

[1173] Step 5:

[1174] The AI ​​module detects elements in the image that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[1175] Step 6:

[1176] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[1177] Step 7:

[1178] The server extracts risk elements and generates a background image that replaces them with other natural background elements.

[1179] Step 8:

[1180] The server generates a number of candidate transformed images.

[1181] Step 9:

[1182] The server sends the generated candidate images to the user's terminal.

[1183] Step 10:

[1184] The terminal displays the received candidate images to the user and provides a confirmation screen.

[1185] Step 11:

[1186] The user reviews the different candidate images presented and selects the best one.

[1187] Step 12:

[1188] If the user desires reconversion, he or she inputs the instruction into the terminal.

[1189] Step 13:

[1190] The server receives the user's instruction for reconversion, generates candidate images again, and sends them to the terminal.

[1191] Step 14:

[1192] The user makes the final selection and sends confirmation to the device.

[1193] Step 15:

[1194] The server finalizes the candidate images selected by the user to generate the final image.

[1195] Step 16:

[1196] The server sends the final image to the user's device.

[1197] Step 17:

[1198] The terminal displays the final image to the user and provides an interface for final confirmation.

[1199] Step 18:

[1200] The user then makes a final confirmation and enters instructions into the device to post to social media.

[1201] Step 19:

[1202] The device retrieves the final image from the server and posts it to a social media platform.

[1203] The above is the flow of image processing.

[1204] Application to video

[1205] Step 1:

[1206] Users select the video they want to upload from their device and click the upload button.

[1207] Step 2:

[1208] The terminal transmits the selected video data to the server.

[1209] Step 3:

[1210] The server receives the uploaded video data.

[1211] Step 4:

[1212] The server passes each frame of the video to the AI ​​module and begins analysis.

[1213] Step 5:

[1214] The AI ​​module detects elements in each frame that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[1215] Step 6:

[1216] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[1217] Step 7:

[1218] The server extracts risk elements for each frame and generates a background image that replaces them with other natural background elements.

[1219] Step 8:

[1220] The server performs the conversion process so that there is no sense of incongruity between consecutive frames, and generates the final video.

[1221] Step 9:

[1222] The server sends the final video to the user's device.

[1223] Step 10:

[1224] The device displays the final video to the user and provides a confirmation screen.

[1225] Step 11:

[1226] The user reviews the final video and enters instructions into the device to post it to social media.

[1227] Step 12:

[1228] The device retrieves the final video from the server and posts it to a social media platform.

[1229] Example 1

[1230] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1231] When posting images or videos to social networking sites, there is a risk that elements such as buildings, address plates, and signs that appear in the background may be identified by a third party. This risk can lead to a violation of privacy, making it difficult for posters to use social networking sites safely. Furthermore, in the case of videos, background conversion technology is also required to prevent the appearance of incongruity between consecutive frames. Therefore, a system is needed that can convert backgrounds to natural ones while reducing the risk of location identification.

[1232] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1233] In this invention, the server includes means for uploading images or videos, means for analyzing the background from each frame of the uploaded image or video and extracting risk elements for location identification, means for converting the extracted risk elements into other natural background elements, means for generating a natural background image using a generative AI model and replacing the risk elements for location identification, means for generating a plurality of candidate converted images or videos and presenting them to the user, and means for generating a final image or video after receiving a selection and confirmation from the user. This makes it possible to achieve conversion to a natural and safe background while reducing the risk of location identification.

[1234] "Means for uploading images or videos" refers to the function for sending image or video data from a user's device to a server.

[1235] "Means for analyzing the background from each frame of an image or video and extracting location-specific risk elements" refers to a function that uses an AI module to analyze each frame of an image or video and detect location-specific risk elements such as buildings, address plates, and signs.

[1236] "Means for transforming location risk elements into other natural background elements" refers to the function responsible for the transformation process to replace location risk elements with other natural backgrounds.

[1237] "Means for generating natural-looking background images using generative AI models to replace location-specific risk elements" refers to a function that uses AI technology to generate new background images and replace them with location-specific risk elements, thereby providing a natural-looking background.

[1238] "Means for generating multiple converted candidate images or videos and presenting them to the user" refers to a function that allows the server to present candidate images or videos generated using multiple conversion patterns to the user.

[1239] "Means for generating a final image or video upon user selection and confirmation" refers to the function that allows the user to finally confirm the image or video selected from the candidates and generate the approved final image or video.

[1240] This invention relates to a system that changes the background of images and videos to a more natural look when they are posted to a social networking site, reducing the risk of location identification. This system is composed of three entities: a server, a terminal, and a user.

[1241] System configuration

[1242] The system mainly uses the following major hardware and software:

[1243] 1. Server

[1244] Hardware: A computer system with a powerful processor, memory, and storage devices.

[1245] Software: Install AI modules (e.g., OpenCV, TensorFlow), image processing libraries, and generative AI models (e.g., DALL-E, GAN).

[1246] 2. Terminal

[1247] Hardware: The devices that users regularly use, such as smartphones, tablets, and PCs.

[1248] Software: A dedicated application or web browser for uploading images and videos and communicating with the server.

[1249] 3. Users

[1250] An individual who uses the system to upload images and videos to be posted on social media and check the final generated content.

[1251] System Features

[1252] Image / Video Upload

[1253] Users select the image or video they want to post from their device and click the upload button. The device then sends the selected image or video data to the server.

[1254] Image / video analysis

[1255] The server passes the received image or video data to an AI module (e.g., OpenCV or TensorFlow) for background analysis and location-specific risk element detection.

[1256] Location-specific element detection and risk assessment

[1257] The AI ​​module detects location-specific risk factors such as buildings, address plates, and signs, and the server performs risk assessment based on these risk factors.

[1258] Converting to a natural background

[1259] The server uses a generative AI model (e.g., DALL-E, GAN) to replace risk elements with a different, more natural background image. Specifically, it uses a generative AI model (e.g., DALL-E, GAN) and inputs the following prompt:

[1260] Example prompt: "Change the background of a photo showing Tokyo Tower to a typical apartment building, and generate an image that eliminates the risk of location identification."

[1261] Generation and presentation of candidate images / videos

[1262] The server generates multiple candidates for the converted images and videos, and sends them to the device, which then presents the candidate images and videos to the user.

[1263] User selection and confirmation

[1264] The user can select and confirm the most suitable image or video from the presented candidates. If necessary, the user can also instruct reconversion.

[1265] Final image / video generation

[1266] The server generates the selected final image or video and sends it to the device, which displays the final output to the user and supports posting to social media.

[1267] This allows users to safely post images and videos with natural-looking backgrounds to social media, while reducing the risk of their location being identified.

[1268] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1269] Step 1:

[1270] Image / Video Upload

[1271] Users use their device to select the image or video they want to post and click the upload button.

[1272] Input: An image or video file selected by the user.

[1273] Output: Image or video data sent from the device to the server.

[1274] Specific operation: Triggered by user operation, the device sends an HTTP request to the server for the selected file, including image or video data.

[1275] Step 2:

[1276] Image / video analysis

[1277] The server passes the received image or video data to the AI ​​module for analysis.

[1278] Input: Image or video data received by the server.

[1279] Output: Analysis results from the AI ​​module (detection of location-specific risk elements such as buildings, house numbers, signs, etc.).

[1280] Specific operation: The server inputs the received image or video data into an AI module such as OpenCV or TensorFlow, and analyzes the location-specific risk elements for each frame, extracting the location information and characteristics of the risk elements as a result.

[1281] Step 3:

[1282] Location-specific element detection and risk assessment

[1283] The AI ​​module assesses the location risk based on the detected elements.

[1284] Input: Analyzed risk elements (buildings, house numbers, signs, etc.).

[1285] Output: Risk level assessment for each risk factor.

[1286] Specific operation: Based on the analysis results, the AI ​​module returns the location-specific risk level of each risk element to the server, which then receives this and performs risk assessment.

[1287] Step 4:

[1288] Converting to a natural background

[1289] The server uses a generative AI model to replace risk elements with alternative, natural-looking background images.

[1290] Input: Risk assessment results and original image or video data.

[1291] Output: The generated natural background image, or the altered image or video data with the risk elements replaced.

[1292] How it works: The server uses a generative AI model (e.g., DALL-E, GAN) to replace risk elements with a different background. Specifically, it generates a prompt sentence and inputs it into the AI ​​model to generate an appropriate background image.

[1293] Example prompt: "Change the background of a photo showing Tokyo Tower to a typical apartment building, and generate an image that eliminates the risk of location identification."

[1294] Step 5:

[1295] Generation and presentation of candidate images / videos

[1296] The server generates multiple candidate images or videos using different transformation patterns and sends them to the terminal.

[1297] Input: Generated background image or modified image / video data.

[1298] Output: Multiple candidate images or videos.

[1299] Specific operation: The server synthesizes the background image obtained by the generative AI model with the original image or video, generates multiple candidates with different conversion patterns, and sends them to the device as an HTTP response.

[1300] Step 6:

[1301] User selection and confirmation

[1302] The user selects and confirms the most suitable image or video from the presented candidates.

[1303] Input: Multiple candidate images or videos, selected by the user.

[1304] Output: User-selected final candidate image or video, with instructions for reconversion if necessary.

[1305] Specific operation: The device receives multiple candidate images or videos from the server, displays them on the user interface, and allows the user to select the most suitable one. Once the user has made their selection, the selection information is sent to the server.

[1306] Step 7:

[1307] Final image / video generation

[1308] The server performs final processing on the selected candidate images and videos and sends them to the terminal.

[1309] Input: Information about the candidate image or video selected by the user.

[1310] Output: The final image or video data.

[1311] Specific operation: The server checks the candidate images and videos selected by the user, performs any necessary final processing, and then sends the final data to the device for user confirmation.

[1312] Step 8:

[1313] Posting to social media

[1314] Users post the final images and videos to social media.

[1315] Input: The final image or video data.

[1316] Output: Images or videos posted to social media.

[1317] Specific operation: The device displays the final image or video to the user, and if the user is satisfied, provides the ability to post it to a social networking site. When the user clicks the post button, the data is uploaded to the social networking site.

[1318] (Application example 1)

[1319] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1320] In modern society, when posting images or videos to social media, there is an increasing risk that individuals and locations may be identified from the content of the posts. It is necessary to resolve these privacy issues and provide an environment where users can share content with peace of mind.

[1321] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1322] In this invention, the server includes means for uploading images or videos, means for analyzing the background from the uploaded images or videos and extracting risk elements for location identification, means for converting the extracted risk elements into other natural background elements, means for generating a plurality of converted candidate images or video frames and presenting them to the user, means for the user to select an appropriate one from the presented candidates, means for generating a final image or final video frame upon receiving the user's selection and confirmation, and means for posting the final image or final video frame to an SNS. This allows users to post images or videos to an SNS with peace of mind while reducing the risk of location identification.

[1323] "Means for uploading images or videos" refers to a function that allows a user to select an image or video from their device and send it to the server.

[1324] "Means for analyzing the background and extracting location-specific risk elements" is a function that uses an AI module to identify and extract location-specific risk elements contained in the background.

[1325] The "means for converting risk elements into other natural background elements" is an algorithm for replacing the extracted risk elements with other natural background elements.

[1326] The "means for generating multiple candidate transformed images or video frames and presenting them to the user" refers to a function for creating different versions of transformed images or video frames and offering them to the user as options.

[1327] The "means for the user to select the appropriate one" is an interface that allows the user to select the most appropriate one from multiple candidate images or video frames presented.

[1328] "Means for generating a final image or video frame upon receiving a user selection and confirmation" refers to functionality for creating a final image or video frame based on the user's selection.

[1329] The "means for posting the final image or final video frame to an SNS" is a function for uploading the generated final image or video frame to an SNS.

[1330] The "means for converting into natural background elements while maintaining the continuity of video frames" is an algorithm for performing background conversion that does not look out of place between a series of video frames.

[1331] A "means for enabling reconversion at the user's command" is an interface that allows the user to request conversion again.

[1332] This invention relates to a system that changes the background of images and videos posted on social networking sites in a natural way, reducing the risk of location identification. The system is mainly composed of three entities: a server, a device, and a user.

[1333] Hardware and software used

[1334] 1. Hardware:

[1335] Server: A server with high-performance computing power.

[1336] Devices: Smartphones, smart glasses, head-mounted displays.

[1337] 2. Software:

[1338] AI module: Uses TensorFlow or PyTorch for image analysis and background conversion.

[1339] Image processing library: OpenCV is used for various editing of generated images.

[1340] Communication and data management: Flask is used for communication between the server and the terminal.

[1341] System Operation

[1342] 1. Upload an image or video:

[1343] Users select images or videos from their smartphones or other devices and upload them to the server through the application.

[1344] 2. Background analysis and risk factor extraction:

[1345] The server receives the uploaded content and passes it to an AI module (e.g., TensorFlow, PyTorch), which analyzes the image or video and extracts location-specific risk factors (e.g., building shapes, house number plates, signs, etc.).

[1346] 3. Background transformation:

[1347] After extracting risk elements, the AI ​​module converts them into other natural background elements. For example, if a photo shows Tokyo Tower, it will replace the Tokyo Tower with natural scenery or a typical apartment building.

[1348] 4. Generate and present the transformed candidate image or video frame:

[1349] The server generates multiple conversion candidates and sends them to the terminal for presentation to the user, who then displays them and offers a choice.

[1350] 5. User Choice and Confirmation:

[1351] The user can select the most suitable one from the presented candidates, and can also instruct reconversion if necessary.

[1352] 6. Generate the final image or video frame:

[1353] The server generates the final image or video frame based on the user's selection and sends it to the user, who then displays it on the device and asks for final confirmation.

[1354] 7. Posting to social media:

[1355] Users can post the final image or video frame to social media.

[1356] Specific examples

[1357] For example, if a user wants to post a photo taken at a tourist spot to a social networking site, the system can detect famous landmarks or signs in the photo and replace them with other natural backgrounds, allowing users to enjoy posting while reducing the risk of being identified.

[1358] Prompt Sentence Examples

[1359] Analyze the input image and detect location risk elements. Next, replace those elements with a natural background and generate an image. Use a method to present multiple candidate images. In this case, the generated image must be consistent between consecutive video frames.

[1360] This will allow users to post images and videos on social media with peace of mind, strengthening privacy protection.

[1361] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1362] Step 1:

[1363] The user selects an image or video from the device and uploads it to the server through the application. The input is the image or video file selected on the device, and the output is the image or video data sent to the server. Specifically, the user clicks the "Upload" button and selects the specified file.

[1364] Step 2:

[1365] The server receives uploaded image or video data and passes it to an AI module (e.g., TensorFlow, PyTorch). The input is the uploaded image or video data, and the output is the analysis request passed to the AI ​​module. Specifically, the server saves the received data in a specified folder and passes the path to the AI ​​module.

[1366] Step 3:

[1367] The AI ​​module analyzes images or videos to extract location-specific risk elements. The input is the uploaded image or video data, and the output is a list of detected risk elements. Specifically, the AI ​​module uses a convolutional neural network (CNN) to identify risk elements in the image, such as building shapes, house number plates, and signs.

[1368] Step 4:

[1369] The server receives the list of risk elements from the AI ​​module and converts them into natural background elements. The input is the list of extracted risk elements, and the output is the converted background image or video frames composited onto the background. Specifically, the server uses the OpenCV library to replace the risk elements with common landscapes and buildings.

[1370] Step 5:

[1371] The server generates multiple candidate transformed images or video frames and sends them to the device for presentation to the user. The input is the transformed background image or video frame, and the output is multiple candidate images or video frames. Specifically, the server generates different background variations, saves each as a candidate, and sends it to the device.

[1372] Step 6:

[1373] The user selects the best one from multiple candidates presented on the device and sends the selection information to the server. The input is multiple candidate images or video frames, and the output is the user's selection information. Specifically, the user selects one of the presented candidate images or video frames and clicks the "Select" button.

[1374] Step 7:

[1375] The server receives the user's selection information and generates the final image or video frame. The input is the user's selection information and the corresponding candidate image or video frame, and the output is the final image or video frame. Specifically, the server performs final processing on the selected background to generate the final image or video.

[1376] Step 8:

[1377] The terminal displays the final image or video frame to the user and asks for the user's final confirmation. The input is the final image or video frame, and the output is the user's final confirmation information. Specifically, the terminal displays the final image or video and provides a "Confirm" button.

[1378] Step 9:

[1379] The user posts the final image or video after confirmation to the social networking site. The input is the image or video frame that has been finalized, and the output is the information to be posted to the social networking site. Specifically, the user clicks the "Post" button to upload the image or video to the social networking site.

[1380] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1381] This invention combines a system that changes the background of images and videos posted to social networking sites in a natural way to reduce the risk of location identification with an emotion engine that recognizes the user's emotions.The system adjusts background conversion candidates according to the user's emotions, allowing it to provide more appropriate images and videos.

[1382] System Overview

[1383] The system is mainly composed of four entities: the server, the device, the user, and the emotion engine. The server is responsible for the main processes, such as image and video analysis, background conversion, candidate generation, and final output generation. The device provides the user interface, uploads images and videos, and displays the results. The user uploads images and videos through the device and selects the best one from the converted candidates. The emotion engine recognizes the user's emotions and provides this information to the server.

[1384] Program processing

[1385] 1. Upload an image

[1386] Users select an image from their device and click the upload button.

[1387] The terminal transmits the selected image data to the server.

[1388] 2. Emotion recognition

[1389] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, input text, etc. to recognize emotions.

[1390] The emotion engine sends the recognized emotion information to the server.

[1391] 3. Image Analysis

[1392] The server passes the received image data to the AI ​​module and begins analysis.

[1393] The AI ​​module detects elements in the image that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[1394] 4. Risk Assessment

[1395] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[1396] 5. Background conversion

[1397] The server extracts risk factors and changes the background based on information from the emotion engine. For example, if the user is in a happy mood, the background will be bright, and if the user is in a calm mood, the background will be changed to a natural landscape.

[1398] For example, if a photo shows Tokyo Tower, and the user is feeling calm, the Tokyo Tower part can be replaced with mountains or a lake.

[1399] 6. Candidate Generation and Presentation

[1400] The server generates multiple candidate conversion images and transmits them to the terminal for presentation to the user.

[1401] The terminal displays the received candidate images to the user and provides a selection screen.

[1402] 7. User Choice

[1403] The user reviews the different candidate images presented and selects the best one.

[1404] If necessary, the user can also instruct reconversion.

[1405] 8. Generating the final image

[1406] The server finalizes the selected candidate images and sends them to the user.

[1407] The device displays the final image to the user for final confirmation.

[1408] 9. Submission

[1409] Users can then post the final image to social media.

[1410] Application to video

[1411] The same process is applied to videos, but the same process is performed on each frame while maintaining continuity, resulting in a natural background transition for the entire video.

[1412] 1. Upload a video

[1413] Users select a video from their device and click the upload button.

[1414] The device transmits the video file to the server.

[1415] 2. Emotion recognition

[1416] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, input text, etc. to recognize emotions.

[1417] The emotion engine sends the recognized emotion information to the server.

[1418] 3. Frame Analysis and Transformation

[1419] The server receives the video data and inputs each frame into the AI ​​module.

[1420] The AI ​​module analyzes each frame, detects elements that pose a risk of location identification, and transforms them into natural backgrounds.

[1421] 4. Maintaining continuity

[1422] The server performs the conversion process so that there is no sense of incongruity between consecutive frames, and generates the final video.

[1423] 5. Final video generation

[1424] The server generates the final video and offers variations based on the user's emotions.

[1425] The device displays the final video to the user and provides a confirmation screen.

[1426] 6. Submission

[1427] Users can then post the final video to social media.

[1428] This will reduce the risk of location identification and provide a system that allows users to safely post images and videos to social media with backgrounds that change naturally based on emotions.

[1429] The processing flow will be explained below.

[1430] Image processing flow

[1431] Step 1:

[1432] Users select the image they want to upload from their device and click the upload button.

[1433] Step 2:

[1434] The terminal transmits the selected image data to the server.

[1435] Step 3:

[1436] The server receives the uploaded image data.

[1437] Step 4:

[1438] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, and input text to recognize their emotions.

[1439] Step 5:

[1440] The emotion engine sends the user's recognized emotion information to the server. For example, if the user is smiling, the emotion of "joy" is recognized.

[1441] Step 6:

[1442] The server passes the received image data to the AI ​​module and begins analyzing the image.

[1443] Step 7:

[1444] The AI ​​module detects elements in the image that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[1445] Step 8:

[1446] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[1447] Step 9:

[1448] The server extracts risk factors and generates a background image that replaces them with other natural background factors. For example, if a risk factor is high, it is set to a tranquil landscape that completely covers it.

[1449] Step 10:

[1450] The server uses emotion information from the emotion engine to adjust the style of background transformation, for example, if the user's emotion is "joy", it selects a light-colored background.

[1451] Step 11:

[1452] The server generates a number of candidate transformed images.

[1453] Step 12:

[1454] The server sends the generated candidate images to the user's terminal.

[1455] Step 13:

[1456] The terminal displays the received candidate images to the user and provides a confirmation screen.

[1457] Step 14:

[1458] The user reviews the different candidate images presented and selects the best one.

[1459] Step 15:

[1460] If the user desires reconversion, he or she inputs the instruction into the terminal.

[1461] Step 16:

[1462] The server receives the user's instruction for reconversion, generates candidate images again, and sends them to the terminal.

[1463] Step 17:

[1464] The user makes the final selection and sends confirmation to the device.

[1465] Step 18:

[1466] The server finalizes the candidate images selected by the user to generate the final image.

[1467] Step 19:

[1468] The server sends the final image to the user's device.

[1469] Step 20:

[1470] The terminal displays the final image to the user and provides an interface for final confirmation.

[1471] Step 21:

[1472] The user then makes a final confirmation and enters instructions into the device to post to social media.

[1473] Step 22:

[1474] The device retrieves the final image from the server and posts it to a social media platform.

[1475] Video processing flow

[1476] Step 1:

[1477] Users select the video they want to upload from their device and click the upload button.

[1478] Step 2:

[1479] The terminal transmits the selected video data to the server.

[1480] Step 3:

[1481] The server receives the uploaded video data.

[1482] Step 4:

[1483] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, and input text to recognize their emotions.

[1484] Step 5:

[1485] The emotion engine sends the recognized user emotion information to the server.

[1486] Step 6:

[1487] The server passes each frame of video data to the AI ​​module and begins analysis.

[1488] Step 7:

[1489] The AI ​​module detects elements in each frame that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[1490] Step 8:

[1491] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[1492] Step 9:

[1493] The server extracts risk elements for each frame and converts them into a background image that replaces them with other natural background elements.

[1494] Step 10:

[1495] The server uses the emotion information from the emotion engine to adjust the style of the background transformation.

[1496] Step 11:

[1497] The server performs the conversion process so that there is no sense of incongruity between consecutive frames, and generates the final video.

[1498] Step 12:

[1499] The server sends the final video to the user's device.

[1500] Step 13:

[1501] The device displays the final video to the user and provides a confirmation screen.

[1502] Step 14:

[1503] The user reviews the final video and enters instructions into the device to post it to social media.

[1504] Step 15:

[1505] The device retrieves the final video from the server and posts it to a social media platform.

[1506] Example 2

[1507] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1508] With conventional SNS posts, the background of images and videos is exposed as is, increasing the risk of location identification. Furthermore, because the background is not changed to take into account the user's feelings, there is also the issue of it being difficult to change the background appropriately. There is a need to solve these problems and provide an environment where users can post images and videos on SNS with peace of mind.

[1509] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1510] In this invention, the server includes means for uploading images, means for recognizing a user's emotion from the uploaded image, means for adjusting background conversion candidates based on the recognized emotion, means for analyzing the uploaded image and extracting risk elements for location identification, means for converting the extracted risk elements into other natural background elements, means for generating a plurality of converted candidate images and presenting them to the user, and means for generating a final image after receiving a selection and confirmation from the user. This allows background conversion according to the user's emotion, reducing the risk of location identification and making it possible to provide images and videos that can be posted safely to SNS.

[1511] "Means for uploading images" is a function that allows a user to send image data selected from the terminal to the server.

[1512] "Means for recognizing user emotions" refers to a function that analyzes emotions from the user's facial expressions, tone of voice, input text, etc., and provides that information to the server.

[1513] The "means for adjusting background conversion candidates" is a function that selects and adjusts suggested background image variations based on the recognized user emotions.

[1514] "Means for extracting risk factors for location identification" is a function that analyzes elements that can lead to location identification from uploaded images and evaluates the risk.

[1515] The "means for converting into background elements" is a function that uses a generative AI model to convert the extracted risk elements into another natural background.

[1516] The "means for generating multiple candidate images and presenting them to the user" is a function for generating multiple converted background variations and displaying them on the user's device.

[1517] The "means for generating a final image" is a function for finalizing the candidate image selected by the user and providing it to the user.

[1518] This invention relates to a system that changes the background of images and videos posted to social networking sites to create a natural look. This system has a function to recognize the user's emotions, reducing the risk of location identification and allowing users to post content with peace of mind. Specific embodiments of this system are described below.

[1519] This system mainly consists of four components: a server, a terminal, a user, and an emotion engine. The server is responsible for analyzing images and videos, converting backgrounds, generating candidates, and generating the final output. The terminal provides the user interface, uploads images and videos, and displays the results. Users upload images and videos through their terminals and select the best one from the converted candidates. The emotion engine recognizes the user's emotions and provides that information to the server.

[1520] Hardware / Software Used

[1521] 1. Server: Analyzes images and videos, converts backgrounds, generates candidates, and generates final output. Specifically, it uses AI frameworks such as TensorFlow and PyTorch.

[1522] 2. Device: Provides the user interface, uploads images and videos, and displays the results. This can be a smartphone or PC.

[1523] 3. Emotion Engine: Recognizes user emotions using computer vision and speech recognition technologies, such as OpenCV and Google Cloud AI's emotion analysis API.

[1524] 4. AI module: Analyzes and evaluates location risks in images. Built using TensorFlow and PyTorch.

[1525] System operation example

[1526] First, the user selects the image they want to post to the SNS from their device and clicks the upload button. The device then sends the selected image data to the server, where it is encoded in Base64 format.

[1527] Next, the device's built-in emotion engine analyzes the user's facial expressions, tone of voice, and input text. For example, if the user is smiling at the camera, the emotion engine recognizes "happiness." The recognition results are sent to the server in JSON format.

[1528] The server inputs the received image data into the AI ​​module, which then analyzes the image's pixel data and uses OCR technology to detect address plates and signs. It also extracts elements that can help identify the location, such as the shape of the building and the background scenery.

[1529] Next, the AI ​​module evaluates the location risk based on each detected element and assigns a score. For example, if an address plate is clearly visible, it is evaluated as a high-risk element. Based on this result, the server lists the risk elements and passes them on to the next conversion process.

[1530] The server applies a conversion algorithm based on this list and emotion data. If the user is in a "happy" mood, the background will be replaced with a "seaside scene." The generated background candidate images are seamlessly synthesized using techniques such as GAN (Generative Adversarial Network).

[1531] The server then generates multiple conversion candidate images and sends them to the terminal for presentation to the user. The terminal displays the received candidate images to the user in thumbnail format and provides a selection screen. The user can select the most appropriate one from the presented candidate images. If necessary, the user can also instruct conversion again.

[1532] Finally, the server processes the selected candidate images, applying edge correction and color balancing before providing them to the user. The device then displays the final image to the user for final confirmation. The user can then post the confirmed image to a social networking site.

[1533] Examples and prompts

[1534] For example, a user selects a family photo from their smartphone's photo library and taps the "Upload" button in the app. The device sends the image to the server and displays the message "Recognizing emotions...". The emotion engine recognizes the user's smile and detects "enjoyment". The AI ​​module marks the house number plate at a specific position in the image as a "high-risk element". Based on the user's emotions, the server generates multiple background images, such as seaside, mountains, and city night views, and displays them on the device. The user selects the best one from these candidates to generate the final image.

[1535] Example prompt sentence:

[1536] "I want to upload photos from a family vacation, but I want to change the background so my home address is not identified. The sentiment is fun."

[1537] "I'm feeling calm right now, so I'd like you to change the background of the videos I upload to natural scenery."

[1538] This allows users to safely post images and videos to social media with reduced risk of their location being identified.

[1539] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1540] Step 1:

[1541] Image upload

[1542] The user selects an image from the file selection dialog on the device and clicks the upload button.

[1543] Input: An image file selected by the user.

[1544] The device sends the selected image data to the server via an HTTP request, where the image data is encoded in Base64 format.

[1545] Output: Base64 encoded image data is sent to the server.

[1546] Step 2:

[1547] emotion recognition

[1548] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, and input text.

[1549] Input: User's real-time facial expression data, voice data, and input text data.

[1550] The device uses emotion recognition algorithms to analyze the user's emotions in real time. For example, if the user smiles at the camera, it will recognize "happiness."

[1551] The emotion engine sends the recognized emotion data in JSON format to the server.

[1552] Output: Emotion data in JSON format is sent to the server.

[1553] Step 3:

[1554] Image analysis

[1555] The server passes the received image data to an AI module (e.g., TensorFlow or PyTorch model) and begins analysis.

[1556] Input: Base64 encoded image data.

[1557] The AI ​​module analyzes the pixel data of the image and uses OCR technology to detect address plates and signs, as well as extracting elements relevant to location identification, such as building shapes and background scenery.

[1558] Output: The analysis results in a list of risk factors.

[1559] Step 4:

[1560] Risk Assessment

[1561] The AI ​​module assesses the location risk of each detected element and assigns a risk score.

[1562] Input: A list of risk factors.

[1563] The server lists elements with high risk scores and passes the data to the next background conversion process.

[1564] Output: A list of elements for which risk assessment has been completed is generated.

[1565] Step 5:

[1566] Background Transformation

[1567] The server applies a transformation algorithm based on the risk factor list and emotion data.

[1568] Input: List of risk-assessed elements, sentiment data.

[1569] The conversion algorithm generates the most appropriate background based on the user's emotions. For example, if the user is in a happy mood, the background will be replaced with a bright seaside scene.

[1570] The generative AI model uses a Generative Adversarial Network (GAN) to generate new backgrounds and seamlessly combine them with the original image.

[1571] Output: Multiple composite background candidate images.

[1572] Step 6:

[1573] Candidate generation and presentation

[1574] The server generates a plurality of candidate conversion images and transmits them to the terminal for presentation to the user.

[1575] Input: Multiple synthesized background candidate images.

[1576] The terminal displays the received candidate images to the user in thumbnail format and provides a selection screen.

[1577] Output: Candidate images in thumbnail format presented to the user.

[1578] Step 7:

[1579] User Selection

[1580] The user selects the best image from the different candidate images presented.

[1581] Input: Candidate images displayed in thumbnail format.

[1582] The terminal accepts the user's selection and transmits the selection information to the server. If the user wishes to reconvert, the terminal can also transmit an instruction to the server.

[1583] Output: Data of selected candidate images.

[1584] Step 8:

[1585] Generating the final image

[1586] The server performs final processing on the selected candidate images, applying edge correction and color balancing.

[1587] Input: The data of the selected candidate image.

[1588] The server sends the final image back to the terminal for display to the user.

[1589] The terminal displays the final image to the user for final confirmation.

[1590] Output: The final image.

[1591] Step 9:

[1592] Post

[1593] Users can then post the final image they have confirmed to social media.

[1594] Input: The final image as viewed by the user.

[1595] The device uses the SNS API to upload the image to the specified account.

[1596] Output: Images posted to social media.

[1597] (Application example 2)

[1598] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1599] Conventional image and video posting systems pose the risk of revealing the location where a user took a photo. In particular, analyzing features such as buildings and signs in the background of a photo can make the location easily identifiable. Furthermore, changing the background without considering the user's emotions can result in lower user satisfaction. A new system is needed to solve these problems and enable safer and more satisfying image and video posting.

[1600] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1601] In this invention, the server includes means for uploading images, means for analyzing the background from the uploaded image and extracting risk elements for location identification, means for converting the extracted risk elements into other natural background elements, means for generating candidates for background conversion based on emotions using an emotion engine that recognizes the user's emotions, means for generating a plurality of converted candidate images and presenting them to the user, and means for generating a final image after receiving a selection and confirmation from the user. This enables a natural background change according to the user's emotions while reducing the risk of location identification.

[1602] The "means for uploading images" provides a function for users to send image data they have taken to the system.

[1603] "Means for analyzing the background and extracting risk elements for identifying a location" provides the functionality to detect and analyze elements (e.g., buildings and signs) that identify a specific location from uploaded images.

[1604] The "means for converting extracted risk elements into other natural background elements" provides a function for replacing detected location-specific risk elements with other safe and natural backgrounds.

[1605] The "emotion engine that recognizes user emotions" is a mechanism for analyzing and recognizing a user's emotional state from facial expressions, tone of voice, input text, etc.

[1606] The "means for generating background conversion candidates based on emotions" provides a function for generating optimal background conversion candidates according to the recognized emotions of the user.

[1607] The "means for generating a plurality of converted candidate images and presenting them to the user" provides a function for displaying a plurality of background conversion candidate images generated based on emotion to the user and making them selectable.

[1608] The "means for generating a final image upon receiving a user's selection and confirmation" provides a function for applying the background conversion candidate selected by the user and generating a final image.

[1609] MODE FOR CARRYING OUT THE INVENTION

[1610] This invention is a system that changes the background of images and videos posted on social media in a natural way to reduce the risk of location identification, and combines it with an emotion engine that recognizes the user's emotions. The system works as follows.

[1611] System Configuration

[1612] This system mainly consists of four components: a server, a device (smartphone), a user, and an emotion engine. The server is responsible for the main processes, such as analyzing images and videos, converting backgrounds, generating candidates, and generating the final output. The device provides the user interface, uploads images and videos, and displays the results. Users upload images and videos through their devices and select the best converted background from the candidates. The emotion engine recognizes the user's emotions and provides that information to the server.

[1613] What the program does

[1614] 1. Image upload: The user selects an image from their smartphone and uploads it to the app. At this time, the image data is read using an image processing library such as OpenCV.

[1615] 2. Emotion recognition: The device's built-in emotion engine (e.g., Google Cloud's Face API) recognizes emotions from the user's facial expressions and input text. This information is sent to the server.

[1616] 3. Background analysis: The server passes the received image data to the AI ​​module, which then begins analysis. The AI ​​module detects elements in the image that can lead to location identification, such as building shapes, signs, and background scenery.

[1617] 4. Risk Assessment: The AI ​​module assesses the risk of location identification based on the detected elements and sets a risk level for each element.

[1618] 5. Background conversion: The server extracts risk factors and converts the background based on information from the emotion engine. For example, if the user is in a happy mood, the background will be changed to a bright one, and if the user is in a calm mood, the background will be changed to a natural landscape.

[1619] 6. Candidate generation and presentation: The server generates multiple background conversion candidate images and sends them to the terminal for presentation to the user.

[1620] 7. User Selection: The user reviews the different candidate images and selects the best one. If necessary, the user can also request reconversion.

[1621] 8. Final image generation: The server finalizes the selected candidate images and sends them to the user. The terminal displays the final image to the user for final confirmation.

[1622] 9. Post: Users can post the final image to social media.

[1623] Hardware and software used

[1624] Hardware: Smartphone (iOS or Android compatible)

[1625] Software: OpenCV, emotion engine libraries (e.g. Google Cloud's Face API), and Python and TensorFlow on the server side for image analysis and background conversion.

[1626] Specific examples

[1627] For example, if a user uploads a smiling selfie, the emotion engine will determine the emotion of "joy." After analyzing risk factors, the server will generate a bright, natural background. The user can then select the best image from the presented candidate images and safely post it to social media.

[1628] Prompt Sentence Examples

[1629] Identify the emotion in an image uploaded by a user and automatically select the background that best matches that emotion to change the image background. For example, change the background to a bright natural scene for a "smiling face." This reduces the risk of location identification based on the original image.

[1630] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1631] Step 1:

[1632] The device provides an interface for the user to select an image from within the smartphone. The user selects an image and clicks the upload button within the app to input the image data into the device. The device then sends the selected image data to the server.

[1633] Step 2:

[1634] The server passes the received image data to the AI ​​module, which begins background analysis. Specifically, it uses OpenCV and an AI model to detect elements in the image that identify specific locations. The AI ​​model identifies location-specific risk elements, such as buildings and signs, and outputs their location information.

[1635] Step 3:

[1636] The server evaluates the risk of the location based on the output from the AI ​​module. The risk assessment is based on the type of element detected and its risk level. The output is a list of the risk level for each element.

[1637] Step 4:

[1638] The emotion engine installed on the device analyzes the user's emotions before or immediately after uploading. This emotion analysis is performed based on the user's facial expressions, voice, input text, etc. The emotion engine sends the analysis results to the server, which receives the user's emotional information as input.

[1639] Step 5:

[1640] The server performs background conversion based on the list of location-specific risk factors and the user's emotional information. Specifically, it removes or converts risk factors and generates an appropriate background based on the emotional information. For example, if the user is in a happy emotional state, the system generates a bright natural landscape.

[1641] Step 6:

[1642] The server prepares the generated multiple background conversion candidates for display and sends them to the device. The device presents the multiple conversion candidate images to the user and displays a selection screen. The user can review the conversion candidate images and select the most appropriate one.

[1643] Step 7:

[1644] The server receives the user-selected background candidate image as input and generates the final image. Specifically, it applies the selected background to the original image, adjusts unnecessary areas, and performs final processing.

[1645] Step 8:

[1646] The server sends the final image to the user's device, which displays it to the user, who can then post it to social media after reviewing it.

[1647] Step 9:

[1648] The user posts the final image to the SNS. Specifically, a dedicated button is provided to upload the generated final image data to the SNS, completing the posting process.

[1649] The above steps reduce the risk of location identification while changing the background according to the user's emotions, creating an environment in which images and videos can be posted to social media more safely.

[1650] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1651] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1652] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1653] [Fourth embodiment]

[1654] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1655] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1656] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1657] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1658] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1659] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1660] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1661] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1662] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1663] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1664] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1665] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1666] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1667] This invention relates to a system that changes the background of images and videos posted to social networking sites in a natural way to reduce the risk of location identification. The program processing of this system is specifically explained below in natural language.

[1668] System Overview

[1669] The system is mainly composed of three entities: the server, the device, and the user. The server is responsible for the main processes, such as image and video analysis, background conversion, candidate generation, and final output generation. The device provides the user interface, uploads images and videos, and displays the results. The user uploads images and videos through the device and selects the best one from the converted candidates.

[1670] Program processing

[1671] 1. Upload an image

[1672] Users select an image from their device and click the upload button.

[1673] The terminal transmits the selected image data to the server.

[1674] 2. Image Analysis

[1675] The server passes the received image data to the AI ​​module.

[1676] The AI ​​module analyzes the images and detects elements that could lead to location identification, such as building shapes, address plates, and signs.

[1677] 3. Risk Assessment

[1678] The AI ​​module assesses the location risk based on the detected elements and assigns a risk level to each element.

[1679] 4. Background conversion

[1680] The server extracts risk elements and generates a background image that replaces them with other natural elements.

[1681] Example: In a photo that shows Tokyo Tower, replace the Tokyo Tower with natural scenery or an ordinary apartment building.

[1682] 5. Candidate Generation and Presentation

[1683] The server generates multiple candidate conversion images and transmits them to the terminal for presentation to the user.

[1684] The terminal displays the received candidate images to the user and provides a selection screen.

[1685] 6. User Choice

[1686] The user checks the presented candidate images and selects the appropriate one.

[1687] If necessary, the user can also instruct reconversion.

[1688] 7. Generating the final image

[1689] The server finalizes the selected candidate images and sends them to the user.

[1690] The device displays the final image to the user for final confirmation.

[1691] 8. Submission

[1692] Users can then post the final image to social media.

[1693] Application to video

[1694] The same process is applied to videos, but the same process is performed on each frame while maintaining continuity, resulting in a natural background transition for the entire video.

[1695] 1. Upload a video

[1696] The user selects a video from their device and clicks the upload button.

[1697] The device transmits the video file to the server.

[1698] 2. Frame analysis and transformation

[1699] The server receives the video data and inputs each frame into the AI ​​module.

[1700] The AI ​​module analyzes each frame, detects elements that pose a risk of location identification, and transforms them into natural backgrounds.

[1701] 3. Maintaining continuity

[1702] The server performs conversion processing that takes into account the continuity of frames in order to generate images that do not look out of place between consecutive frames.

[1703] 4. Final video generation

[1704] The server generates the final video and sends it to the device.

[1705] The device will then display the final video to the user and ask for confirmation.

[1706] 5. Post

[1707] Users can then post the final video to social media.

[1708] This makes it possible to provide a system that reduces the risk of location identification and allows users to safely post images and videos with natural-looking backgrounds to social media.

[1709] The processing flow will be explained below.

[1710] Step 1:

[1711] Users select the image they want to upload from their device and click the upload button.

[1712] Step 2:

[1713] The terminal transmits the selected image data to the server.

[1714] Step 3:

[1715] The server receives the uploaded image data.

[1716] Step 4:

[1717] The server passes the image to the AI ​​module and begins analysis.

[1718] Step 5:

[1719] The AI ​​module detects elements in the image that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[1720] Step 6:

[1721] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[1722] Step 7:

[1723] The server extracts risk elements and generates a background image that replaces them with other natural background elements.

[1724] Step 8:

[1725] The server generates a number of candidate transformed images.

[1726] Step 9:

[1727] The server sends the generated candidate images to the user's terminal.

[1728] Step 10:

[1729] The terminal displays the received candidate images to the user and provides a confirmation screen.

[1730] Step 11:

[1731] The user reviews the different candidate images presented and selects the best one.

[1732] Step 12:

[1733] If the user desires reconversion, he or she inputs the instruction into the terminal.

[1734] Step 13:

[1735] The server receives the user's instruction for reconversion, generates candidate images again, and sends them to the terminal.

[1736] Step 14:

[1737] The user makes the final selection and sends confirmation to the device.

[1738] Step 15:

[1739] The server finalizes the candidate images selected by the user to generate the final image.

[1740] Step 16:

[1741] The server sends the final image to the user's device.

[1742] Step 17:

[1743] The terminal displays the final image to the user and provides an interface for final confirmation.

[1744] Step 18:

[1745] The user then makes a final confirmation and enters instructions into the device to post to social media.

[1746] Step 19:

[1747] The device retrieves the final image from the server and posts it to a social media platform.

[1748] The above is the flow of image processing.

[1749] Application to video

[1750] Step 1:

[1751] Users select the video they want to upload from their device and click the upload button.

[1752] Step 2:

[1753] The terminal transmits the selected video data to the server.

[1754] Step 3:

[1755] The server receives the uploaded video data.

[1756] Step 4:

[1757] The server passes each frame of the video to the AI ​​module and begins analysis.

[1758] Step 5:

[1759] The AI ​​module detects elements in each frame that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[1760] Step 6:

[1761] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[1762] Step 7:

[1763] The server extracts risk elements for each frame and generates a background image that replaces them with other natural background elements.

[1764] Step 8:

[1765] The server performs the conversion process so that there is no sense of incongruity between consecutive frames, and generates the final video.

[1766] Step 9:

[1767] The server sends the final video to the user's device.

[1768] Step 10:

[1769] The device displays the final video to the user and provides a confirmation screen.

[1770] Step 11:

[1771] The user reviews the final video and enters instructions into the device to post it to social media.

[1772] Step 12:

[1773] The device retrieves the final video from the server and posts it to a social media platform.

[1774] Example 1

[1775] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1776] When posting images or videos to social networking sites, there is a risk that elements such as buildings, address plates, and signs that appear in the background may be identified by a third party. This risk can lead to a violation of privacy, making it difficult for posters to use social networking sites safely. Furthermore, in the case of videos, background conversion technology is also required to prevent the appearance of incongruity between consecutive frames. Therefore, a system is needed that can convert backgrounds to natural ones while reducing the risk of location identification.

[1777] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1778] In this invention, the server includes means for uploading images or videos, means for analyzing the background from each frame of the uploaded image or video and extracting risk elements for location identification, means for converting the extracted risk elements into other natural background elements, means for generating a natural background image using a generative AI model and replacing the risk elements for location identification, means for generating a plurality of candidate converted images or videos and presenting them to the user, and means for generating a final image or video after receiving a selection and confirmation from the user. This makes it possible to achieve conversion to a natural and safe background while reducing the risk of location identification.

[1779] "Means for uploading images or videos" refers to the function for sending image or video data from a user's device to a server.

[1780] "Means for analyzing the background from each frame of an image or video and extracting location-specific risk elements" refers to a function that uses an AI module to analyze each frame of an image or video and detect location-specific risk elements such as buildings, address plates, and signs.

[1781] "Means for transforming location risk elements into other natural background elements" refers to the function responsible for the transformation process to replace location risk elements with other natural backgrounds.

[1782] "Means for generating natural-looking background images using generative AI models to replace location-specific risk elements" refers to a function that uses AI technology to generate new background images and replace them with location-specific risk elements, thereby providing a natural-looking background.

[1783] "Means for generating multiple converted candidate images or videos and presenting them to the user" refers to a function that allows the server to present candidate images or videos generated using multiple conversion patterns to the user.

[1784] "Means for generating a final image or video upon user selection and confirmation" refers to the function that allows the user to finally confirm the image or video selected from the candidates and generate the approved final image or video.

[1785] This invention relates to a system that changes the background of images and videos to a more natural look when they are posted to a social networking site, reducing the risk of location identification. This system is composed of three entities: a server, a terminal, and a user.

[1786] System configuration

[1787] The system mainly uses the following major hardware and software:

[1788] 1. Server

[1789] Hardware: A computer system with a powerful processor, memory, and storage devices.

[1790] Software: Install AI modules (e.g., OpenCV, TensorFlow), image processing libraries, and generative AI models (e.g., DALL-E, GAN).

[1791] 2. Terminal

[1792] Hardware: The devices that users regularly use, such as smartphones, tablets, and PCs.

[1793] Software: A dedicated application or web browser for uploading images and videos and communicating with the server.

[1794] 3. Users

[1795] An individual who uses the system to upload images and videos to be posted on social media and check the final generated content.

[1796] System Features

[1797] Image / Video Upload

[1798] Users select the image or video they want to post from their device and click the upload button. The device then sends the selected image or video data to the server.

[1799] Image / video analysis

[1800] The server passes the received image or video data to an AI module (e.g., OpenCV or TensorFlow) for background analysis and location-specific risk element detection.

[1801] Location-specific element detection and risk assessment

[1802] The AI ​​module detects location-specific risk factors such as buildings, address plates, and signs, and the server performs risk assessment based on these risk factors.

[1803] Converting to a natural background

[1804] The server uses a generative AI model (e.g., DALL-E, GAN) to replace risk elements with a different, more natural background image. Specifically, it uses a generative AI model (e.g., DALL-E, GAN) and inputs the following prompt:

[1805] Example prompt: "Change the background of a photo showing Tokyo Tower to a typical apartment building, and generate an image that eliminates the risk of location identification."

[1806] Generation and presentation of candidate images / videos

[1807] The server generates multiple candidates for the converted images and videos, and sends them to the device, which then presents the candidate images and videos to the user.

[1808] User selection and confirmation

[1809] The user can select and confirm the most suitable image or video from the presented candidates. If necessary, the user can also instruct reconversion.

[1810] Final image / video generation

[1811] The server generates the selected final image or video and sends it to the device, which displays the final output to the user and supports posting to social media.

[1812] This allows users to safely post images and videos with natural-looking backgrounds to social media, while reducing the risk of their location being identified.

[1813] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1814] Step 1:

[1815] Image / Video Upload

[1816] Users use their device to select the image or video they want to post and click the upload button.

[1817] Input: An image or video file selected by the user.

[1818] Output: Image or video data sent from the device to the server.

[1819] Specific operation: Triggered by user operation, the device sends an HTTP request to the server for the selected file, including image or video data.

[1820] Step 2:

[1821] Image / video analysis

[1822] The server passes the received image or video data to the AI ​​module for analysis.

[1823] Input: Image or video data received by the server.

[1824] Output: Analysis results from the AI ​​module (detection of location-specific risk elements such as buildings, house numbers, signs, etc.).

[1825] Specific operation: The server inputs the received image or video data into an AI module such as OpenCV or TensorFlow, and analyzes the location-specific risk elements for each frame, extracting the location information and characteristics of the risk elements as a result.

[1826] Step 3:

[1827] Location-specific element detection and risk assessment

[1828] The AI ​​module assesses the location risk based on the detected elements.

[1829] Input: Analyzed risk elements (buildings, house numbers, signs, etc.).

[1830] Output: Risk level assessment for each risk factor.

[1831] Specific operation: Based on the analysis results, the AI ​​module returns the location-specific risk level of each risk element to the server, which then receives this and performs risk assessment.

[1832] Step 4:

[1833] Converting to a natural background

[1834] The server uses a generative AI model to replace risk elements with alternative, natural-looking background images.

[1835] Input: Risk assessment results and original image or video data.

[1836] Output: The generated natural background image, or the altered image or video data with the risk elements replaced.

[1837] How it works: The server uses a generative AI model (e.g., DALL-E, GAN) to replace risk elements with a different background. Specifically, it generates a prompt sentence and inputs it into the AI ​​model to generate an appropriate background image.

[1838] Example prompt: "Change the background of a photo showing Tokyo Tower to a typical apartment building, and generate an image that eliminates the risk of location identification."

[1839] Step 5:

[1840] Generation and presentation of candidate images / videos

[1841] The server generates multiple candidate images or videos using different transformation patterns and sends them to the terminal.

[1842] Input: Generated background image or modified image / video data.

[1843] Output: Multiple candidate images or videos.

[1844] Specific operation: The server synthesizes the background image obtained by the generative AI model with the original image or video, generates multiple candidates with different conversion patterns, and sends them to the device as an HTTP response.

[1845] Step 6:

[1846] User selection and confirmation

[1847] The user selects and confirms the most suitable image or video from the presented candidates.

[1848] Input: Multiple candidate images or videos, selected by the user.

[1849] Output: User-selected final candidate image or video, with instructions for reconversion if necessary.

[1850] Specific operation: The device receives multiple candidate images or videos from the server, displays them on the user interface, and allows the user to select the most suitable one. Once the user has made their selection, the selection information is sent to the server.

[1851] Step 7:

[1852] Final image / video generation

[1853] The server performs final processing on the selected candidate images and videos and sends them to the terminal.

[1854] Input: Information about the candidate image or video selected by the user.

[1855] Output: The final image or video data.

[1856] Specific operation: The server checks the candidate images and videos selected by the user, performs any necessary final processing, and then sends the final data to the device for user confirmation.

[1857] Step 8:

[1858] Posting to social media

[1859] Users post the final images and videos to social media.

[1860] Input: The final image or video data.

[1861] Output: Images or videos posted to social media.

[1862] Specific operation: The device displays the final image or video to the user, and if the user is satisfied, provides the ability to post it to a social networking site. When the user clicks the post button, the data is uploaded to the social networking site.

[1863] (Application example 1)

[1864] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1865] In modern society, when posting images or videos to social media, there is an increasing risk that individuals and locations may be identified from the content of the posts. It is necessary to resolve these privacy issues and provide an environment where users can share content with peace of mind.

[1866] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1867] In this invention, the server includes means for uploading images or videos, means for analyzing the background from the uploaded images or videos and extracting risk elements for location identification, means for converting the extracted risk elements into other natural background elements, means for generating a plurality of converted candidate images or video frames and presenting them to the user, means for the user to select an appropriate one from the presented candidates, means for generating a final image or final video frame upon receiving the user's selection and confirmation, and means for posting the final image or final video frame to an SNS. This allows users to post images or videos to an SNS with peace of mind while reducing the risk of location identification.

[1868] "Means for uploading images or videos" refers to a function that allows a user to select an image or video from their device and send it to the server.

[1869] "Means for analyzing the background and extracting location-specific risk elements" is a function that uses an AI module to identify and extract location-specific risk elements contained in the background.

[1870] The "means for converting risk elements into other natural background elements" is an algorithm for replacing the extracted risk elements with other natural background elements.

[1871] The "means for generating multiple candidate transformed images or video frames and presenting them to the user" refers to a function for creating different versions of transformed images or video frames and offering them to the user as options.

[1872] The "means for the user to select the appropriate one" is an interface that allows the user to select the most appropriate one from multiple candidate images or video frames presented.

[1873] "Means for generating a final image or video frame upon receiving a user selection and confirmation" refers to functionality for creating a final image or video frame based on the user's selection.

[1874] The "means for posting the final image or final video frame to an SNS" is a function for uploading the generated final image or video frame to an SNS.

[1875] The "means for converting into natural background elements while maintaining the continuity of video frames" is an algorithm for performing background conversion that does not look out of place between a series of video frames.

[1876] A "means for enabling reconversion at the user's command" is an interface that allows the user to request conversion again.

[1877] This invention relates to a system that changes the background of images and videos posted on social networking sites in a natural way, reducing the risk of location identification. The system is mainly composed of three entities: a server, a device, and a user.

[1878] Hardware and software used

[1879] 1. Hardware:

[1880] Server: A server with high-performance computing power.

[1881] Devices: Smartphones, smart glasses, head-mounted displays.

[1882] 2. Software:

[1883] AI module: Uses TensorFlow or PyTorch for image analysis and background conversion.

[1884] Image processing library: OpenCV is used for various editing of generated images.

[1885] Communication and data management: Flask is used for communication between the server and the terminal.

[1886] System Operation

[1887] 1. Upload an image or video:

[1888] Users select images or videos from their smartphones or other devices and upload them to the server through the application.

[1889] 2. Background analysis and risk factor extraction:

[1890] The server receives the uploaded content and passes it to an AI module (e.g., TensorFlow, PyTorch), which analyzes the image or video and extracts location-specific risk factors (e.g., building shapes, house number plates, signs, etc.).

[1891] 3. Background transformation:

[1892] After extracting risk elements, the AI ​​module converts them into other natural background elements. For example, if a photo shows Tokyo Tower, it will replace the Tokyo Tower with natural scenery or a typical apartment building.

[1893] 4. Generate and present the transformed candidate image or video frame:

[1894] The server generates multiple conversion candidates and sends them to the terminal for presentation to the user, who then displays them and offers a choice.

[1895] 5. User Choice and Confirmation:

[1896] The user can select the most suitable one from the presented candidates, and can also instruct reconversion if necessary.

[1897] 6. Generate the final image or video frame:

[1898] The server generates the final image or video frame based on the user's selection and sends it to the user, who then displays it on the device and asks for final confirmation.

[1899] 7. Posting to social media:

[1900] Users can post the final image or video frame to social media.

[1901] Specific examples

[1902] For example, if a user wants to post a photo taken at a tourist spot to a social networking site, the system can detect famous landmarks or signs in the photo and replace them with other natural backgrounds, allowing users to enjoy posting while reducing the risk of being identified.

[1903] Prompt Sentence Examples

[1904] Analyze the input image and detect location risk elements. Next, replace those elements with a natural background and generate an image. Use a method to present multiple candidate images. In this case, the generated image must be consistent between consecutive video frames.

[1905] This will allow users to post images and videos on social media with peace of mind, strengthening privacy protection.

[1906] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1907] Step 1:

[1908] The user selects an image or video from the device and uploads it to the server through the application. The input is the image or video file selected on the device, and the output is the image or video data sent to the server. Specifically, the user clicks the "Upload" button and selects the specified file.

[1909] Step 2:

[1910] The server receives uploaded image or video data and passes it to an AI module (e.g., TensorFlow, PyTorch). The input is the uploaded image or video data, and the output is the analysis request passed to the AI ​​module. Specifically, the server saves the received data in a specified folder and passes the path to the AI ​​module.

[1911] Step 3:

[1912] The AI ​​module analyzes images or videos to extract location-specific risk elements. The input is the uploaded image or video data, and the output is a list of detected risk elements. Specifically, the AI ​​module uses a convolutional neural network (CNN) to identify risk elements in the image, such as building shapes, house number plates, and signs.

[1913] Step 4:

[1914] The server receives the list of risk elements from the AI ​​module and converts them into natural background elements. The input is the list of extracted risk elements, and the output is the converted background image or video frames composited onto the background. Specifically, the server uses the OpenCV library to replace the risk elements with common landscapes and buildings.

[1915] Step 5:

[1916] The server generates multiple candidate transformed images or video frames and sends them to the device for presentation to the user. The input is the transformed background image or video frame, and the output is multiple candidate images or video frames. Specifically, the server generates different background variations, saves each as a candidate, and sends it to the device.

[1917] Step 6:

[1918] The user selects the best one from multiple candidates presented on the device and sends the selection information to the server. The input is multiple candidate images or video frames, and the output is the user's selection information. Specifically, the user selects one of the presented candidate images or video frames and clicks the "Select" button.

[1919] Step 7:

[1920] The server receives the user's selection information and generates the final image or video frame. The input is the user's selection information and the corresponding candidate image or video frame, and the output is the final image or video frame. Specifically, the server performs final processing on the selected background to generate the final image or video.

[1921] Step 8:

[1922] The terminal displays the final image or video frame to the user and asks for the user's final confirmation. The input is the final image or video frame, and the output is the user's final confirmation information. Specifically, the terminal displays the final image or video and provides a "Confirm" button.

[1923] Step 9:

[1924] The user posts the final image or video after confirmation to the social networking site. The input is the image or video frame that has been finalized, and the output is the information to be posted to the social networking site. Specifically, the user clicks the "Post" button to upload the image or video to the social networking site.

[1925] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1926] This invention combines a system that changes the background of images and videos posted to social networking sites in a natural way to reduce the risk of location identification with an emotion engine that recognizes the user's emotions.The system adjusts background conversion candidates according to the user's emotions, allowing it to provide more appropriate images and videos.

[1927] System Overview

[1928] The system is mainly composed of four entities: the server, the device, the user, and the emotion engine. The server is responsible for the main processes, such as image and video analysis, background conversion, candidate generation, and final output generation. The device provides the user interface, uploads images and videos, and displays the results. The user uploads images and videos through the device and selects the best one from the converted candidates. The emotion engine recognizes the user's emotions and provides this information to the server.

[1929] Program processing

[1930] 1. Upload an image

[1931] Users select an image from their device and click the upload button.

[1932] The terminal transmits the selected image data to the server.

[1933] 2. Emotion recognition

[1934] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, input text, etc. to recognize emotions.

[1935] The emotion engine sends the recognized emotion information to the server.

[1936] 3. Image Analysis

[1937] The server passes the received image data to the AI ​​module and begins analysis.

[1938] The AI ​​module detects elements in the image that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[1939] 4. Risk Assessment

[1940] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[1941] 5. Background conversion

[1942] The server extracts risk factors and changes the background based on information from the emotion engine. For example, if the user is in a happy mood, the background will be bright, and if the user is in a calm mood, the background will be changed to a natural landscape.

[1943] For example, if a photo shows Tokyo Tower, and the user is feeling calm, the Tokyo Tower part can be replaced with mountains or a lake.

[1944] 6. Candidate Generation and Presentation

[1945] The server generates multiple candidate conversion images and transmits them to the terminal for presentation to the user.

[1946] The terminal displays the received candidate images to the user and provides a selection screen.

[1947] 7. User Choice

[1948] The user reviews the different candidate images presented and selects the best one.

[1949] If necessary, the user can also instruct reconversion.

[1950] 8. Generating the final image

[1951] The server finalizes the selected candidate images and sends them to the user.

[1952] The device displays the final image to the user for final confirmation.

[1953] 9. Submission

[1954] Users can then post the final image to social media.

[1955] Application to video

[1956] The same process is applied to videos, but the same process is performed on each frame while maintaining continuity, resulting in a natural background transition for the entire video.

[1957] 1. Upload a video

[1958] Users select a video from their device and click the upload button.

[1959] The device transmits the video file to the server.

[1960] 2. Emotion recognition

[1961] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, input text, etc. to recognize emotions.

[1962] The emotion engine sends the recognized emotion information to the server.

[1963] 3. Frame Analysis and Transformation

[1964] The server receives the video data and inputs each frame into the AI ​​module.

[1965] The AI ​​module analyzes each frame, detects elements that pose a risk of location identification, and transforms them into natural backgrounds.

[1966] 4. Maintaining continuity

[1967] The server performs the conversion process so that there is no sense of incongruity between consecutive frames, and generates the final video.

[1968] 5. Final video generation

[1969] The server generates the final video and offers variations based on the user's emotions.

[1970] The device displays the final video to the user and provides a confirmation screen.

[1971] 6. Submission

[1972] Users can then post the final video to social media.

[1973] This will reduce the risk of location identification and provide a system that allows users to safely post images and videos to social media with backgrounds that change naturally based on emotions.

[1974] The processing flow will be explained below.

[1975] Image processing flow

[1976] Step 1:

[1977] Users select the image they want to upload from their device and click the upload button.

[1978] Step 2:

[1979] The terminal transmits the selected image data to the server.

[1980] Step 3:

[1981] The server receives the uploaded image data.

[1982] Step 4:

[1983] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, and input text to recognize their emotions.

[1984] Step 5:

[1985] The emotion engine sends the user's recognized emotion information to the server. For example, if the user is smiling, the emotion of "joy" is recognized.

[1986] Step 6:

[1987] The server passes the received image data to the AI ​​module and begins analyzing the image.

[1988] Step 7:

[1989] The AI ​​module detects elements in the image that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[1990] Step 8:

[1991] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[1992] Step 9:

[1993] The server extracts risk factors and generates a background image that replaces them with other natural background factors. For example, if a risk factor is high, it is set to a tranquil landscape that completely covers it.

[1994] Step 10:

[1995] The server uses emotion information from the emotion engine to adjust the style of background transformation, for example, if the user's emotion is "joy", it selects a light-colored background.

[1996] Step 11:

[1997] The server generates a number of candidate transformed images.

[1998] Step 12:

[1999] The server sends the generated candidate images to the user's terminal.

[2000] Step 13:

[2001] The terminal displays the received candidate images to the user and provides a confirmation screen.

[2002] Step 14:

[2003] The user reviews the different candidate images presented and selects the best one.

[2004] Step 15:

[2005] If the user desires reconversion, he or she inputs the instruction into the terminal.

[2006] Step 16:

[2007] The server receives the user's instruction for reconversion, generates candidate images again, and sends them to the terminal.

[2008] Step 17:

[2009] The user makes the final selection and sends confirmation to the device.

[2010] Step 18:

[2011] The server finalizes the candidate images selected by the user to generate the final image.

[2012] Step 19:

[2013] The server sends the final image to the user's device.

[2014] Step 20:

[2015] The terminal displays the final image to the user and provides an interface for final confirmation.

[2016] Step 21:

[2017] The user then makes a final confirmation and enters instructions into the device to post to social media.

[2018] Step 22:

[2019] The device retrieves the final image from the server and posts it to a social media platform.

[2020] Video processing flow

[2021] Step 1:

[2022] Users select the video they want to upload from their device and click the upload button.

[2023] Step 2:

[2024] The terminal transmits the selected video data to the server.

[2025] Step 3:

[2026] The server receives the uploaded video data.

[2027] Step 4:

[2028] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, and input text to recognize their emotions.

[2029] Step 5:

[2030] The emotion engine sends the recognized user emotion information to the server.

[2031] Step 6:

[2032] The server passes each frame of video data to the AI ​​module and begins analysis.

[2033] Step 7:

[2034] The AI ​​module detects elements in each frame that can lead to location identification, such as building shapes, address plates, signs, and background scenery.

[2035] Step 8:

[2036] The AI ​​module assesses location risk based on detected elements and sets a risk level for each element.

[2037] Step 9:

[2038] The server extracts risk elements for each frame and converts them into a background image that replaces them with other natural background elements.

[2039] Step 10:

[2040] The server uses the emotion information from the emotion engine to adjust the style of the background transformation.

[2041] Step 11:

[2042] The server performs the conversion process so that there is no sense of incongruity between consecutive frames, and generates the final video.

[2043] Step 12:

[2044] The server sends the final video to the user's device.

[2045] Step 13:

[2046] The device displays the final video to the user and provides a confirmation screen.

[2047] Step 14:

[2048] The user reviews the final video and enters instructions into the device to post it to social media.

[2049] Step 15:

[2050] The device retrieves the final video from the server and posts it to a social media platform.

[2051] Example 2

[2052] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2053] With conventional SNS posts, the background of images and videos is exposed as is, increasing the risk of location identification. Furthermore, because the background is not changed to take into account the user's feelings, there is also the issue of it being difficult to change the background appropriately. There is a need to solve these problems and provide an environment where users can post images and videos on SNS with peace of mind.

[2054] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2055] In this invention, the server includes means for uploading images, means for recognizing a user's emotion from the uploaded image, means for adjusting background conversion candidates based on the recognized emotion, means for analyzing the uploaded image and extracting risk elements for location identification, means for converting the extracted risk elements into other natural background elements, means for generating a plurality of converted candidate images and presenting them to the user, and means for generating a final image after receiving a selection and confirmation from the user. This allows background conversion according to the user's emotion, reducing the risk of location identification and making it possible to provide images and videos that can be posted safely to SNS.

[2056] "Means for uploading images" is a function that allows a user to send image data selected from the terminal to the server.

[2057] "Means for recognizing user emotions" refers to a function that analyzes emotions from the user's facial expressions, tone of voice, input text, etc., and provides that information to the server.

[2058] The "means for adjusting background conversion candidates" is a function that selects and adjusts suggested background image variations based on the recognized user emotions.

[2059] "Means for extracting risk factors for location identification" is a function that analyzes elements that can lead to location identification from uploaded images and evaluates the risk.

[2060] The "means for converting into background elements" is a function that uses a generative AI model to convert the extracted risk elements into another natural background.

[2061] The "means for generating multiple candidate images and presenting them to the user" is a function for generating multiple converted background variations and displaying them on the user's device.

[2062] The "means for generating a final image" is a function for finalizing the candidate image selected by the user and providing it to the user.

[2063] This invention relates to a system that changes the background of images and videos posted to social networking sites to create a natural look. This system has a function to recognize the user's emotions, reducing the risk of location identification and allowing users to post content with peace of mind. Specific embodiments of this system are described below.

[2064] This system mainly consists of four components: a server, a terminal, a user, and an emotion engine. The server is responsible for analyzing images and videos, converting backgrounds, generating candidates, and generating the final output. The terminal provides the user interface, uploads images and videos, and displays the results. Users upload images and videos through their terminals and select the best one from the converted candidates. The emotion engine recognizes the user's emotions and provides that information to the server.

[2065] Hardware / Software Used

[2066] 1. Server: Analyzes images and videos, converts backgrounds, generates candidates, and generates final output. Specifically, it uses AI frameworks such as TensorFlow and PyTorch.

[2067] 2. Device: Provides the user interface, uploads images and videos, and displays the results. This can be a smartphone or PC.

[2068] 3. Emotion Engine: Recognizes user emotions using computer vision and speech recognition technologies, such as OpenCV and Google Cloud AI's emotion analysis API.

[2069] 4. AI module: Analyzes and evaluates location risks in images. Built using TensorFlow and PyTorch.

[2070] System operation example

[2071] First, the user selects the image they want to post to the SNS from their device and clicks the upload button. The device then sends the selected image data to the server, where it is encoded in Base64 format.

[2072] Next, the device's built-in emotion engine analyzes the user's facial expressions, tone of voice, and input text. For example, if the user is smiling at the camera, the emotion engine recognizes "happiness." The recognition results are sent to the server in JSON format.

[2073] The server inputs the received image data into the AI ​​module, which then analyzes the image's pixel data and uses OCR technology to detect address plates and signs. It also extracts elements that can help identify the location, such as the shape of the building and the background scenery.

[2074] Next, the AI ​​module evaluates the location risk based on each detected element and assigns a score. For example, if an address plate is clearly visible, it is evaluated as a high-risk element. Based on this result, the server lists the risk elements and passes them on to the next conversion process.

[2075] The server applies a conversion algorithm based on this list and emotion data. If the user is in a "happy" mood, the background will be replaced with a "seaside scene." The generated background candidate images are seamlessly synthesized using techniques such as GAN (Generative Adversarial Network).

[2076] The server then generates multiple conversion candidate images and sends them to the terminal for presentation to the user. The terminal displays the received candidate images to the user in thumbnail format and provides a selection screen. The user can select the most appropriate one from the presented candidate images. If necessary, the user can also instruct conversion again.

[2077] Finally, the server processes the selected candidate images, applying edge correction and color balancing before providing them to the user. The device then displays the final image to the user for final confirmation. The user can then post the confirmed image to a social networking site.

[2078] Examples and prompts

[2079] For example, a user selects a family photo from their smartphone's photo library and taps the "Upload" button in the app. The device sends the image to the server and displays the message "Recognizing emotions...". The emotion engine recognizes the user's smile and detects "enjoyment". The AI ​​module marks the house number plate at a specific position in the image as a "high-risk element". Based on the user's emotions, the server generates multiple background images, such as seaside, mountains, and city night views, and displays them on the device. The user selects the best one from these candidates to generate the final image.

[2080] Example prompt sentence:

[2081] "I want to upload photos from a family vacation, but I want to change the background so my home address is not identified. The sentiment is fun."

[2082] "I'm feeling calm right now, so I'd like you to change the background of the videos I upload to natural scenery."

[2083] This allows users to safely post images and videos to social media with reduced risk of their location being identified.

[2084] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2085] Step 1:

[2086] Image upload

[2087] The user selects an image from the file selection dialog on the device and clicks the upload button.

[2088] Input: An image file selected by the user.

[2089] The device sends the selected image data to the server via an HTTP request, where the image data is encoded in Base64 format.

[2090] Output: Base64 encoded image data is sent to the server.

[2091] Step 2:

[2092] emotion recognition

[2093] The device's built-in emotion engine analyzes the user's facial expressions, tone of voice, and input text.

[2094] Input: User's real-time facial expression data, voice data, and input text data.

[2095] The device uses emotion recognition algorithms to analyze the user's emotions in real time. For example, if the user smiles at the camera, it will recognize "happiness."

[2096] The emotion engine sends the recognized emotion data in JSON format to the server.

[2097] Output: Emotion data in JSON format is sent to the server.

[2098] Step 3:

[2099] Image analysis

[2100] The server passes the received image data to an AI module (e.g., TensorFlow or PyTorch model) and begins analysis.

[2101] Input: Base64 encoded image data.

[2102] The AI ​​module analyzes the pixel data of the image and uses OCR technology to detect address plates and signs, as well as extracting elements relevant to location identification, such as building shapes and background scenery.

[2103] Output: The analysis results in a list of risk factors.

[2104] Step 4:

[2105] Risk Assessment

[2106] The AI ​​module assesses the location risk of each detected element and assigns a risk score.

[2107] Input: A list of risk factors.

[2108] The server lists elements with high risk scores and passes the data to the next background conversion process.

[2109] Output: A list of elements for which risk assessment has been completed is generated.

[2110] Step 5:

[2111] Background Transformation

[2112] The server applies a transformation algorithm based on the risk factor list and emotion data.

[2113] Input: List of risk-assessed elements, sentiment data.

[2114] The conversion algorithm generates the most appropriate background based on the user's emotions. For example, if the user is in a happy mood, the background will be replaced with a bright seaside scene.

[2115] The generative AI model uses a Generative Adversarial Network (GAN) to generate new backgrounds and seamlessly combine them with the original image.

[2116] Output: Multiple composite background candidate images.

[2117] Step 6:

[2118] Candidate generation and presentation

[2119] The server generates a plurality of candidate conversion images and transmits them to the terminal for presentation to the user.

[2120] Input: Multiple synthesized background candidate images.

[2121] The terminal displays the received candidate images to the user in thumbnail format and provides a selection screen.

[2122] Output: Candidate images in thumbnail format presented to the user.

[2123] Step 7:

[2124] User Selection

[2125] The user selects the best image from the different candidate images presented.

[2126] Input: Candidate images displayed in thumbnail format.

[2127] The terminal accepts the user's selection and transmits the selection information to the server. If the user wishes to reconvert, the terminal can also transmit an instruction to the server.

[2128] Output: Data of selected candidate images.

[2129] Step 8:

[2130] Generating the final image

[2131] The server performs final processing on the selected candidate images, applying edge correction and color balancing.

[2132] Input: The data of the selected candidate image.

[2133] The server sends the final image back to the terminal for display to the user.

[2134] The terminal displays the final image to the user for final confirmation.

[2135] Output: The final image.

[2136] Step 9:

[2137] Post

[2138] Users can then post the final image they have confirmed to social media.

[2139] Input: The final image as viewed by the user.

[2140] The device uses the SNS API to upload the image to the specified account.

[2141] Output: Images posted to social media.

[2142] (Application example 2)

[2143] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2144] Conventional image and video posting systems pose the risk of revealing the location where a user took a photo. In particular, analyzing features such as buildings and signs in the background of a photo can make the location easily identifiable. Furthermore, changing the background without considering the user's emotions can result in lower user satisfaction. A new system is needed to solve these problems and enable safer and more satisfying image and video posting.

[2145] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2146] In this invention, the server includes means for uploading images, means for analyzing the background from the uploaded image and extracting risk elements for location identification, means for converting the extracted risk elements into other natural background elements, means for generating candidates for background conversion based on emotions using an emotion engine that recognizes the user's emotions, means for generating a plurality of converted candidate images and presenting them to the user, and means for generating a final image after receiving a selection and confirmation from the user. This enables a natural background change according to the user's emotions while reducing the risk of location identification.

[2147] The "means for uploading images" provides a function for users to send image data they have taken to the system.

[2148] "Means for analyzing the background and extracting risk elements for identifying a location" provides the functionality to detect and analyze elements (e.g., buildings and signs) that identify a specific location from uploaded images.

[2149] The "means for converting extracted risk elements into other natural background elements" provides a function for replacing detected location-specific risk elements with other safe and natural backgrounds.

[2150] The "emotion engine that recognizes user emotions" is a mechanism for analyzing and recognizing a user's emotional state from facial expressions, tone of voice, input text, etc.

[2151] The "means for generating background conversion candidates based on emotions" provides a function for generating optimal background conversion candidates according to the recognized emotions of the user.

[2152] The "means for generating a plurality of converted candidate images and presenting them to the user" provides a function for displaying a plurality of background conversion candidate images generated based on emotion to the user and making them selectable.

[2153] The "means for generating a final image upon receiving a user's selection and confirmation" provides a function for applying the background conversion candidate selected by the user and generating a final image.

[2154] MODE FOR CARRYING OUT THE INVENTION

[2155] This invention is a system that changes the background of images and videos posted on social media in a natural way to reduce the risk of location identification, and combines it with an emotion engine that recognizes the user's emotions. The system works as follows.

[2156] System Configuration

[2157] This system mainly consists of four components: a server, a device (smartphone), a user, and an emotion engine. The server is responsible for the main processes, such as analyzing images and videos, converting backgrounds, generating candidates, and generating the final output. The device provides the user interface, uploads images and videos, and displays the results. Users upload images and videos through their devices and select the best converted background from the candidates. The emotion engine recognizes the user's emotions and provides that information to the server.

[2158] What the program does

[2159] 1. Image upload: The user selects an image from their smartphone and uploads it to the app. At this time, the image data is read using an image processing library such as OpenCV.

[2160] 2. Emotion recognition: The device's built-in emotion engine (e.g., Google Cloud's Face API) recognizes emotions from the user's facial expressions and input text. This information is sent to the server.

[2161] 3. Background analysis: The server passes the received image data to the AI ​​module, which then begins analysis. The AI ​​module detects elements in the image that can lead to location identification, such as building shapes, signs, and background scenery.

[2162] 4. Risk Assessment: The AI ​​module assesses the risk of location identification based on the detected elements and sets a risk level for each element.

[2163] 5. Background conversion: The server extracts risk factors and converts the background based on information from the emotion engine. For example, if the user is in a happy mood, the background will be changed to a bright one, and if the user is in a calm mood, the background will be changed to a natural landscape.

[2164] 6. Candidate generation and presentation: The server generates multiple background conversion candidate images and sends them to the terminal for presentation to the user.

[2165] 7. User Selection: The user reviews the different candidate images and selects the best one. If necessary, the user can also request reconversion.

[2166] 8. Final image generation: The server finalizes the selected candidate images and sends them to the user. The terminal displays the final image to the user for final confirmation.

[2167] 9. Post: Users can post the final image to social media.

[2168] Hardware and software used

[2169] Hardware: Smartphone (iOS or Android compatible)

[2170] Software: OpenCV, emotion engine libraries (e.g. Google Cloud's Face API), and Python and TensorFlow on the server side for image analysis and background conversion.

[2171] Specific examples

[2172] For example, if a user uploads a smiling selfie, the emotion engine will determine the emotion of "joy." After analyzing risk factors, the server will generate a bright, natural background. The user can then select the best image from the presented candidate images and safely post it to social media.

[2173] Prompt Sentence Examples

[2174] Identify the emotion in an image uploaded by a user and automatically select the background that best matches that emotion to change the image background. For example, change the background to a bright natural scene for a "smiling face." This reduces the risk of location identification based on the original image.

[2175] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2176] Step 1:

[2177] The device provides an interface for the user to select an image from within the smartphone. The user selects an image and clicks the upload button within the app to input the image data into the device. The device then sends the selected image data to the server.

[2178] Step 2:

[2179] The server passes the received image data to the AI ​​module, which begins background analysis. Specifically, it uses OpenCV and an AI model to detect elements in the image that identify specific locations. The AI ​​model identifies location-specific risk elements, such as buildings and signs, and outputs their location information.

[2180] Step 3:

[2181] The server evaluates the risk of the location based on the output from the AI ​​module. The risk assessment is based on the type of element detected and its risk level. The output is a list of the risk level for each element.

[2182] Step 4:

[2183] The emotion engine installed on the device analyzes the user's emotions before or immediately after uploading. This emotion analysis is performed based on the user's facial expressions, voice, input text, etc. The emotion engine sends the analysis results to the server, which receives the user's emotional information as input.

[2184] Step 5:

[2185] The server performs background conversion based on the list of location-specific risk factors and the user's emotional information. Specifically, it removes or converts risk factors and generates an appropriate background based on the emotional information. For example, if the user is in a happy emotional state, the system generates a bright natural landscape.

[2186] Step 6:

[2187] The server prepares the generated multiple background conversion candidates for display and sends them to the device. The device presents the multiple conversion candidate images to the user and displays a selection screen. The user can review the conversion candidate images and select the most appropriate one.

[2188] Step 7:

[2189] The server receives the user-selected background candidate image as input and generates the final image. Specifically, it applies the selected background to the original image, adjusts unnecessary areas, and performs final processing.

[2190] Step 8:

[2191] The server sends the final image to the user's device, which displays it to the user, who can then post it to social media after reviewing it.

[2192] Step 9:

[2193] The user posts the final image to the SNS. Specifically, a dedicated button is provided to upload the generated final image data to the SNS, completing the posting process.

[2194] The above steps reduce the risk of location identification while changing the background according to the user's emotions, creating an environment in which images and videos can be posted to social media more safely.

[2195] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2196] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2197] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2198] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2199] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2200] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2201] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2202] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2203] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2204] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2205] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2206] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2207] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2208] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2209] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2210] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2211] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2212] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2213] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2214] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2215] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2216] The following is further disclosed regarding the above embodiment.

[2217] (Claim 1)

[2218] How to upload images;

[2219] A means for analyzing the background of uploaded images and extracting location-specific risk factors;

[2220] A means for converting the extracted risk elements into other natural background elements;

[2221] means for generating and presenting a plurality of candidate transformed images to a user;

[2222] The system includes a means for generating a final image upon user selection and confirmation.

[2223] (Claim 2)

[2224] The system of claim 1 , further comprising: means for converting the extracted risk elements into natural background elements while preserving the continuity of the video frames.

[2225] (Claim 3)

[2226] 10. The system of claim 1, further comprising means for enabling user-directed reconversion.

[2227] "Example 1"

[2228] (Claim 1)

[2229] a means for uploading images or videos;

[2230] means for analyzing the background of each uploaded image or video frame to extract location-specific risk factors;

[2231] A means for converting the extracted risk elements into other natural background elements;

[2232] a means for generating natural background images using a generative AI model to replace location-specific risk elements;

[2233] means for generating and presenting a plurality of candidate transformed images or videos to a user;

[2234] The system includes a means for generating a final image or video upon user selection and confirmation.

[2235] (Claim 2)

[2236] The system of claim 1 , further comprising: means for converting the extracted risk elements into natural background elements while preserving the continuity of the video frames.

[2237] (Claim 3)

[2238] 10. The system of claim 1, further comprising means for enabling user-directed reconversion.

[2239] "Application Example 1"

[2240] (Claim 1)

[2241] a means for uploading images or videos;

[2242] A means for analyzing the background of uploaded images or videos and extracting location-specific risk factors;

[2243] A means for converting the extracted risk elements into other natural background elements;

[2244] means for generating and presenting a plurality of candidate transformed images or video frames to a user;

[2245] A means for the user to select an appropriate one from the presented candidates;

[2246] means for generating a final image or video frame upon user selection and confirmation;

[2247] The system includes a means for posting the final image or final video frame to a social networking site.

[2248] (Claim 2)

[2249] The system of claim 1 , further comprising: means for converting the extracted risk elements into natural background elements while preserving the continuity of the video frames.

[2250] (Claim 3)

[2251] 10. The system of claim 1, further comprising means for enabling user-directed reconversion.

[2252] "Example 2: Combining Emotion Engines"

[2253] (Claim 1)

[2254] How to upload images;

[2255] A means for recognizing a user's emotion from an uploaded image;

[2256] means for adjusting candidate background transformations based on the recognized emotion;

[2257] A means for analyzing the uploaded image and extracting location-specific risk factors;

[2258] A means for converting the extracted risk elements into other natural background elements;

[2259] means for generating and presenting a plurality of candidate transformed images to a user;

[2260] The system includes a means for generating a final image upon user selection and confirmation.

[2261] (Clai...

Claims

1. How to upload images; A means for analyzing the background of uploaded images and extracting location-specific risk factors; A means for converting the extracted risk elements into other natural background elements; means for generating and presenting a plurality of candidate transformed images to a user; The system includes a means for generating a final image upon user selection and confirmation.

2. The system of claim 1 , further comprising: means for converting the extracted risk elements into natural background elements while preserving the continuity of the video frames.

3. 10. The system of claim 1, further comprising means for enabling user-directed reconversion.

Images